NATIONAL CENTER FOR EDUCATION STATISTICS

U.S. Department of Education

Office of Educational Research and Improvement NCES 2001-509

TECHNICAL REPORT

THE NAEP 1998
What is The Nation’s Report Card?

THE NATION’S REPORT CARD, the National Assessment of Educational Progress (NAEP), is the only nationally

representative and continuing assessment of what America’s students know and can do in various subject areas. Since 1969,

assessments have been conducted periodically in reading, mathematics, science, writing, history, geography, and other

fields. By making objective information on student performance available to policymakers at the national, state, and local

levels, NAEP is an integral part of our nation’s evaluation of the condition and progress of education. Only information

related to academic achievement is collected under this program. NAEP guarantees the privacy of individual students and

their families.

NAEP is a congressionally mandated project of the National Center for Education Statistics, the U.S. Department of

Education. The Commissioner of Education Statistics is responsible, by law, for carrying out the NAEP project through

competitive awards to qualified organizations. NAEP reports directly to the Commissioner, who is also responsible

for providing continuing reviews, including validation studies and solicitation of public comment, on NAEP’s conduct

and usefulness.

In 1988, Congress established the National Assessment Governing Board (NAGB) to formulate policy guidelines for

NAEP. The Board is responsible for selecting the subject areas to be assessed from among those included in the National

Education Goals; for setting appropriate student performance levels; for developing assessment objectives and test

specifications through a national consensus approach; for designing the assessment methodology; for developing guidelines

for reporting and disseminating NAEP results; for developing standards and procedures for interstate, regional, and national

comparisons; for determining the appropriateness of test items and ensuring they are free from bias; and for taking actions

to improve the form and use of the National Assessment.

The National Assessment Governing Board

Mark D. Musick, Chair

President

Southern Regional Education Board

Atlanta, Georgia

Michael T. Nettles, Vice Chair

Professor of Education

University of Michigan

Ann Arbor, Michigan

Moses Barnes

Secondary School Principal

Fort Lauderdale, Florida

Melanie A. Campbell

Fourth-Grade Teacher

Topeka, Kansas

Honorable Wilmer S. Cody

Former Commissioner of Education

State of Kentucky

Frankfort, Kentucky

Daniel A. Domenech

Superintendent of Schools

Fairfax County Public Schools

Fairfax, Virginia

Edward Donley

Former Chairman

Air Products & Chemicals, Inc.

Allentown, Pennsylvania

Thomas H. Fisher

Director

Student Assessment Services

Florida Department of Education

Tallahassee, Florida

Edward H. Haertel

Professor, School of Education

Stanford University

Stanford, California

Juanita Haugen

Local School Board Member

Pleasanton, California

Honorable Nancy Kopp

State Legislator

Annapolis, Maryland

Honorable Ronnie Musgrove

Governor of Mississippi

Jackson, Mississippi

Roy M. Nageak, Sr.

First Vice-Chair

Alaska Board of Education and

Early Development

Barrow, Alaska

Debra Paulson

Eighth-Grade Mathematics Teacher

El Paso, Texas

Honorable Jo Ann Pottorff

State Legislator

Wichita, Kansas

Diane Ravitch

Research Professor

New York University

New York, New York

Sister Lourdes Sheehan, R.S.M.

Secretary for Education

United States Catholic Conference

Washington, DC

John H. Stevens

Executive Director

Texas Business and Education Coalition

Austin, Texas

Adam Urbanski

President

Rochester Teachers Association

Rochester, New York

Migdania D. Vega

Principal

Coral Way Elementary Bilingual School

Miami, Florida

Deborah Voltz

Assistant Professor

Department of Special Education

University of Louisville

Louisville, Kentucky

Honorable Michael E. Ward

State Superintendent of Public

Instruction

North Carolina Public Schools

Raleigh, North Carolina

Marilyn A. Whirry

Twelfth-Grade English Teacher

Manhattan Beach, California

Dennie Palmer Wolf

Senior Research Associate

Harvard University

Graduate School of Education

Cambridge, Massachusetts

(Ex-Officio)

Assistant Secretary of Education

Office of Educational Research and

Improvement

U.S. Department of Education

Washington, DC

Roy Truby

Executive Director, NAGB

Washington, DC
Nancy L. Allen

John R. Donoghue

Terry L. Schoeps

in collaboration with

Mary Lyn Bourque, Charles Brungardt, Nancy W. Caldwell, James E. Carlson,

Hua-Hua Chang, Patricia L. Donahue, John J. Ferris, David S. Freund,

Elissa A. Greenwald, Lucy M. Gray, Steven P. Isham, Frank Jenkins,

Eugene G. Johnson, Bruce A. Kaplan, Tom Krenzke, Edward Kulick,

Stephen Lazer, Venus Leung, Jo-Lin Liang, Youn-Hee Lim, Robert J. Mislevy,

Norma A. Norris, Ingeborg U. Novatkoski, Jiahe Qian, Katharine E. Pashley,

Timothy Robinson, Alfred M. Rogers, Keith F. Rust, Connie Smith,

Spencer S. Swinton, Mark M. Waksberg, Leslie Wallace, Andrew R. Weiss,

Lois H. Worthington, Ibrahim Yansaneh, and Jinming Zhang

June 2001

U.S. Department of Education

Office of Educational Research and Improvement NCES 2001-509

THE NAEP 1998

TECHNICAL REPORT

NATIONAL CENTER FOR EDUCATION STATISTICS
U.S. Department of Education

Rod Paige

Secretary

National Center for Education Statistics

Gary W. Phillips

Acting Commissioner

June 2001

SUGGESTED CITATION

U.S. Department of Education. Office of Educational Research and Improvement. National Center for

Education Statistics. The NAEP 1998 Technical Report, NCES 2001-509, by Allen, N.L., Donoghue, J.R., &

Schoeps, T.L. (2001). Washington, DC: National Center for Education Statistics.

FOR MORE INFORMATION

Content contact:

Arnold A. Goldstein

202-502-7344

To obtain single copies of this report, while supplies last, or ordering information on other U.S. Department of

Education products, call toll free 1-877-4ED PUBS (877-433-7827), or write:

Education Publications Center (ED Pubs)

U.S. Department of Education

P.O. Box 1398

Jessup, MD 20794-1398

TTY/TDD 1-877-576-7734

FAX 301-470-1244

Online ordering via the Internet: http://www.ed.gov/pubs/edpubs.html

Copies also are available in alternate formats upon request.

This report also is available on the World Wide Web: http://nces.ed.gov/nationsreportcard/

The work upon which this publication is based was performed for the National Center for Education

Statistics, Office of Educational Research and Improvement, by Educational Testing Service.
iii

THE NAEP 1998 TECHNICAL REPORT

♦Table of Contents♦

Introduction James E. Carlson and Nancy L. Allen, Educational Testing Service............................................ 1

Chapter 1 OVERVIEW OF PART I: THE DESIGN AND IMPLEMENTATION OF THE 1998 NAEP

Nancy L. Allen, James E. Carlson, and John R. Donoghue, Educational Testing Service.......... 5

1.1 INTRODUCTION............................................................................................................................ 5

1.2 THE 1998 NAEP DESIGN .............................................................................................................. 6

1.2.1 The 1998 NAEP Samples................................................................................................. 10

1.2.2 NAEP Assessments Since 1969 ....................................................................................... 11

1.3 DEVELOPMENT OF ASSESSMENT OBJECTIVES, ITEMS, AND BACKGROUND

QUESTIONS.................................................................................................................................. 16

1.4 THE 1998 SAMPLE DESIGN ....................................................................................................... 17

1.4.1 Step 1: Primary Sampling Units....................................................................................... 17

1.4.2 Step 2: Selection of Schools............................................................................................. 18

1.4.3 Step 3: Assigning Assessment Session and Sample Type to Schools .............................. 18

1.4.4 Step 4: Sampling Students and Teachers.......................................................................... 19

1.5 ASSESSMENT INSTRUMENTS.................................................................................................. 20

1.6 FIELD OPERATIONS AND DATA COLLECTION ................................................................... 21

1.7 MATERIALS AND DATA PROCESSING .................................................................................. 22

1.8 PROFESSIONAL SCORING ........................................................................................................ 22

1.9 CREATION OF THE DATABASE............................................................................................... 23

Chapter 2 DEVELOPING THE NAEP OBJECTIVES, ITEMS, AND BACKGROUND QUESTIONS

FOR THE 1998 ASSESSMENTS OF READING, WRITING, AND CIVICS

Terry L. Schoeps, Educational Testing Service ............................................................................ 25

2.1 INTRODUCTION.......................................................................................................................... 25

2.2 OVERVIEW OF THE 1998 ASSESSMENT OBJECTIVES AND FRAMEWORKS.................. 26

2.3 GENERAL OVERVIEW OF PROCEDURES FOR DEVELOPING COGNITIVE ITEMS........ 28

2.4 DEVELOPING BACKGROUND ITEMS..................................................................................... 29
iv

Chapter 3 SAMPLE DESIGN FOR THE NATIONAL ASSESSMENT

Keith F. Rust and Tom Krenzke, Westat

Jiahe Qian and Eugene G. Johnson, Educational Testing Service .............................................31

3.1 INTRODUCTION ..........................................................................................................................31

3.1.1 Brief Overview of the Sample Design and Sampling Activities.......................................31

3.1.2 Target Population and Sample Size ..................................................................................32

3.1.3 Highlights of Design Changes for the 1998 Assessment..................................................33

3.2 THE SAMPLE OF PRIMARY SAMPLING UNITS AND SCHOOLS ........................................34

3.2.1 The Definition of Primary Sampling Units.......................................................................34

3.2.2 Definition of PSU Strata...................................................................................................35

3.2.3 Selection of Noncertainty PSUs........................................................................................37

3.2.4 School Sample ..................................................................................................................38

3.2.4.1. Frame Construction..........................................................................................38

3.2.4.2. Assigning Size Measures and Selecting School Samples ...................................39

3.2.4.3. Updating the School Frame and Sample............................................................41

3.2.4.4. School Substitution............................................................................................42

3.2.4.5. School Participation Experience........................................................................42

3.3 ASSIGNMENT OF SESSIONS AND SAMPLE TYPES TO SCHOOLS.....................................43

3.3.1 Description of Session Types ...........................................................................................43

3.3.2 Allocation of Sessions ......................................................................................................43

3.3.2.1 Grade 4 Allocation of Sessions...........................................................................44

3.3.2.2 Grade 8 Allocation of Sessions...........................................................................44

3.3.2.3 Grade 12 Allocation of Sessions.........................................................................45

3.3.3 Assignment of Sample Types ...........................................................................................45

3.3.3.1 Grade 4 Assignment of Sample Types ................................................................46

3.3.3.2 Grade 8 Assignment of Sample Types ................................................................46

3.3.3.3 Grade 12 Assignment of Sample Types ..............................................................46

3.3.3.4 Schools Selected in Both National and State Assessments .................................46

3.4 STUDENT SAMPLE......................................................................................................................46

3.4.1 Updating Estimates of Grade-Eligible Students ...............................................................47

3.4.2 Within-School Sampling Rates.........................................................................................47

3.4.3 The Session Assignment Form (SAF) ..............................................................................47

3.4.4 Updating Session Allocation When Generating SAFs......................................................48

3.4.5 Sample Selection ..............................................................................................................48

3.4.5.1 Oversampling Black and Hispanic Students............................................................49

3.4.5.2 Oversampling SD/LEP Students in Reading ............................................................49

3.4.6 Supporting the Field Staff on Sampling Issues.................................................................50

3.4.7 Excluded Students ............................................................................................................51

3.4.8 Student Participation Results............................................................................................57

3.4.9 Teacher Survey .................................................................................................................59

Chapter 4 SAMPLE DESIGN FOR THE STATE ASSESSMENT

Keith F. Rust and Leslie Wallace, Westat

Jiahe Qian, Educational Testing Service ......................................................................................61

4.1 INTRODUCTION ..........................................................................................................................61
v

Chapter 4 SAMPLE DESIGN FOR THE STATE ASSESSMENT (continued)

4.2 TARGET POPULATIONS AND SAMPLING FRAME FOR THE 1998 STATE

ASSESSMENT ................................................................................................................ 63

4.2.1 Target Population ............................................................................................................. 63

4.2.2 Sampling Frame ............................................................................................................... 63

4.3 STRATIFICATION OF SCHOOLS IN THE SAMPLING FRAME............................................. 66

4.3.1 Stratification Variables..................................................................................................... 66

4.3.2 Missing Stratification Variables....................................................................................... 66

4.3.3 Resources for Stratification Variables.............................................................................. 67

4.3.3.1 Urbanization Classification .............................................................................. 67

4.3.3.2 Minority Classification ...................................................................................... 68

4.3.3.3 Median Household Income ................................................................................ 68

4.3.3.4 Metro Area Status .............................................................................................. 68

4.3.3.5 School Type for Nonpublic Schools ................................................................... 69

4.4 SCHOOL SAMPLE SELECTION................................................................................................. 69

4.4.1 Measure of Size and Sample Selection............................................................................. 69

4.4.2 Sparse State Sample Option ............................................................................................. 70

4.4.3 Control of Overlap of School Samples for National Educational Studies........................ 71

4.4.4 Selection of Schools in Small Jurisdictions...................................................................... 72

4.4.5 Selection of New Public Schools ..................................................................................... 72

4.4.6 Assigning Subject, Sample Type, and Monitor Status ..................................................... 75

4.4.7 School Substitution and Retrofitting ................................................................................ 76

4.5 STUDENT SAMPLE SELECTION............................................................................................... 77

4.5.1 Student Sampling and Participation ................................................................................. 77

4.5.2 The Reduced Sample Option............................................................................................ 77

Chapter 5 FIELD OPERATIONS AND DATA COLLECTION

Lucy M. Gray, Mark M. Waksberg, and Nancy W. Caldwell, Westat.......................................... 79

5.1 INTRODUCTION.......................................................................................................................... 79

5.1.1 Organization of the National Assessment for 1998.......................................................... 79

5.1.1.1 Additional Special Studies ................................................................................. 81

5.1.1.2 Exclusions and Accommodations for Students................................................... 81

5.1.2 Organization of the State Assessment for 1998................................................................ 82

5.2 PREPARING FOR THE ASSESSMENTS.................................................................................... 85

5.2.1 Gaining the Cooperation of Sampled Schools.................................................................. 85

5.2.2 Supervisor Training.......................................................................................................... 88

5.2.3 Contacting Districts and Nonpublic Schools.................................................................... 89

5.2.4 Recruiting, Hiring, and Training Exercise Administrators............................................... 90

5.3 SELECTING THE STUDENT SAMPLES.................................................................................... 91

5.3.1 Selecting the National NAEP Student Samples ............................................................... 91

5.3.2 Selecting the Special Studies Samples ............................................................................. 93

5.3.3 Selecting the State NAEP Student Samples ..................................................................... 94

5.4 CONDUCTING THE ASSESSMENT SESSIONS ....................................................................... 94

5.4.1 Conducting the National Assessments ............................................................................. 94

5.4.2 Conducting the State Assessments ................................................................................... 95
vi

Chapter 5 FIELD OPERATIONS AND DATA COLLECTION (continued)

5.4.3 Participation of DoDEA Schools in State NAEP..............................................................96

5.5 RESULTS OF THE NATIONAL NAEP ASSESSMENT .............................................................97

5.5.1 School and Student Participation......................................................................................97

5.5.2 Assessment Questionnaires...............................................................................................97

5.6 RESULTS OF THE STATE NAEP ASSESSMENT .....................................................................98

5.6.1 School and Student Participation......................................................................................98

5.6.2 Results of the Observations ............................................................................................102

5.7 FIELD MANAGEMENT .............................................................................................................103

Chapter 6 PROCESSING ASSESSMENT MATERIALS

Connie Smith, Charles Brungardt, and Timothy Robinson, National Computer Systems........105

6.1 INTRODUCTION ........................................................................................................................105

6.2 PRINTING....................................................................................................................................105

6.3 PACKAGING AND DISTRIBUTION.........................................................................................107

6.4 PROCESSING ..............................................................................................................................109

6.4.1 Document Receipt and Opening .....................................................................................109

6.4.2 Batching of Booklets ......................................................................................................110

6.4.3 Scanning of Documents..................................................................................................110

6.4.4 Data Transcription ..........................................................................................................110

6.4.4.1 Data Entry........................................................................................................111

6.4.4.2 Data Validation ................................................................................................111

6.5 DATA TRANSMISSION BEFORE SCORING ..........................................................................115

6.6 CLASSROOM-BASED WRITING STUDY ...............................................................................115

Chapter 7 PROFESSIONAL SCORING

Connie Smith, Charles Brungardt, and Timothy Robinson, National Computer Systems........117

7.1 OVERVIEW .................................................................................................................................117

7.2 SELECTION OF TRAINING PAPERS.......................................................................................119

7.3 CALIBRATION POLICIES .........................................................................................................120

7.4 IMAGE SCORING.......................................................................................................................120

7.4.1 Reader Qualification.......................................................................................................120

7.4.2 Backreading Process.......................................................................................................120

7.4.3 Calibration Process .........................................................................................................121

7.4.4 Short-Term Trend Rescoring ..........................................................................................121

7.4.5 Validity Sets Tool...........................................................................................................122

7.4.6 t-Tests .............................................................................................................................122

7.4.7 Procedure for Monitoring Interrater Reliability..............................................................122

7.4.8 Process for Monitoring Frequency Distribution of Scores .............................................122

7.4.9 Process for Monitoring the Rate of Scoring....................................................................123

7.4.10 Scoring Buttons ..............................................................................................................123
vii

Chapter 7 PROFESSIONAL SCORING (continued)

7.5 PAPER SCORING ....................................................................................................................... 123

7.6 LARGE-PRINT BOOKS AND OTHER SPECIAL ACCOMMODATIONS ............................. 123

7.7 TRAINING................................................................................................................................... 124

7.8 SCORING..................................................................................................................................... 124

7.9 INTERRATER RELIABILITY.................................................................................................... 124

7.9.1 Scoring of Reading......................................................................................................... 125

7.9.2 Scoring of Writing.......................................................................................................... 127

7.9.2.1 Selective Rescoring........................................................................................... 127

7.9.2.2 Prewriting Coding ...................................................................................................... 128

7.9.3 Scoring of Civics............................................................................................................ 128

7.10 PREPARATION FOR TAPE CREATION.................................................................................. 129

7.11 UPLOADING OF SCORES TO THE NAEP DATABASE ........................................................ 129

7.12 SD/LEP STUDENT QUESTIONNAIRES .................................................................................. 129

7.13 SCHOOL QUESTIONNAIRES................................................................................................... 130

7.14 TEACHER QUESTIONNAIRE MATCH ................................................................................... 130

7.15 DELIVERY .................................................................................................................................. 130

7.16 STORAGE OF DOCUMENTS.................................................................................................... 130

7.17 QUALITY CONTROL DOCUMENTS....................................................................................... 130

Chapter 8 CREATION OF THE DATABASE, QUALITY CONTROL OF DATA ENTRY, AND

CREATION OF THE DATABASE PRODUCTS

John J. Ferris, Katharine E. Pashley, David S. Freund, and Alfred M. Rogers,

Educational Testing Service........................................................................................................ 131

8.1 INTRODUCTION........................................................................................................................ 131

8.2 CREATION OF THE DATABASE............................................................................................. 131

8.2.1 Merging Files ................................................................................................................. 132

8.2.2 Creating the Master Catalog........................................................................................... 133

8.3 QUALITY CONTROL OF NAEP DATA ENTRY FOR 1998 ................................................... 133

8.3.1 Student Booklet Data ..................................................................................................... 135

8.3.2 SD/LEP Student Questionnaire Data.............................................................................. 135

8.3.3 Teacher Questionnaire Data ........................................................................................... 135

8.3.4 School Characteristics and Policies Questionnaire Data ................................................ 136

8.4 NAEP DATABASE PRODUCTS................................................................................................ 136

8.4.1 File Definition ................................................................................................................ 136

8.4.2 Definition of the Variables............................................................................................. 137

8.4.3 Data Definition............................................................................................................... 137

8.4.4 Data File Catalogs .......................................................................................................... 138

8.4.5 Data File Layouts ........................................................................................................... 139

8.4.6 Data Codebooks............................................................................................................. 139

8.4.7 Control Statement Files for Statistical Packages............................................................ 140

8.4.8 Machine-Readable Catalog Files.................................................................................... 140

8.4.9 NAEP Data on Disk ....................................................................................................... 141
viii

Chapter 9 OVERVIEW OF PART II: THE ANALYSIS OF 1998 NAEP DATA

Nancy L. Allen, James E. Carlson, and John R. Donoghue, Educational Testing Service ......143

9.1 INTRODUCTION ........................................................................................................................143

9.2 SUMMARY OF THE NAEP DESIGN ........................................................................................144

9.3 ANALYSIS STEPS ......................................................................................................................145

9.3.1 Preparation of Final Sampling Weights..........................................................................145

9.3.2 Reliability of Scoring Constructed-Response Items .......................................................145

9.3.3 Teacher Questionnaires...................................................................................................146

9.3.4 Analysis of Item Properties: Background and Cognitive Items......................................146

9.3.4.1 Background Items.............................................................................................................146

9.3.4.2 Cognitive Items.................................................................................................................147

9.3.4.3 Tables of Item-Level Results.............................................................................................148

9.3.4.4 Tables of Block-Level Results...........................................................................................148

9.3.4.5 Differential Item Functioning Analysis of Cognitive Items...............................................149

9.3.5 Scaling ............................................................................................................................152

9.3.5.1 Scaling the Cognitive Items ..............................................................................................153

9.3.5.2 Generation of Plausible Values for Each Scale................................................................153

9.3.5.3 Transformation to the Reporting Metric...........................................................................154

9.3.5.4 Definition of Composites for the Multivariate Scales in Reading.....................................155

9.3.5.5 Tables of Scale Score Means and Other Reported Statistics ............................................155

9.3.6 Dimensionality Analysis.................................................................................................155

9.3.6.1 Previous Dimensionality Analyses of NAEP Data............................................................155

9.3.7 Drawing Inferences from the Results .............................................................................156

9.4 OVERVIEW OF CHAPTERS 10 THROUGH 24 .......................................................................158

Chapter 10 WEIGHTING PROCEDURES AND ESTIMATION OF SAMPLING VARIANCE FOR

THE NATIONAL ASSESSMENT

Jiahe Qian, Bruce A. Kaplan, and Eugene G. Johnson, Educational Testing Service

Tom Krenzke and Keith F. Rust, Westat .....................................................................................161

10.1 INTRODUCTION ........................................................................................................................161

10.2 WEIGHTING PROCEDURES FOR ASSESSED AND EXCLUDED STUDENTS IN

THE NATIONAL SAMPLES ......................................................................................................162

10.2.1 Base Weights ..................................................................................................................163

10.2.2 Adjustment of the Base Weights for Nonresponse .........................................................166

10.2.2.1 Session Nonresponse Adjustment (SESNRF)....................................................166

10.2.2.2 Student Nonresponse Adjustment (STUNRADJ)...............................................167

10.2.3 Variation in Weights.......................................................................................................168

10.2.3.1 Trimming the Weights for Outliers.........................................................................169

10.2.4 Reporting Factors............................................................................................................171

10.2.5 Poststratification .............................................................................................................171

10.2.5.1 The 50-Minute Writing Session ................................................................................173

10.2.6 Final Student Reporting Weights....................................................................................173

10.3 OTHER WEIGHTING PROCEDURES IN THE NATIONAL SAMPLES ................................174

10.3.1 Modular Weights ............................................................................................................174

10.3.2 Linking Weights .............................................................................................................175

10.3.3 School Weights...............................................................................................................177
ix

Chapter 10 WEIGHTING PROCEDURES AND ESTIMATION OF SAMPLING VARIANCE FOR

THE NATIONAL ASSESSMENT (continued)

10.3.4 Reporting Weights with Accommodations..................................................................... 178

10.3.5 Jackknife Replicate Weights .......................................................................................... 179

10.4 POTENTIAL FOR BIAS DUE TO NONRESPONSE................................................................. 179

10.5 VARIANCE ESTIMATION ........................................................................................................ 184

10.5.1 Procedure to Estimate Sampling Variability .................................................................. 184

10.5.2 Approximating the Sampling Variance Using Design Effects ....................................... 187

Chapter 11 STATE WEIGHTING PROCEDURES AND VARIANCE ESTIMATION

Jiahe Qian, Bruce A. Kaplan, and Eugene G. Johnson, Educational Testing Service

Ibrahim S. Yansaneh and Keith F. Rust, Westat........................................................................ 193

11.1 OVERVIEW................................................................................................................................. 193

11.2 CALCULATION OF BASE WEIGHTS...................................................................................... 194

11.2.1 Calculation of School Base Weights .............................................................................. 194

11.2.2 Weighting New Schools................................................................................................. 195

11.2.3 Trimming School Base Weights for New Schools......................................................... 195

11.2.4 Treatment of Substitute Schools..................................................................................... 195

11.2.5 Calculation of Student Base Weights ............................................................................. 196

11.3 ADJUSTMENTS FOR NONRESPONSE.................................................................................... 197

11.3.1 Defining Initial School-Level Nonresponse Adjustment Classes................................... 197

11.3.2 Constructing the Final Nonresponse Adjustment Classes .............................................. 198

11.3.3 School Nonresponse Adjustment Factors....................................................................... 198

11.3.4 Student Nonresponse Adjustment Classes ..................................................................... 199

11.3.5 Student Nonresponse Adjustments................................................................................. 200

11.4 CHARACTERISTICS OF NONRESPONDING SCHOOLS AND STUDENTS....................... 207

11.4.1 Weighted Distributions of Schools Before and After School Nonresponse ................... 208

11.4.2 Characteristics of Schools Related to Response............................................................. 209

11.4.3 Weighted Distributions of Students Before and After Student Absenteeism ................. 215

11.5 VARIATION IN WEIGHTS........................................................................................................ 216

11.6 CALCULATION OF REPLICATE WEIGHTS .......................................................................... 217

11.6.1 Defining Replicate Groups and Forming Replicates for Variance Estimation............... 217

11.6.1.1 Replicate Group Assignments for Non-SD/LEP Students .............................. 218

11.6.1.2 Replicate Group Assignments for SD/LEP Students in Reading .................... 219

11.6.2 School-Level Replicate Weights .................................................................................... 220

11.6.3 Student-Level Replicate Weights................................................................................... 222

11.7 RAKING OF WEIGHTS.............................................................................................................. 223

11.7.1 Raking Dimensions for Full Sample Student Weights................................................... 223

11.7.2 Raking Student Replicate Weights................................................................................. 224

11.8 APPROXIMATING THE SAMPLING VARIANCE USING DESIGN EFFECTS ................... 224
x

Chapter 12 SCALING PROCEDURES

Nancy L. Allen, James E. Carlson, Eugene G. Johnson, and Robert J. Mislevy,

Educational Testing Service ........................................................................................................227

12.1 INTRODUCTION ........................................................................................................................227

12.2 BACKGROUND ..........................................................................................................................227

12.3 SCALING METHODOLOGY .....................................................................................................229

12.3.1 The Scaling Models ........................................................................................................229

12.3.2 An Overview of Plausible Values Methodology ............................................................237

12.3.3 Computing Plausible Values in IRT-Based Scales .........................................................238

12.4 INFERENCES ABOUT PROFICIENCIES..................................................................................240

12.4.1 Computational Procedures..............................................................................................240

12.4.2 Statistical Tests ...............................................................................................................241

12.4.3 Biases in Secondary Analyses ........................................................................................242

12.4.4 A Numerical Example ....................................................................................................243

12.5 DESCRIBING STUDENT PERFORMANCE .............................................................................244

12.5.1 Achievement Levels .......................................................................................................244

12.5.2 Item Mapping Procedures...............................................................................................245

12.6 OVERVIEW OF THE 1998 NAEP SCALES ..............................................................................246

Chapter 13 CONVENTIONS USED IN HYPOTHESIS TESTING AND REPORTING NAEP

RESULTS

Spencer S. Swinton, David S. Freund, and Nancy L. Allen, Educational Testing Service .......247

13.1 OVERVIEW .................................................................................................................................247

13.2 MINIMUM SCHOOL AND STUDENT SAMPLE SIZES FOR REPORTING

SUBGROUP RESULTS...............................................................................................................248

13.3 IDENTIFYING ESTIMATES OF STANDARD ERRORS WITH LARGE MEAN SQUARED

ERRORS.......................................................................................................................................249

13.4 TREATMENT OF MISSING DATA FROM THE STUDENT, TEACHER, AND

SCHOOL QUESTIONNAIRES ...................................................................................................250

13.5 HYPOTHESIS-TESTING CONVENTIONS...............................................................................251

13.5.1 Comparing Means and Proportions for Different Groups of Students............................251

13.5.2 Multiple Comparison Procedures ...................................................................................253

13.5.3 Comparing Proportions Within a Group.........................................................................254

Chapter 14 ASSESSMENT FRAMEWORKS AND INSTRUMENTS FOR THE 1998 NATIONAL

AND STATE READING ASSESSMENTS

Patricia L. Donahue and Terry L. Schoeps, Educational Testing Service.................................255

14.1 INTRODUCTION ........................................................................................................................255

14.2 DEVELOPING THE READING ASSESSMENT FRAMEWORK.............................................255

14.3 READING FRAMEWORK AND ASSESSMENT DESIGN PRINCIPLES...............................256

14.4 FRAMEWORK FOR THE 1998 READING ASSESSMENT .....................................................257
xi

Chapter 14 ASSESSMENT FRAMEWORKS AND INSTRUMENTS FOR THE 1998 NATIONAL

AND STATE READING ASSESSMENTS (continued)

14.5 DEVELOPING THE READING COGNITIVE ITEMS.............................................................. 260

14.6 DEVELOPING THE READING OPERATIONAL FORMS ...................................................... 261

14.7 DISTRIBUTION OF READING ASSESSMENT ITEMS.......................................................... 261

14.8 BACKGROUND QUESTIONNAIRES FOR THE 1998 READING ASSESSMENT ............... 263

14.8.1 Student Reading Questionnaires..................................................................................... 263

14.8.2 Language Arts Teacher Questionnaire ........................................................................... 264

14.9 STUDENT BOOKLETS FOR THE 1998 READING ASSESSMENT....................................... 265

Chapter 15 INTRODUCTION TO THE DATA ANALYSIS FOR THE NATIONAL AND STATE

READING ASSESSMENTS

Jinming Zhang, Jiahe Qian, and Steven P. Isham, Educational Testing Service..................... 269

15.1 INTRODUCTION........................................................................................................................ 269

15.2 DESCRIPTION OF STUDENT SAMPLES, ITEMS, ASSESSMENT BOOKLETS,

AND ADMINISTRATIVE PROCEDURES................................................................................ 269

15.3 SCORING CONSTRUCTED-RESPONSE ITEMS..................................................................... 275

15.4 DIF ANALYSIS........................................................................................................................... 276

15.5 THE WEIGHT FILES.................................................................................................................. 280

Chapter 16 DATA ANALYSIS OF THE NATIONAL READING ASSESSMENT

Jinming Zhang, Steven P. Isham, and Lois H. Worthington, Educational Testing Service..... 283

16.1 INTRODUCTION........................................................................................................................ 283

16.2 NATIONAL ITEM ANALYSES ................................................................................................. 283

16.2.1 Conventional Item and Test Analyses............................................................................ 283

16.2.2 Scoring the Constructed-Response Items....................................................................... 288

16.3 NATIONAL IRT SCALING........................................................................................................ 288

16.3.1 Overview of Item Parameter Estimation ........................................................................ 288

16.3.2 Evaluation of Model Fit ................................................................................................. 289

16.3.2.1 Items Deleted from the Final Scale ................................................................ 289

16.3.2.2 Recoded Polytomous Items............................................................................. 298

16.3.2.3 Item Category Response Functions (ICRF’s) Common Across

Assessment Years ........................................................................................... 299

16.4 GENERATION OF PLAUSIBLE VALUES ............................................................................... 300

16.5 THE FINAL READING SCALES............................................................................................... 302

16.5.1 Purpose-for-Reading Scales ........................................................................................... 302

16.5.2 The Composite Reading Scale........................................................................................ 303

16.6 PARTITIONING OF THE ESTIMATION ERROR VARIANCE .............................................. 304

16.7 READING TEACHER QUESTIONNAIRES.............................................................................. 305
xii

Chapter 17 DATA ANALYSIS OF THE STATE READING ASSESSMENT

Jiahe Qian, Steven P. Isham, Lois H. Worthington, and Jo-Lin Liang,

Educational Testing Service ........................................................................................................307

17.1 INTRODUCTION ........................................................................................................................307

17.2 STATE ITEM AND TEST ANALYSES......................................................................................307

17.3 STATE IRT SCALING ................................................................................................................316

17.3.1 Item Parameter Estimation..............................................................................................318

17.3.2 Recoded Extended Constructed-Response Items............................................................324

17.4 GENERATION OF PLAUSIBLE VALUES................................................................................329

17.5 THE FINAL SCORE SCALES ....................................................................................................336

17.5.1 Linking State and National Scales ..................................................................................336

17.5.2 Producing a Reading Composite Scale ...........................................................................341

17.6 PARTITIONING OF THE ESTIMATION ERROR VARIANCE...............................................342

17.7 READING TEACHER QUESTIONNAIRES ..............................................................................342

Chapter 18 ASSESSMENT FRAMEWORKS AND INSTRUMENTS FOR THE 1998 NATIONAL AND

STATE WRITING ASSESSMENTS

Elissa A. Greenwald and Terry L. Schoeps, Educational Testing Service.................................345

18.1 INTRODUCTION ........................................................................................................................345

18.2 DEVELOPING THE WRITING ASSESSMENT FRAMEWORK .............................................345

18.3 WRITING FRAMEWORK AND ASSESSMENT DESIGN PRINCIPLES................................346

18.4 FRAMEWORK FOR THE 1998 WRITING ASSESSMENT......................................................346

18.5 DEVELOPING THE WRITING COGNITIVE ITEMS...............................................................348

18.6 DEVELOPING THE WRITING OPERATIONAL FORMS .......................................................348

18.7 DISTRIBUTION OF WRITING ASSESSMENT ITEMS ...........................................................349

18.8 BACKGROUND QUESTIONNAIRES FOR THE 1998 WRITING ASSESSMENT ................353

18.8.1 Student Writing Questionnaires......................................................................................353

18.8.2 Language Arts Teacher Questionnaire............................................................................354

18.9 STUDENT BOOKLETS FOR THE 1998 WRITING ASSESSMENT........................................355

 18.10 WRITING CLASSROOM-BASED STUDY IN 1998 .................................................................355

Chapter 19 INTRODUCTION TO THE DATA ANALYSIS FOR THE NATIONAL AND STATE

WRITING SAMPLES

Frank Jenkins, Jiahe Qian, Hua-Hua Chang, and Bruce A. Kaplan,

Educational Testing Service ........................................................................................................359

19.1 INTRODUCTION ........................................................................................................................359

19.2 DESCRIPTION OF STUDENT SAMPLES, ITEMS, ASSESSMENT BOOKLETS, AND

ADMINISTRATIVE PROCEDURES..........................................................................................359

19.3 SCORING CONSTRUCTED-RESPONSE ITEMS.....................................................................367
xiii

Chapter 19 INTRODUCTION TO THE DATA ANALYSIS FOR THE NATIONAL AND STATE

WRITING SAMPLES (continued)

19.4 DIFFERENTIAL ITEM FUNCTIONING ................................................................................... 368

19.5 50-MINUTE WRITING STUDY................................................................................................. 369

19.6 THE WEIGHT FILES.................................................................................................................. 369

Chapter 20 DATA ANALYSIS FOR THE NATIONAL WRITING SAMPLES

Frank Jenkins, Bruce A. Kaplan, and Youn-Hee Lim, Educational Testing Service............... 371

20.1 INTRODUCTION........................................................................................................................ 371

20.2 NATIONAL ITEM ANALYSIS .................................................................................................. 371

20.3 ITEM RESPONSE THEORY (IRT) SCALING .......................................................................... 375

20.3.1 Item Parameter Estimation ............................................................................................. 375

20.3.2 Evaluation of Model Fit ................................................................................................. 376

20.4 GENERATION OF PLAUSIBLE VALUES ............................................................................... 377

20.4.1 Principal Components (NSWEEP Program) .................................................................. 377

20.4.2 Conditioning (BGROUP Program) ................................................................................ 379

20.5 FINAL REPORTING SCALES ................................................................................................... 379

20.6 PARTITIONING OF THE ESTIMATION ERROR VARIANCE .............................................. 380

20.7 WRITING TEACHER QUESTIONNAIRES .............................................................................. 380

Chapter 21 DATA ANALYSIS OF THE STATE WRITING ASSESSMENT

Jiahe Qian, Hua-Hua Chang, Bruce A. Kaplan, Jo-Lin Liang,

and Youn-Hee Lim, Educational Testing Service ...................................................................... 381

21.1 INTRODUCTION........................................................................................................................ 381

21.2 STATE ITEM ANALYSES ......................................................................................................... 382

21.2.1 Conventional Item and Test Analyses............................................................................ 382

21.3 STATE IRT SCALING ................................................................................................................ 385

21.3.1 Samples Used in State IRT Scaling................................................................................ 385

21.3.2 Item Parameter Estimation ............................................................................................. 387

21.4 GENERATION OF PLAUSIBLE VALUES ............................................................................... 388

21.5 FINAL SCORE SCALES............................................................................................................. 393

21.5.1 Linking State and National Scales.................................................................................. 393

21.6 PARTITIONING OF THE ESTIMATION ERROR VARIANCE .............................................. 396

21.7 WRITING TEACHER QUESTIONNAIRES .............................................................................. 398
xiv

Chapter 22 ASSESSMENT FRAMEWORKS AND INSTRUMENTS FOR THE 1998 CIVICS

ASSESSMENT

Andrew R. Weiss and Terry L. Schoeps, Educational Testing Service ......................................399

22.1 INTRODUCTION ........................................................................................................................399

22.2 DEVELOPING THE CIVICS ASSESSMENT FRAMEWORK .................................................399

22.3 CIVICS FRAMEWORK AND ASSESSMENT DESIGN PRINCIPLES....................................400

22.4 FRAMEWORK FOR THE 1998 CIVICS ASSESSMENT..........................................................400

22.5 DEVELOPING THE CIVICS COGNITIVE ITEMS...................................................................401

22.6 DEVELOPING THE CIVICS OPERATIONAL FORMS ...........................................................403

22.7 DISTRIBUTION OF CIVICS ASSESSMENT ITEMS ...............................................................404

22.8 BACKGROUND QUESTIONS FOR THE 1998 CIVICS ASSESSMENT.................................404

22.8.1 Student Civics Questionnaires ........................................................................................405

22.8.2 Civics Teacher Questionnaire.........................................................................................405

22.9 STUDENT BOOKLETS FOR THE 1998 CIVICS ASSESSMENT............................................407

 22.10 CIVICS SPECIAL TREND STUDY IN 1998..............................................................................411

Chapter 23 INTRODUCTION TO THE DATA ANALYSIS FOR THE CIVICS ASSESSMENT

Spencer S. Swinton and Edward Kulick, Educational Testing Service......................................413

23.1 INTRODUCTION ........................................................................................................................413

23.2 DESCRIPTION OF STUDENT SAMPLES, ITEMS, ASSESSMENT

BOOKLETS, AND ADMINISTRATIVE PROCEDURES .........................................................413

23.3 SCORING CONSTRUCTED-RESPONSE ITEMS.....................................................................417

23.4 DIF ANALYSIS ...........................................................................................................................417

23.5 THE WEIGHT FILES ..................................................................................................................420

Chapter 24 DATA ANALYSIS FOR THE CIVICS ASSESSMENT

Spencer S. Swinton, Edward Kulick, and Venus Leung, Educational Testing Service.............421

24.1 INTRODUCTION ........................................................................................................................421

24.2 ITEM ANALYSIS ........................................................................................................................421

24.2.1 Constructed-Response Items...........................................................................................425

24.3 ITEM RESPONSE THEORY (IRT) SCALING...........................................................................425

24.3.1 Evaluating the Fit of the IRT Model...............................................................................426

24.3.2 Derived Background Variables.......................................................................................428

24.4 GENERATION OF PLAUSIBLE VALUES................................................................................435

24.5 TRANSFORMATION OF THE CIVICS CALIBRATION SCALE FOR REPORTING............436

24.6 PARTITIONING OF THE ESTIMATION ERROR VARIANCE...............................................437

24.7 CIVICS TEACHER QUESTIONNAIRE .....................................................................................437
xv

Appendix A STATISTICAL SUMMARY OF THE 1998 NAEP SAMPLES

Bruce A. Kaplan and Youn-Hee Lim, Educational Testing Service.......................................... 439

A.1 INTRODUCTION........................................................................................................................ 439

A.2 MEASUREMENT INSTRUMENTS........................................................................................... 439

A.3 SAMPLE CHARACTERISTICS ................................................................................................. 440

A.4 POPULATION ESTIMATES ...................................................................................................... 440

Appendix B SUMMARY INFORMATION FOR THE NAEP 1998 STATE SAMPLES AND FOR

WEIGHTING THE NAEP 1998 STATE SAMPLES

Keith F. Rust and Leslie Wallace, Westat................................................................................... 473

Appendix C CONSTRUCTED-RESPONSE ITEM SCORE STATISTICS............................................... 563

Appendix D DIFFERENTIAL ITEM FUNCTIONING (DIF) RESULTS ................................................. 585

Appendix E IRT PARAMETERS .................................................................................................................. 587

Appendix F CONDITIONING VARIABLES AND CONTRAST CODINGS........................................... 627

Appendix G REPORTING SUBGROUPS AND SPECIAL VARIABLES FOR THE 1998 NAEP

ASSESSMENT............................................................................................................................ 823

G.1 Major Reporting Subgroups ......................................................................................................... 823

G.2 Writing Derived Variables............................................................................................................ 827

G.3 Civics Derived Variables.............................................................................................................. 828

G.4 Variables Related to Scaling......................................................................................................... 829

G.5 Quality Education Data (QED) Variables .................................................................................... 830

Appendix H ESTIMATION ERROR VARIANCE OF THE MEAN BY GENDER AND

RACE/ETHNICITY................................................................................................................... 831

Appendix I SETTING THE ACHIEVEMENT LEVELS FOR THE 1998 NAEP READING

ASSESSMENT

Mary Lyn Bourque, National Assessment Governing Board..................................................... 841

I.1 INTRODUCTION........................................................................................................................ 841

I.2 1992 Preparation for the Reading Level Setting Meeting............................................................. 842

I.3 1992 Reading Level Setting Panel................................................................................................ 842

I.4 1992 Process for Developing the Achievements Levels............................................................... 843

I.5 1992 Process for Selecting Exemplar Items ................................................................................. 845
xvi

Appendix I SETTING THE ACHIEVEMENT LEVELS FOR THE 1998 NAEP READING

 ASSESSMENT (continued)

I.6 1992 Process for Validating the Levels.........................................................................................847

I.7 Evaluation of the 1992 Levels.......................................................................................................848

I.8 1994 Process for Validating the Levels.........................................................................................848

I.9 1994 Exemplars ............................................................................................................................849

I.10 Mapping the Levels onto the NAEP Scale....................................................................................849

Appendix J SETTING THE ACHIEVEMENT LEVELS FOR THE 1998 NAEP CIVICS AND

WRITING ASSESSMENTS

Mary Lyn Bourque, National Assessment Governing Board .....................................................869

J.1 INTRODUCTION ........................................................................................................................869

J.2 PREPARING THE FINAL DESCRIPTIONS..............................................................................869

J.3 1998 FIELD TRIALS IN CIVICS AND WRITING ....................................................................870

J.4 PREPARATION FOR CIVICS AND WRITING LEVEL SETTING MEETINGS ....................870

J.5 1998 PILOT STUDIES IN CIVICS AND WRITING..................................................................871

J.6 RESULTS OF THE 1998 PILOT STUDIES................................................................................874

J.7 1998 LEVEL-SETTING PANELS ...............................................................................................875

J.8 1998 PROCESS FOR DEVELOPING THE ACHIEVEMENT LEVELS ...................................875

J.9 MAPPING THE LEVELS ONTO THE NAEP SCALE ..............................................................877

J.10 ADDITIONAL ANALYSIS OF THE 1998 DATA .....................................................................878

J.11 SELECTING EXEMPLAR ITEMS .............................................................................................885

J.12 1998 RESEARCH STUDIES ON THE ACHIEVEMENT LEVELS...........................................886

Appendix K PARTICIPANTS IN THE OBJECTIVES AND ITEM DEVELOPMENT PROCESS .......915

References ......................................................................................................................................................921
xvii

THE NAEP 1998 TECHNICAL REPORT

 List of Tables & Figures 

Chapter 1 Overview of Part I: The Design and Implementation of the 1998 NAEP

Figure 1-1 Subsamples of the 1998 NAEP Reading Assessment ....................................................... 8

Table 1-1 NAEP 1998 Student Samples ........................................................................................... 9

Table 1-2 National Assessment of Educational Progress Subject Areas, Grades,

and Ages Assessed: 1969–1998...................................................................................... 12

Table 1-3 An Example of a BIB Design ......................................................................................... 20

Table 1-4 An Example of a PBIB Design ....................................................................................... 21

Chapter 3 Sample Design for the National Assessment

Table 3-1 1998 NAEP National Samples and Target Sample Sizes ............................................... 33

Table 3-2 Definition of NAEP Stratification and Reporting Regions ............................................. 35

Table 3-3 The 22 Largest Primary Sampling Units, by Region 1998 NAEP .................................. 36

Table 3-4 The Number of Noncertainty Strata in Each Major Stratum 1998 NAEP ...................... 37

Table 3-5 Number of Schools Eligible in QED and PSS Sampling Frame Components by

Grade, 1998 Main NAEP................................................................................................ 39

Table 3-6 Participation Rates in 1996 National NAEP................................................................... 40

Table 3-7 Number of Schools in the Original Samples by Major Stratum...................................... 41

Table 3-8 Summary of School Participation Experience for 1998 National NAEP,

Unweighted..................................................................................................................... 43

Table 3-9 Percentage of Assessed and Absent Students Who Were Specified as

SD/LEP National 1998 Reading Samples....................................................................... 50

Table 3-10 Number of Students Per School for Each Subject Type for 1998

National Assessments ..................................................................................................... 53

Table 3-11 Weighted Percentages of Students Excluded (SD and LEP) from 1998

National Reading Assessment......................................................................................... 54

Table 3-12 Weighted and Unweighted Distribution of Students Excluded for 1998

National Assessments, by Reason for Exclusion, Subject, and Grade ............................ 55

Table 3-13 Student Exclusion Rates for 1998 National Assessment by Grade, School Type,

and Sample Type, Weighted........................................................................................... 56

Table 3-14 Comparison of Target Assessments to Actual Assessments for 1998 National

Samples, by Grade.......................................................................................................... 57

Table 3-15 Unweighted Student Participation Rates for National Assessments, by

Grade and School Type .................................................................................................. 57

Table 3-16 Overall Unweighted Participation Rates (School and Student

Combined) for 1998 National Assessments, by Grade ................................................... 58

Table 3-17 Weighted Participation Rates by Grade and Subject Type for the 1998

National Reporting Samples ........................................................................................... 58
xviii

Chapter 4 Sample Design for the State Assessment

Table 4-1 Distribution of Fourth-Grade Schools and Enrollment in Combined

Sampling Frame for 1998 NAEP State Assessments ......................................................64

Table 4-2 Distribution of Eighth-Grade Schools and Enrollment in Combined

Sampling Frame for 1998 NAEP State Assessments ......................................................65

Table 4-3 Estimated Grade Enrollment and Measure of Size, Grade 4 ...........................................69

Table 4-4 Estimated Grade Enrollment and Measure of Size, Grade 8 ...........................................69

Table 4-5 The Effect of the Sparse State Option on Sample Sizes, by Grade for

Jurisdictions Exercising the Option.................................................................................71

Table 4-6 Number of Schools Selected for Both State and National NAEP, by

Grade and School Type...................................................................................................72

Table 4-7 Jurisdictions Where All Schools Were Selected, by Grade and School

Type ................................................................................................................................72

Table 4-8 NAEP 1998 Distribution of New Schools Coming from Districts

Designated as “Medium” or “Large” ..............................................................................74

Chapter 5 Field Operations and Data Collection

Table 5-1 Jurisdictions Participating in the 1998 State Assessment Program .................................83

Table 5-2 Background Questionnaires Received for Schools, Teachers, and

SD/LEP Students in the 1998 National Assessment........................................................98

Table 5-3 School Participation, 1998 State Assessment ...............................................................100

Table 5-4 Student Participation, 1998 State Assessment...............................................................101

Chapter 6 Processing Assessment Materials

Table 6-1 Number of Sessions and Student Booklets Processed for the 1998

National and State Assessments ....................................................................................106

Table 6-2 Questionnaire Totals for the 1998 NAEP Assessment..................................................107

Chapter 7 Professional Scoring

Table 7-1 Processing and Scoring Totals for the 1998 NAEP Assessment...................................117

Figure 7-1 Image Scoring Flow Chart ............................................................................................118

Figure 7-2 Paper Scoring Flow Chart .............................................................................................119

Table 7-2 Interrater Reliability Ranges for the NAEP 1998 Assessment ......................................125

Table 7-3 Number of Constructed-Response Items by Score-Point Levels for the

1998 NAEP Reading Assessment .................................................................................126

Table 7-4 Number of Constructed-Response Items by Score-Point Levels for the

1998 NAEP Writing Assessment ..................................................................................127

Table 7-5 Number of Constructed-Response Items by Score-Point Levels for the

1998 NAEP Civics Assessment ....................................................................................129

Chapter 8 Creation of the Database, Quality Control of Data Entry, and Creation of the Database Products

Table 8-1 Summary of Quality Control Error Analysis for NAEP 1998 Data Entry.....................135
xix

Chapter 10 Weighting Procedures and Estimation of Sampling Variance for the National Assessment

Table 10-1 Reporting Samples for 1998 NAEP National Assessments .......................................... 163

Table 10-2 Session Allocation Weights Used in the 1998 National Assessment............................ 165

Table 10-3 1998 National Assessment Writing and Civics Sample Allocation .............................. 166

Table 10-4 Value of Factor F for Sample Subjects Used in the 1998 National

Assessment ................................................................................................................... 169

Table 10-5 Distribution of Populations of Eligible Students Based on Trimmed

Weights of Assessed Students in Participating Schools, 1998 National

25-Minute Writing Samples.......................................................................................... 170

Table 10-6 1998 National Reading Assessment Reporting Factors for Assessed and

Excluded Students ........................................................................................................ 171

Table 10-7 Major Subgroups for Poststratification in the 1998 National

Assessment ................................................................................................................... 172

Table 10-8 Distributions of Final Student Weights for 1998 National Reporting

Samples......................................................................................................................... 174

Table 10-9 Distribution of Modular Weights Used in the 1998 National

Assessment ................................................................................................................... 175

Table 10-10 First and Second Categorical Variables Used for Raking............................................. 176

Table 10-11 Third Categorical Variable Used for Raking ................................................................ 176

Table 10-12 Percentiles of Raking Adjustments ............................................................................... 177

Table 10-13 Reporting Factors for the Reporting Weights with Accommodations for

the 1998 National Reading Assessment........................................................................ 178

Table 10-14 Distribution of Accommodated Reporting Weights for the 1998

National Reading Assessment....................................................................................... 179

Table 10-15 Distribution of Populations of Eligible Students Based on Full

Weighted Sample of Eligible Schools, Before and After School

Nonresponse Adjustments, 1998 National 25-Minute Writing Samples ...................... 180

Table 10-16 Distribution of Populations of Eligible Students Before and After

Student Nonresponse Adjustments, 1998 National 25-Minute Writing

Samples......................................................................................................................... 181

Table 10-17 Distribution of Populations of Eligible Students Before School and

Student Nonresponse Adjustments, 1998 National 25-Minute Writing

Samples......................................................................................................................... 182

Table 10-18 Weighted Distribution of Absent Students by Nature of Absenteeism

for All Grades, 1998 National 25-Minute Writing Samples ......................................... 183

Table 10-19 Design Effects by Demographic Subgroup and Grade for Mean

Reading Scale Scores.................................................................................................... 189

Table 10-20 Design Effects by Demographic Subgroup and Grade for Mean Writing

Scale Scores.................................................................................................................. 190

Table 10-21 Design Effects by Demographic Subgroup and Grade for Mean Civics

Scale Scores.................................................................................................................. 190

Table 10-22 Within-Grade Mean, Median, and Upper Quartile of the Distribution of

Design Effects for 1998 National Assessments by Subject Area and

Across Subject Areas.................................................................................................... 191
xx

Chapter 11 State Weighting Procedures and Variance Estimation

Table 11-1 Unweighted and Final Weighted Counts of Assessed and Excluded

Students by Jurisdiction, Grade 4 Public Schools, 1998 Reading State

Samples.........................................................................................................................201

Table 11-2 Unweighted and Final Weighted Counts of Assessed and Excluded

Students by Jurisdiction, Grade 8 Public Schools, 1998 Reading State

Samples.........................................................................................................................202

Table 11-3 Unweighted and Final Weighted Counts of Assessed and Excluded

Students by Jurisdiction, Grade 8 Public Schools, 1998 Writing State

Samples.........................................................................................................................203

Table 11-4 Unweighted and Final Weighted Counts of Assessed and Excluded

Students by Jurisdiction, Grade 4 Nonpublic Schools, 1998 Reading

State Samples ................................................................................................................205

Table 11-5 Unweighted and Final Weighted Counts of Assessed and Excluded

Students by Jurisdiction, Grade 8 Nonpublic Schools, 1998 Reading

State Samples ................................................................................................................206

Table 11-6 Unweighted and Final Weighted Counts of Assessed and Excluded

Students by Jurisdiction, Grade 8 Nonpublic Schools, 1998 Writing

State Samples ................................................................................................................207

Table 11-7 Jurisdictions Included in Logistic Regression Analysis of the NAEP

1998 State Assessment..................................................................................................209

Table 11-8 Results of Logistic Regression Analysis of School Nonresponse -

Grade 4, 1998 Reading State Samples ..........................................................................212

Table 11-9 Results of Logistic Regression Analysis of School Nonresponse -

Grade 8, 1998 Reading State Samples ..........................................................................213

Table 11-10 Results of Logistic Regression Analysis of School Nonresponse -

Grade 8, 1998 Writing State Samples ...........................................................................214

Table 11-11 Average Design Effects by Demographic Subgroup for 1998 Mean

State Reading and Writing Scale Scores Averaged Across State

Samples.........................................................................................................................225

Table 11-12 Mean, Median, and Upper Quartile of the 1998 Across-State Average

Design Effects for Mean State Scale Scores (Distribution Across

Demographic Subgroups)..............................................................................................225

Chapter 12 Scaling Procedures

Figure 12-1 Dichotomous Item (R016102) Exhibiting Good Model Fit ..........................................234

Figure 12-2 Polytomous Item (HC00004) Exhibiting Good Model Fit............................................234

Figure 12-3 Dichotomous Item (M017901) Exhibiting Good Model Fit

Across Assessment Years..............................................................................................235

Figure 12-4 Dichotomous Item (M018901) Exhibiting Different Empirical

Item Functions for Different Assessment Years............................................................236

Table 12-1 Estimation Error Variance and Related Coefficients for the 1992

Grade 4 Reading Composite (Based on Five Plausible Values) ...................................244
xxi

Chapter 14 Assessment Frameworks and Instruments for the 1998 National and State Reading Assessments

Figure 14-1 Description of Reading Stances.................................................................................... 258

Figure 14-2 Description of Purposes for Reading ............................................................................ 259

Table 14-1 Percentage Distribution of Items by Reading Purpose as Specified in

the NAEP Reading Framework..................................................................................... 260

Table 14-2 Percentage Distribution of Items by Reading Stance as Specified in the

NAEP Reading Framework .......................................................................................... 260

Figure 14-3 Distribution of Items for the 1998 Reading Assessment............................................... 262

Table 14-3 Percentage Distribution of Assessment Time by Grade and Reading

Purpose for the NAEP 1998 Reading Assessment........................................................ 262

Table 14-4 Percentage Distribution of Assessment Time by Grade and Reading

Stance for the NAEP 1998 Reading Assessment.......................................................... 263

Table 14-5 NAEP 1998 Background Sections of Student Reading Booklets ................................. 264

Table 14-6 NAEP 1998 Reading Grade 4 Booklet Configuration .................................................. 266

Table 14-7 NAEP 1998 Reading Grade 8 Booklet Configuration .................................................. 267

Table 14-8 NAEP 1998 Reading Grade 12 Booklet Configuration ................................................ 268

Chapter 15 Introduction to the Data Analysis for the National and State Reading Assessments

Table 15-1 NAEP 1998 Reading Student Samples ......................................................................... 270

Table 15-2 1998 Reading Blocks and Items Common to the 1992 and 1994

Assessments.................................................................................................................. 271

Table 15-3 Number of Items in Subscales in the Reading Main Assessment, by

Reading Purposes ......................................................................................................... 271

Table 15-4 1998 NAEP Reading Block Composition by Purpose for Reading and

Item Type As Defined Before Scaling, Grade 4 ........................................................... 272

Table 15-5 1998 NAEP Reading Block Composition by Purpose for Reading and

Item Type As Defined After Scaling, Grade 4.............................................................. 272

Table 15-6 1998 NAEP Reading Block Composition by Purpose for Reading and

Item Type As Defined Before Scaling, Grade 8 ........................................................... 273

Table 15-7 1998 NAEP Reading Block Composition by Purpose for Reading and

Item Type As Defined After Scaling, Grade 8.............................................................. 273

Table 15-8 1998 NAEP Reading Block Composition by Purpose for Reading and

Item Type As Defined Before Scaling, Grade 12 ......................................................... 274

Table 15-9 1998 NAEP Reading Block Composition by Purpose for Reading and

Item Type As Defined After Scaling, Grade 12............................................................ 274

Table 15-10 DIF Category for National Samples by Grade for Dichotomous Items ........................ 277

Table 15-11 DIF Category for National Samples by Grade for Polytomous Items........................... 278

Table 15-12 The Category of DIF between Public and Nonpublic Schools for State

Samples, by Grade for Dichotomous Items .................................................................. 279

Table 15-13 The Category of DIF between Public and Nonpublic Schools for State

Samples, by Grade for Polytomous Items..................................................................... 280

Chapter 16 Data Analysis of the National Reading Assessment

Table 16-1 Descriptive Statistics for Item Blocks by Position Within Test Booklet

and Overall Occurrences for the National Main Reading Sample, Grade

4, As Defined After Scaling.......................................................................................... 285
xxii

Table 16-2 Descriptive Statistics for Item Blocks by Position Within Test Booklet

and Overall Occurrences for the National Main Reading Sample, Grade

8, As Defined After Scaling ..........................................................................................286

Table 16-3 Descriptive Statistics for Item Blocks by Position Within Test Booklet

and Overall Occurrences for the National Main Reading Sample, Grade

12, As Defined After Scaling ........................................................................................287

Figure 16-1 Dichotomous Item (R017002) Exhibiting Good Model Fit ..........................................290

Figure 16-2 Polytomous Item (R017104) Exhibiting Good Model Fit.............................................291

Figure 16-3 Polytomous Item (R016603) Exhibiting Unacceptably Poor Model Fit .......................292

Figure 16-4 Polytomous Item (R017110) Exhibiting Poor Model Fit ..............................................293

Figure 16-5 Dichotomous Item (R017110) After Collapsing Categories 1 and 2.............................294

Figure 16-6 Short-Term Trend Polytomous Item (R016210) Demonstrating

Differential Item Functioning Across Assessment Years 1994 and 1998 .....................295

Figure 16-7a Short-Term Trend Polytomous Item (R016210) Fitting Separate Item

Response Functions for Each Assessment Year............................................................296

Figure 16-7b Short-Term Trend Polytomous Item (R016210) Fitting Separate Item

Response Functions for Each Assessment Year............................................................297

Table 16-4 Items Deleted from the Final Scaling............................................................................298

Table 16-5 Recoding of Polytomous Items for Scaling...................................................................298

Table 16-6 Grade 4 Items Scaled Separately by Assessment Years ................................................299

Table 16-7 Grade 8 Items Scaled Separately by Assessment Years ................................................300

Table 16-8 Grade 12 Items Scaled Separately by Assessment Years ..............................................300

Table 16-9 Proportion of Scale Score Variance Accounted for by the Conditioning

Model for the National Main Reading Assessment.......................................................301

Table 16-10 Conditional Correlations and Variances from Conditioning (CGROUP) .....................302

Table 16-11 Marginal Correlations of Reading Scales......................................................................302

Table 16-12 Coefficients of Linear Transformations of the Purpose-for-Reading Scales

from the Calibrating Scale Units to the Units of the Reporting Scale ...........................303

Table 16-13 Weighting of the Purpose-for-Reading Scales on the Reading

Composite Scale............................................................................................................303

Table 16-14 Means and Standard Deviations on the Reading Composite Scale ...............................304

Table 16-15 Estimation Error Variance and Related Coefficients for the National Main

Reading Assessment......................................................................................................304

Chapter 17 Data Analysis of the State Reading Assessment

Table 17-1 Descriptive Statistics for Each Block of Items by Position Within Test

Booklet and Overall – Public Schools, Grade 4............................................................308

Table 17-2 Descriptive Statistics for Each Block of Items by Position Within Test

Booklet and Overall – Nonpublic Schools, Grade 4 .....................................................309

Table 17-3 Descriptive Statistics for Each Block of Items by Position Within Test

Booklet and Overall – Public Schools, Grade 8............................................................310

Table 17-4 Descriptive Statistics for Each Block of Items by Position Within Test

Booklet and Overall – Nonpublic Schools, Grade 8 .....................................................311

Table 17-5 Block-Level Descriptive Statistics for Monitored and Unmonitored

Public-School Sessions, Grade 4...................................................................................312

Table 17-6 Block-Level Descriptive Statistics for Monitored and Unmonitored

Nonpublic-School Sessions, Grade 4 ............................................................................312
xxiii

Table 17-7 Block-Level Descriptive Statistics for Monitored and Unmonitored

Public-School Sessions, Grade 8 .................................................................................. 313

Table 17-8 Block-Level Descriptive Statistics for Monitored and Unmonitored

Nonpublic-School Sessions, Grade 8............................................................................ 313

Table 17-9 Effect of Monitoring Sessions by Jurisdiction: Average Jurisdiction Item

Scores for Monitored and Unmonitored Sessions, Grade 4.......................................... 315

Table 17-10 Effect of Monitoring Sessions by Jurisdiction: Average Jurisdiction

Item Scores for Monitored and Unmonitored Sessions, Grade 8.................................. 316

Table 17-11 Block-Level Descriptive Statistics for Overall Public- and NonpublicSchool Sessions, Grade 4.............................................................................................. 317

Table 17-12 Block-Level Descriptive Statistics for Overall Public- and NonpublicSchool Sessions, Grade 8.............................................................................................. 317

Table 17-13 Extended Constructed-Response Items, 1998 State Assessment in

Reading......................................................................................................................... 319

Figure 17-1 Dichotomous Items (R012106, R012711, and R013405) Exhibiting

Good Model Fit ........................................................................................................... 321

Figure 17-2 Polytomous Item (R013201) Exhibiting Good Model Fit ............................................ 323

Figure 17-3 Polytomous Item (R012111) Before Collapsing Unsatisfactory and

Partial-Response Categories ......................................................................................... 324

Figure 17-4 Polytomous Item (R012111) After Collapsing Unsatisfactory and

Partial-Response Categories ......................................................................................... 326

Figure 17-5 Polytomous Item (R017110) After Collapsing Unsatisfactory and

Partial-Response Categories ......................................................................................... 327

Figure 17-6 Polytomous Item (R016212) After Collapsing Unsatisfactory and

Partial-Response Categories ......................................................................................... 328

Table 17-14 Summary Statistics for State Assessment Conditioning Models, Grade 4 .................... 330

Table 17-15 Summary Statistics for State Assessment Conditioning Models, Grade 8 .................... 331

Table 17-16 Average Correlations and Ranges of Scale Correlations Among the

Reading Scales for 40 Jurisdictions for Grade 8........................................................... 333

Figure 17-7 Plot of Mean Scale Score Versus Mean Item Score by Jurisdiction, Grade 4 .............. 334

Figure 17-8 Plot of Mean Scale Score Versus Mean Item Score by Jurisdiction, Grade 8 .............. 335

Table 17-17 Coefficients of Linear Transformations for the 1998 State Reading

Assessment ................................................................................................................... 338

Figure 17-9 Rootogram Comparing Scale Score Distributions for the State

Assessment Aggregate Sample and the National Linking Sample

for the Reading for Literary Experience Scale, Grade 4............................................... 339

Figure 17-10 Rootogram Comparing Scale Score Distributions for the State

Assessment Aggregate Sample and the National Linking Sample

for the Reading to Gain Information Scale, Grade 4 .................................................... 339

Figure 17-11 Rootogram Comparing Scale Score Distributions for the State

Assessment Aggregate Sample and the National Linking Sample

for the Reading for Literary Experience Scale, Grade 8............................................... 340

Figure 17-12 Rootogram Comparing Scale Score Distributions for the State

Assessment Aggregate Sample and the National Linking Sample

for the Reading to Gain Information Scale, Grade 8 .................................................... 340

Figure 17-13 Rootogram Comparing Scale Score Distributions for the State

Assessment Aggregate Sample and the National Linking Sample

for the Reading to Perform a Task Scale, Grade 8........................................................ 340

Figure 17-14 Rootogram Comparing Scale Score Distributions for the State

Assessment Aggregate Sample and the National Linking Sample

for the Reading Composite Scale, Grade 4................................................................... 341
xxiv

Figure 17-15 Rootogram Comparing Scale Score Distributions for the State

Assessment Aggregate Sample and the National Linking Sample

for the Reading Composite Scale, Grade 8 ...................................................................342

Table 17-18 Estimation Error Variance and Related Coefficients for the Reading

State Assessment, Grade 4 ............................................................................................343

Table 17-19 Estimation Error Variance and Related Coefficients for the Reading

State Assessment, Grade 8 ............................................................................................344

Chapter 18 Assessment Frameworks and Instruments for the 1998 National and State Writing Assessments

Figure 18-1 Description of NAEP 1998 Writing Purposes...............................................................347

Table 18-1 Percentage Distribution of Items by Purpose for Writing as Specified in

the NAEP Writing Framework......................................................................................348

Figure 18-2 NAEP 1998 Forms of Writing ......................................................................................348

Table 18-2 NAEP 1998 Writing Grade 4 Blocks by Title and Purpose ..........................................350

Table 18-3 NAEP 1998 Writing Grade 8 Blocks by Title and Purpose ..........................................351

Table 18-4 NAEP 1998 Writing Grade 12 Blocks by Title and Purpose ........................................352

Table 18-5 Percentage Distribution of Assessment Time by Grade and Purpose for

Writing for the NAEP 1998 Writing Assessment .........................................................353

Table 18-6 NAEP 1998 Background Sections of Student Writing Booklets...................................354

Table 18-7 NAEP 1998 National and State Writing Grade 4 Booklet Configuration .....................356

Table 18-8 NAEP 1998 National and State Writing Grade 8 Booklet Configuration .....................357

Table 18-9 NAEP 1998 National and State Writing Grade 12 Booklet Configuration ...................358

Chapter 19 Introduction to the Data Analysis for the National and State Writing Samples

Table 19-1 NAEP 1998 Writing Student Samples ..........................................................................360

Table 19-2 Number of 25-Minute Items in the National Main Writing Assessment

Within the Three Purposes of Writing ..........................................................................361

Table 19-3 Number of 50-Minute Items in the National Writing Assessment

Within the Three Purposes of Writing ..........................................................................361

Table 19-4 Grade 4: Prompt, Block, and Purpose Correspondence................................................362

Table 19-5 Grade 8: Prompt, Block, and Purpose Correspondence................................................363

Table 19-6 Grade 12: Prompt, Block, and Purpose Correspondence..............................................364

Table 19-7 Correspondence of Prompts, Blocks, and Books: Grade 4 ...........................................365

Table 19-8 Correspondence of Prompts, Blocks, and Books: Grade 8 ...........................................366

Table 19-9 Correspondence of Prompts, Blocks, and Books: Grade 12 .........................................367

Table 19-10 Items with Absolute SMD (Standardized Mean DIF) >.10...........................................369
xxv

Chapter 20 Data Analysis for the National Writing Samples

Table 20-1 Descriptive Statistics for 25-Minute Writing Prompts: Grade 4................................... 372

Table 20-2 Descriptive Statistics for 25-Minute Writing Prompts: Grade 8................................... 373

Table 20-3 Descriptive Statistics for 25-Minute Writing Prompts: Grade 12................................. 374

Figure 20-1 Polytomous Item (W010002) Exhibiting Good Model Fit ........................................... 377

Figure 20-2 Polytomous Item (W008402) Exhibiting Less Than Optimal Model Fit...................... 378

Table 20-4 Proportion of Scale Score Variance Accounted for by the Conditioning

Model for the 1998 National Main Writing Assessment .............................................. 378

Table 20-5 Coefficients of Linear Transformations of the Writing Scales from the

Scaling Metric to the Reporting Metric ........................................................................ 379

Table 20-6 Estimation Error Variance and Related Coefficients for the National

Main Writing Assessment............................................................................................. 380

Chapter 21 Data Analysis of the State Writing Assessment

Table 21-1 Descriptive Statistics Writing Prompts, Writing 25-Minute State

Samples, Grade 8.......................................................................................................... 383

Table 21-2 Descriptive Statistics for Each Item of the Writing State Assessment

Using Senate Weights (Scaled from 0 to 5), Grade 8 ................................................... 384

Table 21-3 Effect of Monitoring Sessions by Jurisdiction: Average Jurisdiction Item

Scores for Monitored and Unmonitored Sessions, Grade 8.......................................... 386

Figure 21-1 Polytomous Item (W006502) Exhibiting Good Model Fit ........................................... 389

Figure 21-2 Polytomous Item (W007602) Exhibiting Good Model Fit ........................................... 390

Table 21-4 Proportion of Scale Score Variance Accounted by Conditioning Model

for the Writing State Assessment, Grade 8 ................................................................... 391

Figure 21-3 Plot of Mean Scale Score Versus Mean Item Score by Jurisdiction, Grade 8 .............. 393

Table 21-5 Coefficients of Linear Transformations for the 1998 State Writing

Assessment ................................................................................................................... 395

Figure 21-4 Rootogram Comparing Scale Score Distributions for the State Assessment

Aggregate Sample and the National Linking Sample for the Composite

Scale, Grade 8............................................................................................................... 396

Table 21-6 Estimation Error Variance and Related Coeffecients for the Writing

State Assessment, Grade 8............................................................................................ 397

Chapter 22 Assessment Frameworks and Instruments for the 1998 Civics Assessment

Table 22-1 Percentage Distribution of Questions by Intellectual Skill as

Recommended in the NAEP Civics Framework........................................................... 401

Table 22-2 Actual Percentage Distribution of Questions by Intellectual Skill................................ 401

Figure 22-1 Description of the NAEP 1998 Civics Framework Components.................................. 402

Table 22-3 NAEP 1998 Civics Assessment Percentage of Student Assessment

Time by Question Format ............................................................................................. 403

Figure 22-2 Distribution of Items for the 1998 Civics Assessment.................................................. 404

Table 22-4 NAEP 1998 Background Sections of Student Civics Booklets .................................... 405

Table 22-5 NAEP 1998 Civics Grade 4 Booklet Configuration ..................................................... 408

Table 22-6 NAEP 1998 Civics Grade 8 Booklet Configuration ..................................................... 409

Table 22-7 NAEP 1998 Civics Grade 12 Booklet Configuration ................................................... 410
xxvi

Chapter 23 Introduction to the Data Analysis for the Civics Assessment

Table 23-1 NAEP 1998 National Main Civics Assessment Student Samples .................................414

Table 23-2 Number of Items in the National Main Civics Assessment by Content

Area...............................................................................................................................415

Table 23-3 1998 NAEP Civics Block Composition As Defined Before Scaling, Grade 4..............415

Table 23-4 1998 NAEP Civics Block Composition After Scaling, Grade 4 ...................................415

Table 23-5 1998 NAEP Civics Block Composition As Defined Before Scaling, Grade 8..............416

Table 23-6 1998 NAEP Civics Block Composition After Scaling, Grade 8 ...................................416

Table 23-7 1998 NAEP Civics Block Composition As Defined Before Scaling, Grade 12............416

Table 23-8 1998 NAEP Civics Block Composition After Scaling, Grade 12 .................................417

Table 23-9 DIF Category by Grade for Dichotomous Civics Items ................................................419

Table 23-10 DIF Category by Grade for Polytomous Civics Items...................................................420

Chapter 24 Data Analysis for the Civics Assessment

Table 24-1 Descriptive Statistics for Item Blocks by Position Within Test Booklet

and Overall Occurrences for the National Main Civics Sample, Grade

4, As Defined After Scaling ..........................................................................................422

Table 24-2 Descriptive Statistics for Item Blocks by Position Within Test Booklet

and Overall Occurrences for the National Main Civics Sample, Grade

8, As Defined After Scaling ..........................................................................................423

Table 24-3 Descriptive Statistics for Item Blocks by Position Within Test Booklet

and Overall Occurrences for the National Main Civics Sample, Grade

12, As Defined After Scaling ........................................................................................424

Table 24-4 1998 Civics Items Receiving Special Treatment...........................................................426

Figure 24-1 Dichotomous Item (P040719) Exhibiting Good Model Fit...........................................428

Figure 24-2 Polytomous Item (P042008) Exhibiting Good Model Fit .............................................429

Figure 24-3 Dichotomous Item (P041209) Exhibiting Moderate Model Misfit ...............................430

Figure 24-4 Polytomous Item (P041902) Exhibiting Moderate Model Misfit..................................431

Figure 24-5 Dichotomous Item (P040506) Exhibiting Poor Model Fit ............................................432

Figure 24-6 Polytomous Item (P040402) Exhibiting Poor Model Fit in the Lower

Two Categories .............................................................................................................433

Figure 24-7 Same Polytomous Item (P040402) with the Lower Two Categories

Collapsed, Now Exhibiting Improved Model Fit ..........................................................434

Table 24-5 Proportion of Scale Score Variance Accounted for by the Conditioning

Model for the National Main Civics Assessment ..........................................................435

Table 24-6 Means and Standard Deviations for the Civics Scale....................................................436

Table 24-7 Transformation Constants for the National Main Civics Assessment ...........................437

Table 24-8 Estimation Error Variance and Related Coefficients for the National

Main Civics Assessment ...............................................................................................437

Appendix A Statistical Summary of the 1998 NAEP Samples

Table A-1 Measurement Instruments Used in 1998 NAEP............................................................441

Table A-2 Number of Items Administered, by Sample and Age Class...........................................442

Table A-3 School Participation in NAEP 1998 Main Samples (All Subsamples)..........................443

Table A-4 School Characteristics in NAEP 1998 Main Samples...................................................444
xxvii

Table A-5 Numbers of Responses to Teacher Questionnaires and Students

Matched with Teacher Data.......................................................................................... 445

Table A-6 Number of Students Assessed, Accommodated, and Excluded, by Reporting

Sample and Grade......................................................................................................... 446

Table A-7 Number of Students in the Reading Reporting Samples, by Subgroup

Classification, National Grades 4, 8, and 12, & State Grades 4 and 8.......................... 447

Table A-8 Number of Students in the Writing 25-Minute and 50-Minute Samples by

Subgroup Classification, Grades 4, 18, and 12, & State Grade 8 ................................. 449

Table A-9 Number of Students in the Civics Main Samples by Subgroup

Classification, Grades 4, 8, and 12 ............................................................................... 451

Table A-10 Number of Excluded Students in the Reading Reporting Samples by

Subgroup Classification, Grades 4, 8, and 12, & State Grades 4 and 8 ........................ 452

Table A-11 Number of Excluded Students in the Writing Samples by Subgroup

Classification, Grades 4, 8, and 12, & State Grade 8.................................................... 454

Table A-12 Number of Excluded Students in the Civics Main Samples by Subgroup

Classification, Grades 4, 8, and 12 ............................................................................... 455

Table A-13 Number of Accommodated Students in the Writing Samples by

Subgroup Classification, Grades 4, 8, and 12, & State Grade 8 ................................... 456

Table A-14 Number of Accommodated Students in the Civics Main Samples by

Subgroup Classification, Grades 4, 8, and 12............................................................... 458

Table A-15 Weighted Percentages of Students in the Reading Reporting Samples

by Subgroup Classification, National Grades 4, 8, and 12, & State

Grades 4 and 8.............................................................................................................. 460

Table A-16 Weighted Percentages of Students in the Writing 25-Minute and

50-Minute Samples by Subgroup Classification, Grades 4, 8, and 12, &

State Grade 8 ................................................................................................................ 462

Table A-17 Weighted Percentages of Students in the Civics Main Samples by

Subgroup Classification, Grades 4, 8, and 12............................................................... 464

Table A-18 Weighted Percentages of Excluded Students in the Reading Reporting

Samples by Subgroup Classification, National Grades 4, 8, and 12, &

State Grades 4 and 8 ..................................................................................................... 466

Table A-19 Weighted Percentages of Excluded Students in the Writing Samples by

Subgroup Classification, Grades 4, 8, and 12, & State Grade 8 ................................... 467

Table A-20 Weighted Percentages of Excluded Students in the Civics Main

Samples by Subgroup Classification, Grades 4, 8, and 12............................................ 468

Table A-21 Weighted Percentages of Accommodated Students in the Writing

Samples by Subgroup Classification, Grades 4, 8, and 12, & State

Grade 8 ......................................................................................................................... 469

Table A-22 Weighted Percentages of Accommodated Students in the Civics Main

Samples by Subgroup Classification, Grades 4, 8, and 12............................................ 471

Appendix B Summary Information for NAEP 1998 State Samples and Weighting NAEP 1998 State Samples

Table B-1 Weighted Mean Values Derived from Sampled Public Schools – Grade 4,

Reading......................................................................................................................... 475

Table B-2 Weighted Mean Values Derived from Sampled Public Schools – Grade 8,

Reading......................................................................................................................... 477

Table B-3 Weighted Mean Values Derived from Sampled Public Schools – Grade 8,

Writing.......................................................................................................................... 479

Table B-4 Weighted Mean Values Derived from Sampled Nonpublic Schools – Grade 4,

Reading......................................................................................................................... 481
xxviii

Table B-5 Weighted Mean Values Derived from Sampled Nonpublic Schools – Grade 8,

Reading .........................................................................................................................482

Table B-6 Weighted Mean Values Derived from Sampled Nonpublic Schools – Grade 8,

Writing ..........................................................................................................................483

Table B-7 Weighted Student Percentages Derived from Sampled Public Schools – Grade 4,

Reading .........................................................................................................................484

Table B-8 Weighted Student Percentages Derived from Sampled Public Schools – Grade 8,

Reading .........................................................................................................................486

Table B-9 Weighted Student Percentages Derived from Sampled Public Schools – Grade 8,

Writing ..........................................................................................................................488

Table B-10 Weighted Student Percentages Derived from All Schools Sampled – Grade 4,

Reading .........................................................................................................................490

Table B-11 Weighted Student Percentages Derived from All Schools Sampled – Grade 8,

Reading .........................................................................................................................492

Table B-12 Weighted Student Percentages Derived from All Schools Sampled – Grade 8,

Writing ..........................................................................................................................493

Table B-13 Final Collapsed Levels Used for Raking Dimensions for All

Jurisdictions – Grade 4, Reading ..................................................................................494

Table B-14 Final Collapsed Levels Used for Raking Dimensions for All

Jurisdictions – Grade 8, Reading ..................................................................................496

Table B-15 Distribution of Selected Public Schools by Sampling Strata, Fourth Grade .................498

Table B-16 Distribution of Selected Public Schools by Sampling Strata, Eighth Grade .................517

Table B-17 Distribution of Selected Nonpublic Schools by Sampling Strata, Fourth Grade...........536

Table B-18 Distribution of Selected Nonpublic Schools by Sampling Strata, Eighth Grade...........541

Table B-19 Weighted School Participation Rates and Sample Counts - Grade 4,

Reading for Public Schools...........................................................................................545

Table B-20 Weighted School Participation Rates and Sample Counts - Grade 4,

Reading for Nonpublic Schools ....................................................................................547

Table B-21 Weighted School Participation Rates and Sample Counts - Grade 8,

Reading and Writing for Public Schools.......................................................................548

Table B-22 Weighted School Participation Rates and Sample Counts - Grade 8,

Reading and Writing for Nonpublic Schools ................................................................550

Table B-23 Weighted Student Participation Rates, Exclusion Rates, and Sample Counts

for the Reporting Samples - Grade 4 Reading for Public Schools ................................551

Table B-24 Weighted Student Participation Rates, Exclusion Rates, and Sample Counts

for the Reporting Samples - Grade 4 Reading for Nonpublic Schools..........................553

Table B-25 Weighted Student Participation Rates, Exclusion Rates, and Sample Counts

for the Reporting Samples - Grade 8 Reading for Public Schools ................................554

Table B-26 Weighted Student Participation Rates, Exclusion Rates, and Sample Counts

for the Reporting Samples - Grade 8 Reading for Nonpublic Schools..........................556

Table B-27 Weighted Student Participation Rates, Exclusion Rates, and Sample Counts

for the Reporting Samples - Grade 8 Writing for Public Schools .................................557

Table B-28 Weighted Student Participation Rates, Exclusion Rates, and Sample Counts

for the Reporting Samples - Grade 8 Writing for Nonpublic Schools...........................559

Table B-29 Results of Logistic Regression Analysis of School Nonresponse - Grade 4

Reading .........................................................................................................................560

Table B-30 Results of Logistic Regression Analysis of School Nonresponse - Grade 8

Reading .........................................................................................................................561

Table B-31 Results of Logistic Regression Analysis of School Nonresponse - Grade 8

Writing ..........................................................................................................................562
xxix

Appendix C Constructed-Response Item Score Statistics

Table C-1 Score Range, Percent Agreement, and Cohen’s Kappa for the

Dichotomously Scored Constructed-Response Reading Items Used in

1998 National Main Assessment Scaling, Grade 4...................................................... 565

Table C-2 Score Range, Percent Agreement, and Cohen’s Kappa for the

Dichotomously Scored Constructed-Response Reading Items Used in

1998 National Main Assessment Scaling, Grade 8....................................................... 566

Table C-3 Score Range, Percent Agreement, and Cohen’s Kappa for the

Dichotomously Scored Constructed-Response Reading Items Used in

1998 National Main Assessment Scaling, Grade12...................................................... 567

Table C-4 Score Range, Percent Agreement, and Intraclass Correlation for the

Polytomously Scored Constructed-Response Reading Items Used in

1998 National Main Assessment Scaling, Grade 4....................................................... 568

Table C-5 Score Range, Percent Agreement, and Intraclass Correlation for the

Polytomously Scored Constructed-Response Reading Items Used in

1998 National Main Assessment Scaling, Grade 8....................................................... 569

Table C-6 Score Range, Percent Agreement, and Intraclass Correlation for the

Polytomously Scored Constructed-Response Reading Items Used in

1998 National Main Assessment Scaling, Grade 12..................................................... 570

Table C-7 Score Range, Percent Agreement, and Cohen’s Kappa for the

Dichotomously Scored Constructed-Response Reading Items from

1994 That Were Rescored in 1998, Grade 4 ................................................................ 572

Table C-8 Score Range, Percent Agreement, and Intraclass Correlation for the

Polytomously Scored Constructed-Response Reading Items from 1994

That Were Rescored in 1998, Grade 4 ......................................................................... 573

Table C-9 Score Range, Percent Agreement, and Cohen’s Kappa for the

Dichotomously Scored Constructed-Response Reading Items from

1994 That Were Rescored in 1998, Grade 8 ................................................................ 574

Table C-10 Score Range, Percent Agreement, and Intraclass Correlation for the

Polytomously Scored Constructed-Response Reading Items from 1994

That Were Rescored in 1998, Grade 8 ......................................................................... 575

Table C-11 Score Range, Percent Agreement, and Cohen’s Kappa for the

Dichotomously Scored Constructed-Response Reading Items from

1994 That Were Rescored in 1998, Grade 12 .............................................................. 576

Table C-12 Score Range, Percent Agreement, and Intraclass Correlation for the

Polytomously Scored Constructed-Response Reading Items from 1994

That Were Rescored in 1998, Grade 12 ....................................................................... 577

Table C-13 Score Range, Percent Agreement, and Intraclass Correlation for the

Polytomously Scored Constructed-Response Writing Items Used in

1998 National Main Assessment Scaling, Grade 4....................................................... 578

Table C-14 Score Range, Percent Agreement, and Intraclass Correlation for the

Polytomously Scored Constructed-Response Writing Items Used in

1998 National Main Assessment Scaling, Grade 8....................................................... 579

Table C-15 Score Range, Percent Agreement, and Intraclass Correlation for the

Polytomously Scored Constructed-Response Writing Items Used in

1998 National Main Assessment Scaling, Grade 12..................................................... 580

Table C-16 Score Range, Percent Agreement, and Intraclass Correlation for the

Polytomously Scored Constructed-Response Civics Items Used in 1998

National Main Assessment Scaling, Grade 4................................................................ 581

Table C-17 Score Range, Percent Agreement, and Intraclass Correlation for the

Polytomously Scored Constructed-Response Civics Items Used in 1998

National Main Assessment Scaling, Grade 8................................................................ 582
xxx

Table C-18 Score Range, Percent Agreement, and Intraclass Correlation for the

Polytomously Scored Constructed-Response Civics Items Used in 1998

National Main Assessment Scaling, Grade 12 ..............................................................583

Appendix D Differential Item Functioning (DIF) Results

Table D-1 1998 Reading Items Identified as "C" or "CC" Items in at Least One

Comparison ...................................................................................................................585

Table D-2 1998 Civics Items Identified as "C" or "CC" Items in at Least One

Comparison ...................................................................................................................585

Appendix E IRT Parameters

Table E-1 IRT Parameters for the 1998 Reading Items Reading for Literary

Experience Scale, Grade 4 ............................................................................................589

Table E-2 IRT Parameters for the 1998 Reading Items Reading to Gain

Information Scale, Grade 4 ...........................................................................................591

Table E-3 IRT Parameters for the 1998 Reading Items Reading for Literary

Experience Scale, Grade 8 ............................................................................................593

Table E-4 IRT Parameters for the 1998 Reading Items Reading to Gain

Information Scale, Grade 8 ...........................................................................................594

Table E-5 IRT Parameters for the 1998 Reading Items Reading to Perform a Task

Scale, Grade 8 ...............................................................................................................596

Table E-6 IRT Parameters for the 1998 Reading Items Reading for Literary

Experience Scale, Grade 12 ..........................................................................................597

Table E-7 IRT Parameters for the 1998 Reading Items Reading to Gain

Information Scale, Grade 12 .........................................................................................598

Table E-8 IRT Parameters for the 1998 Reading Items Reading to Perform a Task

Scale, Grade 12 .............................................................................................................600

Table E-9 IRT Parameters for the 1998 Writing Items, Grade 4 ...................................................602

Table E-10 IRT Parameters for the 1998 Writing Items, Grade 8 ...................................................603

Table E-11 IRT Parameters for the 1998 Writing Items, Grade 12 .................................................604

Table E-12 IRT Parameters for the 1998 Civics Items, Grade 4 .....................................................605

Table E-13 IRT Parameters for the 1998 Civics Items, Grade 8 .....................................................608

Table E-14 IRT Parameters for the 1998 Civics Items, Grade 12 ...................................................613

Table E-15 IRT Parameters for the 1998 State Reading Items Reading for Literary

Experience Scale, Grade 4 ............................................................................................618

Table E-16 IRT Parameters for the 1998 State Reading Items Reading to Gain

Information Scale, Grade 4 ...........................................................................................620

Table E-17 IRT Parameters for the 1998 State Reading Items Reading for Literary

Experience Scale, Grade 8 ............................................................................................622

Table E-18 IRT Parameters for the 1998 State Reading Items Reading to Gain

Information Scale, Grade 8 ...........................................................................................623

Table E-19 IRT Parameters for the 1998 State Reading Items Reading to Perform a

Task Scale, Grade 8 ......................................................................................................625

Table E-20 IRT Parameters for the 1998 State Writing Items, Grade 8 ..........................................626
xxxi

Appendix F Conditioning Variables and Contrast Codings

Table F-1 Description of Specifications Provided for Each Conditioning Variable ..................... 628

Table F-2 Summary Table of the 1998 Reading Conditioning Variable Specifications................ 629

Table F-3 Summary Table of the 1998 Writing Conditioning Variable Specifications ................ 636

Table F-4 Summary Table of the 1998 Civics Conditioning Variable Specifications................... 643

Table F-5 1998 Reading Conditioning Variable Specifications.................................................... 649

Table F-6 1998 Writing Conditioning Variable Specifications..................................................... 690

Table F-7 1998 Civics Conditioning Variable Specifications....................................................... 731

Table F-8 Proportion of Variance of the Conditioning Variable Contrasts

Accounted for by the Principal Components Used in the Conditioning

Model for National Reading Conditioning Variables, Grade 4 .................................... 768

Table F-9 Proportion of Variance of the Conditioning Variable Contrasts

Accounted for by the Principal Components Used in the Conditioning

Model for National Reading Conditioning Variables, Grade 8 .................................... 775

Table F-10 Proportion of Variance of the Conditioning Variable Contrasts

Accounted for by the Principal Components Used in the Conditioning

Model for National Reading Conditioning Variables, Grade 12 .................................. 782

Table F-11 Proportion of Variance of the Conditioning Variable Contrasts

Accounted for by the Principal Components Used in the Conditioning

Model for National Writing Conditioning Variables, Grade 4 ..................................... 786

Table F-12 Proportion of Variance of the Conditioning Variable Contrasts

Accounted for by the Principal Components Used in the Conditioning

Model for National Writing Conditioning Variables, Grade 8 ..................................... 793

Table F-13 Proportion of Variance of the Conditioning Variable Contrasts

Accounted for by the Principal Components Used in the Conditioning

Model for National Writing Conditioning Variables, Grade 12 ................................... 801

Table F-14 Proportion of Variance of the Conditioning Variable Contrasts

Accounted for by the Principal Components Used in the Conditioning

Model for National Civics Conditioning Variables, Grade 4........................................ 805

Table F-15 Proportion of Variance of the Conditioning Variable Contrasts

Accounted for by the Principal Components Used in the Conditioning

Model for National Civics Conditioning Variables, Grade 8........................................ 811

Table F-16 Proportion of Variance of the Conditioning Variable Contrasts

Accounted for by the Principal Components Used in the Conditioning

Model for National Civics Conditioning Variables, Grade 12...................................... 817

Appendix G Reporting Subgroups and Special Variables for the 1998 NAEP Assessment

Table G-1 NAEP Geographic Regions .......................................................................................... 825

Table G-2 Scaling Variables for the 1998 National and State Assessment Samples ..................... 830
xxxii

Appendix H Estimation Error Variance of the Mean by Gender and Race/Ethnicity

Table H-1 Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Reading Grade 4 Literacy Scale............................................................831

Table H-2 Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Reading Grade 4 Information Scale ......................................................831

Table H-3 Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Reading Grade 4 Composite Scale........................................................832

Table H-4 Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Reading Grade 8 Literacy Scale............................................................832

Table H-5 Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Reading Grade 8 Information Scale ......................................................833

Table H-6 Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Reading Grade 8 Perform a Task Scale.................................................833

Table H-7 Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Reading Grade 8 Composite Scale........................................................834

Table H-8 Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Reading Grade 12 Literacy Scale..........................................................834

Table H-9 Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Reading Grade 12 Information Scale ....................................................835

Table H-10 Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Reading Grade 12 Perform a Task Scale...............................................835

Table H-11 Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Reading Grade 12 Composite Scale......................................................836

Table H-12 Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Writing Grade 4 ...................................................................................836

Table H-13 Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Writing Grade 8 ...................................................................................837

Table H-14 Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Writing Grade 12 .................................................................................837

Table H-15 Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Civics Grade 4 ......................................................................................838

Table H-16 Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Civics Grade 8 ......................................................................................838

Table H-17 Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Civics Grade 12 ....................................................................................839

Appendix I Setting the Achievement Levels for the 1998 NAEP Reading Assessment

Table I-1 Results of First Review for Achievement-Level Exemplars..........................................846

Table I-2 Results of Review of Additional Items for Achievement-Level

Exemplars .....................................................................................................................847

Table I-3 Cut Points for Achievement Levels–Grade 4 ................................................................850

Figure I-1 Final Descriptions of 1992 Reading Achievement Levels ............................................851

Figure I-2 Draft Descriptions of the Achievement Levels Prepared by the Original

Level-Setting Panel .......................................................................................................857

Figure I-3 Revised Draft Descriptions of the Achievement Levels Recommended

by the Follow-Up Validation Panel...............................................................................860

Figure I-4 Meeting Participants, NAEP Reading Achievement Level Setting

Original Meeting, St. Louis, Missouri, August 21–25, 1992 ........................................863
xxxiii

Figure I-5 Meeting Participants, NAEP Reading Achievement Level Setting Follow-Up

Validation Meeting, San Diego, California, October 9–11, 1992................................. 865

Figure I-6 Meeting Participants, NAEP Reading Revisit Validation Meeting, St.

Louis, Missouri, October 14–16, 1994 ......................................................................... 866

Appendix J Setting the Achievement Levels for the 1998 NAEP Civics and Writing Assessments

Figure J-1 Sample Reckase Chart Portion ..................................................................................... 872

Figure J-2 Sample Reckase Chart - Complete................................................................................ 873

Table J-1 Pilot Study Cut Scores (Standard Deviations) on the 1998 Civics

NAEP............................................................................................................................ 874

Table J-2 Pilot Study Cut Scores (Standard Deviations) on the 1998 Writing

NAEP............................................................................................................................ 874

Table J-3 Civics Achievement-Level Cut Scores and Standard Deviations, by

Rounds and Percent Correct Data................................................................................. 877

Table J-4 Writing Achievement-Level Cut Scores and Standard Deviations, by

Rounds and Percent Correct Data................................................................................. 878

Table J-5 Mean Cut Scores and Standard Deviations in Writing, by Panelist Type..................... 879

Table J-6 Mean Cut Scores and Standard Deviations in Writing, by Ethnicity ............................ 881

Table J-7 Mean Cut Scores and Standard Deviations in Civics, by Ethnicity .............................. 882

Table J-8 Mean Cut Scores and Standard Deviations in Civics, by Panelist Type ....................... 884

Table J-9 Mean Differences Between Polytomous and Dichotomous Cut Scores

for Civics ...................................................................................................................... 885

Figure J-3 Achievement-Level Descriptions for Civics ................................................................. 888

Figure J-4 Achievement-Level Descriptions for Writing............................................................... 893

Figure J-5 Meeting Participants, NAEP Civics Achievement Level Setting Pilot

Study, St. Louis, Missouri, August 13–17, 1998 .......................................................... 897

Figure J-6 Meeting Participants, NAEP Writing Achievement Level Setting Pilot

Study, St. Louis, Missouri, October 1–5, 1998............................................................. 900

Figure J-7 Meeting Participants, NAEP Civics Achievement Level Setting, St.

Louis, Missouri, November 12–16, 1998 ..................................................................... 903

Figure J-8 Meeting Participants, NAEP Writing Achievement Level Setting Study,

St. Louis, Missouri, December 9–13, 1998 .................................................................. 908

Figure J-9 Meeting Participants, Civics NAEP Similarities Classification Study

Validation Meeting, St. Louis, Missouri, July 9–11, 1999 ........................................... 913

Appendix K Participants in the Objectives and Item–Development Process

Figure K-1 1998 NAEP Reading Item Development Committee.................................................... 915

Figure K-2 1998 NAEP Reading/Writing Standing Committee...................................................... 916

Figure K-3 1998 NAEP Writing Item Development Committee .................................................... 917

Figure K-4 1998 NAEP Civics Item Development Committee....................................................... 918

Figure K-5 1998 NAEP Civics Standing Committee ...................................................................... 920

xxxv

ACKNOWLEDGMENTS

The design, development, administration, analysis, and reporting of the 1998 National Assessment

of Educational Progress (NAEP) program was a collaborative effort among staff from the National Center

for Education Statistics (NCES), the National Assessment Governing Board (NAGB), the Council of Chief

State School Officers (CCSSO), Educational Testing Service (ETS), Westat, and National Computer

Systems (NCS). This report documents NAEP design, administration, and data analysis procedures,

indicating what technical decisions were made and the rationale behind those decisions. The development of

this report and of the national assessment program is the result of the considerable knowledge, experience,

creativity, and dedication of many individuals. I would like to acknowledge these individuals for their

contribution to NAEP.

The 1998 NAEP assessment was funded through NCES, in the Office of Educational Research and

Improvement of the U.S. Department of Education. The NCES staff played a crucial role in all aspects of the

program. We are grateful for the reviews of this report contributed by Marilyn Binkley, Janis Brown, Peggy

Carr, Chris Chapman, Michael Cohen, Arnold Goldstein, Steven Gorman, Elvira Hauskens, Janet Johnson,

Daniel Kaspryzk, Steve Kaufman, Vonda Kiplinger, Andrew Kolstad, Ralph Lee, Larry Ogle, Andrew

Malizio, Marilyn McMillen, Gary Skaggs, Suzanne Triplett, Holly Spurlock, Michael Ross, Bruce Taylor,

Sheida White, and Shi-Chang Wu. Additional reviews were provided by Luz Bay and Eugene Johnson.

Special thanks also go to the members of the National Assessment Governing Board (NAGB) and

the NAGB staff who provided advice and guidance in the preparation of this report, particularly Mary Lyn

Bourque, who provided information for a report appendix.

ETS Management has encouraged high quality work on all NAEP activities. Thanks go to several

members of ETS Management: Nancy Cole, former President of ETS; Paul Ramsey, Vice-President for the

School and College Services Division; Henry Braun, former Vice-President for Research; Charles Davis,

former Director of the Psychometrics and Statistics Research Division; John Barone, Executive Director of

the Division of Data Analysis, Statistics, and Technology Research; and James Carlson, former Group

Leader of the Large-Scale Assessment Research Group. 

ETS management and the NAEP program development and reporting areas within ETS’s School and

College Services Division have been very supportive of NAEP’s technical work. Special thanks go to the

following staff members in the NAEP program area within ETS’s School and College Services Division who

provided direct leadership for the NAEP project: Steve Lazer, Executive Director for NAEP; John Mazzeo,

Center Director, Large-Scale Assessment Research; Beth Durkin, Lauren Fried, and Kim Whittington.

Significant contributions to the project were also received from Nada Ballator, Jay Campbell, Patricia

Donahue, Elissa Greenwald, Christine O’Sullivan, Hilary Persky, Shari Santapau, and Andrew Weiss.

The guidance of the NAEP Design and Analysis Committee on technical aspects of NAEP has been

outstanding. During the period of analysis of the 1998 data, the members were: Anthony Nitko (chair),

Sylvia Johnson (former chair), Albert Beaton, Johnny Blair, Jeremy Finn, Paul Holland, Huynh Huynh,

Edward Kifer, David Lohman, Serge Madhere, Ingram Olkin, Tej Pandey, and Hariharan Swaminathan.

The design and data analysis of the 1998 National Assessment was primarily the responsibility of

the NAEP research and data analysis staff at ETS with significant contributions from NAEP management,

Westat, and NCS staffs. In addition to managing day-to-day data analytic operations, NAEP Large-Scale

Assessment Research staff members have made many innovative statistical and psychometric contributions.

Major contributions were made by Hua-Hua Chang, John Donoghue, Frank Jenkins, Jiahe Qian, Spencer
xxxvi

Swinton, and Jinming Zhang under the leadership of James Carlson. Jo-lin Liang served as research

associate. Eugene Johnson and Robert Mislevy provided valuable statistical and psychometric advice.

The Division of Data Analysis, Statistics, and Technology Research at ETS, under the leadership of

John Barone, was responsible for developing the operating systems and carrying out the data analyses.

David Freund and Alfred Rogers developed and maintained the large and complex NAEP data management

systems, and Katharine Pashley managed database activities. Alfred Rogers developed the production

versions of key analysis and scaling systems. Thanks also go to David Freund, Steven Isham, Bruce Kaplan,

Debbie Kline, Edward Kulick, and Alfred Rogers for their continuing roles as leaders and developers of

innovative solutions to NAEP data analysis challenges. Many other members of this division made

important contributions of their time and talent to NAEP data analyses and analysis software and data

products, including John Ferris, Gerry Kokolis, Laura Jerry, Venus Leung, Youn-Hee Lim, Ting Lu, Mike

Narcowich, Norma Norris, Ingeborg Novatkoski, Tanya Petrovicheva, Steve Szyszkiewicz, Lois

Worthington, and Fred Yan. Special recognition goes to Phillip Leung for his web technology expertise.

The NAEP web site is managed at ETS by Madeline Goodman, Jeffrey Jenkins, and Richard

Bohlander. Pat O’Reilly manages the creation of Internet editions of NAEP reports, including the NAEP

1998 Technical Report. She is ably assisted by Rick Hasney.

The staff at Westat contributed their talents and efforts in all areas of the sample design and data

collection. Particular recognition is due to Nancy Caldwell, Rob Dymowski, Lucy Gray, Brice Hart, Tom

Krenzke, Keith Rust, Renee Slobasky, Debby Vivari, Mark Waksberg, Leslie Wallace, and Dianne Walsh

for directing the sampling and data collection activities. Thanks are also due to Carlos Arieira, John Burke,

Fran Cohen, Karen Dennis, Sharon Hirabayashi, Drew Kistler, Maida Montes, Tom Mule, Lana Ryaboy,

Rick Valliant, Ngoan Vo, and Ibrahim Yansaneh.

Critical to the program was the contribution of NCS, which has been responsible for the printing,

distribution, and processing of the assessment materials, as well as an increased role in professional scoring

of constructed-response items. The leadership roles of Patrick Bourgeacq, Charles Brungardt, William

Buckles, Tom Huencke, Matilde Kennel, Timothy Robinson, Connie Smith, and Bradley Thayer are

especially acknowledged. Thanks also go to Cynthia Malott and Brent Studer.

Cindy Hammel, Joan Stoeckel, Martha Thompson, and Karen Damiano of ETS are acknowledged

for their editorial and administrative assistance during the preparation of this report.

Special recognition and appreciation go to Terry Schoeps and Debbie Kline, editors of this report.

They have been responsible for organizing, scheduling, editing, motivating, and ensuring the cohesiveness

and correctness of the final report.

There are numerous subject-area, technical advisory, policy-related, and state assessment groups

that steer all aspects of the NAEP project. Their work has benefited the project enormously. Finally, NAEP

is grateful to the students and school staff members who participated in the assessment. Without their

efforts, there would be no assessment.

Nancy L. Allen

Center for Large-Scale Assessment Research

Division of Psychometrics and Statistics Research, ETS
1

INTRODUCTION1

James E. Carlson and Nancy L. Allen

Educational Testing Service

The 1998 National Assessment of Educational Progress (NAEP) monitored the performance of

students in United States schools in the subject areas of reading, writing, and civics. The national main

sample involved public- and nonpublic-school students who were in grades 4, 8, or 12. State assessments

were also conducted at grades 4 and 8 in reading and at grade 8 in writing. Nearly 448,000 students were

assessed in the national and state samples. Although a special study was done comparing 1998 civics

results with those for 1988, no NAEP long-term trend (LTT) assessments of reading, writing, math, or

science national samples were conducted in 1998.

For previous assessments in which there were both national (main and/or long-term trend) and

state components, separate technical reports were produced for the national assessment and each state

component (subject area). For 1998, this publication contains technical information about both the state

and national components. Information common to both national and state components is presented in the

first two parts, while later chapters contain detailed information for each subject area and for the national

and state components.

The purpose of this technical report is to provide details on the instrument development, sample

design, data collection, and data analysis procedures for the 1998 assessment. This document provides

information necessary to show adherence to the Standards for Educational and Psychological Testing

(American Educational Research Association [AERA], American Psychological Association [APA], &

National Council on Measurement in Education [NCME], 2000) and to the Educational Testing Service

(ETS) Standards for Quality and Fairness (Educational Testing Service [ETS], 1987). Detailed

substantive results are not presented here but can be found in a series of NAEP reports covering the

status of and trends in student performance; several additional reports provide information on how the

assessment was designed and implemented. The reader is directed to the following reports for 1998

results:

• NAEP 1998 Civics Report Card for the Nation (Lutkus, Weiss, Campbell,

Mazzeo, & Lazer, 1999)

• NAEP 1998 Reading Report Card for the Nation and the States (Donahue,

Voelkl, Campbell, & Mazzeo, 1999)

• NAEP 1998 Reading Report for {each state} (Ballator & Jerry, 1999a)

• NAEP 1998 Writing Report Card for the Nation and the States (Greenwald,

Persky, Campbell, & Mazzeo, 1999)

• NAEP 1998 Writing Report for {each state} (Ballator & Jerry, 1999b)

 

1 James E. Carlson , Nancy L. Allen, and John R. Donoghue were responsible for psychometric and statistical analyses of NAEP

for the 1998 assessment.
2

The Report Card publications highlight results for the nation, states, and selected subgroups. The

frameworks for the 1998 assessment content areas are in:

• Civics Framework for the 1998 National Assessment of Educational Progress

(National Assessment Governing Board [NAGB], 1996a)

• Reading Framework for the National Assessment of Educational Progress: 1992-

1998 (NAGB, 1990)

• Writing Framework and Specifications for the 1998 National Assessment of

Educational Progress (NAGB, 1996b)

Other technical information is in:

• Sampling Activities and Field Operations for 1998 NAEP (Gray, Krenzke, &

Wallace, 2000)

• Report on Data Collection Activities for All States (Westat, 1998)

• 1998 NAEP Assessment Report of Processing and Professional Scoring Activities

(National Computer Systems, 1998)

The NAEP 1998 Reading Data Companion (Rogers, Kokolis, Stoeckel, & Kline, 2000), the

NAEP 1998 Writing Data Companion (Rogers, Kokolis, Stoeckel, & Kline, 2000), and the NAEP 1998

Civics Data Companion (Rogers, Kokolis, Stoeckel, & Kline, 2000) provide information needed to

analyze the 1998 NAEP results, and The NAEP Guide: A Description of the Content and Methods of the

1997 and 1998 Assessments (Calderone, King, & Horkay, 1997) contains a description of the content and

methods used in both the main and state components of the 1998 assessments.

Many of the NAEP reports, including summary data tables, are available on the Internet at

http://nces.ed.gov/nationsreportcard. For information about ordering printed copies of these reports, go

to the Department of Education web page http://www.ed.gov/pubs/edpubs.html, call toll free

1–877– 4ED PUBS (877–433–7827), or write to:

Education Publications Center (ED Pubs)

U.S. Department of Education

P.O. Box 1398

Jessup, MD 20794 –1398

The Frameworks are descriptions and plans for subject-area assessment content. For ordering

information on these reports, write to:

National Assessment Governing Board

800 North Capitol Street NW

Suite 825

Washington, DC 20002

The Frameworks and other NAGB documents are also available through the Internet at http://www.nagb.org.
3

AN OVERVIEW OF NAEP ANALYSIS CHANGES OVER TIME

NAEP strives to maintain its links to the past and still implement innovations in measurement

technology. To that end, long-term trend samples use the same methodology and population definitions

as in previous assessments. Main assessment samples incorporate innovations associated with new NAEP

technology and address current educational issues. Both long-term trend samples and main assessment

samples are nationally represented. The main assessment sample data are used primarily for analyses

involving the current student population, but also to estimate short-term trends for a small number of

recent assessments. Some of the assessment materials administered to the main assessment samples are

periodically administered to state as well as national samples. In continuing to use this two-tiered

approach, NAEP reaffirms its commitment to continuing to study trends while at the same time

implementing the latest in measurement technology and educational advances.

In succeeding assessments, many of the innovations that were implemented for the first time in

1988 were continued and enhanced. For example, a focused balanced incomplete block (focused BIB)

booklet design was used in 1988. Since that time, either focused BIB or focused partially balanced

incomplete block (focused PBIB) designs have been used. Variants of the focused PBIB were used with

the 1998 main national and state assessment samples in reading and writing, and a focused BIB was used

in the 1998 main national civics assessment. Both the BIB and PBIB designs provide for booklets of

interlocking blocks of items, so that no student receives too many items, but all receive groups of items

that are also presented to other students. The booklet design is focused, because each student receives

blocks of cognitive questions in the same subject area. The focused BIB or PBIB design allows for

improved estimation within a particular subject area, and estimation continues to be optimized for groups

rather than individuals.

Since 1984, NAEP has applied the plausible values approach to estimating means for

demographic as well as curriculum-related subgroups. Scale score estimates were drawn from a posterior

distribution that was based on an optimum weighting of two sets of information: the student’s responses

to cognitive questions, and his or her demographic and associated educational process variables. This

Bayesian procedure was developed by Mislevy (1991). An improvement that was implemented first in

1988 and refined for the 1994 assessment continues to be used. This is a multivariate procedure that uses

information from all scales within a given subject area in the estimation of the scale score distribution on

any one scale in that subject area.

To shorten the timetable for reporting results, the period for national main assessment data

collection was shortened in 1992, 1994, 1996, and 1998 from the five-month period (January through

May) used in 1990 and earlier assessments to a three-month period in the winter (January through March,

corresponding to the period used for the winter half-sample of the 1990 national main assessment).

A major improvement introduced in the 1992 assessment, and continued in succeeding

assessments, was the use of the generalized partial-credit model for item response theory (IRT) scaling.

This allowed the incorporation of constructed-response questions that are scored on a multipoint rating

scale into the NAEP scale in a way that utilizes the information available in each response category.

One important innovation in reporting the assessment data that has been continued since 1990 is

the use of simultaneous comparison procedures in carrying out significance tests for the differences

across assessment years. Methods such as the Bonferroni procedure allow one to control for the type I

error rate for a fixed number of comparisons. Beginning with the 1996 assessment, a procedure providing

more powerful statistical tests that control for the false discovery rate (FDR) as applied by Benjamini and

Hochberg (1994) was used for comparisons involving a large number of groups (e.g., state comparisons).

In 1998 the FDR procedure was used for all comparisons in NAEP. While the Bonferroni procedure

controls the probability of making even one false rejection, the FDR procedure used in NAEP controls
4

the expected proportion of falsely rejected hypotheses. The Bonferroni procedure is more conservative

than the Benjamini procedure for large families of comparison.

ORGANIZATION OF THE TECHNICAL REPORT

This report begins with the details of the design of the 1998 main and state assessments,

summarized in Chapter 1. Chapters 2 through 8 provide an overview of the objectives and frameworks

for items used in the assessment, the sample selection procedures, the administration of the assessment in

the field, the processing of the data from the assessment instruments into computer-readable form, the

professional scoring of constructed-response items, and the methods used to create a complete NAEP

database.

The 1998 NAEP data analysis procedures are described in Chapters 9 through 13. Chapter 9

provides a summary of the analysis steps. Subsequent chapters provide a general discussion of the

weighting and variance estimation procedures used in NAEP, an overview of NAEP scaling

methodology, and information about the conventions used in significance testing and reporting NAEP

results.

Details of the reading assessment data analysis are provided in Chapters 14 through 17. These

chapters describe assessment frameworks and instruments, student samples, items, booklets, scoring, DIF

analysis, weights, and item analyses of the main and state assessments. Similar details are provided for

the writing assessment (Chapters 18 through 21) and the civics assessment (Chapters 22 through 24).

The appendices provide detailed information on a variety of procedural and statistical topics.

Appendices I and J explain how achievement levels for the subject areas were set by the National

Assessment Governing Board (NAGB). The last appendix (Appendix K) provides lists of committee

members who contributed to the development of objectives and items.
5

Chapter 1

OVERVIEW OF PART I:

THE DESIGN AND IMPLEMENTATION OF THE 1998 NAEP1

Nancy L. Allen, James E. Carlson, and John R. Donoghue

Educational Testing Service

1.1 INTRODUCTION

 The 1998 National Assessment of Education Progress (NAEP) collected information on the

knowledge and skills of American students in reading, writing, and civics. The 1998 NAEP assessment

included three components: the national main assessments of reading, writing, and civics; the state

assessments of reading and writing; and national special assessments of aspects of writing and civics.

The main assessments were administered to national samples of students. No long-term trend (LTT)

assessment was included in 1998. The basis for the information collected for the national main

assessments was a complex sample survey involving nearly 448,000 students, consisting of national

samples of public- and nonpublic-school students who were in grades 4, 8, and 12. Additional NAEP data

came from the state assessment program, which in 1998 assessed about 300,000 students in reading at

grades 4 and 8 and in writing at grade 8. Grade 4 state samples included public-school students from 40

states, the District of Columbia, the Department of Defense Dependent Elementary and Secondary

Schools (DoDEA/DDESS2

), the Department of Defense Dependents Schools (DoDEA/DoDDS2), and

Virgin Islands, as well as nonpublic-school students from 29 states and Virgin Islands. Grade 8 state

samples for reading included public-school students from 37 states, the District of Columbia,

DoDEA/DDESS, DoDEA/DoDDS, and Virgin Islands, as well as nonpublic-school students from 23

states and Virgin Islands. Grade 8 state samples for writing included public-school students from 36

states, the District of Columbia, DoDEA/DDESS, DoDEA/DoDDS, and Virgin Islands, as well as

nonpublic-school students from 23 states and Virgin Islands. Results for a few of these states and

jurisdictions were not reported because reporting guidelines were not met.

This chapter describes the design for the 1998 main and state assessments and gives an overview

of the steps involved in its implementation, from the planning stage through the creation of edited data

files. The major components of the implementation are presented here with references to other chapters

in Part I that provide greater detail on each aspect of the assessment. The procedures used for the analysis

of the data are summarized in the overview to Part II. The remaining chapters, in Parts III, IV, and V,

detail the data analysis by each subject area. Excluded are details of the analyses of special studies of 50-

minute writing, classroom-based writing, 1988-to-1998 trends in civics, and high school transcripts. The

results from and analyses used in these special studies will be described in separate documents.

 

1 Nancy L. Allen, James E. Carlson, and John R. Donoghue were responsible for the psychometric and statistical analysis of the

1998 national and state NAEP data. The authors are indebted to the authors of Chapters 2 through 8 for portions of this chapter. 2 DoDEA is the Department of Defense Education Activity. Within the DoDEA, two jurisdictions are reported for NAEP: one

for domestic schools (Department of Defense Domestic Dependent Elementary and Secondary Schools [DDESS]) and one for

overseas schools (Department of Defense Dependents Schools [DoDDS]).
6

The organization of this chapter, and of Part I, is as follows:

• Section 1.2 provides an overview of the NAEP design for 1998 and includes a

description of the constituent samples. To provide background information, the

section also includes the assessment schedule from the inception of NAEP in 1969

through the 1998 assessment.

• Section 1.3 provides a summary of the development of the objectives for each

subject area in the assessment and a description of the development and review of

the items written to fit those objectives. Details and results of the objective and item

development processes appear in Chapters 2, 14, 18, and 22.

• Section 1.4 provides a summary of the sampling design used for the 1998 national

and state assessments, with a fuller description provided in Chapters 3 (national) and

4 (state).

• Section 1.5 includes a discussion of the assignment of the cognitive and background

questions to assessment booklets and a description of the complex block designs that

were the basis for assigning cognitive items to assessment booklets and assessment

booklets to individuals. Chapters 14, 18, and 22 provide detailed descriptions of the

assessment booklets for the subject areas of reading, writing, and civics,

respectively.

• Section 1.6 provides a summary of the field administration procedures, including the

processes of training field administrators, attaining school cooperation,

administering the assessment, and conducting quality control. Further details appear

in Chapter 5.

• Section 1.7 includes a description of the flow of data from the receipt of the

assessment materials through data entry, validation, and resolution to the creation of

edited data files. Chapter 6 provides a detailed description of the process.

• Section 1.8 contains a discussion of the professional scoring of students’ responses

to the constructed-response items in the assessment. Details of the process are given

in Chapter 7.

• Section 1.9 provides a summary of the creation of the database, the quality control of

data entry, and lists the 1998 database products. This section also includes a

description of the use of the Internet for dissemination of NAEP information. Further

details appear in Chapter 8.

1.2 THE 1998 NAEP DESIGN

A major purpose of NAEP is the reliable measurement of trends in educational achievement over

time. To do this well, confounding effects due to changes from one assessment to the next in assessment

instrumentation or in assessment procedures must be minimized. This implies a stability in the

measurement process over time. At the same time, the assessment must remain current by allowing the

introduction of new curriculum concepts and changes in educational priorities and by permitting the use

of new measurement technology. The objectives for an assessment are determined through a consensus

process in which committees of subject-matter experts, scholars, and citizens representing many diverse
7

constituencies and points of view are assembled to determine the educational goals that students should

achieve. Satisfying these objectives often requires changes in assessment instrumentation and

methodology.

In order to meet the goals of measuring trends reliably and responding to changes in the current

thinking about subject areas, NAEP has instituted a multicomponent assessment system where each

component is itself a set of assessments designed to accomplish a specific goal. There are four

components in the NAEP design: national main assessments, state assessments, national long-term-trend

assessment in reading, writing, math and science, and special assessments. The national main and state

assessments respond to changes in curriculum on a regular basis, as compared to the long-term trend

assessments, which were administered in 1996 and will be administered again in 1999. The instruments

that measure long-term trends are never changed and measure longer-term trends in a content domain that

is constant over the years.

Several improvements were made in the design of NAEP in the 1984 and succeeding

assessments. Until the 1984 assessment, NAEP was administered using matrix sampling and tape

recorders; that is, by administering booklets of exercises using an aurally presented stimulus that paced

groups of students through the individual assessment exercises in a common booklet. In the 1984

assessment, balanced incomplete block (BIB) spiraling, which does not include aural pacing, was

introduced in place of taped matrix sampling. BIB spiraling is defined in Section 1.5 of this chapter. The

NAEP design now includes sampling grade populations for national main and state assessments, as well

as the age populations that NAEP originally assessed for long-term trend assessments. The definitions of

student age and the time of year in which the assessment takes place have been made uniform so that

students in the fourth, eighth, and twelfth grades are assessed. To shorten the timetable for reporting

results, the period for national main data collection was decreased in assessments since 1990 from the

five-month period used in 1990 to a three-month period in the winter (corresponding to the period used

for the winter half-sample of the 1990 national assessment). To enhance the coverage of the subject areas

assessed, the number of items measuring knowledge and skills was increased for NAEP assessments

since 1990.

A special feature of the 1998 national main and state assessments of reading was the collection

of data from students who were offered accommodations and from students who were not, while using

the new rules (introduced in 1996) for inclusion of students with disabilities (SD) and limited English

proficient (LEP) students in NAEP assessments. Figure 1-1 contains the layout of the pieces of the

sample collected for each grade of the national main and state assessments of reading. In one sample

(sample type 2 in Figure 1-1), accommodations were not offered to students. In the other sample (sample

type 3 in Figure 1-1), students were offered accommodations. Both sample type 2 and sample type 3

schools selected for participation in the 1998 assessments used the new inclusion rules to determine

whether students should be included in the assessment.

For all subject areas, the inclusion rules were applied and accommodations were offered only

when a student had been categorized in his or her individualized education program (IEP) as a student

with disabilities (SD) or as a limited English proficient (LEP) student; all other students were asked to

participate in the assessment. The accommodations provided by NAEP in the national main and state

assessments were meant to match those specified in the student’s IEP or those ordinarily provided in the

classroom for testing situations. The most common accommodation was extended time.

For the 1998 reading national main and state assessments, the sample of students selected for

most analysis and reporting purposes consisted of students from two groups: those who were not

categorized as SD or LEP students (A2 and A3 in Figure 1-1); and those who were categorized as SD or

LEP students and who attended schools providing no accommodations (B2 in Figure 1-1). Test results for

students who were offered accommodations (B3 in Figure 1-1) were not included in the analysis or
8

reporting of the national main and state assessment results for reading, although the results for students

offered accommodations were studied in follow-up analyses. The advantage of the selected reporting

sample is that it preserves trend with previous assessments and it makes use of most of the data from the

assessment. For the writing and civics assessments, NAEP used the new inclusion rules and provided

accommodations to identified students (sample type 3 in Figure 1-1). The information in Chapters 3, 4,

and 5 applies to schools and students in all of the sample types, while the data analysis chapters reflect

schools and students in reporting samples only.

Figure 1-1

Subsamples of the 1998 NAEP Reading Assessment

GROUPS OF SCHOOLS

GROUPS OF STUDENTS

Sample Type 2

- NO ACCOMMODATIONS -

Sample Type 3

- ACCOMMODATIONS -

NOT SD/LEP1 A2 A3

2

INCLUDED SD/LEP1 B2

2 B32

EXCLUDED SD/LEP1

C2

3 C33

1 Students with Disabilities/Limited English Proficient

2 Results for students in subsample B3 were not reported in NAEP 1998 Reading: Report Card

for the Nation and the States. 3 Students in subsamples C2 and C3 were not included in the assessment.

NAEP’s design for 1998 required collecting 19 different samples in order to conduct the

assessments. The various samples collected and reported for the 1998 assessment are summarized in

Table 1-1.
9

Table 1-1

NAEP 1998 Student Samples*

Sample Booklet IDs

Cohort

Assessed

Reporting

Sample Size†

 4 [Reading–Main] R1-R16 Grade 4 7,672

 8 [Reading–Main] R1-R18, R21 Grade 8 11,051

12 [Reading–Main] R1-R18, R21-R22 Grade 12 12,675

 4 [Reading–State] R1-R16 Grade 4 112,138‡

 8 [Reading–State] R1-R18,R21 Grade 8 94,429‡

 4 [Writing–Main] W201-W240 Grade 4 19,816

 8 [Writing–Main] W201-W240 Grade 8 20,586

12 [Writing–Main] W201-W237 Grade 12 19,505

 8 [Writing–50-Minute] W241-W243 Grade 8 6,009

12 [Writing–50-Minute] W241-W243 Grade 12 5,804

 4 [Writing–Classroom Study] —§ Grade 4 2,395**

 8 [Writing–Classroom Study] —§ Grade 8 2,480**

 8 [Writing–State] W201-W240 Grade 8 97,589‡

 4 [Civics–Main] C301-C318 Grade 4 5,948

 8 [Civics–Main] C301-C332 Grade 8 8,212

12 [Civics–Main] C301-C332 Grade 12 7,763

 4 [Civics–Special Trend] CT340†† Grade 4 2,088

 8 [Civics–Special Trend] CT340†† Grade 8 2,055

12 [Civics–Special Trend] CT340†† Grade 12 2,193

 Total without [Writing–Classroom Study]† 438,164

* The 1998 assessment was administered January 5–March 27, 1998. Final makeup sessions

were held March 30–April 3, 1998.

† The reporting samples for reading include students in groups A2, A3, and B2 in Figure 1-1.

Reporting and assessed samples for writing and civics include students designated by A3

and B3. ‡ This sample size includes counts of students from distinct samples for each state or

jurisdiction participating in the assessment.

§ No booklets were administered in the [Writing–Classroom Study]; instead, examples of

classroom-based writing were collected from students participating in this study. ** Because some of the students in this study were included in the [Writing–Main] and

[Writing–50-Minute] samples and others were not included in these samples, the students

in the [Writing–Classroom Study] who are counted here are not included in the reporting

sample size total.

†† These booklets were also administered as a part of the 1988 assessment of civics.
10

Each row of Table 1-1 corresponds to a particular sample and each column of the table indicates

the following major features of that sample:

1. Sample is the sample identifier. The first part of the sample code is a number (the

grade) representing the student cohort included in the sample; the second part, in

brackets, denotes the specific sample type. For example, 4 [Reading–Main] is a

national main assessment reading sample for grade 4. A full description of the

purposes for the various sample types is given in Section 1.2.1.

2. Booklet IDs give the identifier numbers for the booklets used for the assessment of the

particular sample.

3. The cohort assessed denotes the age, grade, or age/grade of the population being

sampled. For example, a grade 4 cohort represents students who are in the fourth

grade; an age 17 cohort consists of students (in any grade) who are 17 years old.

Samples for the 1998 national main assessments were selected on the basis of grade

only. The traditional NAEP samples used in long-term trend estimation were defined

by age only. The definitions of age, and thus the corresponding grade, have changed

in ways that are described in Section 1.2.2.

4. The reporting sample size is the number of students in the sample who were

administered the assessment and whose results were used in the NAEP subject-area

reports. SD/LEP students who were excluded from the assessment (C2 and C3 in

Figure 1-1) are not included in the reporting samples. The reporting samples for the

reading assessment include students who were not categorized as SD or LEP students

(A2 and A3 in Figure 1-1), as well as students who were categorized as SD or LEP

students and attended schools where no accommodations were offered (B2 in Figure

1-1). The reporting sample for the writing and civics assessments include students

who were not categorized as SD or LEP students (A3 in Figure 1-1) and students who

were categorized as SD or LEP students and attended schools where accommodations

were offered (B3 in Figure 1-1).

1.2.1 The 1998 NAEP Samples

The NAEP samples in 1998 consisted of three types: the main samples from the national

assessment, samples from the state assessment, and the special studies samples from the national

assessment. No data from long-term trend (LTT) for reading, writing, math, or science samples were

collected in 1998.

The National Main Assessment Samples. The national main NAEP samples are labeled in Table

1-1 as [Reading–Main], and [Writing–Main], and [Civics–Main]. The samples used complex spiraling

procedures (defined in Section 1.5), and were intended to form the basis for future assessments. Each

sample was assessed in the winter period. In these samples, only grade populations were sampled,

although age/grade populations were assessed in previous assessment years for reading. The national

main assessment samples, and their purposes, are as follows:

[Reading–Main] are grades 4, 8, and 12 national reading assessment samples used for

measuring national reading achievement in 1998. The grade 4 and 8 samples also provided the

comparison groups for the 1998 state assessment of reading in grades 4 and 8 [Reading–State].

These samples used print administration.
11

[Writing–Main] are grades 4, 8, and 12 national writing assessment samples used for

measuring national writing achievement in 1998. The grade 8 samples also provided the

comparison groups for the 1998 state assessment of writing in grade 8 [Writing–State]. These

samples used print administration.

[Civics–Main] are grades 4, 8, and 12 civics national assessment samples used for measuring

national civics achievement in 1998. Civics was not part of the state assessment in 1998. These

samples used print administration.

The State Assessment Samples. In Table 1-1, [Reading–State] and [Writing–State] refer to

samples of public- and nonpublic-school students from each of the states and jurisdictions participating

in the NAEP 1998 state assessments of reading (at grades 4 and 8) and writing (at grade 8). The

assessment booklets were the same print-administered booklets as those used for the matching national

samples [Reading–Main] and [Writing–Main], but the administrative procedures varied from that of the

main assessment in that state personnel collected the data.

The Special Studies Samples. Three sets of samples were collected as part of special NAEP

studies. The samples used special innovative procedures to allow the study of specific aspects of writing

and civics. Each sample was assessed in the winter period. In these samples, only grade populations were

sampled. The special studies samples, and their purposes, are as follows:

[Writing–50-Minute] are samples of specially selected students in grades 8 and 12 who

were administered 50-minute writing blocks in sessions separate from those in which

25-minute blocks were administered.

[Writing–Classroom Study] are samples of grade 4 and grade 8 students in intact

classrooms within schools that participated in the national main writing assessment.

Analyses of the data from the classroom-based writing study are described in the special

report of results from this study. They are not described in this report.

[Civics–Special Trend] are samples of specially selected students in grades 4, 8, and 12

who were administered a booklet from the 1988 civics assessment.

In addition to these special study samples for which different analyses were conducted, the High

School Transcript Study based on the full sample of twelfth grade students required special analyses.

Westat conducted this study and is responsible for analysis of the data. Although the results of this study

are not described in this technical report, documentation is available through Westat in Rockville,

Maryland.

1.2.2 NAEP Assessments Since 1969

Table 1-2 shows the subject areas, grades, and ages assessed since the NAEP project began in

1969. As can be seen, in addition to the 1998 subject areas of reading, writing, and civics, several other

subject areas have been assessed over the years—mathematics, science, social studies, U.S. history,

citizenship, geography, literature, music, career development, art, and computer competence. Many

subject areas are reassessed periodically to measure trends over time.
12

Table 1-2

National Assessment of Educational Progress

Subject Areas, Grades, and Ages Assessed: 1969–1998

Grades/Ages Assessed

Assessment

Year Subject Area(s)

Grade

3

Grade

4

Age

9

Grade

7

Grade

8

Age

13

Grade

11

Grade

12

A

1969–70 Science X X 

Writing X X Citizenship X X 

1970–71 Reading X X 

Literature X X 

1971–72 Music X X 

Social Studies X X 

1972–73 Science X X 

Mathematics X X 

1973–74 Career and Occupational Dvlpt. X X 

Writing X X 

1974–75 Reading X X 

Art X X 

1975–76 Citizenship/Social Studies X X 

Mathematics† X 

1976–77 Science X X 

Basic Life Skills† 

Health† 

Energy† Reading† Science† 

1977–78 Mathematics X X 

Consumer Skills† 

1978–79 Art X X 

Music X X Writing X X 

1979–80 Reading X X 

Literature X X 

* Age 17 students who had dropped out of school or had graduated prior to assessment.

† Small, special-interest assessments conducted on limited samples at specific grades or ages.

(continued)
Table 1-2 (continued)

National Assessment of Educational Progress

Subject Areas, Grades, and Ages Assessed: 1969–1998

Grades/Ages Assessed

Assessment

Year‡ Subject Area(s)

Grade

3

Grade

4

Age

9

Grade

7

Grade

8

Age

13

Grade

11

Grade

12

Ag

17

1981–82 Mathematics X X X

Citizenship/Social Studies X X XScience† XXX

1983–84 Reading X X X X X

Writing X X X X X

1985 Adult Literacy† 

1986 Reading X X X X X X

Mathematics X X X X X XScience X X X X X XComputer Competence X X X X X XU.S. History† X X

Literature† X XReading (long-term trend) X X X X X XMathematics (long-term trend) X X X X X XScience (long-term trend) X X X X X X

‡ It should be noted that somewhat different age definitions were used in the 1984, 1986, and 1988 assessments. In the 1984 assessments, the two 

defined on a calendar-year basis, while the 17-year-olds were defined on an October 1 to September 30 basis. This resulted in modal grades of 4for age cohorts that were exactly four years apart, in the 1986 national main assessment all ages were defined on an October 1 to September 30 

modal grades of 3, 7, and 11. Special studies (Kaplan et al., 1988) were conducted to measure the effect of the changes in age definition. Becaus

encountered in assessing third-graders, in 1988 the ages were defined on a calendar-year basis, with the modal grades being 4, 8, and 12. These definitions used in the 1990, 1992, and 1994 math assessments.

* Age 17 students who had dropped out of school or had graduated prior to assessment.† Small, special-interest assessments conducted on limited samples at specific grades or ages.

(continued)
14

Table 1-2 (continued)

National Assessment of Educational Progress

Subject Areas, Grades, and Ages Assessed: 1969–1998

Grades/Ages Assessed

Assessment

Year‡ Subject Area(s)

Grade

3

Grade

4

Age

9

Grade

7

Grade

8

Age

13

Grade

11

Grade

12

A

1988 Reading X X X X X 

Writing X X X X X Civics X X X X X U.S. History X X X X X Document Literacy† X X X Geography† X Reading (long-term trend) X X X X X Writing (long-term trend) X X X X X Mathematics (long-term trend) X X X Science (long-term trend) X X X 

1990 Reading X X X X X 

Mathematics X X X X X Science X X X X X Reading (long-term trend) X X X X X Writing (long-term trend) X X X X X Mathematics (long-term trend) X X Science (long-term trend) X X Trial State Mathematics X

1992 Reading X X X X X 

Writing X X X X X Mathematics X X X X X Reading (long-term trend) X X X X X Writing (long-term trend) X X X X X Mathematics (long-term trend) X X Science (long-term trend) X X Trial State Mathematics X XTrial State Reading X

‡ It should be noted that somewhat different age definitions were used in the 1984, 1986, and 1988 assessments. In the 1984 assessments, the two younger 

calendar-year basis, while the 17-year-olds were defined on an October 1 to September 30 basis. This resulted in modal grades of 4, 8, and 11. To allow fexactly four years apart, in the 1986 national main assessment all ages were defined on an October 1 to September 30 basis, resulting in modal grades of 

studies (Kaplan et al., 1988) were conducted to measure the effect of the changes in age definition. Because of problems encountered in assessing third-grwere defined on a calendar-year basis, with the modal grades being 4, 8, and 12. These were the age definitions used in the 1990, 1992, and 1994 math as

*Age 17 students who had dropped out of school or had graduated prior to assessment.

(continued)
Table 1-2 (continued)

National Assessment of Educational Progress

Subject Areas, Grades, and Ages Assessed: 1969–1998

Grades/Ages Assessed

Assessment

Year‡ Subject Area(s)

Grade

3

Grade

4

Age

9

Grade

7

Grade

8

Age

13

Grade

11

Grade

12

A

1

1994 Reading X X X X X X

U.S. History X X X X X XGeography X X X X X XReading (long-term trend) X X X X X XWriting (long-term trend) X X X X X XMathematics (long-term trend) X X XScience (long-term trend) X X XTrial State Reading X

1996 Mathematics X X X

Science X X XReading (long-term trend) X X X X X XWriting (long-term trend) X X X X X XMathematics (long-term trend) X X XScience (long-term trend) X X XState Mathematics X XState Science† X

1997 Music X

Theatre XVisual Arts X

1998 Reading X X X

Writing X X XCivics X X XState Reading X XState Writing X

‡ It should be noted that somewhat different age definitions were used in the 1984, 1986, and 1988 assessments. In the 1984 assessments, the two 

defined on a calendar-year basis, while the 17-year-olds were defined on an October 1 to September 30 basis. This resulted in modal grades of 4, age cohorts that were exactly four years apart, in the 1986 national main assessment all ages were defined on an October 1 to September 30 basi

grades of 3, 7, and 11. Special studies (Kaplan et al., 1988) were conducted to measure the effect of the changes in age definition. Because of pro

assessing third-graders, in 1988 the ages were defined on a calendar-year basis, with the modal grades being 4, 8, and 12. These were the age def1990, 1992, and 1994 math assessments.

* Age 17 students who had dropped out of school or had graduated prior to assessment.† Department of Defense Education Activity (DoDEA) schools were assessed at both grades 4 and 8. All other states and jurisdictions in the 1996 

were assessed at grade 8 only.
16

Since its inception, NAEP has assessed 9-year-olds, 13-year-olds, and in-school 17-year-olds,

although the age definitions changed in 1986 and again in 1988. Because of budget restrictions, NAEP no

longer routinely assesses out-of-school 17-year-olds or young adults. (A separate assessment of young

adults of ages 21 to 25 was conducted in 1985 under a separate grant.) Currently, NAEP assesses fourthand eighth-grade students in the national and state assessments, and twelfth-grade students in the national

assessment. Between 1980 and 1996, assessments were administered bi-annually, rather than annually,

due to funding restrictions. National (main and/or long-term trend) assessments are now conducted

annually, and state assessments continue to be conducted bi-annually.

The table also indicates that in 1984, NAEP began gathering data by grade as well as by age, a

practice that had been continued in national main assessments up to 1994; the 1996 and 1998 national

main assessments included data gathered by grade only. It should be noted that somewhat different age

definitions were used in the 1984, 1986, and 1988 assessments. In the 1984 assessment, the two younger

ages were defined on a calendar-year basis, while the 17-year-olds were defined on an October 1 to

September 30 basis. This resulted in modal grades of 4, 8, and 11. To allow for age cohorts that were

exactly four years apart, in the 1986 national main assessment all ages were defined on an October 1 to

September 30 basis, resulting in modal grades of 3, 7, and 11. Special studies (Kaplan, Beaton, Johnson,

& Johnson, 1988) were conducted to measure the effect of the changes in age definition. Because of

problems encountered in assessing third-graders, in 1988 the ages were redefined on a calendar-year

basis, with the modal grades being 4, 8, and 12. These were the age definitions used in the 1990, 1992,

and 1994 national main assessments.

1.3 DEVELOPMENT OF ASSESSMENT OBJECTIVES, ITEMS, AND

BACKGROUND QUESTIONS

In 1998, NAEP conducted national assessments of students at all three grade levels in reading,

writing, and civics. These assessments entailed the generation of a large number of cognitive items—

items measuring knowledge and skills. In addition, a large number of background questions were asked

of students. School, teacher, and instructional questions were asked of principals and teachers. Details on

the item-development procedures for the 1998 national assessment are given in Chapter 2.

In addition to the cognitive items, several questionnaires were developed: a common student

background questionnaire given to all assessed students of a given grade, a subject-specific background

questionnaire, a school characteristics and policies questionnaire, and teacher questionnaires for teachers

of fourth- and eighth-grade students in reading, writing, and civics. A questionnaire for which teachers or

school officials provided information about students with disabilities (SD) or students with limited

English proficiency (LEP) was also developed. Each of these questionnaires was developed through a

broad-based consensus process.

All cognitive and background questions in the assessment underwent extensive reviews by

subject-area and measurement specialists, as well as careful scrutiny to eliminate any potential bias or

lack of sensitivity to any representative group. Further, the items were field tested on a group of students

from across the nation. Based on the results of the field test, items were revised or modified as necessary

and then again reviewed for bias. With the help of staff and outside reviewers, the instrument

development committees selected the items to include in the assessment. After the items were selected

and formed into the final groupings or blocks of items, they were carefully reviewed by the National

Center for Education Statistics (NCES), the Office of Management and Budget (OMB), and the National

Assessment Governing Board (NAGB).
17

The assessment instruments included multiple-choice items, constructed-response items scored

dichotomously, constructed-response items scored polytomously, and cluster items in reading, writing,

and civics. The constructed-response items were professionally scored as described in Chapter 7.

1.4 THE 1998 SAMPLE DESIGN

The sample for the 1998 NAEP assessment was selected using a complex multistage sample

design. The multistage sample design includes the sampling of students from selected schools within

geographic areas (for national NAEP only), called primary sampling units (PSUs), across the United

States. Additional stages in the design are the assignment of assessment sessions to schools and the

assignment of students to sessions. Apart from the assignment of two types of samples in the reading

assessment (one that provided accommodations to certain students and one that did not), the general

sampling design for the 1998 assessment was similar in most respects to that of 1996. The design is

described in detail by Westat, the firm contracted by NCES to select the sample, in the Sampling

Activities and Field Operations for 1998 NAEP (Gray, et al., 2000). The following sections provide an

overview of the steps used to draw NAEP samples using the multistage sample design. Further details are

given in Chapters 3 and 4. Steps 3 and 4 describe the assignment of sample types and assessment sessions

to the second sampling unit schools.

1.4.1 Step 1: Primary Sampling Units

National Assessment

In the first stage of sampling for the national NAEP assessment, the United States (the 50 states

and the District of Columbia) was divided into geographic primary sampling units (PSUs). Each PSU met

a minimum size requirement and generally comprised either a consolidated metropolitan statistical area

(CMSA), a metropolitan statistical area (MSA), a single county, or a group of contiguous counties. The

PSUs were classified into four Regions (Northeast, Southeast, Central, West), each containing about onefourth of the U.S. population. In each region, PSUs were additionally classified as MSA or non–MSA.

This resulted in eight subuniverses of PSUs.

Ninety-four of the PSUs were selected for the 1998 national assessment. Twenty-two PSUs were

designated as certainty units (required to be in the sample) because of their size, and were included in the

sample with certainty. The remaining smaller PSUs were not guaranteed to be selected and were

accordingly designated as noncertainty PSUs. Within each major stratum, further stratification was

achieved by ordering the noncertainty PSUs according to several additional socioeconomic

characteristics, creating a second group of strata. Seventy-two PSUs were selected, one per stratum from

each of the noncertainty strata, with probability proportional to size (total population from the 1990

census). To enlarge the samples of Black and Hispanic students, thereby enhancing the reliability of

estimates for these groups, PSUs from the high-minority noncertainty strata were sampled at twice the

rate of those from the other strata. This was achieved by creating smaller strata within the high-minority

noncertainty strata.

State Assessment

For each jurisdiction in the state assessment, schools were the primary sampling units (PSUs).
18

1.4.2 Step 2: Selection of Schools

National Assessment

In the second stage of sampling for the national assessments, the public schools (including

Bureau of Indian Affairs [BIA] schools and Department of Defense Education Activity [DoDEA]

schools) and nonpublic schools (including Catholic schools) within each of the selected PSUs were listed

according to the grade ranges associated with the three age classes. An independent sample of schools

was selected separately for each of the grades so that some schools were selected for assessment of two

grades, and a few were selected for all three. Schools within each PSU were selected (without

replacement) with probabilities proportional to assigned measures of size with oversampling of

nonpublic schools and of schools with high minority enrollment. Overall probabilities of selection for

high-minority schools were twice those for other schools, while the probabilities of selection for

nonpublic schools were triple those for low-minority public schools of the same size. The increased

probabilities of selection enlarged the samples of Black and Hispanic students and the samples of

students from nonpublic schools, thereby enhancing the reliability of estimates for these groups. Details

of the probabilities used for school selection appear in Chapters 3 and 4. For the national samples, the

overall school cooperation rate was 86 percent for grade 4, 83 percent for grade 8, and 79 percent for

grade 12. In certain instances, refusing schools were replaced by substitutes according to the rules

indicated in Chapters 3 and 4.

State Assessment

For the state samples, the stratification used for sample selection varied by school type (public or

nonpublic). Stratification of public schools involved four primary dimensions, whereas the stratification

of nonpublic schools involved three primary dimensions. Public schools were stratified hierarchically by

small- or large-district status, school size class (measured by student enrollment), urbanization

classification, and minority classification. Nonpublic schools were stratified by school size class, metroarea status, and school type (Catholic or other nonpublic). Public schools were further stratified

implicitly by median household income (i.e., sorted in ascending or descending order) of the ZIP code

area where the school was located, and nonpublic schools were further stratified implicitly by estimated

grade enrollment in order to provide some control over these variables. Schools were randomly sampled

within these stratification classifications.

1.4.3 Step 3: Assigning Assessment Session and Sample Type to Schools

National and State Assessments

Sessions were assigned to the selected schools found to be appropriate at the time of session

assignment, as described in Chapters 3 and 4. Sessions were assigned to schools with three goals in mind.

The first was to distribute students to the different session types across the entire sample for each grade

so that the target numbers of assessed students would be achieved (in each sample type separately in the

national main assessments). The second was to maximize the number of different session types that were

administered within a given selected school, without creating unduly small sessions. The third was to

give each student an equal chance of being selected for a given session type regardless of the number of

sessions conducted in the school.

In order to determine the effect of using different criteria for excluding students from the

assessment, three different sample types were assigned to the schools selected for the national main

assessment in 1996. In sample type 1 schools, the inclusion criteria for the national main samples were

identical to those used in 1990 and 1992. In sample type 2 schools, new 1996 inclusion criteria were
19

used. In sample type 3 schools, the new 1996 inclusion criteria were used and accommodations were

offered to SD/LEP students. In the 1998 national main and state reading assessments, sample types 2 and

3 were assigned to schools. The writing and civics assessments were administered to sample type 3

schools only. More detailed information on assigning sample type to schools is provided in Chapters 3

and 4. Inclusion criteria and accommodations are described in Chapter 5.

1.4.4 Step 4: Sampling Students and Teachers

National and State Assessments

In the final stage of sampling, a consolidated list was prepared for each school of all gradeeligible students for the grade for which the school was selected. To provide the target sample size, a

systematic selection of eligible students was made from this list, if necessary. In small- and medium-sized

schools, all eligible students were in the sample. For schools assigned to more than a single session type,

students were assigned by Westat district supervisors to one of the various session types (audiotape or

print administration) using specified procedures. No student was assigned to more than one session. In

the national main NAEP assessment, students with disabilities and minority students in low-minority

schools were oversampled.

Step 4a: Excluded Students. Despite NAEP’s goal to assess all selected students, certain selected

students were judged by school authorities as being incapable of participating meaningfully in the

assessment. For each student who was excluded, school staff who had knowledge of the student’s

capabilities completed an SD/LEP student questionnaire, listing the reason for exclusion and providing

some background information. For each SD/LEP student who was included in the assessment, school

staff also completed an SD/LEP student questionnaire.

As stated previously, for the national main NAEP samples, the procedures for assessing students

with disabilities (SD) and students of limited English proficiency (LEP) varied by sample type. In sample

type 2 schools (for reading), new 1996 inclusion criteria were used. In sample type 3 schools (for

reading, writing, and civics), the new 1996 inclusion criteria were used and accommodations were

offered to SD/LEP students. The new inclusion criteria were developed to more closely match the

procedures used by many states and school districts in testing situations.

Step 4b: Sampling Teachers. Teachers of students assessed were identified and asked by the

NAEP supervisor to complete a questionnaire (described in Chapter 2) about their background and

instructional practices, by class, for any classes containing assessed students. If the questionnaire was not

collected at the time of the assessment, teachers were asked to return the questionnaire in a postage-paid

envelope.

Step 4c: The School Characteristics and Policies Questionnaires. Before the assessment, Westat

mailed a School Characteristics and Policies Questionnaire to every sampled school for completion by

the principal or school administrator. The Westat supervisor then collected the questionnaires and

returned them to ETS. The school characteristics and policies questionnaire is described in Chapter 2.
20

1.5 ASSESSMENT INSTRUMENTS

Four types of instruments were used in the 1998 assessment:

• Student assessment booklets, containing cognitive items and background

questions (demographic and subject-specific)

• Teacher questionnaires

• School characteristics and policies questionnaires

• SD/LEP questionnaires

For some assessments, NAEP uses a type of matrix sampling called focused balanced

incomplete block (BIB) spiraling to assign blocks or groups of cognitive items to student booklets and

to specific students. For other assessments, NAEP uses focused partially balanced incomplete block

(PBIB) spiraling for the assignment of items to booklets and students. Because of BIB and PBIB

spiraling, NAEP can sample enough students to obtain precise results for each question while

generally consuming an average of about an hour and a half of each student’s time.

The "focused" part of NAEP’s matrix sampling method requires that each student answer

questions from only one subject area. The "BIB" or “PBIB” part of the method ensures that students

receive different interlocking sections of the assessment forms, enabling NAEP to check for any

unusual interactions that may occur between different samples of students and different sets of

assessment questions. "Spiraling" refers to the method by which test booklets are assigned to pupils,

which ensures that any group of students will be assessed using approximately equal numbers of the

different versions of the booklet.

In a BIB design, the cognitive blocks are balanced. Each cognitive block appears an equal

number of times in every possible position. Each cognitive block is also paired with every other

cognitive block in at least one test booklet. (The NAEP BIB design varies according to subject area.)

Table 1-3 presents a simplified example of a BIB design. The full sample of students is

divided into seven equivalent groups, and each group of students is assigned one of the seven test

booklets. In this design, each cognitive block appears only once in each of the three possible positions,

and each block is paired once with every other block. (This example shows only the cognitive blocks,

even though the test booklets also contain background blocks.) The booklets are spiraled in each

packet of booklets, so students in each assessment session received each of the seven booklets.

Table 1-3

An Example of a BIB Design

Booklet

Version

Position 1

Cognitive Block

Position 2

Cognitive Block

Position 3

Cognitive Block

1A B D

2B C E

3C D F

4D E G

5E F A

6F G B

7G A C
21

In a PBIB design, one of the characteristics of a BIB design is not present. Table 1-4 presents

a simplified example of a PBIB design, similar to the NAEP national and state reading assessment

PBIB design. In this case, every block appears in the first and in the second position twice. All blocks

containing items from a content area are paired with every other block with items from that content

area, but is paired with only one block with items from the other content area. In this example, blocks

A, B, C, and D contain items from Content Area 1, and blocks E, F, G, and H contain items from

Content Area 2. The first six booklet versions pair Content Area 1 blocks, and the second six booklet

versions pair Content Area 2 blocks. In the final four booklet versions, every block is paired with a

block of items from the other content area.

For information on the design of specific assessment instruments, see Chapters 2, 14, 18,

and 22.

Table 1-4

An Example of a PBIB Design

Booklet

Version

Position 1

Cognitive Block

Position 2

Cognitive Block

1A C

2B A

3C D

4D B

5A D

6B C

7H E

8E F

9F G

10 G H

11 G E

12 H F

13 C G

14 D H

15 E B

16 F A

1.6 FIELD OPERATIONS AND DATA COLLECTION

Field operations and data collection for the 1998 assessment were the responsibility of Westat,

and are documented in Chapter 5 and in Westat’s Sampling Activities and Field Operations for 1998

NAEP (Gray, et al., 2000). The field operation was conducted by a staff at Westat’s home office and a

larger staff in the field. The Westat home-office staff coordinated all activities related to field operations

and managed materials distribution and home-office receipt of assessment reporting forms. The field staff

consisted of area supervisors, assessment supervisors, and exercise administrators. The assessment

supervisors, who were trained by Westat, were each responsible for the assessment activities in one or

more PSUs. Although ETS made initial contact with participating school districts, each assessment

supervisor was primarily responsible for making follow-up contacts with these districts, recruiting and

training exercise administrators to work with them in administering the assessment sessions, arranging

the assessment sessions, and selecting the sample of students to be assessed within each school. The

assessment supervisors and the exercise administrators administered the assessments, filled out the
22

necessary forms, performed process control, and shipped the assessment booklets and forms to National

Computer Systems (NCS), the subcontractor responsible for processing NAEP materials and data.

Gaining school cooperation was the joint responsibility of Westat and ETS. ETS made the

preliminary contacts preparatory to obtaining school cooperation by first contacting the Chief State

School Officers, informing them that schools within their states had been selected for the assessment, and

in a later letter, listing the selected schools and districts. Later mailings were sent to superintendents of

public schools and parochial schools and principals of other nonpublic schools for all schools selected in

the assessment. These materials provided an explanation of NAEP, a list of the selected schools in the

official’s jurisdiction, and a cover letter explaining that a Westat district supervisor would contact them

to set up an introductory meeting. Westat district supervisors then scheduled and conducted introductory

meetings (both by telephone and in person), worked with the schools to schedule the assessments, and,

with the exercise administrators, conducted the assessments. The unweighted school response rate for the

national main assessments in 1998 was 86 percent overall. The final sample of cooperating schools

included 733 schools at grade 4; 761 schools at grade 8; and 608 schools at grade 12. Further detail on

school participation rates is given in Chapters 3 (national) and 4 (state). An automated management

system tracked and recorded the progress of field work throughout the 1998 assessment period. In

addition, progress was constantly monitored through telephone reports held between the area supervisors

and the assessment supervisors and between the area supervisors and the home-office staff.

Both Westat and ETS participated in the quality control of the field administration, which

involved on-site visits by Westat and ETS staff to verify the sampling of the students and to observe the

conduct of the assessment by the supervisors and the exercise administrators.

1.7 MATERIALS AND DATA PROCESSING

After completing an assessment session, Westat field supervisors and exercise administrators

shipped the assessment booklets and forms from the field to NCS for entry into computer files,

professional scoring, and creating the data files for transmittal to ETS. Careful checking assured that all

data from the field were received. More than 500,000 booklets and questionnaires were received and

processed for the national portion of the 1998 assessment. The extensive processing of these data is

detailed in Chapter 6.

The student data were transcribed into machine-readable form by scanning the student

instruments with an optical scanning machine. An intelligent data-entry system was used for resolution of

the scanned data, the entry of documents rejected by the scanning machine, and the entry of information

from the questionnaires. Additionally, each piece of input data was checked to verify that it was of an

acceptable type, that it was within a specified range or ranges of values, and that it was consistent with

other data values. The entry and editing of materials is discussed in Chapter 6.

1.8 PROFESSIONAL SCORING

Items requiring a written response from the student (constructed-response items) were included

in the national and state assessments in reading and writing and in the national assessment in civics.

More than four million constructed responses were read and marked by the professional scoring staff for

the national and state portions of the 1998 assessment. Image processing and scoring were again used in

1998. Images of students’ responses to the constructed-response items were scanned into computerized

form, then scored online by professional raters.
23

Chapter 7 describes the professional scoring operation, including an overview of the scoring

guides, the training procedures, and the scoring process for each subject area.

1.9 CREATION OF THE DATABASE

Before analyses could begin, the student response data, school, teacher, and SD/LEP student

questionnaire data, and all sampling weights had to be integrated into a coherent and comprehensive

database. This database, which was used for all analyses, was also the source for the creation of two

NAEP database products—the item information database and the secondary-use data files. Secondary-use

data files include sample control statement files for SAS and SPSS statistical software and the NAEP Data

on Disk product suite. The Data on Disk products, including a complete set of secondary-use data files on

CD-ROM, PC-based NAEP data extraction software, and NAEP analysis modules, make secondary use

of NAEP data much easier than it has been in the past. The quality of the data resulting from the

complete data entry system, from the actual instruments collected in the field to the final machinereadable database used in analysis, was verified by selecting field instruments at random and performing

a character-by-character comparison of these instruments with their representations in the final database.

Chapter 8 provides details on the database, quality control activities, and database products.

25

Chapter 2

DEVELOPING THE NAEP OBJECTIVES, ITEMS,

AND BACKGROUND QUESTIONS FOR THE

1998 ASSESSMENTS OF READING, WRITING, AND CIVICS1

Terry L. Schoeps

Educational Testing Service

2.1 INTRODUCTION

In 1998, national main NAEP assessments were conducted in reading, writing, and civics.

Additional data were gathered under the auspices of the state assessment programs in reading and

writing. The state assessment in reading assessed representative samples of public- and nonpublic-school

students from 43 jurisdictions at grades 4 and 8; the state assessment in writing assessed representative

samples of public- and nonpublic-school students from 39 jurisdictions at grade 8 only.

From its inception, NAEP has developed assessments through a consensus process, and the 1998

instruments were no exception. Under the direction of the National Assessment Governing Board

(NAGB), educators, scholars, and citizens representative of many diverse constituencies and points of

view designed assessment frameworks for the writing and civics subject areas. The NAEP reading

framework used in the 1992 and 1994 assessments served as the framework for the 1998 reading

assessment. Copies of the frameworks for these assessments are available on the National Assessment

Governing Board (NAGB) web site at http://www.nagb.org. Staff at Educational Testing Service (ETS)

who are subject-area experts in their respective fields worked with subject-area consultants well versed in

assessment methodology to develop assessment questions appropriate to the objectives. All questions

underwent extensive reviews by subject-matter specialists and measurement specialists, both within and

outside ETS. All questions were also reviewed for bias by staff specially trained in ETS’s fairness review

process. Questions were assembled and printed into booklets suitable for matrix sampling and then

administered either by a trained field staff (for the national program) or by state or local school district

staff (for the state assessment program) to stratified, multistage probability samples of students.

All 1998 assessment development efforts were governed by four major criteria:

1. Each assessment was required to match the content definitions included in the assessment

frameworks, which had been developed through consensus processes conducted under the

auspices of the NAGB.

2. As outlined in the ETS proposal for the administration of the NAEP cooperative agreement

(ETS, 1992), the development of items was guided by an instrument development committee

for each subject area.2

3. As described in the ETS Standards of Quality and Fairness (ETS, 1987), all materials

developed at ETS were in compliance with specified procedures. In particular, all questions

were carefully reviewed for content accuracy, testworthiness, and potential bias.

 

1 Terry L. Schoeps coordinates the production of NAEP technical reports at Educational Testing Service.

2 A list of the consultants who comprised the 1998 instrument development committees is included in Appendix K.
26

4. As per federal regulations, all NAEP cognitive and background items were submitted to a

federal clearance process. This process involved review of all cognitive items by the

National Center for Education Statistics (NCES) and NAGB, and review of all background

questions by the Office of Management and Budget (OMB), the Information Management

Team (IMT) of the Department of Education, and NCES.

The following sections provide an overview of the process of setting objectives and developing

items, as well as specific details about the development of subject-specific objectives and assessments.

2.2 OVERVIEW OF THE 1998 ASSESSMENT OBJECTIVES AND FRAMEWORKS

The subject-area objectives for each NAEP assessment are determined through a legislatively

mandated consensus process. Once objectives are established, frameworks (matrices) are created,

delineating the important content and process areas to be assessed. In addition to these broad

frameworks, the Council of Chief State School Officers (CCSSO) and NAGB provide detailed

descriptions of item types and the numbers of items to be selected for each category. The frameworks for

the 1998 assessments are described below and in Chapters 14 (reading), 18 (writing), and 22 (civics).

The frameworks for the national main 1998 NAEP assessments were developed through

consensus processes and were conducted by the CCSSO in reading and civics, and by the Center for

Evaluation on Research Standards and Student Testing (CRESST) in writing, working under contract to

NAGB. The process involved participation and review by many groups, including teachers, content-area

scholars, educational policy makers, and members of the general public. In addition to people directly

involved in the framework development processes, the documents were reviewed by state education and

testing officials, by representatives of professional associations, and by researchers. In addition, the

frameworks were the subject of testimony at public hearings arranged to allow the widest possible

participation in the consensus process. The objectives resulting from these processes reflect neither a

narrowly defined theoretical framework nor the view of every participant. They do, however, represent

the thinking of a broad cross section of individuals who are deeply committed to improving American

education.

The framework that governed the 1998 NAEP reading assessment was used for the 1992 and

1994 assessments. The NAEP reading assessment was developed in accordance with the Reading

Framework for the National Assessment of Educational Progress, 1992–1998 (NAGB, 1990), making

this the third assessment cycle using this framework. The reading assessment was designed around

questions requiring in-depth analysis of authentic reading materials. A mixture of multiple-choice, short

constructed-response, and extended constructed-response questions made up the assessment. In

aggregate, well over half of the student assessment time was spent answering constructed-response rather

than multiple-choice questions.

The reading framework is organized according to four reading processes that characterize the

ways in which readers gain meaning from text:

• Initial understanding

• Developing an interpretation

• Personal response

• Critical stance
27

In addition, the assessment was designed to measure the three global reading purposes:

• Reading for literary experience

• Reading to gain information

• Reading to perform a task

The assessment measured students’ ability to read based on a variety of passages, including

informational materials, documents, news articles, essays, and stories. Each student in the assessment

was asked to complete either two 25-minute sets (at all three grades) or one 50-minute set (at grades 8

and 12) of reading passages and comprehension questions. A combination of multiple-choice and

constructed-response questions is used to assess students’ understanding of the assessment passages.

The 1998 writing assessment is structured in accordance with the Writing Framework and

Specifications for the 1998 National Assessment of Educational Progress (NAGB, 1996b), the

assessment measured three kinds of writing:

• Informative

• Narrative

• Persuasive

Because the 1998 writing assessment was based on a new framework, it represents the beginning

of a new trend line. Participants responded either to two 25-minute passages or (for some students at

grades 8 and 12) to one 50-minute passage. The writing assessment also contained a special study of

classroom writing. In that study, 100 teachers at grade 4 and 100 teachers at grade 8 were interviewed

about how they teach writing. In addition, for one of their classes, every student was asked to choose and

submit the two best pieces of writing he or she had written for that class. Results of this study will

published in a separate report. Unlike the reading assessment, the writing and civics assessments are

reported along a single within-grade scale.

The framework for the 1998 civics assessment, titled Civics Framework and Specifications for

the 1998 National Assessment of Educational Progress (NAGB, 1996a), is strongly related to the

National Standards for Civics and Government developed by the Center for Civic Education (1994).

Because the 1998 civics assessment was based on a new framework, it represents the beginning of a new

trend line. A combination of multiple-choice, short constructed-response, and extended constructedresponse questions made up the assessment. In addition to the national civics assessment, a special civics

trend study was conducted, in which students were administered instruments from the 1988 NAEP civics

assessment.

According to the framework, the civics assessment was designed to measure three interrelated

components of civics proficiency: knowledge, intellectual and participatory skills, and civic dispositions.

The knowledge component of the framework was divided into five content areas:

• Civic life, politics, and government

• The foundations of the American political system

• The Constitution and American government

• The United States and world affairs

• The roles of United States citizens
28

The framework also divided intellectual skills into three types, ranging roughly from simpler to

higher order thinking skills:

• Identifying and describing

• Explaining and analyzing

• Evaluating, taking, and defending positions

The framework recommended that a special study in civics trend be conducted, in which a

subsample of students participating in the national civics assessment would be administered an intact

portion of the assessment instruments used in the 1988 civics assessment. Results for the portions

administered could then be compared to results of corresponding portions from the 1998 assessment.

2.3 GENERAL OVERVIEW OF PROCEDURES FOR DEVELOPING

COGNITIVE ITEMS

A carefully developed and tested series of steps, similar to those used for past NAEP

assessments, was utilized to create assessment items that reflected reading, writing, and civics objectives

and measured achievement related to them (see Chapters 14, 18, and 22 for information on assessment

instruments for reading, writing, and civics, respectively). The item-development steps for each subject

area were as follows:

1. NAGB provided content frameworks and item specifications in each subject area.

2. Instrument development committees in each subject area provided guidance to

NAEP staff about how the objectives could be measured given the realistic

constraints of resources and the feasibility of measurement technology. The

committees made recommendations about priorities for the assessment (within the

context of the assessment framework) and the types of items to be developed.

3. Items were chosen for the assessment through an extensive selection process that

involved the input of practitioners from across the country as well as from members

of the instrument development committees.

4. Specialists with subject-matter expertise, skills, and experience in creating items

according to specifications were identified from inside and outside ETS to develop

and review the assessment questions.

5. The items and accompanying scoring guides were reviewed and revised by

NAEP/ETS staff and external test specialists.

6. Representatives from the state education agencies met and reviewed all items and

background questionnaires that were scheduled to be part of the state assessment.

7. Editorial and fairness reviews were conducted as required by the ETS Standards for

Quality and Fairness (ETS, 1987).

8. Field test materials were prepared, including those necessary to secure clearance by

the Office of Management and Budget.
29

9. A field test was conducted in many states, the District of Columbia, and Virgin

Islands.

10. Representatives from state education agencies met and reviewed the field test results

for all exercises selected for the state assessment.

11. Based on the field test analyses, new items for the 1998 assessment were revised or

modified where necessary. The items once again underwent the full range of ETS

reviews.

12. The instrument development committees approved the selection of items to include

in the 1998 assessment.

13. After a final review and check to ensure that each assessment booklet and each block

met the overall guidelines for the assessment, the booklets were typeset and printed.

Development of the reading, writing, and civics assessments are described in more detail in

Chapters 14, 18, and 22, respectively.

2.4 DEVELOPING BACKGROUND ITEMS

As part of the assessment, a series of questionnaires was administered to students, teacher, and

school administrators. Similar to the development of the cognitive items, the development of the policy

issues and questionnaire items was a consensual process that involved staff work, field testing, and

review by external advisory groups. A Background Questionnaire Panel drafted a set of policy issues and

made recommendations regarding the design of the items. They were particularly interested in

capitalizing on the unique properties of NAEP and not duplicating other surveys.

The Panel recommended a focused study that addressed the relationship between student

achievement and instructional practices. The issues, items, and field test results were reviewed by the

group of external consultants who identified specific items to be included in the final questionnaires. The

items underwent internal ETS review procedures to ensure fairness and quality and were then assembled

into questionnaires.

Detailed descriptions of the student and teacher questionnaires are given in Chapter 14 (reading),

Chapter 18 (writing), and Chapter 22 (civics). In addition to these, two additional questionnaires were

developed for use across subject areas.

• The School Characteristics and Policies Questionnaire was given to the

principal or other administrator of each school that participated in NAEP. This

questionnaire included questions about characteristics of the school, school

enrollment, absenteeism, drop-out rates, tracking policies, curriculum, testing

practices and use, special priorities and schoolwide programs, availability of

resources, special services, community services, policies for parental

involvement, and schoolwide problems.
30

• The SD/LEP Questionnaire was completed for each student who was selected to

participate in the assessment sample and was classified as a student with a

disability (SD), or was categorized as a limited English proficient (LEP) student.

This questionnaire, which was completed by someone at the school

knowledgeable about the student, asked about the student’s background and the

special programs in which the student participated. This questionnaire was

completed for each SD, LEP, or SD/LEP student in the sample, whether or not

that student included in the assessment.
31

Chapter 3

SAMPLE DESIGN FOR THE NATIONAL ASSESSMENT1

Keith F. Rust and Tom Krenzke

Westat

Jiahe Qian and Eugene G. Johnson

Educational Testing Service

3.1 INTRODUCTION

This chapter details sampling activities of the 1998 National Assessment of Educational Progress

(NAEP). This introduction gives an overview of the sample design and selection activities and provides

some highlights of the current design for the national assessments. Section 3.2 presents detailed

documentation of the 1998 sampling of primary sampling units (PSUs) and of schools within PSUs.

Section 3.3 discusses the allocation of sessions to schools and the assignment of sample types to schools,

and Section 3.4 discusses student sampling within schools. Additional details on the sampling design and

process can be found in Westat’s Sampling Activities and Field Operations for 1998 NAEP (Gray, et al.,

2000).

3.1.1 Brief Overview of the Sample Design and Sampling Activities

The sample for the 1998 national assessment was a multistage probability sample. Counties or

groups of counties were the first-stage sampling units, and elementary and secondary schools were the

second-stage units. The third stage of sampling involved the assignment of sessions by type and of

sample types to sampled schools. The fourth stage involved selection of students within schools and their

assignment to session types.

A total of 94 primary sampling units (PSUs) were included in the national sample; a sample of

733 schools actually participated in the assessment at the fourth grade, 761 schools at the eighth grade,

and 608 schools at the twelfth grade. Various blocks or packages of exercises were administered in these

schools to 36,104 fourth-graders, 48,797 eighth-graders, and 48,588 twelfth-graders, for a total of

133,489 assessed students. Sometimes schools selected for the sample could not participate in the NAEP

assessments (e.g., the schools had closed or no longer taught the appropriate grade level). The

participation rates of schools and students are discussed in Section 3.2.4. The use of partially balanced

incomplete block (PBIB) designs in the assessment booklets, and spiraling in the assembling of booklets

for the assessment is described in Chapter 1.

The weighting procedures for the 1998 NAEP included computing a student’s base weight (i.e.,

the reciprocal of the overall probability that the student was invited to a particular type of session) and

adjusting this base weight for nonresponse. The weights were further adjusted by a poststratification

procedure. Counts of students in various regions and ethnic subclasses were estimated for the 1997–98

school year by age and grade on the basis of information from the Current Population Survey and Census

Bureau tabulations of population distributions. The procedures of poststratifying weights are discussed in

 

1 Keith F. Rust was responsible for overseeing all sampling activities; Tom Krenzke carried out most of the national sampling

activities. Jiahe Qian, in consultation with Eugene G. Johnson, was responsible for the specification and coordination of the

national sampling at ETS.
32

Section 10.2.5. The weights were then adjusted so that the aggregate NAEP estimates would agree with

these estimated counts for each subclass. In all NAEP assessments, including 1998, weights were not

poststratified to the Common Core of Data (CCD) for the following reasons:

• CCD contains only public schools.

• CCD data is not as current as census data.

• CCD collects data at the school level.

• CCD, at that time, did not collect data by grade and race.

• CCD, like other publicly available lists of schools, contains ineligible schools that

were thought to be eligible at the time the CCD was produced.

The CPS estimates and census projections provide independent data sources (i.e., independent

from the source of the NAEP sampling frame), which is commonly used for poststratification in national

surveys.

Variances for NAEP are computed by the jackknife procedure. Westat computed estimates of

summary measures for the samples and their sampling errors in the process of reviewing weights and

weight adjustments. The principal estimates and their variances were computed at ETS.

3.1.2 Target Population and Sample Size

The target population for the 1998 assessment consisted of fourth-grade, eighth-grade, and

twelfth- grade students enrolled in public and nonpublic elementary and secondary schools. Table 3-1

shows the target number of students to be assessed in each grade. The targets were intended to yield

approximately 2,000 completed assessment booklets containing each block of items in the PBIB

assessments for each grade. To allow for the derivation of reliable estimates for nonpublic-school

students, the selection probabilities for nonpublic schools were larger than those of similarly sized public

schools not designated high-minority (see Section 3.2.4.2).
33

Table 3-1

1998 NAEP National Samples and Target Sample Sizes

Subject Target Sample Size

Total 132,000

Grade 4 Civics 6,000

Civics Special Trend 2,000

Reading 8,000

25-Minute Writing 20,000

Grade 4 Total 36,000

Grade 8 Civics 8,000

Civics Special Trend 2,000

Reading 11,000

25-Minute Writing 20,000

50-Minute Writing 6,000

Grade 8 Total 47,000

Grade 12 Civics 8,000

Civics Special Trend 2,000

Reading 13,000

25-Minute Writing 20,000

50-Minute Writing 6,000

Grade 12 Total 49,000

3.1.3 Highlights of Design Changes for the 1998 Assessment

The general sampling design plan for the 1998 assessment was similar in most respects to that of

1996. Four major changes were made:

• The long-term trend assessments of reading, writing, mathematics, and science

were not administered in 1998.

• The samples consisted of three distinct session types (writing/civics, civics

special trend, and reading) for each grade, four distinct subjects for grade 4, and

five distinct subjects for each of grade 8 and 12 (as shown in Table 3-1). Writing

and civics assessments were given in the same session.

• Two sample types (S2, S3) were assigned to subsamples by session in schools.

For S2 students, accommodations were not provided for SD/LEP students, while

for S3 students, accommodations were provided.

• While SD/LEP students were sampled at a higher rate than non-SD/LEP

students, just as in 1996, Black and Hispanic students were also sampled at a

higher rate within schools that were in low-minority geographic areas (see

Section 3.4.5.1).

To aid the reader, a glossary of terms and abbreviations used in this chapter is provided at the

end of the chapter.
34

 3.2 THE SAMPLE OF PRIMARY SAMPLING UNITS AND SCHOOLS

The samples for the 1998 NAEP assessment were selected using a complex multistage sample

design involving the sampling of students from selected schools within 94 selected geographic areas,

called primary sampling units (PSUs), across the United States. The samples were designed to represent

fourth-, eighth-, and twelfth-grade students enrolled in public and nonpublic elementary and secondary

schools. The sample design had four steps in the selection process:

1. Selection of geographic PSUs (counties or groups of counties)

2. Selection of schools within PSUs

3. Assignment of session types and sample types to schools

4. Selection of students for session types within schools

Steps 1 and 2 are documented in this section. Step 3 is discussed in Section 3.3. Step 4 is

discussed in Section 3.4. For area sampling technique, see Kish (1965).

3.2.1 The Definition of Primary Sampling Units

The basic PSU sample design for 1994 NAEP to 2002 NAEP is a stratified probability sample

with one PSU selected per stratum (for each round), with probability proportional to population. A PSU

consists of a consolidated metropolitan statistical area (CMSA), a metropolitan statistical area (MSA), a

New England County metropolitan area (NECMA), a county, or group of contiguous counties in the U.S.

(including Alaska, Hawaii, and the District of Columbia). A total of 94 PSUs per round were selected.

The PSU sampling frame for 1994 NAEP to 2002 NAEP was constructed by grouping counties

following specific rules as follows:

• Each 1990 CMSA, and each MSA that was not part of a CMSA, was considered a separate

PSU. In New England, NECMAs were the metropolitan PSU unit.

• Non-MSA PSUs were made to consist only of non-MSA counties. Whenever possible, each

non-MSA PSU contained geographically contiguous counties with a minimum 1990 total

population of 60,000 persons in the Northeastern and Southeastern regions, and 45,000

persons in the Central and Western regions. The criteria of minimum population for a nonMSA PSU were determined by survey design to achieve similar numbers of PSUs across the

regions.

• Region boundaries were not crossed in the definition of a PSU, not even in the case of

MSAs. If a county in an MSA was in a separate region, it was taken out of the MSA and

grouped with other contiguous counties in its region to define a PSU.

Checks were made to ensure that every county was included in one and only one PSU. The frame

contained 1,027 PSUs: 290 MSAs and 737 non-MSAs.
35

3.2.2 Definition of PSU Strata

Eight major strata were formed by crossing region and MSA status. The PSUs were classified

into four regions, each containing about one-fourth of the U.S. population. These regions were defined

primarily by state (Table 3-2).

Table 3-2

Definition of NAEP Stratification and Reporting Regions

Northeast Southeast Central West

Connecticut Alabama Illinois Alaska

Delaware Arkansas Indiana Arizona

District of Columbia Florida Iowa California

Maine Georgia Kansas Colorado

Maryland Kentucky Michigan Hawaii

Massachusetts Louisiana Minnesota Idaho

New Hampshire Mississippi Missouri Montana

New Jersey North Carolina Nebraska Nevada

New York South Carolina North Dakota New Mexico

Pennsylvania Tennessee Ohio Oklahoma

Rhode Island Virginia* South Dakota Oregon

Vermont West Virginia Wisconsin Texas

Virginia* Utah

Washington

Wyoming

*Those counties and independent cities in Virginia that are part of the Washington, DC, MD-VA

metropolitan statistical area are included in the Northeast region. The remainder of Virginia is included

in the Southeast region.

The 22 largest PSUs were included with certainty because of their large sizes. The inclusion of

these PSUs in the sample with certainty provided an approximately optimal and cost-efficient sample of

schools and students when samples were drawn within them at the required national sampling rate. The

22 largest PSUs by region are presented in Table 3-3.

The remaining smaller PSUs were not guaranteed to be selected for the sample. These were

grouped into a number of noncertainty strata (PSUs in these strata were not included in the sample with

certainty), and one PSU was selected from each stratum. In each region, noncertainty PSUs were

classified as MSA (metropolitan) or non-MSA (nonmetropolitan).
36

Table 3-3

The 22 Largest Primary Sampling Units, by Region, 1998 NAEP

Region Primary Sampling Unit

Northeast

Baltimore, MD MSA

Boston-Lawrence-Salem-Lowell-Brockton, MA NECMA

New York-Northern New Jersey-Long Island, NY-NJ CMSA (excluding that part in CT)

Philadelphia-Wilmington-Trenton, PA-DE-NJ-MD CMSA

Pittsburgh-Beaver Valley, PA CMSA

Washington, DC-MD-VA MSA

Southeast

Atlanta, GA MSA

Miami-Fort Lauderdale, FL CMSA

Tampa-St. Petersburg-Clearwater, FL MSA

Central

Chicago-Gary-Lake County, IL-IN-WI CMSA

Cleveland-Akron, OH CMSA

Detroit-Ann Arbor, MI CMSA

Minneapolis-St. Paul, MN-WI MSA

St. Louis, MO-IL MSA

West

Dallas-Fort Worth, TX CMSA

Denver-Boulder, CO CMSA

Houston-Galveston-Brazoria, TX CMSA

Los Angeles-Anaheim-Riverside, CA CMSA

Phoenix, AZ MSA

San Diego, CA MSA

San Francisco-Oakland-San Jose, CA CMSA

Seattle-Tacoma, WA CMSA

Within each major stratum, further stratification was achieved by ordering the noncertainty PSUs

according to several additional socioeconomic characteristics, yielding 72 strata. The number of such

strata formed within each major stratum is shown in Table 3-4. The strata were defined so that the

aggregate of the measures of size of the PSUs in a stratum was approximately equal for each stratum. The

size measure used was the population from the 1990 Census. The characteristics available for all PSUs,

that were used to define strata were the percent minority population, the percentage change in total

population since 1980, the per capita income, the percent of persons age 25 or over with college degrees,

the percent of persons age 25 or over who have completed high school, and the civilian unemployment

rate. Up to four of these characteristics were used in any one major stratum. For each major stratum the

characteristics used were chosen by modeling NAEP PSU-level mean reading scale scores for years 17,

19, and 21 (1988, 1990, and 1992). The characteristics chosen were the best predictors of PSU-level

mean reading scale scores in these models.
37

Table 3-4

The Number of Noncertainty

Strata in Each Major Stratum 1998 NAEP

Region

Number of Strata for

MSA PSUs

Number of Strata for

Non-MSA PSUs Total

Northeast

Southeast

Central

West

6

12

8

10

4

12

12

8

10

24

20

18

Total 36 36 72

3.2.3 Selection of Noncertainty PSUs

In the first stage of sampling, a sample of PSUs was drawn for the national NAEP samples for

each of the survey years 1994, 1996, 1998, 2000, and 2002. For each survey year, 94 PSUs were selected.

Of the 94 selected PSUs, 22 were included with certainty because they had the largest populations in the

PSU universe. These 22 certainty PSUs were used in the sample for each of the survey years. The rest of

the PSUs in each survey, 72 in total, were selected with a probability proportional to their respective

population size. To select noncertainty PSUs, the remaining PSUs on the sampling frame were further

stratified into 72 noncertainty strata.

Within each of the noncertainty strata, one PSU was selected with probability proportionate to its

1990 population size for each survey year. That is, within each stratum, a PSU’s probability of being

selected was proportional to its population size. The PSUs were selected with probability proportional to

size (PPS) with the twin aims of obtaining approximately self-weighting samples of students and having

approximately equal workloads in each PSU. PSUs were drawn to minimize overlap of the PSUs from

one assessment to the next, except that certainty PSUs were retained in each assessment year, and some

of the larger noncertainty PSUs are in the sample for more than one of these assessment years. Each

sample of 94 PSUs was drawn from a population of about 1,000 PSUs. Primarily because of the use of

MSAs as PSUs, PSUs varied considerably as to their probability of selection, since they varied greatly in

size. In 1998, the 36 selected MSA PSUs had probabilities of selection ranging from 0.03 to 0.56, while

the 36 selected non-MSA PSUs had probabilities ranging from 0.03 to 0.10. Parts of 44 states were

included in the sample PSUs. Since one PSU was selected from each noncertainty stratum, the

distribution of the noncertainty PSUs is the same as the noncertainty strata, as shown in Table 3-4.

Within each stratum the order of the PSUs was randomized. As detailed later in the section, the

selection of PSUs within a stratum was not independent among the survey years. Ordering the PSUs

within a stratum by size, geography, or other variables could have resulted in unintended and possibly

detrimental correlations between survey estimates across years. Since only one PSU is selected for a

given year, the PSU ordering has no effect on sampling variance.

For each PSU within a stratum a normalized measure of size was calculated by dividing the

PSU’s 1990 population by the aggregate 1990 population of all PSUs in the stratum. Next, a cumulative

count of normalized measures of size was calculated for each PSU within a stratum. The cumulative

count for the k

th PSU in the ith stratum, denoted Cik , was equal to 

1

NM

k

j ij =

∑ where NM ij represents the

normalized measure for the j

th PSU in the ith stratum.
38

For each stratum a random number between 0 and 1 was generated. Using this random number,

denoted r, the following sequence of sample designation numbers was generated for the five survey

years:

Survey Year 1994 1996 1998 2000 2002

Sample Designation Number r r + 0.4 r + 0.8 r + 0.2 r + 0.6

Only the noninteger part of any number in the sequence that exceeded 1.0000 was retained. For

example, if r was equal to 0.326743, then r + 0.8 was equal to 1.126743 and 0.126743 became the sample

designation number for 1998.

The first PSU in the stratum whose cumulative count was equal to or greater than r was

designated the 1994 sample PSU. Similarly, the first PSUs in the stratum whose cumulative counts were

equal to or greater than the noninteger portions of r + 0.4, r + 0.8, r + 0.2, and r + 0.6 were designated

the 1996, 1998, 2000, and 2002 sample PSUs, respectively.

The purpose of having the sample designation numbers for 1996, 1998, 2000, and 2002 be

functions of r was to attempt to minimize the overlap among the sets of sample PSUs chosen for the

various survey years. In strata with smaller numbers of PSUs, some PSUs had large enough normalized

measures of size so that they were drawn for two and sometimes even three survey years. As the spacing

between the sample designation numbers for any two consecutive survey years was at least 0.4, selecting

the same PSU in two consecutive survey years was rare.

3.2.4 School Sample

3.2.4.1 Frame Construction

The second-stage sampling is to select a sample of schools within each selected PSU. A list of

schools was formed within each PSU, using a number of sources. The public schools (including Bureau

of Indian Affairs [BIA] schools and Department of Defense Education Activity [DoDEA] schools) and

nonpublic schools (including Catholic schools) were listed according to the three grades. The lists of

schools were obtained from two sources. A list of public, BIA, and DoDEA schools, which is maintained

by Quality Education Data, Incorporated (QED) and included information from the 1994–95 NCES

Common Core of Data (CCD), was obtained in March of 1997. Regular public schools are schools with

students who are classified as being in a specific grade (as opposed to schools having only “ungraded”

classrooms). This includes statewide magnet schools and charter schools. Catholic and other nonpublic

schools were obtained from the 1995-96 Private School Survey (PSS) developed for the National 

Center for Education Statistics. The PSS list of schools is an on-going registry of private schools. 

The registry is updated prior to the survey through two sources. The first source, called the list 

frame, is a conglomeration of a number of lists from several associations, states, etc. Although the list

frame attempts to have complete coverage of the private school universe, it needs to be supplemented

with a second source. The second source uses an area frame to identify and represent schools not 

on the list frame. The area samples are conducted first by randomly selecting primary sampling units

(PSUs), which are single counties or groups of counties from the area frame, which consists of all

counties in the nation. Within each selected PSU, a complete list of schools is gathered from a variety of

means, and schools not on the list frame are identified and are added to the list frame of nonpublic

schools. The majority of the PSS list comes from complete enumeration of schools, a list of schools

obtained from different resources. But a small portion of the PSS list was obtained from a sample of

counties selected for the PSS. For details of PSS area sampling frame, see the Private School Universe
39

Survey, 1995-1996 (Broughman & Colaciello, 1998). The probabilities of selection for schools to be on

the PSS list ranged from 0.06 to 1.00. A weight component was computed, as discussed in Chapter 10, so

that these selected PSS nonpublic schools represent themselves, as well as the non-PSS nonpublic

schools for non-PSS PSUs.

The ID variable NCESSCH is contained in the CCD file and is echoed by the QED file. This is

the unique NCES-assigned school number. The variable NCESSCH is filled in for new schools that were

added to the NAEP samples. It can be used to merge NAEP data back with CCD files. The schools that

do not match will probably be the additional schools, and nonpublic schools.

Table 3-5 shows the numbers of schools included in the various sampling frame components.

The population of eligible schools for each grade was restricted to the selected 94 PSUs. Any school

having one or more of the eligible grades, and located within an appropriate PSU, was included in the

sampling frame of schools (the list of schools from which the samples of schools were drawn) for a given

sample. An independent sample of schools was selected for each of the grades.

Table 3-5

Number of Schools Eligible in QED and PSS Sampling Frame

Components by Grade, 1998 Main NAEP

Sample

QED

Public*

QED

Nonpublic†

PSS

Nonpublic

Grade 4 19,962 20 11,428

Grade 8 7,382 11 10,169

Grade 12 4,513 8 4,845

*

 Public schools, including state-run schools; does not include

DoDEA, BIA schools.

†

 DoDEA, BIA, Catholic, and other nonpublic schools

For each school in each frame, estimates were made of the number of students who were eligible

by grade. The QED and PSS files give total enrollment, enrollment by grade, and the grade range for

each school, thus providing the average enrollment per grade.

A school would appear in the frame for a particular grade without regard to its eligibility status

for either of the two other designated grades. As a result, there is considerable overlap among the three

frames.

Before selecting schools, high-minority public schools were identified for oversampling. If the

percentage of Hispanic and Black students was not reported or if it was less than 10%, the school was

classified as not high-minority; otherwise, the school was classified as high-minority if the percentage of

Hispanic and Black students was greater than 10% (15% for grade 12) and if the number of Hispanic and

Black students was at least 10 (15 for grade 12).

3.2.4.2 Assigning Size Measures and Selecting School Samples

For each grade-level sample, schools were selected (without replacement) across all PSUs

systematically from a sorted list, with probabilities proportional to assigned measures of size. The sorting

variables included NAEP region, private/public classification, type of location, high/low minority

classification, PSU stratum, and estimated grade enrollment. The order of the sort differed depending on
40

public and private school classification and certainty/noncertainty PSU classification. To increase costefficiency in sampling, samples were designed to include more nonpublic schools and high-minority

public schools, and more relatively large schools. Therefore, a measure of size was assigned to each

school according to the following scheme.

Let Si denote the estimated number of grade-eligible students in school i. Let L = 100 for the

assessment of grade 4, L = 125 for the assessment of grade 8, and L = 150 for the assessment of grade 12.

The measure of size was:

.25 ki, if Si was less than 6;

ki Si / 20, if Si was greater than 5 but less than 20;

ki, if Si was greater than 19 but less than 101 (grade 4) or 126

(grade 8) or 151 (grade 12); and

ki Si / L, if Si was greater than L;

where

ki = 3, for nonpublic schools (other than BIA and DoDEA schools);

= 2, for high-minority public schools, and;

= 1, for low-minority public schools.

This procedure was used so as to obtain approximately self-weighting samples of students (i.e.,

students selected with approximately equal overall probabilities) within the oversampling domains at

each grade. Three variations to the overall goal of self-weighting samples were implemented. First,

schools with fewer than 20 estimated grade-eligible students were assigned somewhat lower measures of

size, and thus lower probabilities of selection. This was designed to increase cost efficiency.

Second, each public school designated as high-minority was given double the measure of size of

a public school of similar size not designated high-minority. Such high-minority schools were

oversampled in order to enlarge the sample of Black and Hispanic students, thereby enhancing the

reliability of estimates for these groups. For a given overall sample size, this procedure reduces

somewhat the reliability of estimates for all students as a whole and for those not Black or Hispanic.

Third, each nonpublic school was given triple the measure of size of a public school of similar size not

designated high-minority. These greater probabilities of selection were used to ensure adequate samples

of nonpublic-school students in order to allow the derivation of reliable estimates for such students.

The participation rates used to determine the school and student sample sizes are the

participation and eligibility rates achieved in 1996. They are shown in Table 3-6. In addition, we inflated

the resulting sample sizes by 1.05 to allow for the possibility of decreases in response rate, and for the

inaccuracy of the estimated enrollments.

Table 3-6

Participation Rates in 1996 National NAEP

Grade 4 Grade 8 Grade 12

School Participation Rate 0.86 0.83 0.79

School Eligibility Rate 0.93 0.95 0.96

Student Participation Rate 0.95 0.92 0.80

Overall Participation Rate 0.82 0.76 0.64
41

3.2.4.3 Updating the School Frame and Sample

The QED files do not contain schools that opened between 1996 and the assessment dates.

Therefore, special procedures were implemented to be sure that the NAEP assessment represented

students in new public schools. Small school districts, those that contained only one eligible school for a

given grade, were handled differently from large school districts, which contained more than one eligible

school for a given grade. In small school districts, the schools selected for a given grade were thought to

contain all students in the district who were eligible for the assessment. Districts containing these schools

were asked if other schools with the appropriate grades for the assessment existed, and if so, they were

automatically included in the assessment.

The procedure for obtaining lists of new schools in large districts was coordinated with a similar

procedure used for the 1998 state assessment. For large school districts a district-level frame was

constructed from the schools on the QED file. Then districts were sampled systematically with

probabilities proportional to a measure of size. In most cases, the measure of size was total district

enrollment, but in very small districts a minimum measure of size was used. New schools in small

districts were identified during school recruitment. Each sampled district was asked to update the list of

eligible schools based on information in the QED files. Frames of eligible new schools were then

constructed at each grade level, and samples of new schools were selected systematically with probability

proportional to eligible enrollment using the same sampling rates as for the QED schools. As a result of

this process, 10 new public schools were selected —four at grade 4, three at grade 8, and three at

grade 12.

The number of sampled schools by major stratum is presented in Table 3-7. The counts are

shown for each grade and include new schools selected in the new schools sampling process. It should be

noted that the variables that comprise the major strata (i.e. region, MSA status) were used implicitly as

sorting variables in the school sampling process. Additional counts by geographic and school

characteristics are shown in Table A-4 (for respondent schools).

Table 3-7

Number of Schools in the Original Samples by Major Stratum

Grade Region

MSA

Certainty

PSU

MSA

Noncertainty

PSU

Non-MSA

Noncertainty

PSU Total

4 Northeast 125 54 17 196

Southeast 27 105 61 193

Central 78 80 59 217

West 145 88 50 283

Total 375 327 187 889

8 Northeast 142 60 18 220

Southeast 29 110 70 209

Central 90 84 62 236

West 148 95 49 292

Total 409 349 199 957

12 Northeast 122 45 19 186

Southeast 29 101 79 209

Central 68 59 55 182

West 139 84 52 275

Total 358 289 205 852
42

3.2.4.4 School Substitution

Potential substitute schools were selected for all sampled schools in the 1998 national NAEP

where a close match could be identified by their attributes. An attempt was made to pre-select (before

field processes began) a maximum of two substitute schools for each sampled public school (one indistrict and one out-of-district) and each sampled Catholic school and one for each sampled non-Catholic

private school. A nonparticipating school was replaced by a substitute when the participating school for a

particular grade was considered a final refusal. To minimize bias, a substitute school resembled the

original selection as much as possible.

Substitutes were assigned by matching approximately on the following attributes:

• Affiliation

• Estimated number of grade-eligible students

• Minority composition

A substitute was always selected from the same PSU as the refusing school. When school nonparticipation was due to district refusal, none of the schools in the refusing district were considered

substitute candidates. However, when substituting for refusals due to a principal’s refusal, preference

was given to substitute candidates in the same district.

The net numbers of substitutes added to the sample by the above procedure are shown in

Table 3-8. The number of substitutes was substantially higher than in recent previous rounds of NAEP

because of the efficient preselection method of assigning substitutes. The identity of the substitute

schools was unknown to the field staff until after the corresponding original selection was designated as a

final refusal. This was to protect against any temptation to move on to an “easier” substitute school.

A retrofitting procedure, which used the same criteria as used for the initial substitution

procedure, was implemented midway through the data collection process. This method identified

nonresponding schools that needed substitutes and assigned to them unused substitute schools. Unused

substitute schools are those schools that were initially linked to cooperating original sampled schools.

The same matching rules that were used for assigning substitutes were used in the retrofitting procedure.

3.2.4.5 School Participation Experience

Overall, the 1998 before-substitution school participation rates were lower than school

participation rates encountered in previous years. However, the after-substitution participation rates were

higher than in previous years. Table 3-8 presents a detailed breakdown by participation status of all

schools contacted; 1992, 1994, and 1996 participation rates are also shown based on the same

computations.
43

Table 3-8

Summary of School Participation Experience for 1998 National NAEP, Unweighted

Grade 4 Grade 8 Grade 12 Total Public* Nonpublic†

Total Original Sample 889 957 852 2,698 1,581 1,117

Out-of-Range or Closed 54 79 103 236 29 207

No Eligibles Enrolled 7 7 4 18 0 18

State Tested All Students 1 0 0 1 1 0

District Refused 52 50 50 152 151 1

School Refused 104 118 135 357 162 195

Cooperating 671 703 560 1,934 1,238 696

Cooperation Rate Before Substitution‡ 81% 81% 75% 79% 80% 78%

(1996) 86% 83% 79% 83% 85% 80%

(1994) 86% 86% 79% 83% 82% 85%

(1992) 86% 85% 81% 84% 86% 82%

Cooperating Replacement for Refusals 62 58 48 168 109 59

Total Cooperating Schools 733 761 608 2,102 1,347 755

Cooperation Rate After Substitution 89% 87% 82% 86% 87% 85%

Total Students Assessed 36,104 48,797 48,588 133,489 110,825 22,664 

*

 Public schools including state-run schools; does not include DoDEA, BIA schools.

†

 DoDEA, BIA, Catholic, and other nonpublic schools.

‡

 The percentages shown on this row take into account situations in which a school was cooperative but was unable to participate at

a given grade, because no eligible students were enrolled in that grade at the time of assessment.

3.3 ASSIGNMENT OF SESSIONS AND SAMPLE TYPES TO SCHOOLS

The process of assigning sessions and sample types to schools differed by grade. For grade 4,

sessions and sample types were assigned in the same process, while for grades 8 and 12, sessions were

assigned first, then sample types. For simplicity, allocation of sessions will be explained first, followed

by an explanation of the assignment of sample types.

3.3.1 Description of Session Types

Three different session types were conducted at all grades: writing/civics, reading, and civics

special trend. The writing/civics session type contained two subjects for grade 4 (25-minute writing and

civics), and three subjects for grades 8 and 12 (25-minute writing, 50-minute writing, and civics). The

special civics trend and reading session types contained only one subject in each session type,

respectively.

In the 1998 reading assessment, sample types 2 and 3 were assigned to schools. The writing and

civics assessments were administered to sample type 3 schools only. More detailed information on

assigning sample type to schools is provided in Section 3.3.3.

3.3.2 Allocation of Sessions

The method of determining the number and type of sessions to be administered in a given

selected school varied slightly by grade. Sessions were randomly assigned to the selected schools found
44

to be appropriate at the time of session assignment. First, the number of sessions per school was

established. Four sessions per school were specified for grade 4, and five sessions per school were

specified for grades 8 and 12. This was the maximum number of sessions that could be administered

without creating unduly small session sizes with few eligible students. Schools with fewer than 25 (30 for

grade 12) eligible students were asked to conduct only a single session.

Sessions were assigned to schools with two aims in mind. The first was to distribute students to

the different session types across the whole sample for each grade so that the target numbers of assessed

students would be achieved in each sample type separately. The second was to maximize the number of

different session types that were administered within a given selected school, without violating the

minimum session sizes discussed above.

3.3.2.1 Grade 4 Allocation of Sessions

For grade 4, sessions were allocated to schools in the following way. First, each school was

allocated a number of sessions, based on the estimated number of grade-eligible students, as shown here:

Estimated Number of

Grade-Eligible Students

Number of

Sessions Allocated

1 – 25 1

26 – 50 2

51 – 75 3

76 or More 4

The sessions were allocated to schools by placing schools in the order used for sampling, and

allocating the appropriate number of sessions from the following repeated sequence (W denotes

writing/civics, R denotes reading, and C denotes civics special trend): R, W, W, W, R, W, W, W, R, W,

W, W, R, W, W, C, W, W. This sequence contains 13 W, 4 R, and 1 C. This sequence was designed to

ensure the maximum feasible spread of assessment types among schools, while ensuring that close to 72

percent of the selected students were assigned to writing/civics, 22 percent of the selected students were

assigned to reading, and 6 percent were assigned to civics special trend.

Schools with 26 or more eligible students were always assigned writing/civics. Schools with 76

or more eligible students were almost always assigned reading. Many schools were awarded "multiple"

sessions of writing/civics. This did not necessarily mean that the school had to conduct physically

multiple sessions of writing/civics, but the assignment of session types determined the proportions of

selected students within the school that were assigned to each session type.

3.3.2.2 Grade 8 Allocation of Sessions

For grade 8, sessions were allocated to schools in the following way. First, each school was

allocated a number of sessions, based on the estimated number of grade-eligible students, as shown here:

Estimated Number of

Grade-Eligible Students

Number of

Sessions Allocated

1 – 25 1

26 – 50 2

51 – 75 3

76 – 100 4

101 or more 5
45

The sessions were allocated to schools by placing schools in the order used for sampling, and

allocating the appropriate number of sessions from the following repeated sequence (W denotes

writing/civics, R denotes reading, and C denotes civics special trend): R, W, W, W, R, W, W, W, R, W,

W, W, R, W, W, C, W, W, R, W, W, W, R, W, W, W, R, W, W, W, R, W, W, W, R, W, W, W, R, W, W,

C, W, W, R, W, W. This sequence contains 34 W, 11 R, and 2 C. This sequence was designed to ensure

the maximum feasible spread of assessment types among schools, while ensuring that close to 72 percent

of the selected students were assigned to writing/civics, 23 percent of the selected students were assigned

to reading, and 4 percent were assigned to civics special trend.

Schools with 26 or more eligible students were always assigned writing/civics. Schools with 76

or more eligible students were almost always assigned reading. Many schools were awarded "multiple"

sessions of the same type. This did not necessarily mean that the school had to conduct physically

multiple sessions of a given assessment type, but the assignment of session types determined the

proportions of selected students within the school that were assigned to each session type.

3.3.2.3 Grade 12 Allocation of Sessions

In the same manner, sessions were allocated to grade 12 schools. First, each school was allocated

a number of sessions, based on the estimated number of grade-eligible students, as shown here:

Estimated Number of

Grade-Eligible Students

Number of

Sessions Allocated

1 – 30 1

31 – 60 2

61 – 90 3

91 – 120 4

121 or more 5

The sessions were allocated to schools by placing schools in the order used for sampling, and

allocating the appropriate number of sessions from the following repeated sequence (W denotes

writing/civics, R denotes reading, and C denotes civics special trend): R, W, W, R, W, W, R, W, W, R,

W, W, C, W, W, R, W, W, R, W, W, W, R, W, W, W, R, W, W, R, W, W, R, W, W, C, W, W, R, W, W,

R, W, W, W, R, W, W, W. This sequence contains 34 W, 13 R, and 2 C. This sequence was designed to

ensure the maximum feasible spread of assessment types among schools, while ensuring that close to 69

percent of the selected students were assigned to writing/civics, 27 percent of the selected students were

assigned to reading, and 4 percent were assigned to civics special trend.

Schools with 31 or more eligible students were always assigned writing/civics. Schools with 91

or more eligible students were almost always assigned reading. Many schools were awarded "multiple"

sessions of the same type. This did not necessarily mean that the school had to conduct physically

multiple sessions of a given assessment type, but the assignment of session types determined the

proportions of selected students within the school that were assigned to each session type.

3.3.3 Assignment of Sample Types

In order to determine the effect of using different criteria for excluding students from the

assessment, two different sample types (S2 and S3) were assigned to the subsamples by session in

sampled schools. In sample type 2 schools, the 1996 exclusion criteria were used, but no

accommodations were offered. In sample type 3 schools, the 1996 exclusion criteria were used and
46

accommodations were offered to students with disabilities (SD) and students of limited English

proficiency (LEP). For writing and civics sessions, there was only sample type, S3. For more details of

the exclusion criteria and their implementation, and the accommodations offered students, see Exhibits 4-

1 and 4-2 in Sampling Activities and Field Operations for 1998 NAEP (Gray, et al., 2000). The

information in this chapter applies to both sample types or subsamples.

Sample type was assigned to schools separately for each grade so that 50 percent of the schools

assigned reading were assigned sample type 2 and 50 percent were assigned sample type 3. Then, for

schools that were also selected for the state assessment program, sample type was revised as explained in

Section 3.3.3.4.

3.3.3.1 Grade 4 Assignment of Sample Types

At grade 4, sample type was assigned when allocating sessions to schools. Section 3.3.2

presented the session allocation sequence. The assignment of sample type to the subsamples by session

was incorporated into the sequence as follows: R2, W, W, W, R3, W, W, W, R2, W, W, W, R3, W, W,

C, W, W, where R2 means the school was allocated a reading session and assigned sample type 2, and R3

means the schools was allocated a reading session and assigned sample type 3. Thus, the sequence

contained two reading sessions for sample type 2 (R2) and two reading sessions for sample type 3 (R3).

In this manner, sample type was assigned so that a variety of schools with respect to region, school type,

urbanization, and size were in each sample type.

3.3.3.2 Grade 8 Assignment of Sample Types

For grade 8, the schools were placed in the order of sampling, then sample types were assigned to

subsamples for reading session by alternating sample types 2 and 3. Sample type was assigned so that a

variety of schools with respect to region, school type, urbanization, and size were in each sample type.

3.3.3.3 Grade 12 Assignment of Sample Types

The assignment of sample type to grade 12 schools was done in the same manner as for grade 8.

3.3.3.4 Schools Selected in Both National and State Assessments

For schools selected in both the national samples and state assessment program within the same

grade (only grades 4 and 8 applied), sample type was initially assigned as described above, and then

reassigned for the national samples to be consistent with the state assessment. That is, schools were

ultimately assigned the same sample type as for the state assessment.

3.4 STUDENT SAMPLE

The sample of students within sampled schools was drawn by systematic sampling from schoolprepared lists of eligible students. Student listing forms (SLF) were prepared for each participating

school in a given grade; all enrolled students of the specified grade were to be entered on the SLFs. For

details, see Exhibit 1 of Appendix B in the Sampling Activities and Field Operations for 1998 NAEP

(Gray, et al., 2000). Student samples that also included oversampling of Black and Hispanic students in

low-minority areas, and oversampling SD/LEP students in public schools assigned to reading, were

specified through the use of session assignment forms (SAF).
47

3.4.1 Updating Estimates of Grade-Eligible Students

All assessment components were administered to grade-eligible students. Target numbers of

completed assessment booklets by booklet number played an important role in the sample design.

Preliminary projections of completed test booklets by school were made as a part of the school sample

selection procedure based on estimates of eligible students from frame data (see Section 3.2.4.1).

Up-to-date information on grade enrollment was obtained for sampled schools through two field

processes. Scheduling assessment dates with schools and being on site at the school at the time of

assessment allowed field staff to obtain updated information on the number of grade-eligible students.

3.4.2 Within-School Sampling Rates

Let

MA = Maximum allowable sample size from an individual school

(100, grade 4; 125, grade 8; 150, grade 12); and

Gi = Revised estimate of grade-eligible students for school i.

Then the sampling rate applied to the list of eligible students to select the sample was

given by:

R = A

i

M

G

if Gi > (MA + 10), for grades 4 and 8; or

> (MA + 20), for grade 12;

or R = 1, otherwise.

Students were assigned to the sessions systematically, in proportion to the number of sessions of

each type allocated to the school, as described in Section 3.3.2. Thus, for example, a grade 8 sample

school with an estimated 125 grade-eligible students, assigned sessions W, W, R, W, W, would have

four-fifths of the selected students allocated to writing/civics and one-fifth of the selected students

allocated to reading.

3.4.3 The Session Assignment Form (SAF)

To control the student sampling operations as closely as possible, Westat generated a session

assignment form for each school where sampling was to be carried out. This computer-generated form

specified:

• The types of sessions that were to be administered at the school

• The line numbers (from the SLF) specifying the students to be drawn into the

sample
48

• The minimum and maximum number of students listed on the SLF that could be

accepted without requiring revision to the within-school sampling rates

• Notification of whether there were to be accommodations offered to SD/LEP

students

• Directions and line numbers for oversampling Black and Hispanic students in

public schools with low minority enrollment and SD/LEP students in schools

assigned reading, and

• Special instructions as appropriate for the teacher survey (see Section 3.4.9), the

SD/LEP questionnaire, the NAEP Classroom-Based Writing Study, and the High

School Transcript Study (separate, but related to NAEP).

3.4.4 Updating Session Allocation When Generating SAFs

Due to the presence of updated grade enrollment numbers, it became necessary to revise the

session allocation structure for some smaller-than-expected schools with more than one session type

initially assigned. Smaller-than-expected schools were defined as having a potential of less than 12

students assigned to any particular session type. For example, if two writing/civics and one reading

session were assigned, and the number of grade-eligible students was updated to 30, then there would be

only 10 assessed in reading. In this case, and in general, for smaller-than-expected schools where the

number of grade-eligible students per session type assigned (without regard to the number of sessions

assigned for each type) was 12 or more (15 in the example), all session types were kept and students were

split evenly across the session types. Thus, in the example given here, 15 students would be assigned to

reading and 15 to writing, rather than the initial sample allocation number of 10 and 20, respectively. If

the number of grade-eligible students per session type assigned was less than 12, just one session type

was kept at random, and a weight adjustment factor was computed as the ratio of the number of sessions

assigned to the number of sessions assigned for the session type that was kept. This weight adjustment

accounts for dropping one or more session types.

3.4.5 Sample Selection

In the field operations of sample selection, the district supervisor generally carried out the

sampling of students a week prior to the assessment. Student listing forms (SLF) were prepared for the

applicable grade in each participating school. All enrolled students of the specified grade were to be

entered on the SLF in any order convenient to the school, or the school could produce a computergenerated list. Before carrying out the sampling, the district supervisor reviewed the form and made

comparisons with other information in an effort to make sure that the list included all eligible students.

The sample SLF can be found in Sampling Activities and Field Operations for 1998 NAEP (Gray, et al.,

2000).

The sampling was carried out according to very specific instructions described in the supervisor’s

manual. The sampling statisticians were available by telephone to assist in the resolution of sampling

problems and to generate revised SAFs when necessary.
49

Briefly, the sample selection procedures involved the following:

• Numbering sequentially the lines listed on the SLF or computer-generated list

• Using the line numbers associated with each session type on the SAF, indicating

the sample selection for each session type on the SLF for every student whose

line number corresponded to the line numbers given on the SAF

3.4.5.1 Oversampling Black and Hispanic Students

As discussed in Section 3.2, in public schools with high-minority (Black and Hispanic)

enrollments, schools were assigned a measure of size twice the size of other low-minority schools,

therefore increasing their probability of selection, and indirectly increasing the number of Black and

Hispanic students in the sample.

In public schools with low minority enrollment, an oversample of Black and Hispanic students

was selected. The procedure was as follows. After the initial sample was selected, as discussed in Section

3.4.5, the nonselected Black and Hispanic students were identified and listed. All such extra Black and

Hispanic students were sampled to a total that, as expected, was the same number of Black and Hispanic

students as were already selected. In practice, if the number of nonselected students was less than the

number of selected students, then all nonselected Black and Hispanic students were to be assessed also.

Otherwise, Black and Hispanic students were sampled so that their overall within-school probability of

selection was twice the rate of other students.

Line numbers were generated to split the additional sample of Black and Hispanic students into

sessions as the session allocation rates applied to the initial sampling procedure. Thus, if the school was

assigned two sessions of writing/civics and one of civics special trend, two-thirds of these extra Black

and Hispanic students were assigned to writing/civics, and one-third to civics special trend.

The sampling of additional Black and Hispanic students was carried out using designated line

numbers, indicated on the session assignment form used to generate the samples of students in each

school. In this way, the necessary information as to the selection probability of each student was retained

for use in weighting. No reliance was placed on information generated in the field. Field supervisors had

only to follow the prespecified sampling instructions.

Since the aim was to oversample by a factor of two where possible, but never more than two, the

overall rate of oversampling of Black and Hispanic students was instead less than two. That is because in

smaller low-minority schools there were no students remaining who had not already been assigned to a

session. The weighting procedures ensured that the results were not biased as a result of the relative

underrepresentation of Black and Hispanic students from smaller low-minority schools.

3.4.5.2 Oversampling SD/LEP Students in Reading

As noted in Section 3.1.3, in the reading assessments, the procedures for assessing SD and LEP

students varied by sample type. SD/LEP students in sample type 3 were offered accommodations not

available to other students or to SD/LEP students in sample type 2.

As a measure to ensure an adequate sample size of SD/LEP students from both sample types 2

and 3 for reading, oversampling procedures were applied to SD/LEP students at all three grades. In this

way, comparisons of the effect of offering accommodations to students have enhanced power to detect

effects.
50

The general intent of oversampling within each school that was assigned at least one reading

session was to select SD/LEP students at twice the rate at which non-SD/LEP students were sampled (or

to include all SD/LEP students if there were not sufficient numbers to permit sampling at twice the rate).

There was no oversampling of schools as part of the procedure.

The procedure was as follows. In each school where oversampling of SD/LEP students was to

occur, the initial desired sample of students was drawn for each session assigned, from the full list of

eligible students. In addition, in public schools in low-minority areas, oversampling of Black and

Hispanic students occurred. Among those students not selected for either of the two prior sampling

operations for this school, the SD/LEP students were identified. A sample from among these was drawn,

using a sampling rate that would achieve the double sampling rate required overall. In most cases in

grade 4, this involved selecting all such SD/LEP students in the school. Again, the weighting procedures

ensured that the results were not biased as a result of the relative underrepresentation of SD/LEP students

from smaller schools.

As with the oversampling of Black and Hispanic students, the sampling of additional SD/LEP

students was carried out using designated line numbers.

Table 3-9 shows the results of the oversampling efforts relating to SD/LEP students for each

grade and sample type for reading. The weighted results show the proportion of the sample that would

have been SD/LEP students had no oversampling been attempted. The focus is on sample types 2 and 3

for reading, since this is where the oversampling of SD/LEP students occurred. The extent to which the

unweighted percentage of SD/LEP students exceeds the weighted percentage is a measure of the

effectiveness of the oversampling.

Table 3-9

Percentage of Assessed and Absent Students Who Were Specified as SD/LEP

National 1998 Reading Samples

Sample Grade 4 Grade 8 Grade 12

Type Unweighted Weighted Unweighted Weighted Unweighted Weighted

2 11.0 8.3 12.2 7.2 9.4 4.8

3 13.9 10.8 16.0 9.9 10.5 5.9

Total 12.5 9.5 14.0 8.5 9.9 5.3

As can be seen, the procedure was effective in increasing the sample of SD/LEP students

considerably at grades 8 and 12, and was effective to a lesser extent at grade 4. To increase the sample of

SD/LEP students further at grade 4 would require the assessment of additional schools. The differences

in rates between sample types 2 and 3 show the effects of accommodations being offered. It was expected

that if no accommodations were offered, the rates would be equal; however, since accommodations were

offered in sample type 3, more SD/LEP students were assessed.

3.4.6 Supporting the Field Staff on Sampling Issues

The completed SLF generally contained a number of students, which was different from the

number used in operating the SAF. In order to control the total number of students tested per school, an

acceptable range for that number was specified. Whenever the total number of students listed on the SLF

was outside the specified range, the supervisor used a laptop computer to generate a new set of line
51

numbers. Based on revised sampling rates, a revised SAF was produced. The revised sampling rates were

sent in from the field supervisors and were entered on the weight file.

In order to gain cooperation in some schools, we occasionally granted principals’ special

requests. For example, some large schools divided their students into clusters, and to minimize disruption

among all students in the sampled grade, samples were administered to students within one randomly

selected cluster. Students in the sampled cluster were listed on the SLF and new line numbers were

generated using the cluster’s enrollment. The revised sampling weights were entered on the weight file to

account for sampling one cluster.

Table 3-10 shows the distribution of the number of students per school who were assessed for

each assessment.

Note that, for the various samples, the number of students assessed per item per school is quite

low, even though typically dozens of students were assessed in total in a particular school. Thus, the

extent of clustering of the sample is in general quite modest, because most sampled schools conducted a

few different assessments with a moderate number of students in each. More importantly, the use of BIB

or PBIB spiraling in the administered sessions greatly alleviated the effects of clustering the samples of

students within schools, for item-level data.

3.4.7 Excluded Students

The 1998 assessment, as did previous assessments, excluded students who were functionally

handicapped to the extent that they could not participate in the assessment as it was normally conducted.

Specific groups excluded were:

• Some students identified as having student disability (SD) or equivalent,

• Some students with limited English proficiency (LEP).

Any sample students who were classified SD or LEP (or both) were identified. The school

completed an SD/LEP student questionnaire for each student with this designation. This was a change

from assessments prior to 1996, in which these questionnaires, then called excluded student

questionnaires, were completed only for students who were actually excluded. Then school personnel

determined whether any of these students should be excluded from the assessment based on the criteria

for excluding students.

According to Table 3-10, for the reading reporting population, about half of the SD/LEP students

in grade 4 were excluded. However, for grades 8 and 12, less than half of the SD/LEP students were

excluded. Rates of excluded SD/LEP students are also shown by sample type. Recall from Section 3.3

that students in sample type 2 (S2) were not offered accommodations, while students in sample type 3

were offered accommodations. The exclusion rates for SD/LEP students in sample type 2 are similar to

that of the reporting population. This is because sample type 2 and the reporting populations contain the

same group of SD/LEP students (numerator), but their denominator for the rate calculation differs

slightly due to differing groups of non-SD/LEP students. For students in sample type 3, the rates of

excluded SD/LEP students are lower.

This data collection effort permits national estimates of statistics for SD, LEP, and excluded

students. Table 3-11 shows the distribution of excluded students by reason for exclusion for the three

grades. The dominant reason for exclusion from NAEP across all grades and subjects was a student

disability. The proportion attributable to student disability increased with grade, while the proportion

attributable to limited English proficiency, the second reason, decreased with grade. Table 3-12 presents
52

the weighted student exclusion rates for each grade and subject by school type and sample type. The

exclusion rate decrease as grade increases. The rate for writing and civics are lower than that of civics

special trend, since accommodations were offered if necessary. Likewise, the reading sample type 3 rate

was lower than that of sample type 2 because accommodations were offered. The rates for public schools

are much higher than for private schools.
Table 3-10

Number of Students Per School for Each Subject Type for 1998 National Assessments*

Distribution of Students Per Assessment Per Schoo

Sample Subject Type

Number of

AssessedStudents

Number

Of

Schools Mean Median Minimum Maximum

Grade 4 25-Minute Writing

CivicsReading/S2Reading/S3Civics Special Trend

19,816

5,9484,0484,2042,088

678670217217111

29.2 8.918.719.418.8

28.5

9.0

19.020.019.0

11215

7322304431

Grade 8 25-Minute Writing

50-Minute WritingCivicsReading/S2Reading/S3Civics Special Trend

20,586

6,0098,2126,2255,7102,055

702694697248235104

29.3 8.711.825.124.319.8

30.0

9.0

12.022.023.020.0

111516

165

4866627330

Grade 12 25-Minute Writing

50-Minute WritingCivicsReading/S2Reading/S3Civics Special Trend

19,505

5,8047,7636,6006,7232,193

569564566245241102

34.310.313.726.927.921.5

35.010.514.024.025.021.0

111117

111

3443856479

* The numbers in this table reflect the full samples, including S2 and S3 for reading.† The number varied because reading for grades 8 and 12 was split into 25-minute reading and 50-minute reading. There was a higher pro

assigned to 25-minute reading, and also a larger number of booklets. At grade 8, the number of students per item for the 25-minute readi50-minute reading.
54

Table 3-11

 Weighted Percentages of Students Excluded (SD and LEP) from 1998 National Reading Assessment*

Population Grade Type

Total % of Students

Identified

SD or LEP

Total % of

Students That

Were Excluded

% of Students

Identified w/SD

% of Students

That Were

Excluded and SD

% of Students

Identified w/LEP

Reporting 4 Overall 17.12 9.61 10.05 5.29 7.55 

Public 18.41 10.55 10.63 5.78 8.31 Nonpublic 4.84 0.68 4.59 0.55 0.25 

Reporting 8 Overall 12.39 5.38 9.41 4.63 3.39 

Public 13.51 5.96 10.22 5.13 3.75 Nonpublic 2.23 0.11 2.11 0.11 0.12 

Reporting 12 Overall 7.86 3.08 5.99 2.77 2.14 

Public 8.52 3.33 6.46 3.00 2.32 Nonpublic 1.61 0.69 1.47 0.62 0.36 

S2 4 Overall 17.03 9.56 10.00 5.26 7.50 

Public 18.29 10.48 10.56 5.75 8.25 Nonpublic 4.85 0.68 4.61 0.55 0.25 

S2 8 Overall 12.01 5.21 9.12 4.49 3.29 

Public 13.14 5.80 9.94 4.99 3.65 Nonpublic 2.11 0.10 2.00 0.10 0.11 

S2 12 Overall 7.71 3.02 5.88 2.72 2.10 

Public 8.39 3.28 6.37 2.95 2.29 Nonpublic 1.53 0.66 1.40 0.59 0.34 

S3 4 Overall 16.57 6.48 10.60 4.40 6.46 

Public 18.09 7.10 11.54 4.80 7.09 Nonpublic 1.82 0.49 1.45 0.49 0.38 

S3 8 Overall 13.24 3.70 10.02 2.95 3.67 

Public 14.40 4.07 10.89 3.23 4.00 Nonpublic 2.34 0.29 1.83 0.29 0.51 

S3 12 Overall 7.84 2.10 5.78 1.86 2.19 

Public 8.50 2.29 6.25 2.04 2.40 Nonpublic 1.32 0.13 1.18 0.00 0.13 

* The numbers in this table reflect the full samples, including sample type 2 (S2), and sample type 3 (S3) for reading.
Table 3-12

Weighted and Unweighted Distribution of Students Excluded for 1998 National Assessments, by Reason for Exclusi

Grade 4 Grade 8 G

Reason by Subject

Unweighted

Count

Weighted

Count

Weighted

Percent

Unweighted

Count

Weighted

Count

Weighted

Percent

Unweighted

Count

W

25-Minute WritingSD 717 138,905 64.8 625 116,229 79.2 532 LEP 656 66,657 31.1 213 25,797 17.6 95 SD and LEP 74 8,044 3.8 33 3,611 2.5 16 Other 3 603 0.3 6 1,125 0.8 15 Total 1,450 214,210 100.0 877 146,762 100.0 658 50-Minute WritingSD — — — 186 110,258 78.2 159 LEP — — — 71 27,481 19.5 34 SD and LEP — — — 8 2,753 2.0 3 Other — — — 1 459 0.3 6 Total — — — 266 140,951 100.0 202 CivicsSD 195 125,958 63.0 233 108,922 77.7 201 LEP 197 67,727 33.9 94 27,955 20.0 36 SD and LEP 14 5,900 3.0 14 3,221 2.3 6 Other 1 236 0.1 0 0 0.0 4 Total 407 199,822 100.0 341 140,098 100.0 247 Reading†SD 228 223,674 62.7 490 178,076 85.1 340 LEP 299 122,640 34.4 103 23,461 11.2 87 SD and LEP 11 6,435 1.8 14 2,916 1.4 12 Other 7 3,798 1.1 16 4,694 2.2 3 Total 545 356,547 100.0 623 209,148 100.0 448 Civics Special TrendSD 116 200,458 75.9 71 131,949 81.7 89 LEP 54 58,115 22.0 21 28,631 17.7 12 SD and LEP 6 5,596 2.1 0 0 0.0 2 Other 0 0 0.0 1 998 0.6 0 Total 176 264,169 100.0 93 161,578 100.0 103 

* Weighted counts and percents may not add up exactly to the totals due to rounding.

† Represents the reporting population
56

Table 3-13

Student Exclusion Rates for 1998 National Assessments By Grade, School Type, and Sample Type, Weight

Grade 4 Grade 8 G

Subject/Sample Type Public Nonpublic Total Public Nonpublic Total Public No25-Minute Writing 6.5% 0.3% 5.8% 4.2% 0.4% 3.8% 2.7% 50-Minute Writing* — — — 4.2% 0.1% 3.8% 3.0% 

Civics 6.1% 0.2% 5.5% 4.0% 0.3% 3.7% 2.6% Reading/S2 10.5% 0.7% 9.6% 5.8% 0.1% 5.2% 3.3% Reading/S3 7.1% 0.5% 6.5% 4.1% 0.3% 3.7% 2.3% Civics Special Trend 7.6% 0.0% 6.9% 4.4% 0.0% 4.1% 4.2% 

* 50-minute writing blocks were administered at grades 8 and 12 only.
57

3.4.8 Student Participation Results

The NAEP sample was designed to yield a target number of each of the various assessment

components. Table 3-14 compares the target assessments to the actual assessments for the three grades. The

targets were quite closely met in all cases. Achieving sampling goals precisely is dependent on many factors,

including the reliability of frame enrollment data, and the actual response and exclusion rates encountered.

Table 3-14

Comparison of Target Assessments to Actual Assessments for 1998 National Samples, by Grade

Grade 4 Grade 8 Grade 12

Assessments Target Actual Target Actual Target Actual

Total 36,000 36,104 47,000 48,797 49,000 48,589

25-Minute Writing 20,000 19,816 20,000 20,586 20,000 19,505

50-Minute Writing* — — 6,000 6,009 6,000 5,805

Civics 6,000 5,948 8,000 8,212 8,000 7,763

Reading 8,000 8,252 11,000 11,935 13,000 13,323

Civics Trend 2,000 2,088 2,000 2,055 2,000 2,193

* 50-minute writing blocks were administered at grades 8 and 12 only.

Table 3-15 shows the unweighted student participation rates of invited students. The set of invited

students consists of the selected students, after removing the excluded students. For a given session, a

makeup session was called for when, for various reasons, more than a predetermined tolerable number of

invited students were absent from the originally scheduled session to which they were invited. The

participation rates given in the table express the number finally assessed as a percentage of those initially

invited in the participating schools. Participation rates are shown for public and nonpublic schools

separately.

Table 3-15

 Unweighted Student Participation Rates for National Assessments, by Grade and School Type

1998 Public 1998 Nonpublic 1998 Combined 1996

Grade

Number

Invited

Participation

Rate

Number

Invited

Participation

Rate

Number

Invited

Participation

Rate

Participation

Rate

4 31,400 95.0 6,545 95.8 37,945 95.1 95.4

8 44,171 91.7 8,639 95.9 52,810 92.4 91.5

12 52,148 77.6 8,871 91.4 61,019 79.6 79.9

Overall participation rates are also shown for comparable samples from the 1996 NAEP assessment.

The table shows that student participation rates in 1998 are similar to those experienced in 1996. The rates

increased slightly at grade 8, and remained fairly steady for the other grades. At all grades, the participation

rate of nonpublic-school students exceeds that of public-school students, with the difference, both relative

and absolute, increasing with grade.

The combined impact of school nonparticipation and student absenteeism from sessions within

participating schools is summarized in Table 3-16. The table shows the percentages of students assessed,

from among those who would have been assessed if all initially selected schools had participated and if all

invited students had attended either an initial or make-up session. The results show that, consistent with
58

earlier rounds of NAEP, the overall level of participation decreases substantially with the increase in the

grade of the students.

Table 3-16

 Overall Unweighted Participation Rates (School and Student Combined)

for 1998 National Assessments, by Grade

1998 Sample Grade 4 Grade 8 Grade 12 Overall

School Participation

 Before Substitution 81.1% 80.7% 75.2% 79.2%

 After Substitution 88.6% 87.3% 81.6% 86.0%

Student Participation 95.1% 92.4% 79.6% 88.0%

Overall Student Participation 84.3% 80.7% 65.0% 75.7%

Number of Participating Students 36,104 48,797 48,589 133,490

So far in this section, only unweighted participation rates by grade and school type have been

presented. However, analysis is typically performed separately by grade and subject type, and NCES

standards regarding acceptable potentials for bias are expressed in terms of weighted participation rates.

Therefore, Table 3-17 shows weighted participation rates by grade and subject type. The sample rates are for

students in the reporting populations. Note that the school and student participation rates decrease as grade

increases for different session types. At the school level, session types were assigned, and the writing/civics

session contained two subject types in grade 4 and three subject types in grades 8 and 12, to which students

were assigned. Therefore, the school participation rates for 25-minute writing, 50-minute writing (grades 8

and 12) and civics are identical. The school participation rates (before and after substitution) are fairly

similar across subject types. The overall participation rates are relatively low for twelfth grade samples.

The procedures for taking into account nonparticipating schools and for taking into account absent

students through weighting were designed (so far as feasible) to reduce the biases resulting from school and

student nonparticipation. These procedures are discussed in Chapters 10 and 11.

Table 3-17

Weighted Participation Rates by Grade and Subject Type

for the 1998 National Reporting Samples

Participation 25-Minute 50-Minute Civics Special

(Sample Type) Writing Writing Civics Reading Trend

Grade 4

School Participation

 Before Substitution 79.7% — 79.7% 81.0% 81.1%

 After Substitution 88.6% — 88.6% 89.4% 90.0%

Student Participation 94.9% — 94.8% 96.0% 95.4%

Overall Participation 84.1% — 84.0% 86.0% 86.1%

(continued)
59

Table 3-17 (continued)

Weighted Participation Rates by Grade and Subject Type

for the 1998 National Reporting Samples

Participation 25-Minute 50-Minute Civics Special

(Sample Type) Writing Writing Civics Reading Trend

Grade 8

School Participation

 Before Substitution 77.1% 77.1% 77.1% 76.7% 77.1%

 After Substitution 84.6% 84.6% 84.6% 84.1% 90.7%

Student Participation 92.2% 93.0% 92.3% 92.7% 92.3%

Overall Participation 78.0% 78.7% 78.1% 77.9% 83.7%

Grade 12

School Participation

 Before Substitution 69.7% 69.7% 69.7% 69.7% 68.3%

 After Substitution 78.0% 78.0% 78.0% 78.2% 83.4%

Student Participation 79.7% 80.4% 79.4% 80.1% 82.0%

Overall Participation 62.1% 62.7% 61.9% 62.6% 68.4%

3.4.9 Teacher Survey

For the grade 4 and grade 8 samples, a survey of teachers was conducted to obtain information about

the teachers, their classes, and those of their students who participated in the assessment using the relevant

booklet. The questionnaire gathered information about the teaching practices of teachers of sampled students

in each of the subject areas that were assessed (i.e., reading, writing, and civics) at grades 4 and 8. The

teacher survey was not administered to civics special trend assessments or for assessments in grade 12.

Teachers were asked to complete the questionnaires in order that teachers’ background instructional

practices can be linked to student achievement data.
60

GLOSSARY

AS: The administration schedule was prepared for each session to be held in the school and

served as a student roster to be used by the school coordinator and exercise administrator

(EA) to carry out the session.

BIB design: A design in which all the exercises in the assessment for an age class are divided up into

small blocks. Each exercise block is then assigned to a number of assessment packages

(booklets) such that each block is paired with every other block in some booklet the same

number of times in a balanced incomplete block (BIB) design. Variants of this design are

called partially balanced incomplete block (PBIB) designs.

PSS: Enrollment grade span and other data for individual private schools were aggregated into

data for use in sampling PSUs and schools, and in preliminary session allocation. These data

were obtained from a computer file of schools from the Private School Survey conducted by

NCES.

PSU: Primary sampling units are metropolitan statistical areas, counties, or groups of contiguous

counties in the U.S. that served as the first-stage sampling units (see Section 3.2.1).

QED: Enrollment grade span and other data for individual public schools was aggregated into data

for use in sampling PSUs and schools, and in preliminary session allocation. These data were

obtained from a computer file of schools and school districts from Quality Education Data,

Inc.

SAF: The session assignment form was generated for each cooperating school. It identified the

subjects to be administered and the line numbers on the SLF that identified the sampled

students to be included in each subject.

Session: A group of students reporting for the administration of an assessment. A distinction was

made between the number of invited students and the number completing the assessment.

SLF: The student listing forms were the forms used by the school (or supervisor) to list eligible

students. Students were sampled from these lists.

Spiraling: A procedure for assigning tests to students whereby the test packages that are included in the

spiral administration procedure are systematically interspersed, and are assigned for testing in

this arrangement.

Type of Locale: The type of locale (TOL) code is a Westat code for the location of a school relative to

populous areas.
61

Chapter 4

SAMPLE DESIGN FOR THE STATE ASSESSMENT1

Keith F. Rust and Leslie Wallace

Westat

Jiahe Qian

Educational Testing Service

4.1 INTRODUCTION

This chapter describes sampling activities for the 1998 NAEP state reading and writing

assessments, in which 333,624 students were assessed (see Table 5-4). The 1998 state assessment

program in reading included assessments of fourth- and eighth-grade students. The 1998 state assessment

program in writing was conducted in grade 8 only. Civics was not assessed at the state level. The details

of the sample design and selection procedure can be found in the Sampling Activities and Field

Operations for 1998 NAEP (Gray, et al., 2000). For the eighth grade, the samples selected for both the

reading and writing assessments were selected as part of the same process; and in some schools in the

eighth-grade sample, both sessions of reading and writing were assigned. A representative sample of

public- and nonpublic-school students was drawn in each participating jurisdiction. The samples in each

jurisdiction were selected in two stages, with schools selected at the first stage and students selected at

the second stage. This design was intended to produce aggregate estimates as well as estimates for

various subpopulations of interest for all the participating jurisdictions. The sample for the fourth- and

eighth-grade public-school assessments in each jurisdiction consisted of about 3,150 students (before

attrition) in each subject from about 100 public schools in each case. The target for nonpublic-school

students varied by jurisdiction and was proportional to their representation in the jurisdiction.

The target population for the 1998 state assessment program included students in public and

nonpublic schools who were enrolled in the fourth and eighth grade at the time of assessment. The

sampling frame included public and nonpublic schools having the relevant grade levels in each

jurisdiction. The samples were selected based on a two-stage sample design; selection of schools within

participating jurisdictions, and selection of students within schools. The first-stage samples of schools

were selected with probability proportional to a measure of size based on the estimated grade-specific

enrollment in the schools. Special procedures were used for jurisdictions with many small schools (see

Section 4.4.2), and for jurisdictions having small numbers of grade-eligible schools (See Section 4.4.4).

Note that the 1998 national sample was a four-stage probability sample and the first-stage sampling units

were counties or groups of counties.

Stratification variables were added to the sampling frame prior to sample selection. Public

schools were stratified by urbanization and minority class and nonpublic schools were stratified by metro

area status and type of nonpublic school . The urbanization strata were defined in terms of large or

midsize central city, urban fringe of large or midsize city, large town, small town, and rural areas. Within

urbanization strata, public schools were further stratified explicitly on the basis of minority enrollment in

those jurisdictions with substantial Black or Hispanic student population. Minority enrollment was

defined as the total percent of Black and Hispanic students enrolled in a school. Within minority strata,

public schools were sorted by median household income of the ZIP code area where the school was

 

1

Keith F. Rust was responsible for overseeing all sampling activities; Leslie Wallace carried out most of these activities.

Jiahe Qian was responsible for the specification and coordination of the state sampling at ETS.
62

located. Metro area status was determined by U.S. Bureau of Census definitions as of June 30, 1993.

Other stratification variables were obtained from Quality Education Data, Inc. (QED) and the National

Center for Education Statistics’ Common Core of Data (CCD). For details, see Sections 4.2.2 and 4.3.2.

School type was a dichotomous variable (public, and Catholic or other nonpublic). Within school type,

nonpublic schools were sorted by estimated grade enrollment.

From the stratified frame of public and nonpublic schools within each jurisdiction, a systematic

random sample of grade-eligible schools was drawn with probability proportional to a measure of size

based on the estimated grade-specific enrollment of the school. One or more sessions were sampled

within each school. The number of sessions selected depended on the school’s estimated grade-specific

enrollment, though the overwhelming majority of schools at grade 4 were allocated a single session. In

selection of schools, two sets of inclusion rules for SD/LEP students (S2 and S3 subsamples) were

applied in the state assessment.

For jurisdictions that participated in an earlier trial state assessment, 25 percent of the selected

public and nonpublic schools were designated at random to be monitored during the assessment field

period so that reliable comparisons could be made between sessions administered with and without

monitoring. For jurisdictions that did not participate in an earlier assessment, 50 percent of the selected

public and nonpublic schools were designated to be monitored.

Approximately 3,150 public-school students were targeted for selection for a given grade and

subject in a given jurisdiction. For nonpublic schools, the target for each grade and subject varied by

jurisdiction. On average, 105 public schools and 19 nonpublic schools were selected for fourth grade in

each jurisdiction and 99 public schools and 31 nonpublic schools were selected for eighth grade in each

jurisdiction. The maximum numbers of public and nonpublic schools sampled in a participating

jurisdiction were 121 and 36, respectively, for fourth grade. The minimum numbers of public and

nonpublic schools sampled in a participating jurisdiction were 24 and 10, respectively, for fourth grade.

The maximum numbers of public and nonpublic schools sampled for eighth grade were 125 and 46,

respectively, for eighth grade. The minimum numbers of public and nonpublic schools sampled in a

participating jurisdiction were 6 and 14, respectively, for eighth grade. Each selected school provided a

list of eligible enrolled students, from which a systematic sample of students was drawn. Where possible,

30 students were selected for each session.

For the information of state school samples, Tables B-1 through B-6 in Appendix B provide the

weighted participation rates and the mean values of certain school characteristics for both public and

nonpublic schools, both before and after nonresponse for grade 4 reading, grade 8 reading, and grade 8

writing, respectively. Tables B-15 through B-18 provide the distributions of selected schools by sampling

strata by grades for both public and nonpublic schools.

For the characteristics of interest for state student samples, Tables B-7 through B-12 in

Appendix B provide the weighted student participation rates and a different set of statistics for public

schools and all schools, for both full samples and assessed samples of the state assessments. The

information of the unweighted and final weighted counts of assessed and excluded students can be found

in Tables 11-1 through 11-6 in Chapter 11, both for public and nonpublic schools for each jurisdiction,

grade and subject. For weighting procedures for state samples, including those for excluded students, see

Chapter 11.

The rest of this chapter documents the procedures used to select schools for the 1998 state

assessment. Section 4.2 describes the construction of the sampling frames, including the sources of

school data, missing data problems, and definition of appropriate schools. Section 4.3 includes a

description of the various steps in stratification of schools within participating jurisdictions. Section 4.4
63

describes school sample selection procedures (including new and substitute schools). Section 4.4.6

provides information about the subject sessions, sample type, and monitor status. Section 4.5 includes the

steps involved in selection of students within participating schools.

4.2 TARGET POPULATIONS AND SAMPLING FRAME FOR THE 1998 STATE

ASSESSMENT

4.2.1 Target Population

The target population for the 1998 state assessment included students in public and nonpublic

schools who were enrolled in the fourth or eighth grade. Nonpublic schools included Catholic and other

religious schools, private schools, DoDEA/DDESS (Department of Defense Education

Activity/Department of Defense Domestic Dependent Elementary and Secondary Schools), and Bureau

of Indian Affairs (BIA) schools. Special education schools were not included. Both subsamples of sample

type S2, where accommodations were not offered to SD/LEP students, and sample type S3, where

accommodations were offered, shared this target population.

4.2.2 Sampling Frame

In order to draw the school samples for the 1998 state assessment, it was necessary to obtain a

sampling frame, a comprehensive list of public and nonpublic schools, in each jurisdiction. For each

school, useful information for stratification purposes, reliable information about grade span and

enrollment, and accurate information for identifying the school to the state coordinator (district

membership, name, address) were required.

Based on prior experience with the 1992, 1994, and 1996 trial state assessments, and national

assessments from 1984 to 1996, the file made available by QED was elected as the primary sampling

frame. The QED list covers all U.S. states but not the territories. The CCD school file was used to obtain

schools in Guam and Virgin Islands, and was used to check the completeness of the QED file.

The version of the QED file used was released in early 1997, in time for selection of the school

sample. However, for some schools, the file was missing racial/ethnic minority enrollment and

urbanization data (due to the inability of QED to match these schools with the corresponding CCD file).

Since these variables were to be used for stratification, considerable efforts were undertaken to obtain

these variables for all schools in jurisdictions. These efforts are described in the next section.

For 1998 state assessment, the files of the Private School Universe Survey (PSS), which was

administered by the National Center for Education Statistics, were used as the sampling frame for

nonpublic schools. The QED list was not used to form the sampling frame for nonpublic schools as had

been done in the past. Following the very intensive work of unduplicating these two lists in 1996 and an

evaluation of the 1996 NAEP nonpublic-school sample, it was decided to use PSS as the sole source for

the sampling frame of nonpublic schools.

Tables 4-1 and 4-2 show the distribution of fourth- and eighth-grade schools as well as

enrollment within schools as reported in the combined frame. Grade-specific enrollment was estimated

for each school as the quotient of total school enrollment and the number of grades in the school.
64

Table 4-1

Distribution of Fourth-Grade Schools and Enrollment

in Combined Sampling Frame for 1998 NAEP State Assessments

Public Schools Nonpublic Schools

Jurisdiction

Total

Schools

Total

Enrollment

Total

Schools

Total

Enrollment

Total 40,139 2,877,001 11,487 246,708

Alabama 764 58,729 261 6,154

Arizona 719 62,633 260 4,689

Arkansas 533 35,859 166 2,733

California 4,989 445,937 2,872 61,625

Colorado 808 51,882 277 4,779

Connecticut 571 42,507 253 5,484

Delaware 52 7,983 86 2,126

District of Columbia 113 6,330 68 1,476

DoDEA/DDESS 39 3,215 N/A N/A

DoDEA/DoDDS 103 6,777 N/A N/A

Florida 1,487 173,855 1,073 24,346

Georgia 1,056 108,774 448 9,469

Hawaii 177 15,343 99 2,589

Illinois 2,268 152,948 1,195 27,633

Iowa 752 37,515 224 4,677

Kansas 798 36,548 191 3,747

Kentucky 782 47,576 289 6,717

Louisiana 793 60,398 377 11,794

Maine 385 17,128 106 1,213

Maryland 804 62,012 459 10,818

Massachusetts 1,039 74,564 473 9,836

Michigan 1,919 130,496 909 18,291

Minnesota 844 64,029 469 8,647

Mississippi 458 40,674 166 4,163

Missouri 1,123 68,180 529 11,236

Montana 455 13,485 75 932

Nebraska 883 22,147 194 3,753

Nevada 254 23,038 59 1,167

New Hampshire 266 16,562 93 1,374

New Mexico 387 25,607 176 2,855

New York 2,250 207,021 1,656 42,214

North Carolina 1,140 97,817 429 7,963

Oklahoma 941 50,649 128 2,389

Oregon 751 42,503 247 3,738

Rhode Island 181 12,086 89 1,933

South Carolina 554 50,729 256 4,971

Tennessee 926 71,198 370 6,557

Texas 3,304 291,812 970 21,139

Utah 441 35,513 54 934

Virgin Islands 24 1,831 27 543

Virginia 1,051 86,583 384 7,729

Washington 1,065 74,783 390 7,122

West Virginia 532 23,168 118 1,305

Wisconsin 1,137 66,170 846 14,256

Wyoming 221 7,654 33 319
65

Table 4-2

Distribution of Eighth-Grade Schools and Enrollment

in Combined Sampling Frame for 1998 NAEP State Assessments

Public Schools Nonpublic Schools

Jurisdiction

Total

Schools

Total

Enrollment

Total

Schools

Total

Enrollment

Total 17,660 2,796,611 5,378 121,361

Alabama 484 56,743 232 5,443

Arizona 364 59,746 235 4,355

Arkansas 352 36,434 126 1,968

California 1,719 393,472 2,417 53,298

Colorado 342 51,100 229 3,929

Connecticut 208 36,775 250 5,754

Delaware 30 8,506 78 1,951

District of Columbia 33 4,421 64 1,438

DoDEA/DDESS 12 1,625 N/A N/A

DoDEA/DoDDS 65 5,093 N/A N/A

Florida 499 168,930 911 21,194

Georgia 420 104,295 399 8,357

Hawaii 52 13,183 85 3,127

Illinois 1,370 144,236 1,121 26,481

Kansas 421 36,269 147 2,958

Kentucky 347 50,454 254 5,986

Louisiana 441 59,009 367 13,757

Maine 232 16,617 101 1,168

Maryland 239 60,756 426 10,218

Massachusetts 401 65,981 468 10,452

Minnesota 448 64,025 358 7,073

Mississippi 780 121,964 140 3,848

Missouri 652 67,282 477 10,696

Montana 319 13,277 69 841

Nebraska 580 23,402 160 3,400

Nevada 93 21,028 50 1,061

New Mexico 154 25,227 131 2,393

New York 1,020 192,295 1,496 40,224

North Carolina 521 92,213 368 6,347

Oklahoma 613 49,440 107 2,103

Oregon 338 41,762 228 3,376

Rhode Island 52 11,409 91 2,327

South Carolina 255 51,632 220 4,186

Tennessee 532 67,373 347 6,618

Texas 1,519 284,146 756 16,975

Utah 154 38,971 57 1,022

Virgin Islands 6 2,368 20 411

Virginia 343 84,608 343 7,397

Washington 430 73,529 326 6,115

West Virginia 206 23,826 99 1,143

Wisconsin 520 64,855 751 12,815

Wyoming 94 8,334 28 234
66

4.3 STRATIFICATION OF SCHOOLS IN THE SAMPLING FRAME

4.3.1 Stratification Variables

The stratification used for sample selection varied by school type (public or nonpublic), because

the availability of information and the feasibility of performing sampling are different for public and

nonpublic schools. Stratification of public schools involved four primary dimensions, whereas the

stratification of nonpublic schools involved three primary dimensions. Public schools were stratified

hierarchically by small or large district status, school size classification (measured by student

enrollment), urbanization classification, and minority classification. For details of the resources for

stratification variables, see Section 4.3.3. Nonpublic schools were stratified by school size classification,

metro area status, and school type (Catholic or other nonpublic).

Public schools were further stratified implicitly by median household income (i.e., sorted in

ascending or descending order) of the ZIP code area where the school was located, and nonpublic schools

were further stratified implicitly by estimated grade enrollment, in order to provide some control over

these variables.

Prior to the selection of the school samples, the public schools were sorted by their four

stratification variables (small or large district status, school size classification, urbanization

classification, and minority classification) in an order such that changes occur on only one variable at a

time (also known as a serpentine order). This is accomplished by alternating between ascending and

descending sort order on each variable successively through the sort hierarchy. Within this sorted list, the

schools were sorted, in serpentine order, by the median household income. This final stage of sorting

resulted in implicit stratification of median household income.

The counts of sampled schools by the primary stratification variables can be found in Tables B15 through B-18 in Appendix B.

4.3.2 Missing Stratification Variables

As stated earlier, the sampling frame for the 1998 state assessment was the combination of the

most recent version of the QED file available and the 1995 PSS list of nonpublic schools. The CCD file

was used to extract information on urbanization (“type of location”) for public schools where this

information was missing on the QED file. Any public schools with remaining missing values in

urbanization or minority enrollment had their data imputed.

 Schools with missing values in urbanization data were assigned the urbanization of other school

records within the same state, county, and city when urbanization did not vary within the given city. Any

schools still missing urbanization were assigned the modal value of urbanization within their city. Any

remaining missing values were assigned individually based on city, using U.S. Bureau of Census

publications.

Schools with missing values in minority enrollment data were assigned the average minority

enrollment within their school district. Any schools still missing minority enrollment data were assigned

values individually, using ZIP code and U.S. Bureau of Census data. The minority data were extracted

only for those schools in jurisdictions in which minority stratification was performed.

Metro area status was assigned to each nonpublic school based on U.S. Bureau of Census

definitions as of June 30, 1993, based on Federal Information Processing Standard (FIPS) county code,
67

and was found for all schools in the sampling frame. The Catholic school flag was assigned to each

nonpublic school based on the PSS school type and was found for all schools in the sampling frame.

Median household income was assigned to every school in the sampling frame by merging on

ZIP code with a file from Donnelly Marketing Information Services. Any schools still missing median

household income were assigned the mean value of median household income for the three-digit ZIP

code prefix or county within which they were located.

4.3.3 Resources for Stratification Variables

The procedures used to compile or create the stratification variables for sampling schools are described

below. The resulting classifications for urbanization, minority stratification, metro area status, and school

type for schools used within each participating jurisdiction can be found in Tables B-15 through B-18 in

Appendix B.

4.3.3.1 Urbanization Classification

Urbanization classification was created based on the NCES type of location variable. The type of

location variable contains at most seven levels:

1. Large Central City: A central city of a metropolitan statistical area (MSA) with a

population greater than or equal to 400,000, or a population density greater than or

equal to 6,000 persons per square mile;

2. Midsize Central City: A central city of an MSA but not designated as a large central

city;

3. Urban Fringe of Large City: A place within an MSA of a large central city and

defined as urban by the U.S. Bureau of Census;

4. Urban Fringe of Midsize City: A place within an MSA of a midsize central city and

defined as urban by the U.S. Bureau of Census;

5. Large Town: A place not within an MSA, but with a population greater than or equal

to 25,000 and defined as urban by the U.S. Bureau of Census;

6. Small Town: A place not within an MSA, with a population less than 25,000, but

greater than 2,499 and defined as urban by U.S. Bureau of Census; and

7. Rural: A place with a population of less than 2,500 and defined as rural by the U.S.

Bureau of Census.

Urbanization classification was created by collapsing type of location categories as necessary

and according to specific rules until each urbanization stratum included a minimum of 10 percent of

eligible students in the participating jurisdiction. The specific rules used were to first try collapsing

categories 1 and 2, 3 and 4, or 5 and 6. If that did not work, categories 1-4 or 5-7 were collapsed. For an

explanation of the rules used, see Westat’s Sampling Activities and Field Operations for 1998 NAEP

(Gray, et al., 2000).
68

4.3.3.2 Minority Classification

Minority classification was created within urbanization strata and was based on a school’s

percentages of Black and Hispanic students. Three different minority classification schemes were used

and are described as follows:

• Case 1: Urbanization strata with less than 10 percent Black students and 7 percent

Hispanic students were not stratified by minority enrollment (Level 0).

• Case 2: Urbanization strata with greater than or equal to 10 percent Black students

or 7 percent Hispanic students, but not more than 20 percent of each, were stratified

by ordering percent minority enrollment (Black plus Hispanic) within the

urbanization classes and dividing the schools into three groups with about equal

numbers of students per minority classification (Levels 1, 2, and 3).

• Case 3: In urbanization strata with greater than 20 percent of both Black and

Hispanic students, minority strata were formed with the objective of providing equal

strata with emphasis on the minority group (Black or Hispanic) of higher

concentration. The stratification was performed as follows. The higher percentage

minority group provided the primary stratification variable; the other group gave the

secondary stratification variable. Within urbanization class, the schools were first

sorted based on the primary stratification variable; then they were divided into two

groups of schools containing approximately equal numbers of students based on

estimated grade enrollment. Within each of these two groups, the schools were

sorted by the secondary stratification variable and subdivided into two subgroups of

schools containing approximately equal numbers of students. As a result, within

urbanization strata there were four minority classifications (e.g., low Black/low

Hispanic, low Black/high Hispanic, high Black/low Hispanic, and high Black/high

Hispanic (Levels 4, 5, 6, and 7).

The minority groups and classifications were formed solely for the purpose of creating efficient

stratification design at this stage of sampling. These classifications are not directly used in analysis and

reporting of the data, but will act to reduce sampling errors for scale score estimates.

4.3.3.3 Median Household Income

The data on median household income was related to the ZIP code area in which the school is

located. The data were derived from the 1990 Census and were obtained from Donnelly Marketing

Information Services.

4.3.3.4 Metro Area Status

All schools in the sampling frame were assigned a metro area status based on their Federal

Information Processing Standard (FIPS) county code and Office of Management and Budget (OMB)

metropolitan area Definitions as of June 30, 1993. This field indicated if a school was located within a

metropolitan area or not.
69

4.3.3.5 School Type for Nonpublic Schools

All nonpublic schools were assigned a school type (Catholic or other nonpublic) based on their

PSS school-type variable.

4.4 SCHOOL SAMPLE SELECTION

When the public and nonpublic schools in the sampling frame were stratified within each

jurisdiction, a sample of about 100 grade-eligible schools was drawn with probability proportional to a

measure of size (PPS) based on the estimated grade-specific enrollment of the school. In practice, the

PPS sampling was implemented by the PPS systematic sampling. The number of schools selected

generally did not vary by the sizes of jurisdictions. In each selected school, students were selected by

systematic sampling. The PPS sampling schools and systematic sampling for students would give each

student an equal probability of selection (Kish, 1965).

One or more sessions were sampled within each school. The number of sessions selected

depended on the school’s estimated grade-specific enrollment, though the overwhelming majority of

schools at grade 4 were allocated a single session.

4.4.1 Measure of Size and Sample Selection

For each grade-eligible school, an estimated grade enrollment (EGE) was obtained by dividing

the school’s total student enrollment by the school’s number of grades. Based on previous assessments,

the EGE provided appropriate estimates for the sampling process. The estimated grade enrollment was

not used directly in sample selection as the measure of size of grade students in schools. Instead, the

measure of size was based on the following function of estimated grade enrollment. Tables 4-3 and 4-4

define the relationship between the estimated grade enrollment and measure of size in sample selection

for grades 4 and 8.

Table 4-3

Estimated Grade Enrollment and Measure of Size, Grade 4

Estimated Grade Enrollment (EGE) Measure of Size

EGE < 10 15

10 ≤ EGE < 20 1.5 × EGE

20 ≤ EGE < 33 30

33 ≤ EGE EGE

Table 4-4

Estimated Grade Enrollment and Measure of Size, Grade 8

Estimated Grade Enrollment Measure of Size

EGE < 10 30

10 ≤ EGE < 20 3 × EGE

20 ≤ EGE < 65 60

65 ≤ EGE EGE
70

Schools were designated as being in “small” or “large” districts and were assigned to one of two

school size classifications. A large district was defined as a district containing 20 percent or more of a

jurisdiction’s student population. All other districts were considered small. Schools were assigned to the

large school size classification if their estimated grade enrollment was greater than 19 students.

Otherwise, schools were assigned to the small school size classification.

A sample of schools was then selected for each jurisdiction with probability proportional to each

school’s measure of size. The sampling frame of schools was sorted in systematic order prior to sample

selection, as follows:

• Public schools

♦ Small or large district status

♦ School size classification

♦ Urbanization stratum

♦ Minority stratum

♦ Median household income

• Nonpublic schools

♦ School size classification

♦ Metro area status

♦ Catholic/nonCatholic

♦ Estimated grade enrollment

Sorting the sampling frame in a specific order prior to systematic sample selection ensures that

the sampled schools represent a variety of population subgroups. Tables B-15 through B-18 in Appendix

B provide the distributions for the counts of selected schools by sampling strata by grades for both public

and nonpublic schools. Tables B-19 through B-22 show weighted school participation rates and counts of

sampled schools by jurisdiction, grade, and subject for both public and nonpublic schools.

4.4.2 Sparse State Sample Option

The standard NAEP sample design requirements are burdensome for jurisdictions whose student

populations are largely concentrated in small schools. In these jurisdictions, large numbers of schools

must be selected in order to reach the required student sample sizes. Thus these jurisdictions bear an

exceptionally large burden in school recruitment and assessment administrations, but are not eligible for

any reduction in sample size under the reduced sample option, which is described in Section 4.5.2. In an

effort to address this problem, while at the same time ensuring that adequate sampling standards for

representation and precision were assured, the sparse state sample option was offered to qualifying

jurisdictions for the first time in 1998. The jurisdictions eligible for this option were those that would

have had at least 120 public schools selected under the full sample. Under the option, a proportional

sample of schools was selected and the school and student sample sizes were reduced such that the

following conditions held:

1. The number of schools selected was at least 115 (noting that many states have been

assigned sample sizes close to this in the past).

2. The number of schools selected for each individual subject was at least 80 (so as to

assure reliable sample inferences can be made for each subject).
71

3. The sampling probability of each individual school was at least half as great as for a

full sample (this is to ensure that all parts of the jurisdiction’s student population are

adequately represented).

4. The largest schools were all retained in the sample, and the student sample sizes in

these schools were also retained.

Note that the third and fourth conditions taken together imply that all of the large schools were

retained and at least half of the small schools were retained. In practice, this usually meant that

jurisdictions had their samples reduced from over 120 schools to 115, since the first condition is usually

the most restrictive. Also, the student sample would be at least a half sample, and usually was

substantially more than that. The eligible jurisdictions were Alaska, Kansas, Montana, Nebraska, North

Dakota, Oklahoma, and South Dakota at grade 4; and Alaska, Montana, Nebraska, North Dakota, South

Dakota, Vermont, and Wyoming at grade 8. The effect of the Sparse State Sample Option on sample

sizes is shown in Table 4-5 for participating jurisdictions exercising the option. Note that Alaska,

Nebraska, and North Dakota at grade 4, and Nebraska and North Dakota at grade 8 also requested the

option, but later decided not to participate (at least in the public-school portion of the assessment).

Table 4-5

The Effect of the Sparse State Option on Sample Sizes, by Grade

for Jurisdictions Exercising the Option

Grade Jurisdiction

Original

School

Sample

Reduced

School

Sample

Reduced Student Sample

as a Percentage of the

Original Student Sample

4 Montana 132 115 88%

8 Montana 139 116 89%

8 Oklahoma 130 115 89%

4.4.3 Control of Overlap of School Samples for National Educational Studies

The issue of school sample overlap has been relevant in all rounds of NAEP in recent years. To

avoid excessive burden on individual schools, NAEP has developed a policy for 1998 of avoiding

overlap between national and state samples. This was to be achieved without unduly distorting the

resulting samples by introducing bias or substantial variance. The procedure used was an extension of the

method proposed by Keyfitz (1951). The general approach is given in the Technical Report of the NAEP

1994 Trial State Assessment Program in Reading (Mazzeo, Allen, & Kline, 1995). It is summarized

briefly as follows.

To control overlap between NAEP state and national samples, a procedure was used that

conditions on the national NAEP PSU sample. This simply means that national school selection

probabilities that were conditional on the selection of national sample PSUs (i.e., within PSU school

selection probabilities) were used in determining state NAEP school selection probabilities. No

adjustments were made to state NAEP school selection probabilities in jurisdictions where there were no

national NAEP PSUs selected. This procedure reduces the variance of the state samples, although it leads

to a greater degree of sample overlap than if unconditional national selection probabilities had been used

in the procedure for controlling overlap between state and national samples. The procedure also

recognizes the impact of the heavy within-PSU sampling in noncertainty PSUs in some jurisdictions.

Schools to be included with certainty in the state sample are not subject to overlap control, as such

schools are self-representing in the state sample. Excluding such schools on a random basis would add
72

extra variance to the state estimates. For actually drawing the state samples, a conditional probability of

selection was used that was conditional on the selection of PSUs for the national NAEP samples. This

procedure in general gave state NAEP conditional selection probabilities that are smaller than the

unconditional state selection probabilities for schools that had been selected for the national sample. The

state NAEP conditional selection probabilities were such that the unconditional probabilities obtained by

integrating over the national sampling process were the required state NAEP probabilities, had overlap

control not been implemented. Thus, a school’s unconditional probability of selection for state NAEP

was the same regardless of whether overlap control had been implemented. Counts of school selection for

both state and national NAEP are found in Table 4-6.

Table 4-6

Number of Schools Selected for Both State and National

NAEP, by Grade and School Type

State NAEP National NAEP Grade

Grade School Type 4 8 12

4 Public 11 4 2

4 Nonpublic 0 18 4

8 Public 6 38 9

8 Nonpublic 15 3 28

4.4.4 Selection of Schools in Small Jurisdictions

All schools in jurisdictions with small numbers of public schools were selected. This was also

true for the nonpublic schools in two jurisdictions. The jurisdictions and grades are shown in Table 4-7.

Table 4-7

Jurisdictions Where All Schools Were Selected, by Grade and School Type

Public Nonpublic

Jurisdiction Grade 4 Grade 8 Grade 4 Grade 8

Delaware * * — —

District of Columbia * * — *

DoDEA/DDESS * * — —

DoDEA/DoDDS * * — —

Hawaii — * — —

Rhode Island — * — —

Virgin Islands ** **

4.4.5 Selection of New Public Schools

A sample of new public schools was drawn to properly reflect additions to the target population

occurring after the sampling frame building information was created. A district-level file was constructed

from the QED school-level file. The district-level file was divided into a “small” districts file that was

not used in the selection of new schools, and a “medium and large” districts file that was used for this

purpose. Small districts consisted of those districts in which there were at most three schools on the

aggregate frame and no more than one fourth-, one eighth-, and one twelfth-grade school. New schools in
73

small districts were identified during school recruitment. The remainder of districts were denoted as

“medium and large” districts.

A sample of medium and large public-school districts was drawn in each jurisdiction. All

districts were selected in Delaware, the District of Columbia, Hawaii, and Rhode Island. The remaining

jurisdictions in the file of medium and large districts (eligible for sampling) were divided into two files

within each district. Two districts were selected per jurisdiction with equal probability among the smaller

districts with combined enrollment of less than or equal to 20 percent of the state enrollment in the

medium and large districts file. From the rest of the file, eight districts were selected per jurisdiction with

probability proportional to enrollment. The breakdown given above applied to all jurisdictions that had at

least eight large districts. In jurisdictions with fewer than 8 large districts, all of the large districts were

selected, and then enough small districts were selected to make 10 districts selected altogether. The 10

selected districts in each jurisdiction were then sent a listing of all their schools that appeared on the file,

and were asked to provide information about the new schools not included in the file. These listings,

provided by selected districts, were used as sampling frames for selection of new public schools.

The eligibility of a school was determined based on the grade span. A school was also classified

as “new” if a change of grade span was such that the school status changed from ineligible to eligible.

The average grade enrollment for these schools was set to the average grade enrollment before the gradespan change. The schools found eligible for sampling due to the grade-span change were added to the

new school selection frame.

The probability of selecting a school was

sampling rate measure of size minimum , 1 P(district)

  ⋅    

,

where P(district) was the probability of selection of a district and the sampling rate was the rate used for

the particular jurisdiction in the selection of the original sample of schools. For example, in a state where

the sampling rate is .005, a school with 100 eligible students in a district selected with probability .75

would have a probability of selection of .67 [(.005 x 100)/.75].

In each jurisdiction, the sampling rate used for the main sample of grade-eligible schools was

used to select the new schools. Additionally, all new eligible schools coming from small districts (those

with at most one grade 4 and one grade 8 school and at most three schools on the aggregate frame) that

had a school selected in the regular sample for the fourth grade were included in the sample with

certainty. In the 1998 state assessment, there were no such schools.

Table 4-8 shows the number of new schools coming from the medium and large and small

districts for the fourth- and eighth-grade samples.
74

Table 4-8

NAEP 1998 Distribution of New Schools Coming from

Districts Designated as “Medium” or “Large”*

Jurisdiction Grade 4 Samples Grade 8 Samples

Total 70 49

Alabama 2 3

Alaska — 0

Arizona 5 5

Arkansas 0 1

California 1 1

Colorado 3 2

Connecticut 3 0

Delaware 13 2

District of Columbia 1 5

Florida 0 0

Georgia 0 0

Guam — 0

Hawaii 2 4

Illinois 0 —

Indiana — 0

Iowa 1 0

Kansas 0 —

Kentucky 1 0

Louisiana 4 4

Maine 3 2

Maryland 0 2

Massachusetts 6 1

Michigan 1 0

Minnesota 1 0

Mississippi 0 0

Missouri 2 2

Montana 0 0

Nebraska 4 2

Nevada 6 1

New Hampshire 0 0

New Jersey — 0

New Mexico 1 1

New York 1 3

North Carolina 1 2

North Dakota — 0

Oklahoma 0 —

Oregon 0 0

Rhode Island 0 0

South Carolina 2 0

Tennessee 0 0

* In the 1998 assessment, there were no sampled schools designated “small”.

(continued)
75

Table 4-8 (continued)

NAEP 1998 Distribution of New Schools Coming from

Districts Designated as “Medium” or “Large”*

Jurisdiction Grade 4 Samples Grade 8 Samples

Texas 1 3

Utah 1 0

Vermont — 0

Virgin Islands 1 —

Virginia 0 0

Washington 0 0

West Virginia 0 0

Wisconsin 0 1

Wyoming 0 2

DoDEA/DDESS 2 0

DoDEA/DoDDS 1 0

* In the 1998 assessment, there were no sampled schools designated “small”.

4.4.6 Assigning Subject, Sample Type, and Monitor Status

For the sampled schools, one or more subject sessions were assigned within each school. The

number of sessions selected depended on the school’s estimated grade-specific enrollment, though the

overwhelming majority of schools at grade 4 were allocated a single session.

Rules for assigning subjects (reading at grades 4 and 8; writing at grade 8 only) varied by grade.

All fourth-grade schools were assigned to participate in reading assessments. All eighth-grade schools

with 25 or more students were assigned to participate in both reading and writing assessments. Schools

with fewer than 25 students were assigned one randomly selected subject.

The 1998 state assessment used the inclusion rules from 1996 for SD/LEP students (see Chapter

3) for two different sets of schools (S2 and S3 subsamples). The S2 subsample was not given the option

of taking the assessment with accommodations. The S3 subsample was given the option of offering

SD/LEP students accommodations. A sample type variable was created to reflect which set of rules to

use within a given school. The sample type variable applied to reading only because writing was always

administered using S3 rules including accommodations.

The schools assigned reading were sorted by stratum (public and nonpublic) and school ID and

then assigned sample type in an alternating pattern within the sorted list. The inclusion rules for SD/LEP

students are described in Chapter 3.

Since the state assessments were given by local administration, Westat monitored field

assessments in some of the schools in the state assessments as they did in the national assessments to

make reliable comparisons between both assessments. Jurisdictions received 25 or 50 percent monitoring

of sessions depending on previous participation in the state assessments. All jurisdictions received 25

percent monitoring except Kansas, where 50 percent monitoring was used. The sampled schools were

sorted by stratum, subject, sample type, and school ID and then assigned the two levels of monitoring in

an alternating pattern.
76

4.4.7 School Substitution and Retrofitting

A substitute school was assigned to each sampled school (to the extent possible) prior to the field

period through an automated substitute selection mechanism that used distance measures as the matching

criterion. Schools were also required to be of the same type (i.e., public, nonpublic, BIA, and DoDEA

schools were only allowed to substitute for each other), and substitutes for nonpublic, BIA, and DoDEA

schools were required to come from within the same district. Public-school substitutes were required to

come from different districts. Two passes were made at the substitution, with the second pass raising the

maximum distance measure allowed and removing the different district assignment requirement for

public schools. This strategy was motivated from the fact that most public-school nonresponse occurs at

the school district level.

A distance measure was used in each pass and was calculated between each sampled school and

each potential substitute. The distance measure was equal to the sum of four squared standardized

differences. The differences were calculated between the sampled and potential substitute school’s

estimated grade enrollment, median household income, percent Black enrollment and percent Hispanic

enrollment. Each difference was squared and standardized to the population standard deviation of the

component variable (e.g., estimated grade enrollment) across all grade-eligible schools and jurisdictions.

The potential substitutes were then assigned to sampled schools by order of increasing distance measure.

An acceptance limit was put on the distance measure of .60 for the first pass. A given potential substitute

was assigned to one and only one sampled school. Some sampled schools did not receive assigned

substitutes (at least in the first pass) because the number of potential substitutes was less than the number

of sampled schools or the distance measure for all remaining potential substitutes from different districts

was greater than .60.

In the second pass, the different district constraint for public schools was lifted and the maximum

distance allowed was raised to .75. This generally brought in a small number of additional assigned

substitutes. Although the selected cutoff points of .60 and .75 on the distance measure were somewhat

arbitrary, they have been used since 1994 after being decided upon for the 1994 trial state assessment by

a group of statisticians reviewing a large number of listings beforehand and finding a consensus on the

distance measures at which substitutes began to appear unacceptable.

Jurisdictions that did not receive substitutes for all selected schools were allowed to retrofit

unused substitutes after part of the field period elapsed. Substitutes that were assigned to cooperating or

ineligible original selections were free to be assigned to other original selections that did not receive

substitutes. These free substitutes were put back into the substitute selection mechanism described above

and allowed to pair up with other original selections.

The information about the number of substitutes provided and the number participating in each

jurisdiction can be found in the report Sampling Activities and Field Operations for 1998 NAEP (Gray, et

al., 2000). Of the 45 participating jurisdictions, 42 were provided with at least one substitute at grade 4,

and 41 were provided with at least one substitute at grade 8. Among jurisdictions receiving no

substitutes, the majority had 100 percent participation from the original sample. The total number of

substitutes associated with nonparticipating original schools were 524, 600, and 400 for grade 4 reading,

grade 8 reading, and grade 8 writing, respectively. The numbers of substitutes that participated were 153,

93, and 97, respectively.
77

4.5 STUDENT SAMPLE SELECTION

4.5.1 Student Sampling and Participation

To select a student sample, schools initially sent a complete list of students to a central location

in November 1997. They were not asked to list students in any particular order, but were asked to

implement checks to ensure that all grade-eligible students were listed. Based on the total number of

students on this list, the student listing form, sample line numbers were generated for student sample

selection. To generate these line numbers, the sampler entered the number of students on the form and

the number of sessions into a personal computer that had been programmed with the sampling algorithm.

The program generated a random start that was used to systematically select the student line numbers (30

per session). To compensate for new enrollees not on the student listing form, extra line numbers were

generated for a supplemental sample of new students.

After the student sample was selected, the administrator at each school identified students who

were incapable of taking the assessment either because they were identified as students with disabilities

(SD) or because they were classified as being of limited English proficiency (LEP). New inclusion rules,

which were first used in 1996, were used. These rules were meant to clarify the procedure for identifying

whom to exclude from NAEP and to provide wider inclusion of SD and LEP students. More details on

the procedures for student exclusion are presented in Chapter 5 of this report and in Westat’s Sampling

Activities and Field Operations for 1998 NAEP (Gray, et al., 2000).

When the assessment was conducted in a given school, a count was made of the number of

nonexcluded students who did not attend the session. If this number exceeded three students, to reduce

nonresponse error, the school was instructed to conduct a makeup session, to which all students who

were absent from the initial session were invited. A summary of the distribution of the student samples,

student exclusion rates, and response rates by grade, school type, and jurisdiction can be found in Tables

B-23 to B-28 in Appendix B.

4.5.2 The Reduced Sample Option

Jurisdictions with fewer than 100 schools, and schools assigned more than two sessions at grade

4 or more than three sessions at grade 8 were given the option to reduce the expected student sample size

in order to reduce testing burden and the number of multiple-testing sessions for participating schools. If

jurisdictions chose to exercise this option, the estimates obtained from the assessment were more variable

than they otherwise would have been. In general, jurisdictions could reduce student sample sizes by

adjusting the number of sessions with participating schools subject to the following constraints:

• The minimum number of sessions per school had to be equal to 1.

• The maximum number of sessions per school had to be equal to 2 at the fourth grade

and 3 at the eighth grade.

• The expected student size from the reduced sample was greater than or equal to half

of the original student sample size.

To reduce testing burden and the number of testing sessions for participating schools, Delaware

exercised the reduced sample option at both grade levels.

79

Chapter 5

FIELD OPERATIONS AND DATA COLLECTION1

Lucy M. Gray, Mark M. Waksberg, and Nancy W. Caldwell

Westat

5.1 INTRODUCTION

This chapter describes the field operations and data collection activities for the 1998 National

Assessment of Educational Progress (NAEP). Traditionally, NAEP is comprised of main national

samples, long-term trend (LTT) national samples, and state samples. For 1998, LTT was not scheduled,

however, so the 1998 assessment program consisted of main, national, and state samples, as described in

this chapter. The national NAEP component typically involves new assessment items, and may include

new subject areas and innovative features. The national assessments are based on national probability

samples of schools and students that allow for regional and national reporting only. The state assessment,

the other major component of NAEP for 1998, comprises the state program that uses national NAEP

assessment materials and involves much larger sample sizes per state (or jurisdiction), so that results can

be reported for each participating state or jurisdiction.

The organization and operation of 1998 NAEP field activities are described in the remaining

sections of this chapter. For all components, NAEP guarantees the anonymity of participants, and student

or teacher names are never recorded on assessment booklets nor removed from the schools. NAEP results

are reported on the national level, by region of the country, by state, or by demographic subgroup.

5.1.1 Organization of the National Assessment for 1998

The 1998 national assessment was conducted in a sample of approximately 2,700 public and

nonpublic schools located in 94 geographic areas called primary sampling units (PSUs) throughout the

states and the District of Columbia. The PSUs were selected by Westat to represent the nation as a whole.

 Assessments for national NAEP were conducted from January through March at grades 4, 8, and

12. Students were assessed in reading, writing, and civics, and this included a special assessment in

civics only, which established a trend line (but not long-term trend) from the earlier civics assessment in

1988. The civics special trend assessment was conducted at the same time and in some of the same

schools as national NAEP. Three session types were administered in 1998:

• Reading: The reading assessment was based on the existing frameworks, which

established a new trend line in 1992 (NAGB, 1990). The reading booklets

included the background questions in the front of the booklet.

 

1 Lucy M. Gray and Mark M. Waksberg develop survey operations and procedures and monitor field activities for the NAEP

assessments under the direction of Nancy W. Caldwell.
80

• Writing/Civics: The writing and civics assessments were combined into one

session, with the different booklets spiraled together. These assessments were

based on new frameworks developed for the 1997 field test (Center for the

Evaluation, Standards, and Student Testing [CRESST], 1996; Council of Chief

State School Officers [CCSSO], 1996)

• Civics Special Trend: The civics special trend study was based on the

frameworks developed for the 1988 assessment (CCSSO, 1996), and was distinct

from the civics assessment included with the writing tests. These sessions used

the same materials used in 1988, including an answer sheet separate from the test

booklets.

• Most schools had two of the possible three types of sessions administered in

1998 (reading, writing/civics, and/or civics special trend). In some of the

smallest schools, only one of the types of sessions was administered. Following

the precedent established in 1996, accommodations (described in Section

5.1.1.2) were offered for the writing/civics sessions and for half of the reading

sessions, but none for the civics special trend.

In order to reduce the burden on the participating schools, NAEP field staff performed most of

the work associated with the assessments. Introductory contacts and meetings (if needed) occurred in the

fall of 1997 to enlist cooperation and explain the assessment procedures to district and school

representatives and to set a mutually agreed-upon assessment date for each school. The assessment

supervisor visited the school a week or two before the assessment to select the sample of students. The

assessment sessions were conducted by exercise administrators, also members of the NAEP field staff,

under the direction of the assessment supervisor. At the conclusion of the assessment in a school, field

staff coded demographic information on the booklet covers and shipped the completed materials to

National Computer Systems (NCS), the processing subcontractor for NAEP (see Chapter 6 for more

detailed information on processing assessment materials). For reference, the national NAEP field staff

administrative structure is summarized in the chart below.

WESTAT NATIONAL NAEP FIELD STAFF ADMINISTRATIVE STRUCTURE

Field Director

Oversees all aspects of field operations

Field Managers

Report to Westat Field Director and oversee supervisors who have direct contact with schools

Field Supervisors

Report to a specific field manager, gain cooperation of schools, select student samples, arrange

and supervise assessments, assigning assessments to exercise administration

Exercise Administrators

Conduct assessment sessions and assist with field paperwork/record keeping under direct

supervision of a field supervisor
81

5.1.1.1 Additional Special Studies

Apart from the civics special trend study, two other special studies, each requiring additional

interaction with school personnel, were carried out in conjunction with the national 1998 assessment. A

classroom-based writing study was designed to explore methods of assessing students’ writing abilities at

grades 4 and 8 by using written assignments that students had completed as part of their school

curriculum. A High-School Transcript Study, similar to the transcript study that took place in 1994, was

conducted in a number of grade 12 schools included in the main assessment.

These results from these two studies will be available in forthcoming reports. More information

about the studies is provided in section 5.3.2.

5.1.1.2 Exclusions and Accommodations for Students

Historically, a small proportion (less than 10%) of the sampled students have been “excluded”

from NAEP assessment sessions because, according to school records, they are students with either

disabilities (SD) or limited English language proficiency (LEP) who have been determined to be

incapable of participating meaningfully in the assessment. More recently, especially with the passage of

the Individuals with Disabilities Education Act, increased attention has been given to these students and

to including as many of them as possible in NAEP sessions. NAEP addressed these concerns through a

1996 special study (Mazzeo, Carlson, Voelkl, & Lutkus, 1999) that used both old and new “inclusion”

criteria and (in some schools) offered accommodations for testing students with disabilities, limited

English proficiency, or both (SD/LEP).

Results of the 1996 assessment indicated that the revision of the criteria for including students

had little impact on the numbers of students included; therefore, for 1998 and beyond, the revised criteria

were used because they are most current. The 1996 data also indicated that providing accommodations

resulted in greater inclusion of students who might previously have been excluded from NAEP.

The inclusion criteria used in the 1998 NAEP assessments fell into two categories—students

with disabilities (SD) and students with limited English proficiency (LEP). A student identified as having

a disability (SD), that is, a student with an Individualized Education Plan (IEP) or equivalent

classification, was to be excluded from the NAEP assessment if any of the three following conditions

applied:

• The IEP team or equivalent group determined that the student was unable to

participate in assessments such as NAEP.

• The student’s cognitive functioning was so severely impaired that he or she could

not participate.

• The student’s IEP required that the student be tested with an accommodation that is

not permitted by NAEP, and the student could not demonstrate his or her proficiency

in reading, writing, or civics without that accommodation.

A student who was identified as limited English proficient (LEP) and was a native speaker of a language

other than English was to be excluded from the NAEP assessment only if both of the following

conditions applied:

• The student received language arts instruction primarily in English for less than

three school years including the current year.
82

• The student was unable to demonstrate his or her proficiency in reading, writing, or

civics, even with an accommodation permitted by NAEP.

Decisions on exclusion were made by the assessment supervisor in consultation with school staff

and were guided by the SD/LEP questionnaires completed by the school staff. This questionnaire, which

was completed for each SD/LEP student in the sample by someone at the school knowledgeable about

the student, asked about the student’s background and the special programs in which the student

participated.

Because the 1998 reading assessment results were to be compared to those from the 1992

assessment, one group of students was assessed under conditions similar to those in 1992. Thus, in half

of the 1998 reading sessions, accommodations were not permitted. To be able to evaluate the differences

in results that occur when students are assessed with accommodations, accommodations were permitted

in the other half of the reading sessions.

For the writing/civics sessions, because new trend lines are being established, accommodations

were made available to all students, if needed or appropriate. Finally, for civics special trend sessions,

accommodations were not permitted for any students.

Accommodations included but were not limited to extended time to answer the test questions,

large-print booklets, bilingual dictionaries, scribe or use of computer to record answers, session in which

the test administrator would read the test questions aloud, sessions with a smaller number of students

than in the regular sessions, and one-on-one test administrations.

5.1.2 Organization of the State Assessment for 1998

Forty-four states, the District of Columbia, Virgin Islands, and Guam volunteered for the 1998

state assessment, as did the Department of Defense Domestic Dependent Elementary and Secondary

Schools (DoDEA/DDESS) and the Department of Defense Dependents Schools (DoDEA/DoDDS).

Table 5-1 identifies the jurisdictions participating in the state assessment. For the state program,

assessments were conducted in one subject, reading, at the fourth grade and in reading and writing at the

eighth grade.

Data collection for the 1998 state assessment involved a collaborative effort between the

participating jurisdictions and the NAEP contractors, especially Westat, the field administration

contractor. Westat’s responsibilities included:

• Selecting the sample of schools and students for each participating jurisdiction

• Developing the administration procedures and manuals

• Training state and school personnel to conduct the assessments, and

• Conducting an extensive quality assurance program which involves observing

and monitoring 25 percent of the state NAEP sessions conducted by school staff.
83

Table 5-1

Jurisdictions Participating in the 1998 State Assessment Program

Alabama Guam Missouri South Carolina

Alaska Hawaii Montana Tennessee

Arizona Illinois2 Nebraska Texas

Arkansas Indiana Nevada Utah

California Iowa New Hampshire Vermont

Colorado Kentucky New Jersey Virginia

Connecticut Louisiana New Mexico Washington

Delaware Maine New York West Virginia

DoDEA/DDESS1 Maryland North Carolina Wisconsin

DoDEA/DoDDS1 Massachusetts North Dakota Wyoming

District of Columbia Michigan Oregon

Florida Minnesota Pennsylvania

Georgia Mississippi Rhode Island

1 DoDEA refers to the Department of Defense Education Activity. Its domestic schools (Department of

Defense Domestic Dependent Elementary and Secondary Schools [DDESS]) and its overseas schools

(Department of Defense Dependents Schools [DoDDS]) participated in the state assessment program.

2 Illinois participated in the assessment; however, results were not reported due to low school participation

rates prior to the addition of substitute schools.

Each jurisdiction volunteering to participate in the 1998 program was asked to appoint a state

coordinator. In general, the coordinator was the liaison between NAEP/Westat staff and the participating

schools. In particular, the state coordinator was asked to:

• Gain the cooperation of the selected schools

• Assist in the development of the assessment schedule in the selected schools

• Receive the lists of all grade-eligible students from the schools

• Coordinate the flow of information between the schools and NAEP

• Provide space for the Westat state supervisor to use when selecting the samples

of students

• Notify assessment administrators about training and send them their assessment

manuals, and

• Send the lists of sampled students to the schools.

Westat hired and trained six field managers for the state assessment. Each field manager was

responsible for working with the state coordinators of seven to eight jurisdictions and for overseeing

assessment activities. The primary tasks of the field managers were to:

• Obtain information from state coordinators about cooperation and scheduling

• Make sure the arrangements for the assessments were set and assessment

administrators identified, and

• Schedule the assessment administrator training sessions.
84

Westat also hired and trained a state supervisor for each jurisdiction. The 1998 state assessment

involved about the same number of state supervisors (Westat staff) as the 1992, 1994, and 1996

assessments, since approximately the same number of jurisdictions were involved each year. In addition,

three troubleshooters were trained in case any state supervisor was unable to complete their assignment.

The primary tasks of the state supervisor were to:

• Select the samples of students to be assessed

• Recruit and hire the quality control monitors throughout their jurisdiction

• Conduct in-person assessment administration training sessions, and

• Coordinate the monitoring of the assessment sessions and makeup sessions.

At the school level, an assessment administrator(s) was appointed (by the school), and this

person, often a teacher, was responsible for preparing for and conducting the assessment session(s) in

one or more schools. These individuals were usually school or district staff and were trained by Westat

staff. The assessment administrator’s responsibilities included:

• Receiving the list of sampled students from the state coordinator

• Identifying sampled students who should be excluded

• Distributing assessment questionnaires to appropriate school staff and collecting

them upon their completion

• Notifying sampled students and their teachers

• Administering the assessment session(s)

• Completing assessment forms, and

• Preparing and shipping the completed assessment materials.

• Decisions on exclusion of students (if any) were made in consultation with

school staff and were guided by the SD/LEP questionnaires completed by the

school staff.

In addition, Westat hired several quality control (QC) monitors in each jurisdiction to monitor

assessment sessions. The number of QC monitors varies, from about 4 to 6, by state according to the

number of schools samples in a state. The QC monitors report to Westat supervisors and are responsible

for observing a subset of the state NAEP sessions conducted by the school staff. For reference, the state

NAEP field staff administrative structure is summarized in the following chart.
85

WESTAT STATE NAEP FIELD STAFF ADMINISTRATIVE STRUCTURE

Field Director

Oversees all aspects of field operations

Field Managers

Work directly with state coordinators on gaining cooperation of schools and oversee state

supervisors (Westat staff) who select student samples and supervise QC monitors

Field Supervisors

Select student samples at state coordinators office, train assessment administrators (chosen by

schools) to conduct assessments, schedule and oversee assessment observation visits made by

quality control monitors

Assessment Administrators

Are school (or district) staff appointed by the school to conduct one or more state NAEP

assessment sessions in that school

Quality Control Monitors

Are hired and trained by Westat field managers and field supervisors, interview each school for

feedback on the assessment and to visit a specific subsample of schools to observe the

administration of the NAEP session by school staff; report directly to field supervisor

5.2 PREPARING FOR THE ASSESSMENTS

5.2.1 Gaining the Cooperation of Sampled Schools

The process of gaining cooperation of the schools selected for the NAEP assessments, both

national and state, began in August 1997 with a series of letters and contacts with state and district-level

officials. The National Center for Education Statistics (NCES) first sent each jurisdiction a letter

announcing NAEP plans for 1998. Westat then contacted the state test directors or NAEP state

coordinators in each sampled state to notify them of the districts and schools selected in their states. In

the 41 jurisdictions participating in the state assessment that also had schools sampled for the national

assessment, the state received the list of districts and schools sampled for both the national and state

assessments.

From September through early December 1997, Westat sent lists of schools sampled for the

assessments and other NAEP materials to district superintendents, diocesan superintendents of Catholic

schools, and principals or heads of schools in other nonpublic schools, inviting their participation. These

initial mailings paved the way for telephone contacts by NAEP field supervisors who were assigned the

task of gaining cooperation and scheduling assessment dates.
86

The schedule for project activities for the 1998 national and state assessments was as follows:

August 1997 Department of Education sent first letter to chief state

school officers and state test directors.

Westat sends state coordinators the lists of schools

selected for 1998 state assessments along with

informational materials. Similar mailings continue, to

state test directors, through mid-September 1997 for

national NAEP schools.

August/September 1997 Westat field managers visit states to train state

coordinators to use computerized state NAEP field

management system for recording participation status

of the state NAEP schools.

September 24–27, 1997 Training session held for national assessment

schedulers.

Mid-to-Late September 1997 Westat sent samples and informational materials to

school districts, if not already sent by state

coordinators.

Mid-September – December 1, 1997 Supervisors contacted districts and schools to secure

cooperation and to schedule assessments in national

NAEP schools.

Supervisors conducted introductory meetings for the

national NAEP assessment, by telephone (or in person

if requested by districts or schools). Westat selected

substitutes for refusals.

Supervisors recruited, hired, and trained exercise

administrators for national NAEP.

September – November 1997 State coordinators obtained cooperation from districts

and public schools for state NAEP samples. State

coordinators reported participation status to Westat

field managers via hardcopy lists or computer files.

Westat field staff secured cooperation from sampled

nonpublic schools (for state NAEP samples).

State coordinators sent summary of school tasks,

student listing forms, and new enrollee student listing

forms to participating public schools in state NAEP

samples.

October 6 – November 12, 1997 Westat sent student listing forms and new enrollee

listing forms to participating nonpublic schools in

state NAEP samples.
87

November 5 – 8, 1997 Training session for state NAEP supervisors.

Early December 1997 Supervisors sent informational materials to principals

and school coordinators and Westat send letters

confirming assessment schedules to each national

NAEP school.

December 1 – 12, 1997 State NAEP supervisors visited state coordinator

offices to select student samples and prepare

administration schedules listing the students selected

for each session in public schools selected for state

NAEP. The state supervisor prepared a package to be

sent to each public school containing the

administration schedules and the instructions for

assessing students with disabilities and/or limited

English proficiency.

December 1 – 5, 1997 Westat provided schedule of state NAEP assessment

administrator (AA) training sessions and copies of the

Manual for Assessment Administrators to state

coordinators for distribution.

Westat distributed state NAEP AA training schedules

and manuals directly to nonpublic schools.

December 8, 1997 – January 2, 1998 State coordinator notified state NAEP AAs of the date

and time of training and sent each a copy of the

Manual for Assessment Administrators.

December 9 – 15, 1997 National NAEP assessment supervisor training session

was held.

January 5 – March 27, 1998 Student samples were selected for national NAEP and

assessments were administered. Makeup sessions, if

needed, were held from March 30 to April 3, 1998.

January 7 – 10, 1998 Training session was conducted for quality control

monitors (see Section 5.4.2) who observe state NAEP

AAs in 25% of state NAEP sessions.

January 12 – 30, 1998 Westat state NAEP supervisors conducted assessment

administrator training sessions.

Student samples were selected for nonpublic schools in

state NAEP training sessions for state NAEP AAs.
88

January 19 – February 13, 1998 State coordinators sent packages containing

administration schedules and instructions for

assessing students with disabilities and/or limited

English proficiency to each public school two weeks

before the scheduled assessment date for state NAEP.

NCS sent assessment materials to each school two

weeks before the scheduled assessment date for state

NAEP.

February 2 – 27, 1998 State NAEP assessments were conducted and

monitored, with makeup sessions held the week of

March 2–6, 1998.

5.2.2 Supervisor Training

Training for assessment supervisors was multiphased and involved separate sessions conducted

in August, September, and December 1997. In addition, a large state NAEP training session for quality

control monitors was held in early January 1998. All training was conducted by the Westat project

director, field director, and home office staff. Also in attendance were representatives from Educational

Testing Service (ETS), NCS, and NCES.

The first training session was held September 24 – 27, 1997 for 40 field staff assigned to gaining

cooperation phase of the project. After an introduction to the study, which included the background and

history of NAEP, an overview of the 1998 assessments, and the 1997–1998 assessment schedule, the

training continued with a thorough presentation of NAEP’s activities for contacting schools and gaining

their cooperation. This is a lengthy process of contacting states, districts, and schools regarding their

participation in and scheduling for NAEP; several demonstration phone calls, role plays, and exercises

were used to provide some practical experience during this part of the training. Other training topics

included: supervisory responsibilities, setting the assessment schedule, recruiting and training exercise

administrators, and administrative forms and procedures. The scheduling supervisors also received a full

day of training on using the reporting system installed on the laptop computers assigned to each of them

for the gaining cooperation and scheduling phase. The reporting system is Westat's computerized field

system used throughout national NAEP to record and update the participation status of each school and

the attendance at each assessment session.

The 75 NAEP supervisors who were responsible for national NAEP assessment activities were

trained again, in a second session, held December 9–15, 1997. The training began with a review of the

preliminary activities during the fall, including results of gaining cooperation with districts and schools,

scheduling of assessments, and the status of exercise administrator (EA) recruitment. (The role of EAs

who conduct the assessments is discussed in Section 5.2.4.) The main focus of the training was a

thorough discussion of assessment activities: sampling procedures, inclusion of SD/LEP students, teacher

surveys, providing testing accommodations, conducting the sessions, and administrative forms and

procedures. Westat’s classroom management videotape, which is a 40-minute presentation on student

behavior/attitudes and suggested approaches to "handling" students at various grade levels, was also

shown at this training session. Key portions of the December training were devoted to carefully

presenting the procedures involved in each of the two special studies, and each of these studies required a

full day of training. These special studies, High School Transcript and Classroom-Based Writing, were

initiated during the sampling visit to each school and continued on the assessment day, with certain
89

follow-up activities performed after the assessments. A full day of training on Westat’s computerized

NAEP field reporting system was also offered at the December training session.

The national NAEP and state assessment field managers were present at the December session to

support training activities and answer questions from supervisors (who work under the field managers)

concerning districts and schools that fell into the samples for more than one component of the

assessment. Each supervisor also met with the person who completed the scheduling in their area, as a

first step in preparing for the new supervisors’ contacts with each school (and district, if needed).

The state NAEP supervisors attended a training session held November 5–8, 1997. This training

session focused on the state supervisors’ immediate tasks—selecting the student samples and hiring

quality control monitors. Supervisors were given the training script and materials for the assessment

administrators’ training sessions they would conduct in January so they could become familiar with these

materials.

Approximately 400 quality control monitors were trained for state NAEP in a session held in

early January 1998. The first day of the training session was devoted to a presentation of the assessment

administrators’ training program by the state supervisors, which not only gave the monitors an

understanding of what assessment administrators were expected to do, but gave state supervisors an

opportunity to practice presenting the training program. The remaining days of the training session were

spent reviewing the quality control monitor observation form and the role and responsibilities of the

quality control monitors.

5.2.3 Contacting Districts and Nonpublic Schools

Once the supervisors were trained in September 1997, they began working on obtaining

cooperation for national NAEP. In the states both sampled for national NAEP and participating in the

state assessment, the national NAEP supervisor first spoke with the state NAEP field manager to

determine what contacts, if any, had already been made with districts about NAEP. The approach the

supervisors took when calling superintendents depended on whether the district had been notified about

national NAEP by the state coordinator and whether the district also had schools selected for the state

assessment. For districts that had been contacted by the state coordinator, the supervisor began by

referring to that contact. Gaining specific cooperation in "state NAEP" schools was the responsibility of

the state coordinators, while the Westat supervisors gained cooperation from all other schools, that is, the

national NAEP schools and the nonpublic schools in state NAEP.

In previous national assessments, the supervisors offered and usually held “introductory

meetings” with representatives from the superintendents’ offices and the selected schools, typically the

superintendent and the principals. These served as both an introduction to NAEP and a presentation on

what would be asked of the school. The meetings were also used to establish a schedule for the sampling

visits and the assessments in the schools.

Over the years, however, these meetings have become somewhat redundant, since many districts

have fallen into the national sample more than one time. It has also become more and more difficult to

schedule these meetings, as district and school officials find it harder to allot time away from their

offices. Thus, during the fall preparations for both the 1996 and 1998 NAEP studies, the material was

almost always presented to the superintendents and principals during telephone calls rather than in formal

meetings. Generally, an in-person meeting was held only if specifically requested by the district or school

officials, or if the supervisor felt that such a meeting would provide a better chance for convincing a

district to participate.
90

As the supervisors contacted superintendents, principals, and nonpublic-school officials to

introduce NAEP and determine the schools’ cooperation status, they completed two forms and entered

the school status in the receipt control system installed on their laptop computers. The results of contact

form was completed to document the discussion the supervisor had with each administrator concerning

the district’s willingness to participate and any special circumstances regarding the schools’ cooperation

or assessments.

The supervisor also completed portions of a school control form. This form was preprinted with

the number and types of national assessment sessions assigned to the school, so that this information

could then be shared with district and school officials. Information gathered during the phone call,

including the name of the person designated to be the school coordinator, the number of students in the

designated grade, tentative dates for the sampling visit and assessment, and other information that could

have some bearing on the assessment, was recorded on the form. This information was used to update

records in the home office. In December, the forms were provided to the supervisors who would be

conducting the assessments.

A small number of in-person introductory meetings were held. The New York City and Los

Angeles City school districts have previously used these meetings to present information about the

national NAEP assessments to the officials of all the selected schools and to encourage their

participation, and wished to continue that practice for the current assessment. A small number of other

school districts also requested such a meeting, involving representatives from their selected schools so

that they would have a full understanding of what the assessments entailed.

During the telephone presentation or the introductory meeting, the supervisor discussed

arrangements for the national assessments with representatives from each school. Within the weeks

scheduled for the PSU, the supervisor had the flexibility to set each school’s assessment date in

coordination with school staff. The staff sometimes expressed preferences for a particular day or dates or

had particular times when the assessment could not be scheduled. Their preferences or restrictions

depended on the events that had already been scheduled on their school calendar. Using this information

from the schools, the supervisors set up the assessment schedule for each PSU.

The supervisor usually learned during the introductory contact whether a school required some

form of parental notification or permission. Three versions of standard NAEP letters were offered for the

school’s use, and each letter could be produced for selected students only or for all eligible students. The

first version informs parents about the assessment. The second assumes parental consent unless parents

send the form back stating that they do not want their child to participate in the assessment. The third

version requires that parents sign and return the form before students can be assessed. All versions of the

letter were available to the schools, although when the issue of parental permission came up in

discussion, supervisors offered the least restrictive version that met the requirements of the school or

district. In addition, Spanish language versions of the parent information letter were made available to the

schools. Schools could also send out their own letters and notices if they preferred not to use those

offered through NAEP. Information on whether the school required parent letters and the type of letter

used was recorded on the school control form.

5.2.4 Recruiting, Hiring, and Training Exercise Administrators

During the fall, while the supervisors were contacting schools and scheduling assessments, their

other major responsibility was to recruit and hire exercise administrators, who would administer the

assessment sessions for national NAEP (for state NAEP, the school or district provides the assessment

staff, known as assessment administrators). Exercise administrators for national NAEP were recruited

from many sources. Each supervisor was given a PSU-by-PSU computerized list of exercise
91

administrators and other field staff who had worked previously on education studies for Westat. People

who had served as exercise administrators before, with good evaluations from their previous supervisors,

were usually the first considered for hiring. Subsequently, during contacts with the schools, the

supervisors asked the school principals and other staff to recommend potential exercise administrators.

These referrals were frequently retired teachers or substitutes. Finally, where necessary, ads were placed

in local newspapers and the employment service was notified.

Supervisors were told that, in general, four to five exercise administrators should be hired for

each PSU, although a variety of factors might influence the actual number. The number of schools in a

PSU, the size of the student sample in each school, distances to be traveled, the geography of the area,

and weather conditions during the assessment period were all factors taken into consideration by

supervisors in developing their plan for hiring exercise administrators.

A few supervisors, whose NAEP assignments contained contiguous PSUs, hired the same

exercise administrators to work in all their PSUs. Other supervisors, whose assignments comprised PSUs

that were not geographically connected, tended to hire teams of exercise administrators for each PSU.

Supervisors were encouraged to hire locally and to hire individuals with teaching experience and the

ability to handle classroom situations.

The scheduling supervisors, all of whom were experienced NAEP supervisors, had complete

responsibility for recruiting, hiring, and training all of the exercise administrators, including ones who

would report to different assessment supervisors. The training was standardized so that all supervisors

used a prepared script and exercises to train the exercise administrators.

Each exercise administrator received an exercise administrator manual, which covered the full

range of their job responsibilities. After studying the manual, they attended a half-day training session.

During the training, the supervisor reviewed all aspects of the exercise administrators’ job, including

preparing materials, booklets, and administration schedules for assessments; the actual conduct of the

session; post-assessment collection of materials; coding booklet covers; recordkeeping; and

administrative matters. In January 1998, each exercise administrator attended a shorter, refresher training

session, conducted by the assessment supervisor, to gain further experience with the specific procedures

and materials to be used in the assessment sessions.

For state NAEP, assessment administrators (AAs), rather than exercise administrators, conducted

the NAEP sessions in each school. These persons were appointed by the school (or the district), usually

from school staff, at the request of the state coordinator who gained cooperation and established the

assessment arrangements for state NAEP schools. All of these arrangements were made during October–

December 1997. Manuals on conducting the assessment were shipped to AAs by the state coordinators.

Then, in January 1998, each AA attended a half-day assessment administrator training conducted by

Westat supervisors for state NAEP. Many of the assessment procedures addressed in these AA training

sessions are thoroughly demonstrated in person via film and through exercises.

5.3 SELECTING THE STUDENT SAMPLES

5.3.1 Selecting the National NAEP Student Samples

After securing cooperation from the school, the first scheduled visit to each national NAEP

school was made to select the sample of students to take part in the national assessments, and to conclude

the arrangements for the actual testing. This visit was made in January by the supervisor responsible for

the assessments in the school. Upon arriving at the school (rarely, sampling was done at the district office

instead of in the school), the supervisor first reviewed the list of grade-eligible students and confirmed
92

verbally with the school coordinator that all eligible students were listed. If any eligible students were

omitted, sampling could not proceed until the list was completed. Instructions for preparing the student

list, which essentially should contain all students (even those not normally tested) enrolled in the grade to

be assessed, are mailed to schools late in the fall term prior to the national assessments.

Using the session assignment form (SAF) produced by Westat for the national assessment, the

supervisor selected the sample of students to be assessed. The SAF is specific to a given NAEP school

and provides detailed written sampling instructions for the school; it specifically documents the number

and type(s) of sessions to be administered, the anticipated number of students to be assessed, the

expected number of students eligible for the assessment, and a series of line numbers designating the

students to be sampled for each session type. Those eligible students on the school’s master list whose

line numbers were shown on the SAF were selected for the assessment. After making sure that all eligible

students had been listed, the supervisor numbered the students on the master list. If the total number of

eligible students was within the minimum and maximum limits indicated on the SAF, the supervisor

could proceed to select the sample. If the number was outside the limits, the supervisor called Westat for

additional sampling instructions. With either the original instructions or revised line numbers, the

supervisor proceeded to select the sample of students. The SAFs provided step-by-step instructions for

sampling, indicating not just the line number of each student to be selected, but the type of assessment

session for which each student was selected.

Once students were assigned to national NAEP sessions, the supervisor and exercise

administrators filled out an administration schedule for each session. The administration schedule is the

primary control document for the assessment. It is used to list each sampled student and is the only link

between booklets and students. The sample was designed so that about 30 students were assigned to each

national NAEP session. The supervisor discussed the final schedule of the sessions with the school

coordinator and the date, time, and location of each session were filled in on the administration

schedules. Because student names were recorded on the administration schedules, those forms remained

in the schools after the sample was drawn.

The supervisor then asked the school coordinator to identify any students in the sample with an

Individualized Education Program (IEP) (for reasons other than being gifted and talented) or who were

designated as LEP. Any student with either (or both) of these designations was to be indicated on the

administration schedules. The school was asked to complete an SD/LEP student questionnaire for each

student with this designation. This was to be completed by a teacher, counselor or other school official

who knew the designated student well.

The school coordinator was also asked to determine whether any of these students should be

excluded from national sessions based on the criteria for assessing SD/LEP students (the use of the

criteria for each NAEP session type are discussed more specifically in Section 5.1.1.2). If the school

coordinator could not identify the excluded students while the supervisor was at the school, the

instructions were left with the coordinator along with blank copies of the SD/LEP student questionnaire.

In those cases, the coordinator consulted with other school officials and informed the supervisor as to

who was to be excluded when the coordinator returned for the national assessment.

For the 1998 assessment, the sampling process generated, in total, 149,880 students to be

assessed in those schools cooperating in national NAEP. These counts include the SD/LEP students

whom the schools determined should participate in the assessments. Accommodations were provided for

an estimated 3,270 students. The most frequently provided accommodations were small-group, extendedtime (untimed testing), and one-one-one testing. Detailed information on SD/LEP results and on the

specific numbers of students actually assessed are provided earlier in Chapter 3 of this report, beginning

with Table 3-8 and continuing in subsequent tables.
93

At the end of the sampling visit, if requested by the school, the supervisor or exercise

administrators made lists of the sampled students for the teachers and/or completed appointment cards

notifying students about their assessment schedule. Teacher notification letters were also prepared in

some schools, which explained the assessment and listed the students who had been selected.

5.3.2 Selecting the Special Studies Samples

Two special studies, requiring added steps in the sampling process, were included in the national

assessment for 1998. One of these special studies involved some of the students in writing assessments.

The other involved collecting high school transcripts for grade 12 students. In the case of both studies, no

student names or other identifiers were taken out of the schools.

The classroom-based writing study involved the random selection during the national NAEP

sampling visit of one English/language arts classroom from each fourth- and eighth-grade school in

which a writing assessment was to be conducted. At the same time, the students in that classroom were

listed on a writing study linkage form so that the classroom students who also took the national writing

assessment could be identified. The classroom’s English/language arts teacher was asked to work with the

students and have them select two examples of their best classroom writing. The students were asked to

answer a few questions about each selection. The teachers completed an interview with the supervisor

who collected the writing materials after the assessment. A full report on this study is due to be published

in the year 2001.

The High School Transcript Study (HSTS) involved a subsample of most of the NAEP public

high schools and one-third of the private high schools selected for the original 1998 national NAEP

sample. This subsample comprised approximately 350 schools. Sampled schools were included

regardless of whether they participated in national NAEP in order to minimize nonresponse bias. The

HSTS student sample included all eligible twelfth-grade students who were sampled for the 1998

national assessment. This included students who were either excluded or absent, though not those who

had withdrawn or were ineligible. Approximately 23,000 student transcripts were collected in this

sample. Seven steps of the HSTS process were completed by Westat field supervisors at the time of the

NAEP sampling visit, and these seven steps are as follows:

• Discuss the HSTS with the school coordinator prior to sampling visit.

• Complete the school information form concerning the organization of course

offerings and course credits at this school, in an interview with school

coordinator.

• Obtain and review course catalogs.

• Complete the course catalog check sheet.

• Obtain and review three examples of student transcripts.

• Mask all identifiers on the sample transcripts.

• Identify and mark the sampled students’ files.

The actual collecting of the transcripts for the sampled twelfth-grade students was performed after the

end of the 1997–1998 school year. The HSTS is conducted periodically to provide educational policy

makers with information regarding course offerings and course-taking patterns, including links to the

NAEP assessment results, in the nation’s secondary schools. The 1998 results will be provided in detail

at a later date in a separate HSTS report prepared by Westat.
94

5.3.3 Selecting the State NAEP Student Samples

Following their November training, the state NAEP supervisors’ first task was to complete the

selection of the sample of students who were to be assessed in each school. All participating schools

were asked to send a list of their grade-eligible students to the state coordinator by November 14.

Sample- selection activities were conducted in the state coordinator’s office unless the state coordinator

preferred that the lists be taken to another location.

Using a sampling package on their laptop computers, the supervisors generally selected a sample

of 30 students per session type per school, with three exceptions: in schools with fewer than 30 students

in the grade to be assessed, all of the students were selected; in schools in which more than one session

was scheduled, 60 students (or some multiple of 30 students) were selected; and in schools with no more

than 33 students in the grade, all students were selected for the assessment.

After the sample was selected, the supervisor completed an administration schedule for each

session, listing the students to be assessed. The administration schedules for each school were put into an

envelope and given to the state coordinator to send to the school two weeks before the scheduled

assessment date. Included in the envelope were instructions for sampling students who had enrolled at

the schools since the creation of the original list.

5.4 CONDUCTING THE ASSESSMENT SESSIONS

5.4.1 Conducting the National Assessments

The primary responsibility for conducting national NAEP assessment sessions was given to the

exercise administrators. Supervisors were required to observe the first session each exercise

administrator conducted to ensure that they followed the procedures properly. Supervisors were also

required to be present in all schools with more than one small session to be conducted. The supervisor

plays an important role as the liaison between the national assessment and school staff, ensuring that the

assessments go smoothly.

To ensure that sessions were administered in a uniform way, the exercise administrator was

provided with scripts for each session type. The scripts were read verbatim, and began with a brief

introduction to the study. The exercise administrator then distributed the booklets, being careful to match

the student with the preassigned booklet.

After the booklets were distributed, some additional, scripted directions were read. Students were

asked to write in the NAEP school ID (except in grade 4, where NAEP staff entered the ID on the cover

of the booklet) and were given some general directions for completing the assessment. For fourth-grade

students, all of the background questions were read aloud by the exercise administrator; at the upper

grades, the first question, which asks the students’ race/ethnicity, was read by the exercise administrator,

and the students read the rest to themselves. After the background questions were completed, the students

were told that any further questions they might have could not be answered by the exercise administrator,

and that they were to begin the first cognitive section of the assessment. This process (along with the

script) was modified somewhat for writing/civics sessions where the background questions were at the

end of the assessment booklet, and none of the items was read aloud at grades 8 or 12.

During the sessions, the exercise administrators walked around the room, monitoring the students

to make sure they were working in the correct section of their booklet and to discourage them from

looking at a neighbor’s or excluded booklet.
95

At the end of each assessment session, booklets were collected and students dismissed according

to the school’s policy. The exercise administrator was then responsible for completing the information at

the top of the administration schedule, totaling the number of participating students, and coding the

covers of all booklets, including those booklets assigned to absent students.

5.4.2 Conducting the State Assessments

During the months of November and December 1997, the state supervisors also recruited and

hired quality control monitors to work in their jurisdictions. It was the quality control monitor’s job to

observe the sessions designated to be monitored, to complete an observation form on each session, and to

intervene when the correct procedures were not followed. Because earlier results indicated little

difference in performance between monitored and unmonitored schools, and in an effort to reduce costs,

the percentage of public schools to be monitored was maintained at 25 percent (i.e., the reduced

monitoring rate initiated in 1994). The monitoring rate for nonpublic schools was also maintained at 25

percent (and reduced from the 50% rate used in 1994, which was the first year that nonpublic schools

were assessed by NAEP). As has been customary in the past, monitoring was conducted at 50 percent for

jurisdictions that were new to the state assessment in 1998. The schools to be monitored were known

only to contractor staff; it was not indicated on any of the listings provided to state staff.

Almost immediately following the quality control monitor training, supervisors began conducting

training for assessment administrators. Each quality control monitor attended at least two training

sessions, to assist the state supervisor and to become thoroughly familiar with the assessment

administrator’s responsibilities. To ensure uniformity in the training sessions, Westat developed a highly

structured three-day training program involving a script for trainers, a videotape, and a training example

to be completed by the trainees. The training package, developed for previous state assessments, was

revised to reflect the subjects and grades assessed in 1998. The supervisors were instructed to read the

script verbatim as they proceeded through the training, ensuring that each trainee received the same

information. The script was supplemented by the use of overhead transparencies, displaying the various

forms that were to be used and enabling the trainer to demonstrate how they were to be filled out.

Two weeks prior to the scheduled assessment date, the state NAEP assessment administrator

received the administration schedule and assessment questionnaires and materials. Five days before the

assessment, the quality control monitor made a call to the administrator and recorded the results of the

call on the quality control form for monitored schools, because the assessment administrators were not

supposed to know in advance which sessions were designated to be monitored. The preassessment call

was conducted in exactly the same way regardless of whether the school was to be monitored or not. For

example, directions to the school were obtained even if the school was in the unmonitored sample. Most

of the questions asked in the preassessment call were designed to gauge whether the assessment

administrator had received all materials needed and had completed the preparations for the assessment.

If the sessions in a school were designated to be monitored, the quality control monitor was to

arrive at the school one hour before the scheduled beginning of the assessment to observe preparations

for the assessment. To ensure the confidentiality of the assessment items, the booklets were packaged in

shrink-wrapped bundles and were not to be opened until the quality control monitor arrived or 45 minutes

before the session began, whichever occurred first.

In addition to observing the opening of the bundles, the quality control monitor used the quality

control form to check that the following had been done correctly: sampling newly enrolled students,
96

reading the script, distributing and collecting assessment materials, timing the booklet sections,

answering questions from students, and preparing assessment materials for shipment. After the

assessment was over, the quality control monitor obtained the assessment administrator’s opinions of

how the session went and how well the materials and forms worked.

If four or more students were absent from the session, a makeup session was to be held. If the

original session had been monitored, the makeup session was also monitored. This required coordination

of scheduling between the quality control monitor and assessment administrator.

5.4.3 Participation of Department of Defense Education Activity Schools in State NAEP

The schools run by the Department of Defense at military bases and other installations around

the world participated in the NAEP state assessment for the third time in 1998. The participation of the

selected schools was mandated by the Department of Defense Education Activity (DoDEA) schools. To

accommodate the geographic diversity of DoDEA schools, some minor adaptations were made in the

preparatory activities used for the other jurisdictions.

For 1998, as in 1996, the data collection in DoDEA schools was expanded from the 1994 model

so that both the DoDEA’s Department of Defense Elementary and Secondary Schools (DDESS), which

includes domestic schools, and the DoDEA’s Department of Defense Dependents Schools (DoDDS),

which includes overseas schools, were surveyed. In 1994, only the schools at overseas installations were

sampled as part of the state assessment.

Many of the quality control monitors hired for the DoDEA schools were based overseas, and

many had previous experience working within the DoDEA system. They were referred to Westat by

DoDEA. All quality control monitors for the DoDEA schools attended the quality control training in Los

Angeles and several assessment administrator training sessions in the geographic areas in which they

worked.

The samples of students to be assessed in the DoDEA schools were selected in the Westat home

office, using standard NAEP procedures, from lists of students produced in the DoDEA offices in

northern Virginia. Due to privacy concerns, only student ID numbers and not student names appeared on

the DoDEA lists. Thus, after sampling, the administration schedules contained only the ID numbers, and

the assessment administrators consulted school records and added the names of the students to the

administration schedules prior to the assessments.

Two field supervisors were hired specifically to conduct assessment administrator trainings and

monitor quality control monitors in the DoDEA/DoDDS schools. The DoDEA liaison in northern

Virginia, who essentially functioned as the state coordinator, arranged the assessment administrator

training sessions, all of which were held in schools or other facilities on the bases. In many cases, the

quality control monitors were required to obtain special clearances through DoDEA to visit the bases for

training and the assessments. The assessments in DoDEA schools were conducted using the same

procedures as in all state assessment schools.
97

5.5 RESULTS OF THE NATIONAL NAEP ASSESSMENT

5.5.1 School and Student Participation

The unweighted school response rate for the national assessments in 1998 was 86 percent

overall. This rate reflects the final sample of cooperating schools including 731 schools at grade 4; 753

schools at grade 8; and 599 schools at grade 12. Table 3-8 in Chapter 3 provides detailed counts and

response rates.

The school response rates increased for 1998, which reverses the small declines in national

assessment school response rates that occurred between 1990 and 1996. The gains were most likely due

to persistent efforts to convert schools and districts that indicated that they were not interested in

participating in the assessments. Both Westat field managers and ETS staff were employed in these

conversion efforts.

 Although school response rates for 1998 reached their highest levels since 1990, the most

frequently stated reason for school and district refusals, historically, has been the increase in testing

throughout the jurisdictions and the resulting difficulty in finding time in the school schedule to conduct

the NAEP assessments. With so many states now mandating their own testing, school schedules are

becoming tighter, and administrators are finding it increasingly difficult to accommodate outside testing.

Despite the increased visibility and publicity surrounding NAEP, schools are reluctantly finding it

necessary to decline participation as a result of the increasing demands on their students’ time.

Of the 160,480 students sampled for the 1998 assessment, roughly 5 percent overall were

excluded by schools. Altogether, 133,489 students were assessed across all three grades: 36,104 students

were assessed at fourth grade, 48,797 were assessed at eighth grade, and 48,588 students were assessed at

twelfth grade. The final student participation rate was 89 percent and this reflects students who

participated in the NAEP session, based on "students to be assessed", that is, after eliminating any

students withdrawn from the school, not eligible, or excluded by the school.

The student response rate at which supervisors were required to conduct a makeup session was

90 percent (lower rates were used prior to 1996); that is, any session (or group of sessions within the

same subject area) at which fewer than 90 percent of the eligible students were assessed would require a

makeup session. For 1998 NAEP sessions, about 23,200 of the roughly 150,000 students to be assessed

were absent from the original sessions. Almost 7,000 of the absent students were assessed in makeup

sessions, which represents about 30 percent of those absent from the original sessions. The makeup

assessments added an estimated 4.5 percentage points to the overall student response rate for all grades

combined, and it is further estimated that the makeups were conducted in 25 to 30 percent of the schools,

with some variation according to the grade level assessed.

5.5.2 Assessment Questionnaires

Westat provided each school with a school questionnaire a few weeks before the assessment was

scheduled to be conducted (i.e., at the time of sampling). At the same time, supervisors prepared an

SD/LEP student questionnaire for each sampled student with either an IEP or an LEP designation, with

the request that it be completed by someone at the school knowledgeable about that student.

For fourth grade and eighth grade, selected teachers in the subject areas of language arts and

civic education were asked to fill out teacher questionnaires. The teachers asked to participate were the

reading, writing, or civics teachers of those students selected for the assessment so that the teacher data

could be linked to student performance data. The teacher questionnaire for grade 4 was combined into
98

one form, since it is recognized that at this grade level the same teacher would probably teach all of the

subjects. For grade 8, there were two distinct questionnaires, one for language arts teachers and the other

for civics teachers. At grade 12, teacher questionnaires were not used in 1998 NAEP.

The NAEP supervisor requested that the teacher questionnaires be distributed as quickly as

possible after the sampling so that they could be returned by the day of the assessment. Additional

introductory materials were included with the teacher questionnaires, in response to questions that

teachers have had in the past about the importance of completing the questionnaires and about NAEP in

general. Teachers received a letter explaining the purpose of the teacher questionnaire, along with

background materials about NAEP.

If the teacher addressed questionnaires were not complete at the time of the assessment, the

supervisor left a postage-paid envelope to NCS to be used to return the questionnaires. Table 5-2 shows

the number of questionnaires distributed and the number completed.

Table 5-2

Background Questionnaires Received for Schools, Teachers,

and SD/LEP Students in the 1998 National Assessment*

Teacher Questionnaires

School

Questionnaire

Language

Arts/Civics

(Grade 4 only)

Language

Arts Civics

SD/LEP Student

Questionnaire

Grade 4

Number Expected 731 2,145 — — 7,066

Number Received 700 2,081 — — 6,830

Percent Received 96% 97% — — 97%

Grade 8

Number Expected 753 — 2,303 1,594 7,942

Number Received 722 — 2,170 1,489 7,575

Percent Received 96% — 94% 93% 95%

Grade 12

Number Expected 599 — — — 6,588

Number Received 570 — — — 6,214

Percent Received 95% — — — 94%

* Every cooperating school was given a school questionnaire, but some schools failed to complete their questionnaires, so that

the number of completed questionnaires is smaller than the number of participating schools.

5.6 RESULTS OF THE STATE NAEP ASSESSMENT

5.6.1 School and Student Participation

Table 5-3 shows the results of the state coordinators’ efforts to gain the cooperation of the

schools selected for state NAEP.

Overall, for the 1998 state assessment in reading, 4,594 public schools and 570 nonpublic

schools for grade 4 participated. For eighth grade, 3,805 public schools and 453 nonpublic schools

participated in reading, and 3,688 public and 450 nonpublic participated in writing assessments.
99

Participation results for students in the 1998 state assessments are given in Table 5-4. Nearly

139,000 fourth-grade students and over 237,000 eighth-grade students were sampled. As can be seen

from the table, the original sample, which was selected by the NAEP state supervisors, comprised

approximately 135,000 (or 97%) of the total number of students sampled for grade 4, and approximately

231,500 (or 98%) of the total number of students sampled for grade 8. The original sample size was

increased somewhat after the supplemental samples had been drawn (from students newly enrolled since

the creation of the original list of students).

When queried, the quality control monitors felt most positive about the attitudes of the

assessment administrators and somewhat less positive about the attitudes of other school staff and the

students toward the assessment. The QC monitors’ evaluations, impressions, and observations are

recorded in the QC monitoring form provided to them for each school.

Quality control monitors concluded the summary section of their QC monitoring form by

assigning a final rating of the assessment administrator’s performance. With this rating, the quality

control monitor reconsidered the session from the vantage point of how well it would have gone without

the quality control monitor’s presence. Eighty-four percent of the assessment administrators in monitored

sessions were self-reliant or needed to consult the quality control monitors for only one or two minor

items. Between four and five percent cited serious difficulty conducting the session (that is, relied on the

quality control monitor to initiate procedures or conduct the session).
100

Table 5-3

School Participation, 1998 State Assessment*

Grade 4 Reading Grade 8 Reading 

Public Nonpublic Public Nonpublic 

Schools in original sample 4,594 570 3,805 453 Schools not eligible (closed or no sampled grade) 73 68 85 71 

Eligible schools in original sample 4,521 502 3,720 382 

Noncooperating† 440 131 397 90 

Cooperating 4,081 371 3,323 292 

Participating substitutes for noncooperating schools 125 27 84 8 Total of schools participating (after substitution) 4,206 398 3,407 300 

* Corresponding data for national NAEP schools are provided in Chapter 3 of this report.† e.g., school, district, or state refusal
101

Table 5-4

Student Participation, 1998 State Assessment*

GRADE 4 READING GRADE 8 READING 

Public Nonpublic Public Nonpublic 

Number Sampled 130,230 8,621 113,789 5,922 

Original Sample 126,414 8,551 110,995 5,880 Supplemental Sample 3,816 70 2,794 42 Percent Increase in Original Sample 3.0% 0.8% 2.5% 0.7% 

Number of Originally Sampled Students Withdrawn 5,628 88 5,357 57 Percent of Originally Sampled Students Withdrawn 4.4% 1.0% 4.8% 1.0% 

Number of Students Excluded† 9,186 64 6,068 43 

Number of Sampled Students Identified as SD 15,040 210 12,750 157 Percent of Sampled Students Identified as SD 11.5% 2.4% 11.2% 2.7% Number of Sampled Students Excluded as SD 7,181 54 5,039 27 Percent of Sampled Students Excluded as SD 5.5% 0.6% 4.4% 0.5% Number of Sampled Students Identified as LEP 5,514 53 3,338 64 Percent of Sampled Students Identified as LEP 4.2% 0.6% 2.9% 1.1% Number of Sampled Students Excluded as LEP 2,406 13 1,260 19 Percent of Sampled Students Excluded as LEP 1.8% 0.2% 1.1% 0.3% 

Number of Students To Be assessed 115,416 8,469 102,364 5,822 Number of Students Assessed 109,149 8,101 93,229 5,554 

Original Sessions 108,145 8,020 91,614 5,511 Makeup Sessions 1,004 81 1,615 43 

Student Participation Rates – Before Makeups 93.7% 94.7% 89.5% 94.7% Student Participation Rates – After Makeups 94.6% 95.7% 91.1% 95.4% 

* Corresponding data for national NAEP schools are provided in Chapter 3 of this report.† To be excluded, a student had to be designated as SD or LEP and judged incapable of participating in the assessment. A student could be identified a

in this number being less than the sum of the students excluded as SD or LEP.
102

5.6.2 Results of the Observations

During the state NAEP assessment sessions, the quality control (QC) monitors observed whether

the assessment environment was adequate or inadequate based on factors such as room size, seating

arrangements, noise from hallways or adjacent rooms, and lighting. (If the room was unsuitable,

however, the quality control monitors did not routinely ask the assessment administrator to make other

arrangements.) Of the approximately 3,300 monitored assessment sessions, the quality control monitors

felt that at least 96 percent of the sessions were held in suitable surroundings. This evaluation of the

assessment environment is recorded in the QC monitoring form provided to them for each school

observed, that is, the QC monitors’ observations are recorded systematically in the pre-printed form

during their observations of the sessions.

The Manual for Assessment Administrators encouraged assessment administrators to use an

assistant during the assessment session, a suggestion that came from the earliest state assessment in 1990.

To measure how frequently that advice was heeded, quality control monitors noted whether an assistant

was used in the monitored sessions. The results indicate that assistants were used for about 52 percent of

the public-school sessions. In nonpublic schools, however, an assistant was employed less often (19–29%

of the time), which is possibly a reflection of fewer staff resources and generally smaller session sizes in

nonpublic schools; the largest occurrence of assistants in public schools (29%) was at grade 4.

Assessment administrators used assistants in varying capacities. The Manual for Assessment

Administrators was very emphatic that only a NAEP-trained person could actually administer the

assessment session. In most cases, assistants helped to supervise the session and to prepare, distribute,

and collect assessment materials and booklets.

The assessment administrators were asked to estimate the total time that they spent on the

preparations for and the conduct of the assessment, including their attendance at the training session.

Estimates for 1998 were similar to those for previous years. In 1998, a majority of the assessment

administrators with grade 4 sessions (73% in public schools and 90% in nonpublic schools) stated that

they spent less than 20 hours on the assessment. For grade 8, however, only 40 percent of the assessment

administrators in public schools, compared to 88 percent of those in nonpublic schools, spent fewer than

20 hours. The variation in time distribution for grade 8 public schools, particularly compared to public

schools at grade 4, is most likely due to the fact that two session types (reading and writing) were usually

conducted in each grade 8 school for state NAEP, but only one session type (reading) was held at grade

4. This does not appear to hold true for nonpublic schools, however, where the distribution of time spent

is more similar for grades 4 and 8. It is evident that assessment administrators in nonpublic schools spent

fewer hours overall on the assessment than did assessment administrators in public schools. Potential

explanations might be the generally smaller sessions sizes in nonpublic schools (i.e., fewer materials to

prepare and ship) and the possibility that some grade 8 schools may have used more than one assessment

administrator, with each assessment administrator conducting one session (but compiling a larger total

time for all sessions combined).

Quality control monitors observed that assessment booklet bundles were opened at the proper

time in about 98 percent of sessions. In a few sessions, however, the bundle opening was not observed

due to quality control monitor error (e.g., the quality control monitor was late, in the wrong place, or

miscommunicated with the assessment administrator); presumably, some (or probably most) of these

bundles were opened at the correct time. For a few other sessions, the quality control monitors were

unable to observe the bundle opening that occurred early due to assessment administrator error (e.g., the

assessment administrator misunderstood the procedures, felt more time was needed, had scheduling

conflicts, or needed to prepare for multiple sessions starting at the same time).

After the conclusion of the state NAEP assessment sessions, Westat mailed state coordinators a

short survey to obtain their reactions to the operations associated with the 1998 state assessment and any
103

suggestions they had for improving the program. Thirty-one of the forty-four state coordinators who were

mailed the survey (or about 70 percent) responded by returning the survey or by providing their

responses over the telephone. A detailed summary of the state coordinators’ responses is contained in the

Report on Data Collection Activities for All States (Westat, 1998), which was distributed to state

coordinators in October 1998. Some of the responses from the state coordinators included:

• Eleven of the 31 reporting jurisdictions mandated participation in the 1998 state

assessment.

• Only two jurisdictions reported that they helped gain the cooperation of

nonpublic schools. One had success contacting parochial schools, but requested

assistance from NAEP staff for recruiting other nonpublic schools. Most

coordinators preferred that NAEP staff contact the nonpublic schools.

• All 31 jurisdictions responding (of the 44 jurisdictions sampled) used the

computer system during the field period. Five jurisdictions used the system

initially but not necessarily during the entire assessment period. The jurisdictions

seemed to be comfortable with the computer system and were able to use it

effectively. Typically, the reason for discontinuing use of the computer was that

coordinators had completed their data-entry tasks and had turned responsibility

back to the state supervisor who was coordinating requests for assessment date

changes.

• Of the jurisdictions reporting on staff time devoted to NAEP, state coordinators

spent an average of 28 days on NAEP activities, and in addition, other staff spent

an average of 25 days.

• Reactions to the 1998 state assessment were quite positive. Most of the state

coordinators who expressed an opinion said that the assessments went “very

well” or “well”—with very few problems.

5.7 FIELD MANAGEMENT

Two field managers monitored the work of about 25 scheduling supervisors who worked during

fall 1997 to gain cooperation of districts and schools for the national assessment. During the national

assessment period, these staff were expanded to about 80 supervisors and 5 field managers. All

supervisors reported directly to their field managers who, in turn, reported to Westat’s field director.

These contacts were made at least weekly.

An automated management system was developed and maintained in Westat’s home office. The

national NAEP scheduling supervisors working to contact schools during the fall used this system on

their portable computers. The system contained a record for each sampled school. A disposition code

structure was developed to indicate the status of each school’s participation (e.g., school cooperating,

decision pending, school refusal, district refusal, school closed, etc.). As a school’s status was

determined, the scheduling supervisors entered the status of the school into their computers, and this

information was downloaded into the home office system on a weekly basis. Disposition reports were

then generated from the receipt system once a week so that home office staff could review the progress of

securing cooperation from the sampled schools.

 These reports were an invaluable tool for the sampling statisticians as well as for the field

director and field management staff. They provided the statisticians with the information needed to

determine whether or not the response rates were high enough for the sample of schools to produce
104

representative results. Based on the information contained in these reports, the sampling statisticians

selected substitute schools to replace some of the noncooperating schools.

After national NAEP assessments were completed, the system was used to enter data from the

school worksheets (for national NAEP) on the number of students to be assessed, the number assessed,

and the number absent for each school. Data on completed questionnaires received was provided by

NCS. The system was also used to alter school assessment dates, particularly when bad weather required

a change in schedule, and to monitor plans for and progress in conducting makeup sessions. Reports were

generated weekly during the assessment period, allowing the project staff to monitor the progress of the

assessments both in terms of checking that the schools were assessed on schedule as well as assuring that

a high response rate was achieved. The sampling statisticians used these reports to monitor the sample

yield by school, PSU, and age or grade level.

Progress of the national NAEP assessments was constantly monitored through telephone reports

held between NAEP supervisors, field managers, and home office staff. During these phone

conversations, the supervisors’ schedules were reviewed and updated, and any problems that the

supervisors were experiencing were discussed. Progress of the fieldwork was also monitored during

quality control visits made to the field by Westat and ETS office staff.

The supervisors who traveled filled out a work schedule for a one- to two-week period, showing

their whereabouts, so that they could be contacted if necessary. It also allowed field managers and project

staff to review the supervisors’ schedules and the distribution of work.
105

Chapter 6

PROCESSING ASSESSMENT MATERIALS1

Connie Smith, Charles Brungardt, and Timothy Robinson

National Computer Systems

6.1 INTRODUCTION

In the spring of 1998, the National Assessment of Educational Progress (NAEP) assessed

students in reading, writing, and civics at grades 4, 8, and 12 at the national level. At the state level,

reading was assessed at grades 4 and 8, and writing was assessed at grade 8 only. Civics was not assessed

at the state level. National Computer Systems (NCS), under subcontract to Educational Testing Service

(ETS), completed the following activities related to test-materials processing for both the national and

state components of the 1998 assessment:

• Printing of test booklets and questionnaires

• Materials packaging and distribution

• Receipt control

• Data capture through image and optical mark recognition scanning

• Data editing and validation

• Performance scoring of constructed-response (open-ended) items

• Data file creation

• Inventory control and materials storage

NCS received and processed a total of 447,377 assessed student booklets and 113,676

questionnaires for the three grades and subjects assessed. A total of 4,272,139 readings of student

constructed responses were conducted via image-based on-line scoring. This allowed for item-by-item

scoring and on-line, real-time monitoring of both interrater reliabilities and the performance of each

individual reader. Session and booklet information for the 1998 national and state assessments is given in

Table 6-1. Table 6-2 provides information on questionnaires expected, received, and processed. Further

detail is provided in NCS’s 1998 NAEP Assessment Report of Processing and Professional Scoring

Activities (National Computer Systems, 1998).

6.2 PRINTING

For the 1998 assessments, 284 unique documents were designed. NCS printed more than

1,500,000 booklets and forms, totaling more than 60 million pages. This was a collaborative effort

involving staff from ETS, Westat, and NCS. ETS created camera-ready blocks using NCS’s

DesignExpertTM software for the test booklets and questionnaires. Using ETS’s booklet maps, which

specified the order of blocks in each booklet, NCS assembled electronic components into complete

 

1 Connie Smith was the NCS project manager for 1998 NAEP, Charles Brungardt was the NCS project director for 1998 NAEP

scoring, and Timothy Robinson was the NCS senior processing coordinator for 1998 NAEP.
106

booklets. NCS then forwarded proofs to ETS, while conducting simultaneous quality control itself. Upon

approval, final-form test booklets and questionnaires were produced and accounted for in the NCS

inventory control system.

Table 6-1

Number of Sessions and Student Booklets Processed

for the 1998 National and State Assessments

Grade

Session

Type

Number of

Sessions

Assessed

Booklets

Absent

Booklets

Excluded

Booklets

National

4

Reading 470 8,280 330 924

Writing 1,519 25,816 1,317 1,880

Civics 116 2,088 98 180

Total 2,105 36,184 1,745 2,984

8

Reading 623 11,970 937 977

Writing 1,925 34,858 2,827 1,508

Civics 114 2,055 161 96

Total 2,662 48,833 3,925 2,581

12

Reading 694 13,417 3,393 729

Writing 1,769 33,106 8,373 1,207

Civics 114 2,193 500 100

Total 2,577 48,716 12,266 2,100

State

4

Reading 4,915 117,237 6,363 9,317

Total 4,915 117,237 6,363 9,317

8

Reading 4,389 98,776 9,236 6,176

Writing 4,375 97,603 9,338 97,603

Total 8,764 196,479 18,574 103,799
107

Table 6-2

Questionnaire Totals for the 1998 NAEP Assessment

Expected Received Percent

National

Grade 4

Language Arts/Civics Teacher Questionnaire 2,145 2,081 97.0%

School Questionnaire 731 700 95.8%

SD/LEP Questionnaire 7,066 7 96.7%

Grade 8

Language Arts Teacher Questionnaire 2,303 2,170 94.2%

Civics Teacher Questionnaire 1,594 1,489 93.4%

School Questionnaire 753 722 95.9%

SD/LEP Questionnaire 7,942 7,575 95.4%

Grade 12

School Questionnaire 599 570 95.2%

SD/LEP Questionnaire 6,588 6,214 94.3%

State

Grade 4

Language Arts Teacher Questionnaire 16,597 16,339 98.4%

School Questionnaire 4,593 4,550 99.1%

SD/LEP Questionnaire 18,711 18,310 97.8%

Grade 8

Language Arts Teacher Questionnaire 14,854 14,370 96.7%

School Questionnaire 3,935 3,858 98.0%

SD/LEP Questionnaire 28,515 27,798 97.5%

6.3 PACKAGING AND DISTRIBUTION

The distribution effort for the 1998 NAEP assessment involved packaging and mailing documents

and associated forms and materials to the Westat supervisors for the national assessment and to individual

schools for the state assessment. The NCS materials distribution system (MDS) was utilized again in

1998. Files in the MDS system contained shipping addresses, scheduled assessment dates, and a listing of

all materials available for use by a participant in a particular subject area. Changes to any of this

information were made directly in the MDS file either manually or via file updates provided by Westat.

Bar code technology continued to be utilized in document control, as has been done since the

1990 NAEP assessment. NCS identified each document with a unique 10-digit identification number. This

number consisted of the 3-digit booklet number or form type, a 6-digit sequential number, and a check

digit. Each form was assigned a range of identification numbers. Bar codes reflecting this identification

number were applied to the front covers of documents by NCS bar code processes and high-speed ink-jet

printers.

Spiraling of the NAEP booklets was done according to the pattern specified by ETS (see

Section 1.5) to capture the sample size needed for each subject per grade. One booklet type from each

grade and subject was designated as an accommodation booklet. These booklets were grouped in bundles

of three.
108

Using sampling files provided by Westat, NCS assigned bundles to sessions and customized the

packing lists. File data was coupled with the file of bundle numbers and the corresponding booklet

numbers. This file was then used to preprint all booklet identification numbers, school name, school

number, and session type directly onto the scannable administration schedule. This increased the quality

level of the booklet accountability system by enabling NCS to identify where any booklet should be at any

time during the assessments. To assist Westat supervisors with sampling in the schools, NCS distributed

the preprinted administration schedules and questionnaires for the national assessment in December 1997.

Preprinted administration schedules for the state assessment were sent to the appropriate state supervisor

for distribution during training of the assessment administrators in January and February 1998.

NCS was also responsible for packaging and distributing bulk and session materials to Westat

supervisors for the national assessment. Bulk shipments included materials that could be used by

supervisors from one session to another, such as ancillary items and additional booklets.

Distribution of materials for the national assessment was accomplished in two phases. In the first

phase, bulk supplies of materials were distributed to each supervisor. The second phase was the

distribution of session specific materials by supervisor region and primary sampling unit (PSU). Each

session box of materials contained the assigned bundles of booklets and the appropriate ancillary items.

For additional materials, Westat supervisors were instructed to contact NCS using the NAEP toll-free

line or the NAEP e-mail address.

Session materials were sent to individual schools in the NAEP state assessment. Distribution of

materials was accomplished in five waves of shipment dates. Except for wave “zero,” session materials

were sent to schools two weeks before their scheduled assessment date. All school materials were sent

directly to an assessment administrator at a school or school district. Materials for Hawaii, Virgin

Islands, and DoDEA/DoDDS (Department of Defense Education Activity’s Department of Defense

Dependents Schools) were distributed in wave “zero”. These shipments required an alternate carrier to

ensure timely delivery.

Initially, 6,933 individual sessions were shipped to 3,814 schools for the national assessment. For

the state assessment, 13,586 sessions were mailed to 12,253 schools. Approximately 450 additional

shipments of booklets and miscellaneous materials were also sent out for the national assessment and

3,000 for the state assessment.

To request additional materials for the 1998 NAEP assessment, Westat supervisors used either

the NCS/NAEP toll-free telephone number or the NCS/NAEP e-mail address. After all the appropriate

information had been entered, the system produced a packing list and mailing labels for NCS’s packaging

staff, who filled and sent the order.

State assessment administrators (AAs) were given two options also, a toll-free telephone number

or a toll-free fax number. This year NCS created a materials request form and included it in the school

shipment to be used either as a guide for ordering materials over the phone or as a fax order form. A form

was created for each grade and great care was taken to group items by session type to simplify the

process for the AAs.

NCS clerical staff also responded to calls or e-mail concerning shipment delivery dates, lost

shipments, and general questions concerning the NAEP assessment.
109

6.4 PROCESSING

NCS staff created a set of predetermined rules and specifications that was to be followed by the

processing departments within NCS. Project staff performed a variety of procedures on materials

received from the assessment administrators before releasing these materials into the NCS/NAEP

processing system. Control systems were used to monitor all NAEP materials returned from the field.

The NAEP Process Control System (PCS) contained the status of sampled schools for all sessions and

their scheduled assessment dates. As materials were returned, the PCS was updated to indicate receipt

dates, to record counts of materials returned, and to document any problems discovered in the shipments.

As documents were processed, the system was updated to reflect processed counts. NCS report programs

were utilized to allow ETS, Westat, and NCS staff to monitor progress in the receipt control operations.

An alerts process was utilized to record, monitor, and categorize all discrepant or problematic situations.

Throughout the processing cycle, alert situations were identified based on the processing specifications.

NCS’s Work Flow Management system (WFM) was used to track batches of student booklets

through each processing step, allowing project staff to monitor the status of all work in progress. It was

also used by NCS to analyze the current work load, by project, across all workstations. Through routine

monitoring of this data, NCS’s management staff was able to assign priorities to various components of

the work and to monitor all phases of the data receipt and processing.

6.4.1 Document Receipt and Opening

Shipments were to be returned to NCS packaged in their original boxes. The bar-coded label

applied during the distribution phase containing the NAEP school identification number was scanned

into a personal computer (PC) file upon receipt. The PC file was then transferred to the mainframe, and

the shipment receipt date was applied to the appropriate school within the PCS system. This provided the

status of receipts regardless of any processing delays. Each receipt was reflected on the PCS status report

provided to the NCS receiving department and supplied to Westat weekly via electronic file transfer and

in hard-copy format. ETS also received a hard copy. The PCS file could be manually updated to reflect

changes. The shipment was then forwarded to the opening area.

Opening personnel checked the shipment to verify that the contents of the box matched the

school and session indicated on the label. Each shipment was checked for completeness and accuracy.

Any shipment not received within three days of the scheduled assessment date was flagged in the PCS

system and annotated on the PCS report. The administration status of these delayed shipments was

checked, and in some cases a trace was initiated on the shipment.

NCS was required to open all shipments within 48 hours of their receipt and to key-enter

preliminary processing information into the PCS system from the administration schedule. The

preliminary information was written on the administration schedule by Westat assessment administrators

and consisted of the following:

• School number

• Session number

• Original test date

• Total number of students to be assessed

• Total number of students assessed

• Completeness flag
110

This preliminary information, used to provide Westat with timely student response rates, was

updated with actual data when materials passed error-free through processing. The shipment was checked

by NCS opening staff to see if any part of the shipment was missing, held for makeup, not administered,

or refused. The shipment was also checked to verify that all booklets whose numbers were preprinted or

handwritten on the administration schedule were returned with the shipment and that all administration

codes matched from booklet cover to the administration schedule.

For all makeup sessions and for any missing materials not returned, the documents were placed

on holding carts until the other documents arrived. These sessions were flagged on the PCS system and

Westat was informed of this information. If the materials were not being returned, processing continued

and the appropriate administration code was applied to the administration schedule. All questionnaires

received were matched against the roster of questionnaires, which was a checklist of all types of

questionnaires used in the assessment.

6.4.2 Batching of Booklets

Once all student booklets listed on the administration schedule for a session were verified as

being present, the entire session (both the administration schedule and booklets) was forwarded to the

batching administration area. Booklet batches were created by grade level, subject area, and session type.

Each batch was assigned a unique batch number. This number, created on the Image Capture

Environment (ICE) system for all image-scannable documents, facilitated the internal tracking of the

batches and allowed departmental resource planning. All other scannable documents—school

questionnaires, teacher questionnaires, SD/LEP (students with disabilities/limited English proficient)

questionnaires, and the roster—were batched by document type in the same manner.

6.4.3 Scanning of Documents

The 1998 NAEP assessment used four rosters—one for each grade and one supplemental

SD/LEP roster—to account for all questionnaires. Rosters of questionnaires were used to record the

distribution and return of SD/LEP questionnaires, teacher questionnaires, and school questionnaires.

Batches of school questionnaires and rosters, which are image scannable documents, were created on the

ICE system. Batches of teacher and SD/LEP questionnaires, image scannable for the first time in the

1998 NAEP cycle, were also created on the ICE system. Batches were then forwarded to scanning, where

all information on the rosters or questionnaires was scanned into the system.

6.4.4 Data Transcription

The transcription of the student response data into machine-readable form was achieved through

the use of the following two systems: data entry (image scanning, intelligent character recognition [ICR],

and key entry), and data validation (edit). NCS used the same format as in prior NAEP assessments and

field tests to set up the document definition files for the number of unique documents used in the 1998

assessment. To do the proper edits, a detailed document definition procedure was designed to allow NCS

to define an item once and use it in many blocks and to define a block once and use it in many

documents.
111

6.4.4.1 Data Entry

The data-entry process was the first point at which booklet-level data were directly available to

the computer system. Depending on the NAEP document, one of three methods was used to transcribe

NAEP data to a computerized form. The gridded data on scannable documents were collected using NCS

optical-scanning equipment, which also captured images of the constructed-response (open-ended) items

and ICR fields in a single pass.

Optical Mark Recognition (OMR) Scanning. The data values were captured from the

booklet covers and administration schedules and were coded as numeric data. Unmarked

fields were coded as blanks and editing staff were alerted to missing or uncoded critical

data. Fields that had multiple marks were coded as asterisks (*). The data values for the

item responses and scores were returned as numeric codes. The multiple-choice singleresponse format items were assigned codes depending on the position of the response

alternative; that is, the first choice was assigned the code “1,” the second “2,” and so

forth. The mark-all-that-apply items were given as many data fields as response

alternatives; the marked choices were coded as “1,” while the unmarked choices were

recorded as blanks.

Image Scanning. The images of constructed-response (open-ended) items were saved as

a digitized computer file. The area of the page that needed to be saved was defined prior

to scanning through the document definition process. The fields from unreadable pages

were coded “X” as a flag for resolution staff to correct. Any image document or sheet

unreadable by the image scanning system was taken to a flat-bed scanner to be scanned

into the system. In addition to capturing the student responses, the bar code identification

numbers used to maintain process control were decoded and transcribed to the NAEP

computerized data file.

Intelligent Character Recognition. The intelligent character recognition (ICR) engine

was again utilized to read various hand and machine printing on the front cover of the

booklet and supervisor documents for the 1998 assessment. Some information from

student documents, administration schedule, roster of questionnaires, and some questions

in the school questionnaires, were read by the ICR engine and verified by an on-line keyentry operator. In all, the ICR engine read 1,994,416 characters for the 1998 assessment.

Use of the ICR engine saved NAEP field staff a significant amount of time, since they

did not have to grid rows and columns of data.

In all three cases, the data were edited, and suspect cases were resolved before further

processing.

6.4.4.2 Data Validation

Each dataset produced by the scanning system contained data for a particular batch. These data

had to be validated (or edited) for type and range of response. The data-entry and resolution system used

was able to simultaneously process a variety of materials from all age groups, subject areas, control

documents, and questionnaires as the materials were submitted to the system from scannable and

nonscannable media.

The data records in the scan file were organized in the same order in which the paper materials

were processed by the scanner. A record for each batch header preceded all data records for that batch.

The document code field on each record distinguished the header record from the data records.
112

When a batch-header record was read, a preedit data record and an edit log entry was generated.

As the program processed each record within a batch from the scan file, it wrote the edited and

reformatted data records to the preedit file and recorded all errors on the edit log. The data fields on an

edit log record identified each data problem by the batch sequence number, booklet serial number,

section or block code, field name or item number, and data value. After each batch had been processed,

the program generated a listing or on-line edit file of the data problems and resolution guidelines. An edit

log listing was printed at the termination of the program for all nonimage documents. Images requiring

editing were routed to on-line editing stations for those documents that were image scanned.

When the entire document was processed, the completed string of data was written to the data

file. When all the documents in the batch were processed, the program generated an edit listing for

nonimage and key-entered documents. Image-scanned items that required correction were displayed at an

on-line editing terminal.

For rapid resolution, the edit criteria for each item in question appeared on the screen along with

the suspect item. Corrections were made immediately. The system employed an edit/verify system that

ultimately meant two different people viewed the same suspect data and operated on it separately. The

verifier made sure the two responses (one from either the entry operator or the ICR engine) were the

same before the system accepted that item as being correct. If the editor could not determine the

appropriate response, he or she escalated the suspect situation to a supervisor. For errors or suspect

information that could not be resolved by supervisory staff, a product-line queue was created, allowing

supervisors in the processing area to escalate edits to project staff for resolution.

Once an entire batch was through the edit phase, it became eligible for the count-verification

phase. The administration schedule data were examined systematically for booklet identification numbers

that should have been processed (assessed administration codes). All documents under that

administration schedule were then inspected to ensure that all of the booklets were included.

With the satisfactory conclusion of the count-verification phase, the edited batch file was

uploaded to the mainframe, where it went through yet another edit process. A paper edit log was

produced and, if errors remained, was forwarded to another editor. When this edit was satisfied, the PCS

and WFM tracking systems were updated.

The teacher and SD/LEP questionnaires were edited on paper. Machine edits performed during

data capture verified that each sheet of each document was present and that each field had an appropriate

value.

Data editing took place after these checks. This consisted of a computerized edit review of each

respondent’s document and the clerical edits necessary to make corrections based on the computer edit.

This data-editing step was repeated until all data were correct.

Suspect data that were investigated during the edit phase consisted of, but were not limited to,

the following by document types:

Administration Schedule

a) Verification that all assessed student booklets are present in a processed batch: If an

administration code of 10-14, 20-24, or 71-79 was present on the administration

schedule, the editor verified that a booklet was present. If the booklet was missing, the

booklet was located and processed before the batch can continue to be processed.
113

b) Verification that the booklet bar code number was valid: NAEP booklet bar code

numbers for the 1998 assessment were 10-digits long and fell within a certain range of

numbers by grade. If, on a hand-written administration schedule, the booklet bar code

written was less than 10-digits or out of range for the grade being processed, NAEP

project staff corrected the bar code number as appropriate to match the booklet being

processed.

c) Verification that the School number was valid: If the school number was blank or not

on the PCS file, the school number was corrected by NAEP project staff.

Student Booklets

a) Investigating suspect bar codes, duplicate bar codes, or invalid check-digits: If the

bar code number was read incorrectly by the scanner, the bar code was corrected to

match the bar code on the booklet in question.

b) Investigating suspected absent students: If a booklet had an administration code

indicating an assessed student, yet no multiple-choice responses were read by the

scanning equipment, the editor manually checked the booklet for any multiple-choice

responses. If a student had penciled in his or her multiple-choice responses too lightly

for the scanners to read, the editor key entered the responses into the student data

record. If no multiple-choice responses were present, but open-ended responses were,

the booklet was sent through processing unchanged. If no multiple-choice or openended responses were present, the administration code was changed to indicate that

there were no responses in the booklet, and the booklet was sent through processing

with the updated administration code.

c) Investigating responses within the valid range: An example of a range check would

be verifying that the birth month of the respondent falls with the range of 01-12. If the

birth month is not within the valid range and a correct birth month can be determined

from either the administration schedule or booklet cover, the birth month is corrected.

If a valid response cannot be determined, the birth month is blanked out. The same

type of range check is done for the birth year when specific years are valid by grade.

A computerized edit list, produced after NAEP documents were scanned, and all the supporting

documentation sent from the field were used to perform the first phase of the edit function. The hardcopy edit list contained all the vital statistics about the batch: number of students, school code, type of

document, assessment code, suspect cases, and record serial numbers. Using the information, the data

editor verified that the batch had been assembled correctly and that each school number was correct.

During data entry, counts of processed documents were generated by type. These counts were compared

against the information captured during scanning.

In the second phase of data editing, experienced editing staff used a predetermined set of

specifications to review the field errors and record necessary corrections to the student data file. The

computerized edit list used in phase one was used to perform this function. The editing staff reviewed the

computer-generated edit log and the area of the source document that was noted as being suspect or as

containing possible errors. The composition of the field was shown in the edit box. The editing staff

checked this piece of information against the NAEP source document. At that point, one of the following

took place:
114

(a) Correctable error: If the error was correctable by the editing staff according to the editing

specifications, the correction was noted on the edit log for later correction via key entry.

(b) Alert: If an error was not correctable according to the specifications, an alert was issued to

NAEP project staff for resolution. Once the correction information was obtained, the

correction was noted on the edit log for key-entry correction.

(c) Noncorrectable error: If a suspected error was found to be correct as stated and no alteration

was possible according to the source document and specifications, no corrective action was

taken. The programs were tailored to allow this information to be accepted into the data

record.

The corrected edit log was then forwarded to the key-entry staff for processing. When all

corrections were entered and verified for a batch, an extract program pulled the corrected records into a

mainframe data set. At this point, the mainframe edit program was initiated. The edit criteria were again

applied to all records. If there were further errors, a new edit listing was printed and the cycle was

repeated.

When the edit process produced an error-free file, the booklet identification number was posted

to the NAEP tracking file by age, assessment, and school. This permitted NCS staff to monitor the NAEP

processing effort by accurately measuring the number of documents processed by form. The posting of

booklet identification numbers also ensured that a booklet identification number was not processed more

than once.

To provide another quality check on the image scanning and scoring system, NCS staff

implemented a quality check process by creating a stamp with a valid score designated on it. Each unique

document type scored via the image system had two quality assurance documents stamped with valid

scores for the items present. The QA booklets were batched and processed together with student

documents of the same type. During the process of scoring, valid score points could be changed or

dropped due to revision in the scoring rubrics. NCS provided ETS with documentation as to what score

points on these items were no longer valid. When an image quality assurance stamp was displayed to a

reader that contained a score point that was no longer valid, the reader assigned the response a score

point of OT (off-task).

NCS also produced various status reports. The Receipt Control Status Report was designed to

track the receipt of material from the schools. It was sorted by school number and displayed the

following information: participation status, scheduled administration date and the shipment receipt date.

The comment field in this report showed any school for which a shipment had not been received within

three days of the scheduled test date.

The Processing Status Report was divided into two sections. The first was sorted by school and

grade within each assessment. The following preliminary data for each were entered from the

administration schedule as the shipment was opened by the receiving department: school number, session

code, test date, preliminary count date, preliminary to-be-assessed counts, preliminary total-assessed

counts, and completeness flags. The actual to-be-assessed count, actual total-assessed count, actual

withdrawn ineligible count, actual count date, actual number excluded, and actual absent count were

entered programmatically following the completion of processing. The second section of the Processing

Status Report sorted and totaled the various documents by form within each grade and assessment.

The PCS Exceptions Report listed all schools and sessions with discrepancies, that is, materials

not returned within three days, school or session given a completeness flag. Once all discrepancies were

resolved for a school, the school would be removed from the report.
115

NCS transmitted electronic files containing the above data to Westat weekly. Hard copy of the

PCS Exception Report, Alerts, and Documents Processed Report were also sent to ETS and Westat

weekly.

6.5 DATA TRANSMISSION BEFORE SCORING

Delivery of data to the scoring center was accomplished via T1 transmission lines that linked the

mainframe computers and the NAEP servers at the document-scanning site in the NCS main facility with

the scoring servers that were dedicated to distributing work to the professional readers at the scoring

center. The actual task of scheduling items for downloading was accomplished using a code written by

the Image Software Development team. This code enabled the person scheduling the download to choose

a team of readers and select the scheduled items from a list of all items that the team would be scoring

throughout the scoring project. This process was repeated for all teams of readers until all anticipated

work was scheduled.

6.6 CLASSROOM-BASED WRITING STUDY

Approximately 200 schools participating in the national writing assessments also conducted the

Classroom-Based Writing Study. This study involved collecting two examples of student writing from an

intact classroom at the selected schools. Participating students were also asked to complete a brief

questionnaire of the assignment for which the writing samples were written. Teachers of participating

classes were interviewed and an audiotape of the interview was shipped to NCS for transcription. Details

of this study will be published in a forthcoming NAEP report.

117

Chapter 7

PROFESSIONAL SCORING1

Connie Smith, Charles Brungardt, and Timothy Robinson

National Computer Systems

7.1 OVERVIEW

The 1998 NAEP assessment required the scoring of constructed responses in reading, writing,

and civics at grades 4, 8, and 12 on the national level. At the state level, constructed responses were

scored at grades 4 and 8 for reading and grade 8 for writing. All preparations were completed and scoring

accomplished on a schedule that allowed faster reporting and delivery of data than in previous years.

Also, to measure longitudinal trends in reading, the project required National Computer Systems (NCS)

to replicate scoring from the 1994 NAEP reading assessment for most of the reading items and to

demonstrate that scoring of this subject was statistically comparable across years.

To accomplish the task of scoring the constructed responses, NCS’s Performance Assessment

Scoring Center (PSC) employed more than 300 professional and 82 clerical scorers on a two-shift

schedule. The professional scorer is required to have, at a minimum, a baccalaureate degree from a fouryear college or university; an advanced degree, scoring experience, and/or teaching experience is

preferred. The clerical scorers, who coded the pre-writing exercise, have at least a high school diploma.

NCS worked with Educational Testing Service (ETS) to prepare training materials and carry out the

training of the scoring teams. Table 7-1 lists the processing and scoring totals for each subject and grade.

Table 7-1

Processing and Scoring Totals for the 1998 NAEP Assessment

Booklets

Processed

Number of

Constructed

Responses*

Number of

Discrete

Response

Items†

Number of

Scorers and

Team

Leaders‡

Dates of

Training and

Scoring

Total 447,961 3,770,952 335 — —

National & State Grade 4 Reading 125,517 1,535,479 46 160 / 16 3/23/98 – 4/24/98

National & State Grade 8 Reading 110,746 1,470,932 69 100 / 10 3/23/98 – 4/24/98

National Grade 12 Reading 13,431 195,444 76** 40 / 4 3/23/98 – 4/24/98

National Grade 4 Writing 19,937 49,347 20 30 / 3 4/28/98 – 7/1/98

National & State Grade 8 Writing 124,346 268,238 23 129 / 12 4/28/98 – 7/1/98

National Grade 12 Writing 25,433 55,695 23 30 / 3 4/28/98 – 7/1/98

National Grade 4 Civics 8,087 52,454 21 27 / 3 4/27/98 – 5/11/98

National Grade 8 Civics 10,337 72,450 28 27 / 3 4/27/98 – 5/11/98

National Grade 12 Civics 10,031 70,913 29 36 / 4 4/27/98 – 5/11/98

* This is the number of student responses to the constructed-response items. These scored responses include those that were rescored

for reliability estimation.

† This is the number of discrete constructed-response items in assessment booklets.

‡ Because readers scored items from all grades and all types of booklets, it is not possible to break the numbers down by how

many scored each classification of items.

** This included 75 image and 1 paper.

 

1

 Connie R. Smith was the NCS project manager for 1998 NAEP, Charles Brungardt was the NCS project director for 1998

NAEP scoring, and Timothy Robinson was the NCS senior processing coordinator for 1998 NAEP.
118

Figures 7-1 and 7-2 provide flowcharts for image scoring (see Section 7.4) and paper scoring (see

Section 7.5). Further detail is provided in NCS’s 1998 NAEP Assessment Report of Processing and

Professional Scoring Activities (National Computer Systems, 1998).

Figure 7-1

Image Scoring Flow Chart

Scoring Guides

Receipt of

Assessed

Documents

Entry of Items and

Scoring Guides

into Database

Update Scoring

Guides

Review Sample

Responses

Orientation Training

Review Scoring

Guides

Score and Review

Practice Responses

Reliability

Satisfactory?

No

Scoring

Yes

Scoring

First Scoring

of Items

Reliability Scoring

of Items

Back-Reading of

Responses by

Table Leader

Reliability

Coefficient

Reports

No

Continue Scoring

Yes

Select Anchor,

Training, and

Calibration

Papers

Reliability

 Satisfactory?

Upload

Files
119

Figure 7-2

Paper Scoring Flow Chart

Scoring Guides

Receipt of

Assessed

Documents

Entry of Items and

Scoring Guides

into Database

Update Scoring

Guides

Review Sample

Responses

Orientation Training

Review Scoring

Guides

Score and Review

Practice Responses

Reliability

Satisfactory?

No

Scoring

Yes

Scoring

First Scoring

of Items

Reliability Scoring

of Items

Back-Reading of

Responses by

Table Leader

Reliability

Coefficient

Reports

No

Continue Scoring

Yes

Select Anchor,

Training, and

Calibration

Papers

Reliability

Satisfactory?

Batch Scoring

Sheets

Scan Scoring

Sheets

Edits?

No

Yes

Resolve

Edits

Upload

Files

Warehouse

Scoring

Sheets

Input Scores for

Paper-Based Scoring

7.2 SELECTION OF TRAINING PAPERS

Clerical staff began the process of copying all responses for rangefinding and creation of anchor

and training sets in November of 1997 by copying all the responses (approximately 400 per prompt) for

the writing prompts that did not change wording or format between the field test and operational

assessment. In January and February of 1998, the clerical staff copied more sample responses, including

approximately 300 responses for each writing item that had undergone changes in wording or format, 200

responses for each writing item that remained the same since the field test, 200 responses for each new

reading item, and 150 responses for each civics item. NCS clerical staff wrote the booklet identification

numbers on each page of each response so that the training samples could be linked back to the

identification numbers of the booklet they came from. They then sorted the papers by item and sent the

samples to ETS for the rangefinding, while keeping the samples in Iowa City for those items to be

reviewed at NCS.

Rangefinding2

 and creation of training sets took place at ETS for the three new reading blocks,

all the writing prompts, and those civics blocks assigned to ETS staff for training. The process took place

 

2 Rangefinding is the process of interpreting the scoring guide onto student responses. These scored responses are then used in

the various training sets (i.e., anchor, practice, calibration, and qualification papers.)
120

in Iowa City for civics blocks assigned to NCS trainers. After review by each subject’s coordinator, ETS

returned the training sets to NCS staff, who reproduced them for scoring. Correct scores were written on

all the anchor papers, while only the table leaders and trainers had keys for the practice, calibration (see

Section 7.4.3), and qualification sets. Trainers also kept annotations, explaining the thought process

behind each score assigned. If any of these changed during training for scoring, the table leaders kept

notes explaining the reason.

7.3 CALIBRATION POLICIES

When scoring was expected to last longer than a few hours (for example, items with a state

sample), a calibration set was created to refresh the training and avoid scorer drift. Responses were

chosen from the current sample (see Section 7.4.3). The table leader invoked the calibration tool in the

backreading tool (see Section 7.4.2) to create calibration sets. In general, each team scored calibration

sets whenever they took a break longer than 15 minutes, such as when returning from lunch.

7.4 IMAGE SCORING

 During processing, images of the student responses to each of the constructed-response items

were digitized, placed in an image archive, and grouped according to scoring purpose (e.g., grade 4

reading, grade 4 writing, and validity). Two of the significant advantages of the image-scoring system

were the ease of regulating the flow of work to scorers and the ease of monitoring scoring. The image

system provided table leaders with tools to determine scorer qualification, to backread scores, to

determine scorer calibration, to monitor interrater reliability, and to gauge the rate at which scoring was

being completed. These tools are described in Sections 7.4.1 through 7.4.10.

7.4.1 Reader Qualification

Teams used copies of paper sets to determine whether each individual scorer was sufficiently

prepared to score. All extended items in reading and civics and all items in writing required scorers to

qualify. Short items in reading and civics did not require special qualification. Once scorers demonstrated

readiness for scoring, either through the trainer’s perception during the training of short constructedresponse items or the formal 80 percent correct on the qualification set for extended constructed-response

items, the table leader used the qualification tool to route work to the team. To make sure that all scorers

had a common understanding of the training, the teams usually gathered around one terminal at the

beginning of scoring, read several papers aloud, and scored them as a group. Then the teams broke into

pairs for scoring, followed by individual scoring.

7.4.2 Backreading Process

After scoring began, NCS table leaders reviewed each scorer’s progress using a backreading

utility that allowed the table leader to review papers scored by each scorer on the team. Typically, a table

leader reviewed approximately 10 percent of all responses scored by each scorer. Table leaders made

certain to note the score the scorer awarded each response as well as the score a second scorer gave that

same paper. This was done as an interrater reliability check. Alternatively, a table leader could choose to

review all responses given a particular score to determine if the team as a whole was scoring consistently.

Both of these review methods used the same display screen and showed the identification number of the

scorer and the scores awarded. If the table leader disagreed with the score given an item, he or she

discussed it with the scorer for possible correction. This discussion was used as a training tool to ensure
121

that all scorers assigned the same score to similar responses. Whether or not the table leader agreed with

the score, he or she assigned a table-leader score in backreading. If this score agreed with the first score,

the score was recorded only for statistical purposes. If the scores disagreed, then the table-leader score

overrode the first score as the reported score.

7.4.3 Calibration Process

During backreading, the table leader had a pool of 300 responses for each item, which were

available to use in the calibration process. The table leader viewed samples of these responses together

with the scores assigned by the first and, if applicable, second scorer. From this pool, the table leader

chose which responses to put into the pool that would be available to scorers during calibration, choosing

responses that were scored correctly and were a good measure to keep scoring on track. From this pool,

the table leader built sets with the desired number of responses, usually between 5 and 10, to be

displayed to scorers for calibration. When the scorers invoked the calibration window, all scorers

received the same responses and scored them. After scorers had finished scoring this pool, the table

leader could look at reliability reports, which included only the data from the calibration set just run.

Thus, this type of calibration served to refresh training and avoid drift in scoring. Because paper

calibration sets from 1994 reading still existed, some reading teams used hard copies to calibrate scorers.

7.4.4 Short-Term Trend Rescoring

To measure comparability of this year’s reading scoring to the scoring of the same items done in

1994, a minimum of 600 on-task responses per item from 1994 were scanned and loaded into the system

with their scores from 1994 as the first score.

“On-task” responses generate scores of 1, 2, 3, 4, 5, 6, or 7. “Off-task” scores are received when

the response

• is blank,

• is “I don’t know,”

• is totally erased,

• contains only comments for the test developer or scorer, or

• contains other unelicited remarks, drawings, or both.

These responses were loaded into a separate computer application to keep the data separate from regular

scoring. At staggered intervals during the scoring process, the table leader released items from the 1994

cycle for scorers to read and score. Since the 1994 scores were preloaded as first scores, this year’s teams

in effect scored 100 percent of them a second time. Typically, the table leaders released 100 responses

after training was finished but before beginning the scoring of current-year responses. The table leader

and trainers then looked at reliability reports and t-tests and performed backreading to gauge consistency

with 1994 scoring and make adjustments in scoring where appropriate. The remainder of the responses

were released in equal amounts when scoring was one-third finished, two-thirds finished, and 90 percent

finished. Note that the time intervals between rescored sessions varied with the number of responses to

be scored per item.

Cross-year reliability results for each constructed-response item used in both 1998 and 1994 are

provided in Tables C-7 through C-12 in Appendix C.
122

7.4.5 Validity Sets Tool

In order to score a validity set, the table leader updated the scorers’ qualification to the same item

they were regularly scoring for the validity application. Then, when scorers opened the scoring window,

they received the validity papers. Validity papers, student responses prescored by the trainer during

rangefinding, were used to prevent reader drift over the course of scoring. All scorers were in effect

second scoring against the preloaded first scores. Unlike calibration sets, where all scorers read the same

responses, with the validity sets, each scorer received different responses. Since the validity papers were

under a separate application, the reliability reports and t-tests and backreading were available

independently of the regular scoring. Before the next time the validity sets were used, the table leader

used a tool to reset the items to make them available for scoring again, and also reset the reliability

statistics. They accomplished this by executing a command in the report menu that then prompted them

for a topic name. When the system carried out this command, it reset scoring and statistics only for the

batch involved in the validity process.

7.4.6 t-Tests

To perform a t-test, the table leader executed a command in the report window that prompted the

table leader for the item, the application, and the cubicle to which the item was assigned. The system then

displayed an analysis of the data, which could be printed. The test results were based only on responses

for which both scores were on-task. The display showed number of scores compared, number of scores

with exact agreement, percent of scores with exact agreement, mean of the preloaded scores, mean of the

currently assigned scores, mean difference, variance of the mean difference, standard error of the mean

difference, and the t value.

7.4.7 Procedure for Monitoring Interrater Reliability

During the scoring of an item or the scoring of a calibration set, table leaders monitored progress

using interrater reliability. This was done using a computer display that functioned in either of two

modes: (1) to display information of all first readings versus all second readings, or (2) to display all

readings of an individual that were also scored by other scorers versus the scores assigned by those other

scorers. The information was displayed as a matrix, with scores awarded during first readings displayed

in rows and scores awarded during second readings displayed in columns for mode one and the

individual’s scores in rows and all other scorers in columns for mode two. In this format, instances of

exact agreement fell along the diagonal of the matrix. For completeness, data in each cell of the matrix

contained the number and percentage of cases of agreement (or disagreement). The display also

contained information on the total number of second readings and the overall percentage of reliability on

the item. Also, the computer program provided on demand a separate calculation for exact and adjacent

agreement rates for each writing item. Since the interrater reliability reports were cumulative, a printed

copy of the reliability of each item was made periodically and compared to previously generated reports.

Scoring staff saved printed copies of all final reliability reports and archived them with the training sets.

7.4.8 Process for Monitoring Frequency Distribution of Scores

For each topic, table leaders could run a report that showed the frequency distribution of scores.

The report displayed separate frequencies for first and second scores. For each score level, the report

showed the number of responses as an integer and as a percentage of the total. The report could be

updated and printed on demand.
123

7.4.9 Process for Monitoring the Rate of Scoring

The table leaders were able to monitor work flow for each item using a status tool that displayed

the number of responses scored, the number of responses first-scored that still needed to be secondscored, the number of responses remaining to be first-scored, and the total number of responses

remaining to be scored. This allowed the team leaders and performance assessment specialists to

accurately monitor the rate of scoring and to estimate the time needed for completion of the various

phases of scoring.

7.4.10 Scoring Buttons

To assign a score, scorers clicked the mouse over a button displayed in the scoring window.

Since buttons included only valid score values, there was no editing for out-of-range scores.

7.5 PAPER SCORING

The 1998 NAEP assessment used paper scoring only for one item, the “tax form” item in grade

12 reading. The tax form items were packaged into sets of 20. The development staff printed score sheets

with the identification numbers for the 20 books contained in each packet on a score sheet. Separate

score sheets were printed for the responses selected for second scoring. As soon as the last student

response on any score sheet was completed, the score sheets were collected and taken to a central clerical

support area to be scanned on the NCS paper-based scoring system using OpScan 7 scanners. As each

sheet was processed, the scanning system edited the incoming data against tables to ensure that all

responses were scored with one and only one valid score, and that only raters who were qualified to score

an item scored it. Any discrepancies (e.g., no score assigned, double gridding, out-of-range scores, or

invalid scorer identification numbers) were flagged and resolved before the data from that sheet were

accepted into the scoring system database. Interrater agreement reports were generated on demand.

All score data were stored on personal computers at NCS as the responses were scanned. When

scoring was completed, the scanner operator ran a query to make sure that all score sheets were

accounted for. Once all edits were corrected, the PC file was renamed and put into an export file, which

automatically created the mainframe file. This file was then uploaded to the mainframe to be merged

with the mainframe student files.

7.6 LARGE-PRINT BOOKS AND OTHER SPECIAL ACCOMMODATIONS

NCS’s Performance Assessment Scoring Center (PSC) scored responses for a number of students

whose special accommodations made the books nonscannable. These included large-print books as well

as responses typed on a separate sheet of paper outside the booklet. Altogether, there were 37 such books

for reading, 3 for civics, and 61 for writing.

Since the books were nonscannable, they were transported to the scoring center after processing.

Clerical staff created a log to account for all the special accommodations books and a score sheet for

each book listing the constructed-response items in that book. The books were routed to the table leaders

in charge of the different items in each book. As the team scored an item, the table leader marked the

score for that response, his or her scorer identification number, and the date scored. Once all items in

each book for a given subject were scored, the scoring sheets were returned to development staff to enter

those scores manually into the records for those books.
124

7.7 TRAINING

The training on each item was conducted by subject-area specialists from ETS and NCS. Dates

for training and scoring can be found in Table 7-1. All of the assessments were scored item-by-item so

that each scorer worked with only one set of rubrics at a time. After scoring all available responses, a

team then proceeded with training and scoring the next item.

Training involved explaining the item and its scoring rubric to the team and discussing types of

student responses that represented the various score points in the guide. Typically, two or three student

responses were chosen to anchor each score point. When review of the anchor packet was completed, the

scorers scored 10 to 20 “practice papers,” previously scored by subject-area specifications that

represented the entire range of score points the item could receive. The trainer then led the team in a

discussion of the practice papers to focus the scorers on how the scoring rubrics should be interpreted.

After the trainer and table leader determined that the team had reached consensus, the table leader then

released work on the image-scoring system to the scorers. The scorers initially took turns reading aloud

their first “live” responses to the team or worked in pairs as a final check before beginning work

individually. Once the practice session was completed, the formal scoring process began.

During training, scorers and the table leader kept notes of scoring decisions. The table leader was

then responsible for compiling those notes and ensuring that all scorers were in alignment with the

decisions. Teams varied greatly in the amount of time spent scoring as a group before breaking into

individual scoring. This time ranged from five minutes to five hours.

7.8 SCORING

All scoring for each item was conducted via computer image except for the grade 12 reading “tax

form” item. During scoring, the table leaders continued to compile notes on scoring decisions for the

scorers’ reference and guidance. Additionally, table leaders closely monitored interrater reliability using

both team and individual statistics as a reference. Consistently throughout the scoring of each item, the

table leaders also performed backreading duties in which they reviewed a sample of the responses scored

by each scorer on the team. The table leaders and performance assessment specialists continuously

monitored the progress of each team and noted all scoring-related decisions to ensure that training and

scoring progressed smoothly and in a timely manner.

7.9 INTERRATER RELIABILITY

A subsample of the reading, writing, and civics responses for each item were scored by a second

scorer to obtain statistics on interrater reliability. In general, items administered only to the national

sample received 25 percent second scoring, while those given in both the national and state samples

received less. Thus, all civics items received 25 percent second scoring; all grade 12 reading received 25

percent second scoring; grades 4 and 8 reading items received 6 percent second scoring; grades 4 and 12

writing received 25 percent second scoring, and grade 8 writing items received 10 percent second

scoring, except for the three 50-minute prompts, which received 25 percent second scoring because they

were administered only in the national sample. The reliability information obtained from second scoring

was also used by the team leaders to monitor the capabilities of all scorers and maintain uniformity of

scoring across scorers. Reliability reports were generated on demand by the table leader, team leader, or

performance assessment specialist as needed. They were displayed at a computer workstation. Printed

copies were reviewed daily by both NCS and ETS lead scoring staff. In addition to the immediate

feedback provided by the on-line reliability reports, each table leader could also review the actual

responses scored by a scorer by using the backreading tool (see Section 7.4.2). In this way, the table
125

leader was able to monitor each scorer carefully and correct difficulties in scoring almost immediately

with a high degree of efficiency. Table 7-2 provides the interrater reliability ranges.

Table 7-2

Interrater Reliability Ranges for the NAEP 1998 Assessment

Number and Percentage of Items in Percentage Exact Agreement Range

60–69% 70–79% 80–89% Above 90%

Grade

Total Number

of

Unique Items Number Percent Number Percent Number Percent Number Percent

Reading

4 46 — — 3 6.5 16 34.7 27 58.6

8 69 1 1.4 4 5.8 28 40.6 36 52.2

12 76 1 1.3 4 5.2 36 47.4 35 46.1

Writing

4 20 4 20.0 16 80.0 — — — —

8 23 18 78.3 4 17.4 — — — —

12 23 10 43.5 9 39.1 3 13.0 — —

Civics

4 21 — — 3 14.3 11 52.4 7 33.3

8 28 1 3.6 6 21.4 17 60.7 4 14.3

12 29 — — 8 27.6 20 70.0 1 3.4

Detailed results of interrater scoring reliability for the reading, writing, and civics constructedresponse items are provided in Appendix C.

7.9.1 Scoring of Reading

The reading portion of the 1998 NAEP assessment included a total of 154 discrete constructedresponse items. Four items were scored on an accelerated schedule between March 23 and 27. Scoring

for the rest of the items took place between March 30 and April 24. The items scored included shortanswer constructed responses and extended constructed responses. Each constructed-response item had a

unique scoring rubric that identified the range of possible scores for the item and defined the criteria to

be used in evaluating student responses. Note that these numerical values were for scoring only; they do

not reflect the IRT-based scores used in analysis of the data. Chapter 15 describes the IRT values used in

the data analysis.

During the course of the project, each team scored constructed-response items using a 2-, 3-, or

4-point scale as outlined below:

Dichotomous Items

1 = unacceptable response

2, 3, or 4 = acceptable response
126

(Items that originated in the 1992 NAEP used 1 and 4 for dichotomously scored items; items from the

1994 NAEP used 1 and 3; items developed in the 1997 field test used 1 and 2.)

Short Three-Point Items

1 = evidence of little or no comprehension

2 = evidence of partial or surface comprehension

3 = evidence of full comprehension

Extended Items

1 = unsatisfactory

2 = partial

3 = essential

4 = extensive

Table 7-3 lists the number of reading constructed-response items by item type and score-point

level.

Table 7-3

Number of Constructed-Response Items by Score-Point Levels

for the 1998 NAEP Reading Assessment

2- 3- 4-

Item Type Grade Category Category Category Total

Reading Items – Total

4 19 11 6 36

4/8 8 — 2 10

8 11 16 5 32

8/12 13 9 5 27

12 22 19 8 49

Reading Items – New in 1998

4 32 1 6

4/8 — — — —

8 14 1 6

8/12 2 4 1 7

12 — — 1* 1

Reading Items – Trend from 1994

4 16 9 5 30

4/8 8 — 2 10

8 10 12 4 26

8/12 11 5 4 20

12 22 19 7 48

* Even though the grade 12 tax form stimulus had been used in previous assessments, it is counted here

as a new item, because no rescoring was done and it was not used to measure trend.

Note: “—” indicates that this category was not applicable.
127

7.9.2 Scoring of Writing

The writing portion of the 1998 NAEP assessment included a total of 66 discrete constructedresponse items. Scoring was conducted from April 28 to July 1. The amount of space given students to

respond ranged from four pages for the 25-minute prompts to eight pages for the 50-minute prompts.

Trainers used generic holistic scoring guides for each grade that identified the range of possible scores

for the item and defined the criteria to be used in evaluating student responses. Note that these numerical

values were for scoring only; they do not reflect the IRT-based scores used in analysis of the data.

Chapter 19 describes the IRT values used in the data analysis.

All writing scoring rubrics used a six-point scale as follows:

6 = excellent response

5 = skillful response

4 = sufficient response

3 = uneven response

2 = insufficient response

1 = inappropriate (grade 4) or unsatisfactory (grade 8 and 12) response

The IRT numerical values used in analysis of the data are described in Chapter 19. Table 7-4 lists

the number of writing constructed-response items by item type and score-point level.

Table 7-4

Number of Constructed-Response Items by Score-Point Levels

for the 1998 NAEP Writing Assessment

Item Type Grade 6-Category Total

Writing Items

4 20 20

8 23 23

12 23 23

Prewriting Items

4 20 20

8 23 23

12 23 23

7.9.2.1 Selective Rescoring

To address problems of low reliability at the upper-score levels, the ETS staff chose 13 prompts

at grade 4, 9 at grade 8, and 8 at grade 12 to conduct a selective rescoring of responses. For each prompt

involved in the selective rescoring, all responses that received either a first or second score of 5 or 6 were

downloaded again to the scoring center. Specially selected trainers prepared additional training material

focusing on the upper-level scores. One trainer did all of the grade 4 selectively rescored items with the

team that the trainer had worked with throughout the project. Three trainers, each with a specially

selected team of 10 scorers, prepared and carried out the rescoring for the grade 8 responses. One team

rescored responses to narrative prompts, another rescored responses to informative prompts, and the third

worked exclusively on persuasive prompts. At grade 12, one trainer and team rescored responses to six of

the prompts, while another trainer and group rescored two. Scores of 5 and 6 from the original scoring

were deleted from the active files, though copies were maintained to provide an audit trail. All frequency
128

distributions and interrater agreement reports attached to this report show the status of the items after the

selective rescoring was finished.

7.9.2.2 Prewriting Coding

All students were given a blank page to use for prewriting planning. Codes were developed for

the type of prewriting planning students did during the assessment. Prewriting coding took place during

the evening shift from May 11 through 26, working 41/2 hours from 6:00 p.m. to 10:30 p.m. The first

evening, the ETS writing coordinator trained the table leaders, who in turn trained their teams of clerical

scorers the following evening.

The coders classified the prewriting strategies for all items using the same coding guide, anchor

set, and practice papers. All coding was completed by May 26.

The codes used to classify prewriting were as follows:

1 = rough draft

2 = list

3 = outline

4 = diagram

6 = picture

7 = multiple

Note that when a response showed multiple prewriting strategies the different, specific strategies

used by a student were not recorded by the coders. Also note that the code value of “5” was originally

planned to indicate that the student used a table as a prewriting strategy. However, that category was

eliminated before training began.

7.9.3 Scoring of Civics

The civics portion of the 1998 NAEP assessment included a total of 78 discrete constructedresponse items. It was scored from April 27 to May 11 on an evening shift that ran from 6:00 p.m. to

10:30 p.m. The items scored included short-answer constructed responses and extended constructed

responses. Each constructed-response item had a unique scoring rubric that identified the range of

possible scores for the item and defined the criteria to be used in evaluating student responses.

During the course of the scoring, each team scored constructed-response items using a 3- or 4-

point scale as outlined below:

Short Item

1 = unacceptable

2 = partial

3 = acceptable
129

Extended Items

1 = unacceptable

2 = partial

3 = acceptable

4 = complete

The IRT numerical values used in analysis of the data are described in Chapter 23. Table 7-5 lists

the number of constructed-response items by item type and score-point level.

Table 7-5

Number of Constructed-Response Items by Score-Point Levels

for the 1998 NAEP Civics Assessment

3- 4-

Item Type Grade Category Category Total

Civics Items

4 15 6 21

8 22 6 28

12 23 6 29

7.10 PREPARATION FOR TAPE CREATION

The 1998 NAEP assessment data collection resulted in several classes of data files—student,

school, teacher, SD/LEP student, student/teacher match, and student-response information. Studentresponse information included response data from all assessed students in 1998. Data resolution activities

occurred prior to the submission of data files to ETS and Westat to resolve any irregularities that existed.

7.11 UPLOADING OF SCORES TO THE NAEP DATABASE

An important quality control component of the image-scoring system was the inclusion, for

purposes of file identification, of an exact copy of the student edit record, including the student booklet

identification number, with every image of a student’s response to a constructed-response item. When all

the responses for an individual item had been scored, the system automatically submitted all item scores

assigned during the scoring, along with their edit records, to a queue to be transmitted to the mainframe.

A custom edit program matched the edit records of the scoring files to those of the original edit records

on the mainframe. As matches were confirmed, the scores were applied to those individual files.

7.12 SD/LEP STUDENT QUESTIONNAIRES

SD/LEP questionnaires were completed for those students who were selected to participate in the

assessment sample and were classified as students with disabilities (SD), or were categorized as limited

English proficient (LEP) students. This questionnaire, which was completed by someone at the school

knowledgeable about the student, asked about the student’s background and the special programs in

which the student participated. NCS processed the SD/LEP student questionnaires via optical mark

recognition (OMR) scanning. Edits performed on the questionnaires assured that responses to questions

fell within the valid range for that question. SD/LEP questionnaires were then matched to a student

record. SD/LEP questionnaires that were not matched to a student document were cross-referenced with
130

the corresponding administration schedule, roster of questionnaires, and student data files to correct, if

necessary, the information needed to result in a match.

7.13 SCHOOL QUESTIONNAIRES

In 1998, NCS continued to use intelligent character recognition (ICR) technology to capture

percentage figures written by school personnel directly in boxes on the school questionnaire, rather than

requiring the school official to grid ovals in a matrix. The data were then verified by an edit operator.

7.14 TEACHER QUESTIONNAIRE MATCH

The same processes that were followed in previous cycles were used in 1998 to achieve the best

possible student/teacher match rate. Student identification numbers that were not matched to a teacher

questionnaire were cross-referenced with the corresponding administration schedule and roster of

questionnaires to verify (and change, if necessary) the teacher number, teacher period, and questionnaire

number recorded on these control documents. The NAEP school identification numbers listed on the

roster of questionnaires and teacher questionnaire were verified and corrected, if necessary. Once these

changes were made, any duplicate teacher numbers existing within a school were, if possible, crossreferenced for resolution with the roster(s) of questionnaires. Since this information was located together

on a single, central control document, the ability to match and resolve discrepant or missing fields was

simplified.

7.15 DELIVERY

After all data-processing activities were completed, data cartridges, or diskettes were created and

shipped via overnight delivery to ETS or Westat. NCS maintains a duplicate archive file for security and

back-up purposes.

7.16 STORAGE OF DOCUMENTS

After batches of processed documents had successfully passed the editing process, they were sent

to the NCS warehouse for storage. Due to the large number of rescoring projects done with NAEP

material, the documents were unspiraled and sequenced by grade and book type after all of the

processing and scoring was completed. This allows for efficient document retrieval to fill requests for

specific booklets or book types for future projects. Unspiraled and sequenced booklets were then

assigned a new inventory number by grade and book type and were sent back to the warehouse for

storage. The storage locations of all documents were recorded on the inventory control system.

7.17 QUALITY CONTROL DOCUMENTS

ETS required that a random sample of books be pulled for an additional quality control check.

The 1998 NAEP assessment of reading, writing, and civics documents to be scored were all image

scanned (aside from the exception noted previously). For image-scanned documents, a scoring sheet was

not used, so ETS used scores sent to them on a data tape to verify the accuracy of applied scores. All of

these documents were selected prior to sending the booklets to storage and were then sent to ETS to

verify the accuracy and completeness of the data. A random sample of all the questionnaires used in the

1998 NAEP assessment was also sent to ETS along with the quality assurance booklets used for

processing and scoring. The quality control analyses of these booklets are discussed in Chapter 8.
131

Chapter 8

CREATION OF THE DATABASE, QUALITY CONTROL OF

DATA ENTRY, AND CREATION OF THE DATABASE PRODUCTS1

 John J. Ferris, Katharine E. Pashley, David S. Freund, and Alfred M. Rogers

Educational Testing Service

8.1 INTRODUCTION

The data-processing, scoring, and editing procedures described in Chapters 6 and 7 resulted in

the generation of disk and tape files containing various data for students (assessed and excluded),

teachers, schools, and SD/LEP (students with disabilities and students with limited English proficiency)

information. The weighting procedures described in Chapters 10 and 11 resulted in the generation of data

files that included the sampling weights required to make valid statistical inferences about the population

from which the 1998 fourth-, eighth- and twelfth-grade NAEP samples were drawn. These files were

merged into a comprehensive, integrated database. The creation of the database is described in Section

8.2.

Section 8.2.2 describes a central repository or master catalog of this information. The master

catalog is accessible by all analysis and reporting programs and provides correct parameters for

processing the data fields and consistent labeling for identifying the results of the analyses.

To evaluate the effectiveness of the quality control of the data-entry process, the corresponding

portion of the final integrated database was verified in detail against a sample of the original instruments

received from the field. The results of this procedure are given in Section 8.3.

The integrated database was the source for the creation of the NAEP item information database

and the NAEP secondary-use data files. These are described in Section 8.4.

8.2 CREATION OF THE DATABASE

The data processing conducted by National Computer Systems (NCS) resulted in the transmittal

to ETS of four data files for each of fourth, eighth and twelfth grade: one file for the student background

and item-response data and one file for each of the three questionnaires—teacher, school characteristics

and policies, and SD/LEP. The sampling weights, derived by Westat, comprised additional files for each

grade. (See Chapters 10 and 11 for a discussion of the sampling weights.) These files at each grade were

the foundation for the analysis of the 1998 NAEP data. Before data analyses could be performed, these

data files had to be integrated into a coherent and comprehensive database.

 The database ultimately comprised four files per cohort: three student files (reading, writing, and

civics) and a single school file. The student files were separated by subject area to improve maintenance

and efficiency of the databases and data analyses. Each record on the student file contained a student’s

responses to the particular assessment booklet the student was administered (in the case of excluded

 

1 John J. Ferris was responsible for the evaluation of the quality of the database and the data-entry process; Katharine E. Pashley

was responsible for database generation under the supervision of David S. Freund; Alfred M. Rogers created the secondary-use

data files.
132

students, a booklet was assigned, but the student-response fields contain a special code indicating no

response), and the information from the questionnaire that the student’s teacher completed. Additionally,

for a student (assessed or excluded) who was identified as a student with a disability (SD) or of limited

English proficiency (LEP), the data from the SD/LEP questionnaire are included. This questionnaire is

filled out for all students both assessed and excluded, identified as SD, LEP, or both. (See Chapter 2 for

information regarding assessment instruments.) Also added to the student files were variables with

school-level information supplied by Quality Education Department, Inc. (QED), including demographic

information about schools such as distributions of student populations by race/ethnicity. Since the

teacher data are not from a representative sample of teachers and since the focus of NAEP is to report

student-level results, the teacher-response data were added to the student records in cases where the

student’s teacher responded to a teacher questionnaire. The school data were on separate files that could

be analyzed on their own and could also be linked to the student files through the unique school

identification code.

The creation of the student data files for fourth, eighth, and twelfth grade began with the

reorganization of the data files received from NCS. This involved two major tasks:

1. The files were restructured, eliminating unused (blank) areas to reduce the size of the

files.

2. In cases where students had chosen not to respond to an item, the missing responses

were recoded as either “omit” or “not reached,” as discussed in Chapter 12 of this

report.

8.2.1 Merging Files

Following the reorganization of data files, the student-response data were merged with the

student-weights files. The resulting file was then merged with the SD/LEP and teacher data. In all

merging steps, the 10-digit booklet identification (the 3-digit booklet number common to every booklet

with the same block of items, a 6-digit serial number unique to the booklet a student was given, and a

single check digit) was used as the matching criterion. The teacher data can be linked to the student data

through four data variables: primary sampling unit (PSU), school code, teacher ID, and classroom period.

The school file for each grade was created by merging the school characteristics and policies

questionnaire file with the file of school weights and school variables, supplied by Westat. The PSU and

school codes were used as the matching criteria. Since some schools did not return a questionnaire, some

of the records in the school file contained only school-identifying information and sampling-weight

information. The school data can be linked to the student data through the PSU and school code

variables.

When the student and school files for each grade had been created, the database was ready for

analysis. In addition, whenever new data values (such as composite background variables or plausible

values) were derived, they were added to the appropriate database files using the same matching

procedures described above.

For archival purposes and to provide data for outside users, restricted-use data files and

codebooks for each jurisdiction in the state assessment were generated from this database. The restricteduse data files contain all responses and response-related data from the assessment, including responses

from the student booklets, teacher questionnaires, and school characteristics and policies questionnaires,

scale scores, sampling weights, and variables used to compute standard errors.
133

8.2.2 Creating the Master Catalog

A critical part of any database is its processing control and descriptive information. Having a

central repository for this information, which may be accessed by all analysis and reporting programs,

will provide correct parameters for processing the data fields and consistent labeling for identifying the

results of the analyses. The NAEP master catalog file was designed and constructed to serve these

purposes for the NAEP database.

Each record of the master catalog contains the processing (e.g., response options), labeling,

classification (e.g., content), and location information for each assessment exercise and other data

variables in the NAEP database. The control parameters are used by the access routines in the analysis

programs to define the manner in which the data values are to be transformed and processed.

Each data variable has a 50-character label in the master catalog describing the contents of the

variable and, where applicable, the source of the variable. The variables with discrete or categorical

response values (e.g., multiple-choice items and professionally scored items, but not weight variables)

have additional label fields in the catalog containing 8- and 20-character labels for those response values.

These short labels can be used for reporting purposes as a concise description of the responses for these

discrete items.

The classification area of the master catalog record contains distinct fields corresponding to

predefined classification categories (e.g., reading purpose and reading stance) for the data variables. For

a particular classification variable, a nonblank value indicates the code of the subcategory within the

classification category for the data variable. This classification area permits the grouping of identically

classified items or other variables by performing a selection process on one or more classification fields

in the master catalog.

According to NAEP design, it is possible for assessment exercises to appear in more than one

student sample and in more than one block of exercises within each sample. The location fields of the

catalog record contain age cohort, block, and, where applicable, the order within the block for each

appearance of the assessment exercise.

The master catalog file was constructed concurrently with the collection and transcription of the

national and state assessment data so that it would be ready for use by analysis programs when the

database was created. As new data fields were derived and added to the database, their corresponding

descriptive and control information were entered into the master catalog.

8.3 QUALITY CONTROL OF NAEP DATA ENTRY FOR 1998

This section describes the evaluation of the data-entry process for the 1998 national assessment.

As in past years, the NAEP database was found to be more than accurate enough to support the analyses

that were done. Overall, the observed error rates were comparable to those of past assessments, including

those of the teacher questionnaires, which returned to more typical levels after displaying a somewhat

elevated error rate in 1996. Derived error rate limits were around one error per thousand responses except

for the school questionnaire data, which was nearly five per thousand (see discussion below).

The purpose of the analysis reported in this section is to assess the quality of the data resulting

from the complete data-entry system, beginning with the actual instruments collected in the field and

ending with the final machine-readable database used in the analyses. The process involved the selection

of instruments at random from among those returned from the field and the comparison of each entire
134

instrument, character by character, with its representation in the final database. In this way, we were able

to measure the error rates in the data as well as the success of the data-entry system.

Of course the observed error rate cannot be taken at face value. For example, the sample of

school questionnaires that happened to be selected for close inspection contained two errors out of a total

of 2,251 characters. To conclude that the entire school questionnaire database has an error rate of 2

2,251 ,

or .0009, would be too optimistic; we may simply have been lucky (or unlucky) with this particular

random sample. What is needed is an indication of how bad the true error rate might be, given what we

observed. Such an indication is provided by confidence limits. Confidence limits indicate how likely it is

that a value falls inside a specified range in a specified context or distribution. In our analysis, the

specified range is an error rate between zero and some maximum value beyond which we are confident at

a specified level (traditionally 99.8%) that the true error rate does not lie (for the school questionnaires,

this error rate is .0046). The specified context or distribution turns out to be the cumulative binomial

probability distribution. An example will demonstrate this technique:

Let us say that 1,000 booklets were processed, each with 100 characters of data

transcribed for a total of 100,000 characters. Let us say further that 5 of these characters

were discovered to be in error in a random sample of 50 booklets that were completely

checked; in other words, five errors were found in a sample of 5,000 characters. The

following expression may be used to establish the probability that the true error rate is

.0025 or less, rather than the single-value estimate of the observed rate, one in a thousand

(.001):

( )( ) 5

5000

0

5000 .0025 1 .0025 .0147 j j

j j −

=

 

  × ×− =    

∑

This is the sum of the probability of finding five errors plus the probability of finding

four errors plus . . . etc. . . . plus the probability of finding zero errors in a sample of

5,000 with a true error rate of .0025; that is, the probability of finding five or fewer

errors by chance when the true error rate is .0025. Notice that we did not use the size of

the database in this expression. Actually, the assumption here is that our sample of 5,000

was drawn from a database that is infinite. The smaller the actual database is, the more

confidence we can have in the observed error rate; for example, had there been only

5,000 in the total database, our sample would have included all the data, and the

observed error rate would have been the true error rate. The result of the above

computation allows us to say, conservatively, that .0025 is an upper limit on the true

error rate with 98.53 percent (i.e., 1 - .0147) confidence; that is, we can be quite sure that

our true error rate is no larger than .0025. As noted above, in NAEP quality control we

use a more stringent confidence limit of 99.8%, which yields an even more conservative

upper bound on the true error rate; with 99.8% confidence, we would state that the true

error rate in this example is no larger than .0031, rather than .0025.

Calculations of true probabilities based on a combinatorial analysis have been done (e.g., Grant,

1964). Even when the sample was as much as 10% of a population of 50, the estimate of the probability

based on the binomial theorem was not much different from the correct probability. NAEP does not

sample at a rate greater than about 2%. Thus, the computations of the upper limits on the true error rates

based on the binomial theorem are likely to be highly accurate approximations.
135

The individual instruments are briefly discussed in the following sections and a summary table

(Table 8-1) gives the upper 99.8 percent confidence limit for the error rate for each of the instruments as

well as the sampling information. The 99.8 percent confidence limit and the selection rates indicated

were chosen to make these results comparable to those of administrations since 1983, all of which used

the same parameters.

Table 8-1

Summary of Quality Control Error Analysis for NAEP 1998 Data Entry

 Instrument/ Sample

Selection

Rate

Different

Booklets

Number

of

Booklets

Sampled

Number

of

Characters

Sampled

Number

of

Errors

Observed

Error

Rate

Upper

99.8%

Confidence

Limit

 Student Booklets – Nat’l. Main

 SD/LEP Student Questionnaires

 Teacher Questionnaires

School Characteristics and

 Policies Questionnaires

1/278

1/77

1/68

1/53

266

 3

 4

 3

509

217

 131

40

29,802

19,964

 14,811

 2,251

16

 8

6

2

.0005

.0004

.0004

.0009

.0011

.0010

.0012

.0046

8.3.1 Student Booklet Data

Data from about 140,000 students were processed across all samples in this assessment. Roughly

one booklet in 278 was selected for close examination, which is a somewhat higher rate than that used in

past assessments, when a rate of approximately one in 350 was used. The higher selection rate improves

the chance of drawing sufficient numbers of each booklet when there is a large number of different

books. The student data error rates were consistently low in all subject areas and across all three grades,

typically involving an occasional multiple response taken as a single one. The overall quality of the data

was very high.

8.3.2 SD/LEP Student Questionnaire Data

In this assessment, 16,703 SD/LEP student questionnaires were scanned. The quality control

sampling rate was 1 in 77, a somewhat higher rate than that used in previous assessments. The data

showed about the same error rate as that in the previous assessment—comparable to the rate for the

student data. The few problems encountered involved the scanner’s mistaking an erasure for a genuine

response or failing to identify a multiple response as such.

8.3.3 Teacher Questionnaire Data

In this assessment, 8,959 teacher questionnaires were collected and scanned. About 1.5 percent

of these questionnaires was sampled for the quality control procedure. The error rates for these

questionnaires were about the same as for the student categories of data, and much improved over the

1996 error rates. Since there has been no significant change in the format of these questionnaires, the

improved error rates may be attributable to improved administration procedures.
136

8.3.4 School Characteristics and Policies Questionnaire Data

In this assessment, 2,102 school characteristics and policies questionnaires were collected. They

were sampled at a rate of about 1 in 53. Only two scanning errors were found in these questionnaires,

both of which involved the scanner’s failing to pick up a valid response. In spite of this apparently good

error rate of less than one in a thousand, the application of the binomial theorem yields an upper bound

on the true error rate of .0046 (at the same confidence level). While this may seem surprisingly high, an

error rate limit derived from an application of the binomial theorem is appropriate here, since the sample

population is large, as noted in the above discussion of the application of this technique.

8.4 NAEP DATABASE PRODUCTS

The NAEP database described to this point serves primarily to support analysis and reporting

activities that are directly related to the NAEP contract. This database has a singular structure and access

methodology that is integrated with the NAEP analysis and reporting programs. One of the directives of

the NAEP contract is to provide secondary researchers with a nonproprietary version of the database that

is portable to any computer system. In the event of transfer of NAEP to another client, the contract

further requires ETS to provide a full copy of the internal database in a format that may be installed on a

different computer system.

The secondary-use data files are designed to enable any researcher with an interest in the NAEP

database to perform secondary analysis on the same data as those used at ETS. The data, documentation,

and supporting files are distributed on CD-ROM media. For each sample in the assessment, the following

files are provided: the response data file, a printable data file layout and codebook file, a file of control

statements that will generate an SPSS system file, a file of control statements that will generate a SAS

system file, and a machine-readable catalog file. Each codebook is in portable document file (PDF)

format, which may be browsed, excerpted, and printed using the Adobe Acrobat Reader program on a

variety of platforms. Each machine-readable catalog file contains sufficient control and descriptive

information to permit the user who does not have either SAS or SPSS to set up and perform data analysis.

The remainder of this section summarizes the procedures used in generating the data files and

related materials.

8.4.1 File Definition

The design of the 1998 assessment perpetuates two features of the 1990, 1992, 1994, and 1996

assessment design: the focused BIB or PBIB booklet design and the direct matching of teacher

questionnaires to student assessment instruments. In addition, the sample of students who were excluded

from the assessment is now incorporated into the appropriate assessed student subject-area sample.

The focused BIB or PBIB design within the main assessment isolates the primary subject areas to

separate groups of booklets. This permits the division of the main sample into subject-specific

subsamples. The data files generated from these subsamples need only contain the data that are relevant

to their corresponding subject areas and are therefore smaller and more manageable than their

counterparts in previous assessments.

The intent of the 1998 assessment design was to collect data from the reading, writing, or civics

teachers of fourth-grade and eighth-grade students who participated in the assessments of, respectively,

reading, writing, or civics. A portion of the teacher questionnaire contained questions that were directly

related to each matched student. This change in the design afforded a very high matching rate between
137

student and teacher data. Therefore, for those subject areas in each grade cohort for which teacher data

were collected, the teacher responses were appended to each student record in the secondary-use data

files.

8.4.2 Definition of the Variables

The initial step in the variable definition process was the generation of a labels file of descriptors

of the variables for each data file to be created. Each record in a labels file contains, for a single data

field, the variable name, a short description of the variable, and processing control information to be used

by later steps in the data-generation process. This file could be edited for deletion of variables,

modification of control parameters, or reordering of the variables within the file. The labels file is an

intermediate file only; it is not included on the released data files.

The variables on all data files are grouped and arranged in the following order: identification

information, weights, derived variables, scale scores (where applicable), and response data. On the

student data files, these fields are followed by the teacher-response data and the SD/LEP student

questionnaire data, where applicable. The identification information is taken from the front covers of the

instruments. The weight data include sample descriptors, selection probabilities, nonresponse

adjustments, and replicate weights for the estimation of sampling error. The derived data include sample

descriptions from other sources and variables that are derived from the response data for use in analysis

or reporting.

For each subject area of the 1998 assessment, the item-response data within each block of

questions (see Section 1.5) were left in their order of presentation. The responses to cognitive blocks that

were not present in a given booklet were left blank, signifying a condition of “missing by design.”

In order to process and analyze the spiral sample data effectively, the user must also be able to

determine, from a given booklet record, which blocks of item response data were present and their

relative order in the instrument. This problem was remedied by the creation of a set of control variables,

one for each block, which indicated not only the presence or absence of the block but its order in the

instrument. These control variables are included with the derived variables.

8.4.3 Data Definition

To enable the data files to be processed on any computer system using any procedural or

programming language, it was desirable that the data be expressed in numeric format. This was possible,

but not without the adoption of certain conventions for re-expressing the data values.

During creation of the NAEP database, the responses to all multiple-choice items were

transcribed and stored in the database using the letter codes printed in the instruments. This scheme

afforded the advantage of saving storage space for items with 10 or more response options, but at the

expense of translating these codes into their numeric equivalents for analysis purposes. The response data

fields for most of these items would require a simple alphabetic-to-numeric conversion. However, the

data fields for items with 10 or more response choices would require “expansion” before the conversion,

since the numeric value would require two column positions. One of the processing control parameters

on the labels file indicates whether or not the data field is to be expanded before conversion and output.

The ETS database contained special codes to indicate certain response conditions: “I don’t

know” responses, multiple responses, omitted responses, not-reached responses, and unresolvable

responses, which include out-of-range responses and responses that were missing due to errors in printing
138

or processing. The scoring guides for the reading, writing, and civics constructed-response items included

additional special codes for ratings of “illegible,” “off task,” or nonrateable by the scorers. All of these

codes had to be re-expressed in a consistent numeric format.

The following convention was adopted and used in the designation of these codes: The

“illegible” response codes were converted to 5, the “off task” response codes were converted to 6, the “I

don’t know” and nonrateable response codes were converted to 7, the “omitted” response codes were

converted to 8, the “not reached” response codes were converted to 9, and the multiple-response codes

were converted to 0, and the out-of-range and missing responses were coded as blank fields,

corresponding to the “missing by design” designation.

This coding scheme created conflicts for those multiple-choice items that had seven or more

valid response options as well as the “I don’t know” response and for those constructed-response items

whose scoring guide had five or more categories. These data fields were also expanded to accommodate

the valid response values and the special codes. In these cases, the special codes were “extended” to fill

the output data field: The “I don’t know” and nonrateable codes were extended from 7 to 77, the omitted

response codes were extended from 8 to 88, and so on.

Each numeric variable on the secondary-use files was classified as either continuous or discrete.

The continuous variables include the weights, scale scores, identification codes, and questionnaire

responses where counts or percentages were requested. The discrete variables include those items for

which each numeric value corresponds to a response category. The designation of “discrete” also

includes those derived variables to which numeric classification categories have been assigned. The

constructed-response items were treated as a special subset of the discrete variables and were assigned to

a separate category to facilitate their identification in the documentation.

8.4.4 Data File Catalogs

The catalog file is created by the GENCAT program from the labels file and the 1998 master

catalog file. Each record on the labels file generates a catalog record by first retrieving the master catalog

record corresponding to the field name. The master catalog record contains usage, classification, and

response code information, along with positional information from the labels file, field sequence number,

output column position, and field width. Like the labels file, the catalog file is an intermediate file and is

not included on the released data files.

The information for the response codes consists of the valid data values for the discrete numeric

fields, and a 20-character description of each. The GENCAT program uses additional control information

from the labels file to determine if extra response codes should be generated and saved with each catalog

record. The first flag controls generation of the “I don’t know” or nonrateable response code; the second

flag regulates omitted or not-reached code generation; and the third flag denotes the possibility of

multiple responses for that field and sets up an appropriate response code. All of these control

parameters, including the expansion flag, may be altered in the labels file by use of a text editor, in order

to control the generation of data or descriptive information for any given field.

The catalog file supplies control and descriptive information for many of the subsequent

secondary-use data-processing steps.
139

8.4.5 Data File Layouts

The data file layouts are the first user product to be generated in the secondary-use data files

process. The generation program, GENLYT, uses a catalog file as input and produced a printable file.

The layout file is little more than a formatted listing of the catalog file.

Each line of the layout file contains the following information for a single data field: sequence

number, field name, output column position, field width, number of decimal places, data type, value

range, key or correct response value, and a short description of the field. The sequence number of each

field is implied from its order on the labels file. The field name is an 8-character label for the field that is

to be used consistently by all secondary-use data files materials to refer to that field on that file. The

output column position is the relative location of the beginning of that field on each record for that file,

using bytes or characters as the unit of measure. The field width indicates the number of columns used in

representing the data values for a field. If the field contains continuous numeric data, the value under the

number of decimal places entry indicates how many places to shift the decimal point before processing

data values.

The data type category uses five codes to designate the nature of the data in the field: Continuous

numeric data are coded “C”; discrete numeric data are coded “D”; constructed-response item data are

coded either “OS” (if the item was dichotomized for scaling) or “OE” (if it was scaled under a

polytomous response model). Additionally, the discrete numeric fields that include “I don’t know”

response codes are coded “DI.” If the field type is discrete numeric, the value range is listed as the

minimum and maximum permitted values separated by a hyphen to indicate range. If the field is a

response to a scorable item, the correct option value, or key, is printed. If the field is an assigned score

that was scaled as a dichotomous item using cut-point scoring, the range of correct scores is printed. Each

variable is further identified by a 50-character descriptor.

8.4.6 Data Codebooks

The data codebook is a printed document containing complete descriptive information for each

data field. Most of this information originates from the catalog file, while the remaining data comes from

the counts file and the IRT parameters file.

Each data field receives at least one line of descriptive information in the codebook. If the data

type is continuous numeric, no more information is given. If the variable is discrete numeric, the

codebook lists the response codes, response-code labels, and frequencies of each value in the data file.

Additionally, if the field represents an item used in IRT scaling, the codebook lists the parameters used

by the scaling program.

Certain blocks of cognitive items in the 1998 assessment that are to be used again in later

assessments for trend comparisons have been designated as nonreleased. In order to maintain their

confidentiality, generic labels have been substituted for the response category descriptions of these items

in the data codebooks and the secondary-use files.

The frequency counts are not available on the catalog file, but must be generated from the data.

The GENFREQ program creates the counts file using the field name to locate the variable in the

database, and the response code values to validate the range of data values for each field. This program

also serves as a check on the completeness of the response codes in the catalog file, as it flags any data

values not represented by a value and label.
140

The IRT parameter file is linked to the catalog file through the field name. Printing of the IRT

parameters is governed by a control flag in the classification section of the catalog record. If an item has

been scaled for use in deriving the scale score estimates, the IRT parameters are listed to the right of the

values and labels, and the score value for each response code is printed to the immediate right of the

corresponding frequency.

The layout and codebook files are written by their respective generation programs to print-image

disk data files. Draft copies are printed and distributed for review before the production copy is

generated. The production copy combines the layout and codebook files for each sample in a portable

document file (PDF) format. This file may be browsed, excerpted and printed using the Adobe Acrobat

Reader program on a variety of platforms and operating systems.

8.4.7 Control Statement Files for Statistical Packages

An additional requirement of the NAEP cooperative agreement is to provide, for each

secondary-use data file, a file of control statements each for the SAS and SPSS statistical systems that

will convert the raw data file into the system data file for that package. Two separate programs, GENSAS

and GENSPX, generate these control files using the catalog file as input.

Each of the control files contains separate sections for variable definition, variable labeling,

missing value declaration, value labeling, and creation of scored variables from the cognitive items. The

variable definition section describes the locations of the fields, by name, in the file, and, if applicable, the

number of decimal places or type of data. The variable label identifies each field with a 50-character

description. The missing value section identifies values of those variables that are to be treated as

missing and excluded from analyses. The value labels correspond to the response codes in the catalog

file. The code values and their descriptors are listed for each discrete numeric variable. The scoring

section is provided to permit the user to generate item score variables instead of the item response

variables.

Each of the code generation programs combines three steps into one complex procedure. As each

catalog file record is read, it is broken into several component records according to the information to be

used in each of the resultant sections. These record fragments are tagged with the field sequence number

and a section sequence code. They are then organized by section code and sequence number. Finally, the

reorganized information is output in a structured format dictated by the syntax of the processing

language.

The generation of the system files accomplishes the testing of these control statement files. The

system files are saved for use in special analyses by NAEP staff. These control statement files are

included on the distributed data files to permit users with access to SAS and/or SPSS to create their own

system files.

8.4.8 Machine-Readable Catalog Files

For those NAEP data users who have neither SAS nor SPSS capabilities, yet require processing

control information in a computer-readable format, the distribution files also contain machine-readable

catalog files. Each machine-readable catalog record contains processing control information, IRT

parameters, and response codes and labels. The machine-readable catalog files are described in and are

available as part of the secondary-use data files package for use in analyzing the data with programming

languages such as SAS and SPSS (see the NAEP 1998 Reading Data Companion, [Rogers, Kokolis,
141

Stoeckel, & Kline, 2000], the NAEP 1998 Writing Data Companion, [Rogers, Kokolis, Stoeckel, &

Kline, 2000], and the NAEP 1998 Civics Data Companion, [Rogers, Kokolis, Stoeckel, & Kline, 2000]).

8.4.9 NAEP Data on Disk

The complete set of secondary-use data files described above are available on CD-ROM as part

of the NAEP Data on Disk product suite. This medium is ideal for researchers and policy makers

operating in a personal computing environment.

The NAEP Data on Disk product suite includes two other components that facilitate the analysis

of NAEP secondary-use data. The PC-based NAEP data extraction software, NAEPEX, enables users to

create customized extracts of NAEP data and to generate SAS or SPSS control statements for preparing

analyses or generating customized system files. The NAEP analysis modules, which currently run under

SPSS

 for Windows, use output files from the extraction software to perform analyses that incorporate

statistical procedures appropriate for the NAEP design (e.g., minimum sample size requirements,

appropriate row-wise and column-wise t-tests, and automatic calculation of correct and consistent

standard errors and degrees of freedom).

143

Chapter 9

OVERVIEW OF PART II: THE ANALYSIS OF 1998 NAEP DATA1

Nancy L. Allen, James E. Carlson, and John R. Donoghue

Educational Testing Service

9.1 INTRODUCTION

The purpose of this chapter is to summarize some information from previous chapters that is

integral to the analysis of NAEP data, to summarize the analysis steps used for all subjects, and to

indicate what information is in each of the remaining chapters. The overview of the analyses conducted

on the 1998 NAEP data focuses on the common elements of the analyses used across the subject areas of

the assessment. Some of this information is available only within this chapter. Details by subject area are

provided in Chapters 14 through 24.

The organization of this chapter is as follows:

• Section 9.2 provides a short overview of the NAEP design for 1998. To provide

additional background information, the section also provides a short description of the

samples selected for 1998. Chapters 1 through 7 provide this same information in

much more detail.

• Section 9.3 summarizes the steps in analysis common to all subject areas. Some of

this information is described in more detail in other chapters. The rest is included

only within this chapter. The topics covered are as follows:

♦ Section 9.3.1 briefly describes the preparation of the final sampling weights.

Detailed information about the weighting procedures is given in Chapters 10

and 11. Detailed information about the sampling design is in Chapters 3 and 4.

♦ Section 9.3.2 provides information about the scoring reliability of

constructed-response items. It provides information about the reliability

measures used with the NAEP data during analysis. Chapter 7 contains

information about the reliability procedures used during the scoring process.

♦ Section 9.3.3 summarizes the information provided by the teacher

questionnaires, and indicates its use during the analysis process.

♦ Section 9.3.4 provides a description of the item properties examined for

background questions and for cognitive items. It includes a description of the

classical item statistics examined for both dichotomously (right versus

wrong) and polytomously (more than two response categories) scored items.

It also includes a description of the item-level results available from

summary data tables. Chapter 13 contains more information about the

conventions used in creating these summary tables. Finally, a thorough

description of differential item functioning analyses is provided.

 

1

 Nancy L. Allen, James E. Carlson, and John R. Donoghue were responsible for the psychometric and statistical analysis of the

1998 national and state NAEP data.
144

♦ Section 9.3.5 summarizes the steps used to scale NAEP data. The steps

include item response theory (IRT) scaling of the items, generating plausible

values to account for measurement error, transforming the results to the final

reporting scale, creating composite scores if necessary, and providing tables

of reported statistics. Details of the theory behind these steps are available in

Chapter 12.

♦ Section 9.3.6 provides some information about previous results of

dimensionality analyses.

♦ Finally, Section 9.3.7 gives an introduction to hypothesis testing and

drawing correct conclusions about NAEP data. Specific information about

which hypothesis test procedures were used for different purposes is

provided in Chapter 13.

• Section 9.4 contains a description of the information provided in Chapters 10 through

24 of this report.

9.2 SUMMARY OF THE NAEP DESIGN

As described in Chapter 1, the 1998 NAEP comprised three components. One component

encompassed major assessments in reading, writing, and civics, providing detailed information about

student scale scores at the fourth-, eighth-, and twelfth-grade levels of nonpublic and public schools. The

second major component was the state assessment at the fourth- and eighth-grade levels in reading and at

the eighth-grade level in writing. In addition to the two major components, special studies—a civics

special trend study, a 50-minute writing study, and a classroom-based study of writing—were conducted.

The results from and procedures used in these special studies are reported in separate documents.

Results from the analyses described in the following chapters were published in the following

reports:

• The NAEP 1998 Reading Report Card for the Nation and the States (Donahue et

al., 1999), which provides both public- and nonpublic-school data for major

NAEP reporting subgroups for all of the jurisdictions that participated in the state

assessment program, as well as selected results from the 1998 national reading

assessment.

• The NAEP 1998 Writing Report Card for the Nation and the States (Greenwald et

al., 1999), which provides both public- and nonpublic-school data for major

NAEP reporting subgroups for all of the jurisdictions that participated in the state

assessment program, as well as selected results from the 1998 national writing

assessment.

• The NAEP 1998 Civics Report Card for the Nation (Lutkus et al., 1999), which

provides both public- and nonpublic-school results for major NAEP reporting

subgroups from the 1998 national civics assessment.

Because the samples of students included in the 1998 NAEP assessment are listed and described

in detail in Chapter 1, only a brief description of these samples is given here. The 1998 national samples

consisted of the main NAEP samples for reading, writing, and civics, which were based on a common set

of assessment procedures including grade-level samples, and samples for these special studies; a study of
145

trends in civics performance (1988–1998); a study in which students were administered a 50-minute

writing assessment; and a study of classroom writing.

As described in Chapters 1 and 2, for each subject area in the main and state assessments, blocks

of items were used to create a large number of different assessment booklets according to a focused

design. The 1998 civics assessment used a focused balanced incomplete block (BIB) design. The 1998

reading and writing assessments used focused partially balanced incomplete block (focused PBIB)

designs. In a focused BIB design, each block of cognitive items appears in the same number of booklets.

To balance possible block-position main effects, each block appears an equal number of times in each

position. In addition, the focused BIB design requires that each block of items be paired in a booklet with

every other block of items. If one of the features that define a focused BIB design is not evident, then the

design is called a focused partially balanced incomplete block (PBIB) design.

9.3 ANALYSIS STEPS

 Because the analysis methods are not identical across subject areas, a separate analysis chapter

has been included for each major assessment. The procedures used depended on whether assessment

items were scored dichotomously (right versus wrong) or polytomously (more than two categories of

response) and whether links across grade levels were required. Basic procedures common to most or all

of the subject area analyses are summarized here. The order is essentially that in which the procedures

were carried out.

9.3.1 Preparation of Final Sampling Weights

 Because NAEP uses a complex sampling design (Chapters 3 and 4) in which students in certain

subpopulations have different probabilities of inclusion in the sample, the data collected from each

student must be assigned a weight to be used in analyses. The 1998 NAEP weights were provided by

Westat, the NAEP contractor in charge of sampling. Detailed information about the weighting procedures

is available in Chapters 10 and 11 and in Westat’s Sampling Activities and Field Operations for 1998

NAEP (Gray, et al., 2000).

9.3.2 Reliability of Scoring Constructed-Response Items

A minimum of 25 percent of the responses for reading, writing, and civics items involved only in

the national assessment and 6 percent of the responses for reading and writing items involved in both the

national and state assessments were scored by a second reader to obtain statistics on interreader

(interrater) reliability. Ranges for percentage of exact agreement for the combined state and national

assessments of reading, writing, and civics can be found in Table 7-2. This reliability information was

also used by the team leaders to monitor the capabilities of all readers and maintain uniformity of scoring

across readers. More information about this use of the reliability information is provided in Chapter 7.

In addition to reliability information calculated and used during the scoring process, several

additional reliability measures are calculated for constructed-response items after the item response data

has been placed in the NAEP database. They appear in Appendix C. These include a final percentage

exact agreement, the intraclass correlation, Cohen’s Kappa (Cohen, 1968), and the product-moment

correlation between the scores for the first and second readers. These measures are summarized in Zwick

(1988), Kaplan and Johnson (1992), and Abedi (1996). Each measure has advantages and disadvantages

for use in different situations. In this report, the percentage exact agreement is reported for all
146

constructed-response items, Cohen’s Kappa is reported for dichotomously scored constructed-response

items, and the intraclass correlation is reported for polytomously scored constructed-response items.

9.3.3 Teacher Questionnaires

Teachers of assessed students were asked to complete a two-part questionnaire. The first part of

the questionnaire pertained to the teacher’s background and training. The second part pertained to the

procedures used by the teacher for specific classes containing assessed students. See Chapter 2 for a

description of the teacher questionnaires.

To analyze the data from the teacher questionnaires at grades 4 and 8 with respect to the

students’ data, each teacher’s questionnaire had to be matched to all of the sampled students who were

taught by that teacher. In the subsequent chapters, two separate match rates for each grade are given. The

first is the percentage of students that could be matched to both the first and second parts of the teacher

questionnaire. For these students, information is available about the background and training of their

teachers and about the methods used in the particular class they attended. The second match rate is the

percentage of students that could be matched to the first part of the teacher questionnaire. This match rate

is larger because more students could be matched with information about a teacher than with information

about the particular class they attended. Note that these match rates only reflect the student-level missing

data. They do not reflect the additional missing data due to item-level nonresponse on the part of

teachers. Variables derived from the teacher questionnaires were used as reporting variables at the

student level and as variables that contributed to conditioning for the appropriate samples.

Teachers of students who were in the grade 4 assessment sample were asked to complete a twopart questionnaire. As with the grade 8 teacher questionnaire, the first part pertained to the teacher’s

background and training. Unlike the grade 8 teacher questionnaire, the second part pertained to only a

single class that the teacher taught. In development of the questionnaires, it was thought that fourth-grade

teachers would teach one class in each subject. In practice, that was found to be untrue for a number of

teachers. A single student-teacher match rate matching students to the first part of the questionnaire is

reported for grade 4 in the following chapters.

9.3.4 Analysis of Item Properties: Background and Cognitive Items

The first step in the analysis of the 1998 data was item-level analysis of all instruments. Item

analyses were performed separately for each grade on each item in each subject area. Each block of items

was analyzed separately by grade, with the total score on the block (including the analyzed item) used as

the criterion score for statistics requiring such a score. In the cases where final weights were not

available, preliminary weights were used in these preliminary analyses. The item analysis of cognitive

items was repeated after scaling of the items was completed.

9.3.4.1 Background Items

For each NAEP background item, the unweighted and weighted percent of students who gave

each response were examined, as well as the percent of students who omitted the item and the percent

who did not reach the item. The number of respondents was also tabulated. These preliminary analyses

were conducted within grade cohorts and within major reporting categories. If unexpected results were

found, the item data and the encoding of responses were rechecked.
147

9.3.4.2 Cognitive Items

All NAEP cognitive items were subjected to analyses of item properties. These analyses included

conventional item analyses and incorporated examinee sampling weights. Item analysis was conducted at

the block level so that the “number correct” scores for students responding to an item, selecting each

option of an item, omitting an item, or not reaching an item, is the average number of correct responses

for the block containing that item. Because of the inclusion of polytomously scored items in the cognitive

instruments, it was necessary to use special procedures for these items. The resulting statistics are

analogous to those for the dichotomously scored items, as listed below.

Dichotomously Scored Items. These items were analyzed using standard procedures that result

in a report for each item that includes:

• for each option of the item, for examinees omitting and not reaching the item, and for

the total sample of examinees:

♦ the number of examinees,

♦ the percentage of examinees,

♦ the mean of number-correct scores for the block in which the item appears, and

♦ the standard deviation of number-correct scores for the block in which the item

appears;

• the percentage of examinees providing a response that was “off-task”;

• p+, the proportion of examinees who received a correct score on the item (ratio of

number correct to number correct plus wrong plus omitted);

• , the inverse-normally transformed p+ scaled to mean 13 and standard deviation 4;

• the biserial correlation coefficient between the item and the number-correct scores

for the block in which the item appears; and

• the point-biserial correlation coefficient between the item and the number-correct

scores for the block in which the item appears.

Polytomously Scored Items. Enhanced procedures were employed for polytomously scored

items. Methods parallel to those used for dichotomously scored items resulted in values reported for each

distinct response category for the item. Response categories for each item were defined in two ways—

one based on the original codes for responses as specified in the scoring rubrics used by the scorers, and

one used in defining the item response theory (IRT) model scales. The latter was based on a scoring

guide developed by subject-area and measurement experts and it defined the treatment of each response

category in scaling. For example, a constructed-response item with four response categories would

initially have seven categories (not-reached, omitted, off-task, and the four valid response categories).

Another set of statistics resulted from mapping the response categories (excluding not-reached) into a

new set of categories reflecting the scoring guide for the items as scaled. A constructed-response item

with ordered categories, for example, would be mapped into a set of integers in a corresponding order.

The scoring guide could result in the collapsing of (combining of) some response categories. The

response categories, based on the final scoring guide developed by subject-area and measurement

experts, were used to calculate the polytomously scored item statistics.

The following statistics, analogous to those for dichotomously scored items, were computed:

• The percentage of examinees providing a response that was “off-task.”
148

• In place of p+, the ratio of the mean item score to the maximum-possible item score

was used.

• In place of , the inverse-normally transformed ratio of the mean item score to the

maximum-possible item score scaled to mean 13 and standard deviation 4.

• The polyserial correlation coefficient was used in place of the biserial.

• The Pearson correlation coefficient, or R-polyserial was used in place of the pointbiserial.

9.3.4.3 Tables of Item-Level Results

Tables were created of the percentages of students choosing each of the possible responses to

each item within each of the samples administered in 1998. The results for each item were crosstabulated against the basic reporting variables such as region, gender, race/ethnicity, public/nonpublic

school, and parental education. All percentages were computed using the sampling weights. These tables

are referred to as the test question section of the electronically available summary data tables for each

sample. In the summary data tables, the sampling variability of all population estimates was obtained by

the jackknife procedure used by ETS in previous assessments.

9.3.4.4 Tables of Block-Level Results

Tables summarizing the item statistics for all of the items within each block are provided in

Chapters 16, 17, 20, 21, and 24. These tables contain statistics calculated using student weights to

account for NAEP’s complex sampling of students, as well as the unweighted sample size. Weighted

summary statistics estimate the results for the whole population of students in the NAEP sampling frame.

• The unweighted sample size is the number of students in the reporting sample who receive

each block in the assessment. It is the number of students contributing to the other statistics

presented in the tables.

• The weighted average item score for the block is the average, over items, of the score

means for each individual weighted items in the block. Missing responses to polytomous

items before the last observed response in a block are also considered intentional omissions

and scored so that the response is in the lowest category. Occasionally, extended constructedresponse items are the last item in a block of items. Because considerably more effort is

required of the student to answer these items, nonresponse to an extended constructedresponse item at the end of a block is considered an intentional omission (and scored as the

lowest category) unless the student also did not respond to the item immediately preceding

that item. In that case, the extended constructed-response item is considered not reached and

treated as if it had not been presented to the student. In the case of the main and state writing

assessment, there is a single constructed-response item in each separately-timed block. In the

writing assessment when a student does not respond to the item or when the student provides

an off-task response, the response is also treated as if the item had not been administered.

Scaling areas in NAEP are determined a priori by grouping items into content areas for

which overall performance is deemed to be of interest, as defined by the frameworks

developed by the National Assessment Governing Board (NAGB). A scale score k is

defined a priori by the collection of items representing that scale. What is important,

therefore, is that the models capture salient information in the response data to effectively

summarize the overall performance on the content area of the populations and

subpopulations being assessed in the content areas.
149

• The weighted average R-polyserial correlation is the average, over items, of the item-level

R-polyserial correlations (R-biserial for dichotomous items) between the item and the

number-correct block score. For each item-level R-polyserial, total block number-correct

score (including the item in question, and with students receiving zero points for all notreached items) was used as the criterion variable for the correlation. The number-correct

score was the sum of the item scores for a student where correct dichotomous items are

assigned 1 and correct polytomous (or multiple-category) items are assigned the score

category for the response. Data from students classified as not reaching the item were

omitted from the calculation of the statistic.

• The weighted alpha reliability is the average of the polyserial correlations for polytomous

items and the biserial correlation for the dichotomous items within a block. As for the weighted

average R-polyserial correlations, the total block number-count score was used as the criterion.

• The weighted proportion of students attempting the last item of a block (or, equivalently,

one minus the proportion of students not reaching the last item) is often used as an index of

the degree of speededness associated with the administration of that block of items. Mislevy

and Wu (1988) discussed these conversions.

9.3.4.5 Differential Item Functioning Analysis of Cognitive Items

Differential item functioning (DIF) analysis refers to procedures that assess whether items are

differentially difficult for different groups of examinees. DIF procedures typically control for overall

between-group differences on a criterion, usually test scores. Between-group performance on each item is

then compared within sets of examinees having the same total test scores.

DIF analyses were conducted for items in the national main assessments in reading, writing, and

civics that had not previously been studied for differential item functioning. Each set of analyses

involved three reference group/focal group comparisons: male/female, White/Black, and White/Hispanic.

The Mantel-Haenszel Procedure. The DIF analyses of the dichotomous items were based on the

Mantel-Haenszel chi-square procedure (Mantel & Haenszel, 1959), as adapted by Holland and Thayer

(1988). The procedure tests the statistical hypothesis that the odds of correctly answering an item are the

same for two groups of examinees that have been matched on some measure of proficiency (usually

referred to as the matching criterion). The DIF analyses of the polytomous items were completed using

the Mantel-Haenszel ordinal procedure which is based on the Mantel procedure (Mantel, 1963), (Mantel

& Haenszel, 1959). These procedures compare proportions of matched examinees from each group in

each polytomous item-response category.

For both types of analyses, the measure of proficiency used is typically the total item score on

some collection of items. Since, by the nature of the BIB or PBIB design, booklets comprise different

combinations of blocks, there is no single set of items common to all examinees. Therefore, for each

student, the measure of proficiency used was the total item score on the entire booklet. These scores were

then pooled across booklets for each analysis. This procedure is described by Allen and Donoghue (1994,

1996). In addition, because research results (Zwick & Grima, 1991) strongly suggest that sampling

weights should be used in conducting DIF analyses, the weights were used.

For each dichotomous item in the assessment, an estimate of the Mantel-Haenszel common odds

ratio,αMH , expressed on the ETS delta scale for item difficulty, was produced. The estimates indicate the

difference between reference group and focal-group item difficulties (measured in ETS delta scale units),

and typically run between about +3 and -3. Positive values indicate items that are differentially easier for

the focal group than the reference group after making an adjustment for the overall level of proficiency in
150

the two groups. Similarly, negative values indicate items that are differentially harder for the focal group

than the reference group. It is common practice at ETS to categorize each item into one of three

categories (Petersen, 1988): “A” (items exhibiting no DIF), “B” (items exhibiting a weak indication of

DIF), or “C” (items exhibiting a strong indication of DIF). Items in category “A” have Mantel-Haenszel

common odds ratios on the delta scale that do not differ significantly from 0 at the alpha = .05 level or

are less than 1.0 in absolute value. Category “C” items are those with Mantel-Haenszel values that are

significantly greater than 1 and larger than 1.5 in absolute magnitude. Other items are categorized as “B”

items. A plus sign (+) indicates that items are differentially easier for the focal group; a minus sign (-)

indicates that items are differentially more difficult for the focal group.

The ETS/NAEP DIF procedure for polytomous items uses the Mantel-Haenszel ordinal

procedure (Mantel & Haenszel, 1959). The summary tables of identified polytomous items contain

generalizations of the dichotomous A, B, and C categories: “AA,” “BB,” or “CC.”

SIBTEST Procedure. For the first time in the 1998 assessment, ETS introduced the SIBTEST

(Shealy & Stout, 1993) DIF procedure into the analyses of NAEP items. All items new in 1998 were

examined using both Mantel-Haenszel and SIBTEST procedures for DIF. Like the Mantel-Haenszel

procedure, SIBTEST seeks to compare the performance of the focal and reference group members of

similar ability. The Mantel-Haenszel procedure uses matching on total score to establish comparability;

SIBTEST uses a linear "regression correction" (see [Shealy & Stout, 1993] for details) to obtain more

accurate matching of the groups. Simulation results (Chang, et al., 1995; Roussos & Stout, 1996) indicate

that the Mantel-Haenszel procedure and SIBTEST function similarly for most items, although SIBTEST

maintains better Type I error control for items with extreme discrimination IRT(a-parameters).

Like the Mantel-Haenszel procedure, SIBTEST analyses used the entire booklet score in forming

the matching variable. These results were then pooled across the booklets using a procedure described by

Chang, et al. (1995) and implemented by Donoghue (1998b). Sampling weights were used for SIBTEST

analyses.

The SIBTEST measure of DIF, β , is in the metric of Dorans and Kulick’s (1986) standardized

mean difference (SMD). As an effect size measure, the SMD divided by the item standard deviation was

used (as was done for polytomous items with the Mantel procedure). For an item to receive the

designation C (dichotomous items) or CC (polytomous items), two criteria had to be met: (a) the estimate

of β had to be significantly different from zero, and (b) the absolute value of the effect size (SMD/std.

dev.) had to be at least .25.

In 1998, results for the SIBTEST procedure were quite similar to those for the Mantel-Haenszel

procedure. All but 1 C or CC item identified by the Mantel-Haenszel procedure was also identified by

SIBTEST. No C or CC items were uniquely identified by SIBTEST. All C or CC items identified by

either procedure were referred to DIF committees (described below).

Standardization Method. In standard DIF analyses such as Mantel-Haenszel and SIBTEST, it is

well established that a moderately long matching test is required for the procedures to be valid (i.e.,

identify DIF in items unconfounded by other irrelevant factors [e.g., Donoghue, Holland, & Thayer,

1993]). In the main and state NAEP writing assessments, the booklets contain two 25-minute blocks, with

one writing prompt per block. Thus, each examinee has (at most) two responses on six-category prompts.

This is too little information for the test statistics associated with Mantel (1963) or SIBTEST (Shealy &

Stout, 1993) procedures to function effectively. Thus, standard DIF approaches based on statistical tests

of items are likely to function poorly, and so were not used in the writing assessment analysis.
151

In the writing assessment, the standardization method of Dorans and Kulick (1986) was used to

produce descriptive statistics. The matching variable was the total score on the booklet. As in other

NAEP DIF analyses, the statistics were computed based on pooled booklet matching; the results are

accumulated over the booklets in which a given item appears (e.g., Allen & Donoghue, 1996). This

analysis was accomplished using the standard NAEP DIF program NDIF that also calculates the MantelHaenszel statistic. The statistic of interest appears under the label SMD for "standardized mean

difference." First, differences in the item score between the two comparison groups are calculated for

each level of the booklet score. Then, the SMD for the item is the average of these differences divided by

their standard deviation.

Significance testing was not performed, due to the low reliability of the matching variable.

Instead, the standardized mean difference values were used descriptively, to identify those items that

demonstrate the most evidence of DIF. A rough criterion used in the past to describe DIF for polytomous

items has been to create the ratio of the SMD to the item’s standard deviation and flag any item with a

ratio of at least .25. A criteria of at least .10 could also be arbitrarily used to identify items with the most

evidence of DIF.

All NAEP DIF Procedures. All NAEP DIF analyses used rescaled sampling weights. A separate

rescaled weight was defined for each comparison as

where the total sample size is the total number of students for the two groups being analyzed (e.g., for the

White/Hispanic comparison, the total number of White and Hispanic examinees in the sample at that

grade), and the sum of the weights is the sum of the sampling weights of all the students in the sample for

the two groups being analyzed. Three rescaled weights were computed for White examinees—one for the

gender comparison and two for the race/ethnicity comparisons. Two rescaled overall weights were

computed for the Black and Hispanic examinees—one for the gender comparison and another for the

appropriate race/ethnicity comparison. The rescaled weights were used to ensure that the sum of the

weights for each analysis equaled the number of students in that comparison, thus providing an accurate

basis for significance testing.

In the calculation of total item scores for the matching criterion, not-reached, off-task, and

omitted items were considered to be wrong responses. Polytomous items were weighted more heavily in

the formation of the matching criterion, proportional to the number of score categories. For each item,

calculation of the Mantel-Haenszel statistic did not include data from examinees who did not reach the

item in question.

Each DIF analysis was a two-step process. In the initial phase, total item scores were formed and

the calculation of DIF indices was completed. Before the second phase, the matching criterion was

refined by removing all identified C or CC items, if any, from the total item score. The revised score was

used in the final calculation of all DIF indices. Note that when analyzing an item classified as C or CC in

the initial phase, that item score is added back into the total score for the analysis of that item only.

Following standard practice at ETS for DIF analyses conducted on final forms, all C or CC items

were reviewed by a committee of trained test developers and subject-matter specialists. Such committees

are charged with making judgments about whether or not the differential difficulty of an item is unfairly

related to group membership. The committees assembled to review NAEP items include both ETS staff

and outside members with expertise in the field. The committees carefully examine each identified item

Total Sample Size Rescaled Weight = Original Weight Sum of the Weights

•
152

to determine if either the language or contents would tend to make the item more difficult for an

identified group of examinees. As pointed out by Zieky (1993):

It is important to realize that DIF is not a synonym for bias. The item response

theory based methods, as well as the Mantel-Haenszel and standardization

methods of DIF detection, will identify questions that are not measuring the same

dimension(s) as the bulk of the items in the matching criterion . . . .Therefore,

judgment is required to determine whether or not the difference in difficulty

shown by a DIF index is unfairly related to group membership. The judgment of

fairness is based on whether or not the difference in difficulty is believed to be

related to the construct being measured . . . .The fairness of an item depends

directly on the purpose for which a test is being used. For example, a science item

that is differentially difficult for women may be judged to be fair in a test

designed for certification of science teachers because the item measures a topic

that every entry-level science teacher should know. However, that same item,

with the same DIF value, may be judged to be unfair in a test of general

knowledge designed for all entry-level teachers. (p. 340)

9.3.5 Scaling

Scales based on item response theory (IRT) were derived for each subject area. Three scales

were created for national main reading grade 8 and grade 12 assessment data, one for each purpose for

reading. Only two of these scales—Reading for Literary Experience and Reading to Gain Information—

were assessed at grade 4. A single scale was created for national main writing assessment data, and one

scale was created for national main civics assessment data. NAEP uses the methodology of multiple

imputations (plausible values) to estimate characteristics of the scale score distributions. Chapter 12

describes in detail the theoretical underpinnings of NAEP’s scaling methods and the required estimation

procedures. The basic analysis steps are outlined here.

1. Use the NAEP BILOG/PARSCALE computer program (described in Chapter 12) to estimate

the parameters of the item response functions on an arbitrary provisional scale. This program

uses an IRT model incorporating the two- and three-parameter logistic forms for

dichotomously scored items and the generalized partial-credit form for polytomously scored

items. In order to select starting values for the iterative parameter-estimation procedure for

each dataset, the program is first run to convergence, imposing the condition of a fixed

normal prior distribution of the scale score variable. Once these starting values are computed,

the main estimation runs model examinee scale score ability as a multinomial distribution.

That is, no prior assumption about the shape of the scale score distribution is made. In

analyses involving more than one population, estimates of parameters are made with the

overall mean and standard deviation of all subjects’ proficiencies specified to be 0 and 1,

respectively.

2. Use a version of the MGROUP program (described in Chapter 12), which implements the

method of Mislevy (see Chapter 10 or Mislevy, 1991) to estimate predictive scale score

distributions for each respondent on an arbitrary scale, based on the item parameter estimates

and the responses to cognitive items and background questions.

3. Use random draws from these predictive scale score distributions (plausible values, in NAEP

terminology) for computing the statistics of interest, such as mean proficiencies for

demographic groups.
153

4. Determine the appropriate metric for reporting the results and transform the results as

needed. This includes the linking of current scales to scales from the past or the selection of

the mean and variance of new scales. After scale score distributions for the scaling are

transformed, composite scale score distributions are created for the reading, writing, and

civics assessments.

5. Use the jackknife procedure to estimate the standard errors of the mean proficiencies for the

various demographic groups.

As explained in Chapter 10, the plausible values obtained through the IRT approach are not

optimal estimates of individual scale score; instead, they serve as intermediate values to be used in

estimating subpopulation characteristics. Under the assumptions of the scaling models, these

subpopulation estimates are statistically consistent, which would not be true of subpopulation estimates

obtained by aggregating optimal estimates of individual scale score.

9.3.5.1 Scaling the Cognitive Items

The data from the national main assessment samples were scaled using IRT models. For

dichotomously scored items two- and three-parameter logistic forms of the model were used, while for

polytomously scored items the generalized partial-credit model form was used. These two types of items

and models were combined in the NAEP scales. Item parameter estimates on a provisional scale were

obtained using the NAEP BILOG/PARSCALE program. The fit of the IRT model to the observed data

was examined within each scale by comparing the empirical item response functions with the theoretical

curves, as described in Chapter 12. Plots of the empirical item response functions and theoretical curves

were compared across assessments for items in the reading trend assessment. The DIF analyses

previously described also provide information related to the model fit across subpopulations.

The national main assessments of reading, writing, and civics each have special characteristics

that determine the procedures that were followed for the scaling of each subject. For reading, a key

consideration was the degree of similarity between the 1998 assessment and earlier assessments in terms

of the populations assessed and the characteristics of the assessment instrument used. The civics and

writing scales were not linked to any previously defined scales.

The frameworks for the different subject areas dictate differences in the numbers of scales. For

reading, item parameter estimation was performed separately for each of three scales defined in its

framework, using data from each grade sample separately.

9.3.5.2 Generation of Plausible Values for Each Scale

After the scales were developed, plausible values were drawn from the predictive distribution of

scale score values for each student (this process is called conditioning). For the writing and civics scales,

plausible values were drawn separately for each grade. For the reading scale, vectors of multivariate

plausible values were drawn from the joint distribution of scale score values for the assessed student. The

scales within an assessment are correlated. Multivariate generation utilizes this shared variation among

the scales in generating the plausible values. This procedure properly reflects the dependency between

the scale proficiencies. Multivariate plausible values were computed separately for each grade. All

plausible values were later rescaled to the final scale metric using appropriate linear transformations.

The variables used to calculate plausible values for a given national main assessment scale or

group of scales included a broad spectrum of background, attitude, and experiential variables and

composites of such variables. All standard reporting variables were included. To enhance numerical
154

stability for the national main assessment scales, the original background variables were standardized and

transformed into a set of linearly independent variables by extracting principal components from the

correlation matrix of the original contrast variables. The principal components, rather than the original

variables, were used as independent variables to calculate plausible values for those scales. Details of the

conditioning process and of the NAEP BGROUP and NAEP CGROUP (Thomas, 1994) computer

programs that implement the process are presented in Chapter 12. The variables used in conditioning are

listed in Appendix F.

9.3.5.3 Transformation to the Reporting Metric

Reading short-term trend scales were linked to previous assessment scales via common

population linking procedures described in the subject-specific data analysis chapters. Essentially, the

1994 and 1998 data were calibrated together. Data from the two assessments were scaled together in the

same BILOG/PARSCALE run, specifying the samples for each assessment as coming from different

populations. For each scale, the mean and standard deviation of the 1994 data from this joint calibration

were matched to the mean and standard deviation of the 1994 data as previously reported. This then

linked the 1998 data to the previously established scale. New scales were established for the writing and

civics national main assessment. Then the metrics for the newly established scales were set to have a

mean of 150 and a standard deviation of 35.

The transformations were of the form

target = A • calibrated + B

where

target = scale level in terms of the system of units of the final scale used for

reporting;

calibrated = scale level in terms of the system of units of the provisional

NAEP-BILOG/PARSCALE scale;

A = SDtarget / SDcalibrated ;

B =Mtarget - A • Mcalibrated ;

SDtarget = the estimated or selected standard deviation of the scale score

distribution to be matched;

SDcalibrated = the estimated standard deviation of the sample scale score distribution

on the provisional NAEP-BILOG/PARSCALE scale;

Mtarget = the estimated or selected mean of the scale score distribution to be

matched; and

Mcalibrated = the estimated mean of the sample scale score distribution on the

provisional NAEP-BILOG/PARSCALE scale.

After the plausible values were linearly transformed to the new scale, any plausible value less than 0 was

censored to 0. For the reading assessment, any value greater than 500 was censored to 500; for the
155

writing and civics assessments, any value greater than 300 was censored to 300. Fewer than 1 percent of

the students in any sample were censored in this way. The final transformation coefficients for

transforming each provisional scale to the final reporting scale are given in subsequent chapters.

9.3.5.4 Definition of Composites for the Multivariate Scales in Reading

In addition to the plausible values for each scale, a composite of the individual reading

assessment scales was created as a measure of overall proficiency. The composite scale score was a

weighted average of the plausible values of the individual scales. The weights reflected the relative

importance of the scales and were provided in the framework developed by the subject-area committee.

The weights are approximately proportional to the number of items in each scale at a given grade level.

9.3.5.5 Tables of Scale Score Means and Other Reported Statistics

Scale scores and trends in scale scores were reported by grade for a variety of reporting

categories. Additionally, the percentages of the students within each of the reporting groups who were at

or above achievement levels were reported to provide information about the distribution of achievement

within each subject area. All estimates based on scale score values have reported variances or standard

errors based on scale score values, including the error component due to the latency of scale score values

of individual students as well as the error component due to sampling variability. These tables are part of

the electronically delivered summary data tables.

9.3.6 Dimensionality Analysis

Over the years a number of studies have been conducted in order to seek answers to the question

of how many dimensions underlie the various NAEP assessment instruments, and whether there is a

sufficiently strong first dimension to support inferences about a composite scale in subjects such as

reading. For the 1992 mathematics and reading assessments, a study was conducted (Carlson, 1993) to

determine whether the increasing emphasis on extended constructed-response items that are scored

polytomously has any effect on the dimensionality. It was determined that for the 1992 NAEP data, item

type was not related to any of the dimensions identified.

9.3.6.1 Previous Dimensionality Analyses of NAEP Data

In an early study, the dimensionality of NAEP reading assessment data collected during the

1983–84 academic year was examined by Zwick (1986, 1987). Zwick also studied simulated data

designed to mirror the NAEP reading item response data but having known dimensionality. Analysis of

the simulated datasets allowed her to determine whether the BIB spiraling design artificially increases

dimensionality. Zwick found substantial agreement among various statistical procedures, and that the

results using BIB spiraling were similar to results for complete datasets. Overall she concluded that “it is

not unreasonable to treat the data as unidimensional” (1987, p. 306).

Rock (1991) studied the dimensionality of the NAEP mathematics and science tests from the

1990 assessment using confirmatory factor analysis. His conclusion was that there was little evidence for

discriminant validity except for the geometry scale at the eighth-grade level, and that “we are doing little

damage in using a composite score in mathematics and science” (p. 2).

A second-order factor model was used by Muthn (1991) in a further analysis of Rock’s

mathematics data, to examine subgroup differences in dimensionality. Evidence of content-specific

variation within subgroups was found, but the average (across seven booklets) percentages of such
156

variation was very small, ranging from essentially 0 to 22, and two-thirds of these percentages were

smaller than 10.

Carlson and Jirele (1992) examined 1990 NAEP mathematics data. Analyses of simulated onedimensional data were also conducted, and the fit to these data was slightly better than that to the real

NAEP data. Although there was some evidence suggesting more than one dimension in the NAEP data,

the strength of the first dimension led the authors to conclude that the data “are sufficiently

unidimensional to support the use of a composite scale for describing the NAEP mathematics data, but

that there is evidence that two dimensions would better fit the data than one” (p. 31).

Carlson (1993) studied the dimensionality of the 1992 mathematics and reading assessments. The

relative sizes of fit statistics for simulated as compared to actual data suggested that lack of fit may be

more due to the BIB spiraling design of NAEP than the number of dimensions fitted. Kaplan (1995)

similarly found that the chi-squared goodness of fit statistic in the maximum likelihood factor analysis

model was inflated when data were generated using a BIB design. The sizes of the fit statistics for

incomplete simulation conditions (a BIB design as in the actual NAEP assessment) were more like those

of the real data than were those of the case of simulation of a complete data matrix. Consistent with

findings of Zwick (1986, 1987), however, the incomplete design for data collection used in NAEP does

not appear to be artificially inflating the number of dimensions identified using these procedures.

9.3.7 Drawing Inferences from the Results

Drawing correct inferences from the results of the assessments depends on several components.

First, the hypothesis of no difference between groups must be tested statistically. For the 1998

assessment, the use of t-tests was introduced for most comparisons. These tests are more appropriate than

z-tests based on normal distribution approximations when the statistics that are being compared are from

distributions with thicker tails than those from the normal distribution. The statistical significance tests

used in NAEP are described in detail in Chapter 13.

A second component contributing to drawing correct inferences is the way in which error rates

are controlled when multiple comparisons are made. If we wish to make a number of comparisons in the

same analysis, say White students versus Black, Hispanic, Asian/Pacific Island, and American Indian

students, the probability of finding “significance” by chance for at least one comparison increases with

the family size or number of comparisons. By the Bonferroni inequality, for a family size of 4, for

example, the probability of a false positive (Type I error) using α = 0.05 is less than or equal to 4 × 0.05

= 0.20, larger than most decision makers would accept.

One general method for controlling error rates in multiple comparisons is based on the

Bonferroni inequality. In this method, the Bonferroni inequality is applied and α is divided by the family

size, n. Now α = .05/4 = .0125, and using α, the combined probability of one or more errors in the four

comparisons remains controlled at less than or equal to .05. Note that dividing the probability by n is not

the same as multiplying the critical value or the confidence band by n. Indeed, in moving from a family

size of 1 to 4, we increase the critical value only from 1.960 to 2.498, a 27.4 percent increase. Doubling

the family size again, to 8, increases the critical value to 2.735, an additional 9.5 percent increase. To

double the initial critical value to 3.92, the family size would have to be increased to 564.

The power of the tests thus depends on the number of comparisons planned. There may be cases

for which, before the data are seen, it is determined that only certain comparisons will be conducted. As

an example, with the five groups above, interest might lie only in comparing the first group with each of

the others (family size 4), rather than comparing all possible pairs of groups (family size 10). This means
157

that some possibly significant differences will not be found or discussed, but the planned comparisons

will have greater power to identify real differences when they occur.

In 1998, a different criterion was used to increase the power of statistical tests in NAEP. Unlike

other multiple-comparison procedures (e.g., the Bonferroni procedure) that control the familywise error

rate (i.e., the probability of making even one false rejection in the set of comparisons), the false discovery

rate (FDR) controls the expected proportion of falsely rejected hypotheses. So, if an α of .05 is selected,

about 95 percent of the hypothesis tests made rejected or accepted the hypothesis correctly, while about 5

percent of the hypothesis tests made rejected or accepted the hypothesis incorrectly. Familywise

procedures are considered conservative for large families of comparisons. Therefore, the FDR procedure

is more suitable for multiple comparisons in NAEP than other procedures (Williams, Jones, & Tukey,

1999). The FDR procedure used in NAEP has been described by Benjamini and Hochberg (1994). These

methods for controlling error rates in multiple comparisons are described in Chapter 13.

A third component contributing to drawing correct inferences is limiting comparisons to those

for which there are adequate data. In NAEP reports and data summaries, estimates of quantities such as

composite and content area scale score means, percentages of students at or above the achievement

levels, and percentages of students indicating particular levels of background variables (as measured in

the student, teacher, and school questionnaires) are reported for the total population as well as for key

subgroups determined by the background variables. In some cases, sample sizes were not large enough to

permit accurate estimation of scale score or background variable results for one or more of the categories

of these variables.

For results to be reported for any subgroup in NAEP, a minimum sample size of 62 is required.

This number was arrived at by determining the sample size required to detect an effect size of 0.5 with a

probability of .8 or greater. The effect size of 0.5 pertains to the “true” difference in mean scale score

between the subgroup in question and the total population, divided by the standard deviation of scale

score in the total population. In addition, subgroup members must represent at least five primary

sampling units (PSUs).

A fourth component contributing to drawing correct inferences is limiting comparisons to those

comparing statistics with standard errors that are estimated well. Standard errors of mean proficiencies,

proportions, and percentiles play an important role in interpreting subgroup results and comparing the

performances of two or more subgroups. The jackknife standard errors reported by NAEP are statistics

whose quality depends on certain features of the sample from which the estimate is obtained. In certain

cases, typically when the number of students upon which the standard error is based is small or when this

group of students all come from a small number of participating schools, the mean squared error

associated with the estimated standard errors may be quite large. In the summary reports, estimated

standard errors subject to large mean squared errors are followed by the symbol "!".

The magnitude of the mean squared error associated with an estimated standard error for the

mean or proportion of a group depends on the coefficient of variation (CV) of the estimated size of the

population group, denoted as N. The coefficient of variation is estimated by:

where Nˆ is a point estimate of N and ( ) ˆ SE N is the jackknife standard error of Nˆ .

Experience with previous NAEP assessments suggests that when this coefficient exceeds 0.2, the

mean squared error of the estimated standard errors of means and proportions based on samples for this

ˆ ˆ ˆ

SE(N) CV(N)=

N
158

group may be quite large. Therefore, the standard errors of means and proportions for all subgroups for

which the coefficient of variation of the population size exceeds 0.2 are followed by "!" in the tables of

all summary reports. These standard errors, and any confidence intervals or significance tests involving

them, should be interpreted with caution. (Further discussion of this issue can be found in Johnson &

Rust, 1993.)

A final component contributing to drawing correct inferences pertains to comparisons involving

extreme proportions. When proportions are close to zero or one, their distributions differ greatly from tor z-distributions. For this reason, hypothesis tests of the sort used by NAEP are not appropriate in these

cases. Under these conditions, no test is made. Chapter 13 includes the specific definition of extreme

proportion used in the analysis of 1998 data.

9.4 OVERVIEW OF CHAPTERS 10 THROUGH 24

The remaining chapters of this report are as follows:

Chapters 10 and 11: The 1998 national assessment used a stratified multistage probability

sampling design that provided for sampling certain subpopulations at higher rates (see Chapters 3 and 4).

Because probabilities of selection are not the same for all assessed students, sampling weights must be

used in the analysis of NAEP data. Also, in NAEP’s complex sample, observations are not independent.

As a result, conventional formulas for estimating the sampling variance of statistics are inappropriate.

Chapters 10 and 11 describe the weighting procedures and methods for estimating sampling variance that

are necessitated by NAEP’s sample design. Further detail on sampling and weighting procedures is

provided in the NAEP 1994 Sampling and Weighting Report (Wallace & Rust, 1996), published by

Westat, the NAEP contractor in charge of sampling.

Chapter 12: A major NAEP innovation introduced by ETS is the reporting of subject-area results

in terms of IRT-based scales. Scaling methods can be used to summarize results even when students

answer different subsets of items. For purposes of summarizing item responses, NAEP developed a

scaling technique that has its roots in IRT and in the theories of imputation of missing data. Chapter 12

describes this scaling technique, the underlying theory, and the application of these methods to 1998

NAEP data. The final section of Chapter 12 gives an overview of the NAEP scales that were developed

for the 1998 assessment.

Chapter 13: The 1998 assessment analyses included changes in the methods, procedures, and

conventions used in making group comparisons. Chapter 13 highlights these changes and provides details

about which results were reported.

Chapter 14: The 1998 reading assessment was based on a framework developed by the National

Assessment Governing Board for the 1992 reading assessment. This framework was used in the 1994 and

1998 assessments. Chapter 14 discusses the framework and assessment instruments used in the 1998

assessment.

Chapters 15, 16, and 17 describe analyses of the reading data for national and state assessments.

This analysis included a study of the cognitive variables and student background variables. At grades 4

and 8, background information and data on instructional methods were collected from teachers, and the

relation of these variables to reading scale scores was examined. The reading results appear in the NAEP

1998 Reading Report Card for the Nation and the States (Donahue et al., 1999).
159

Chapter 18: The 1998 writing assessment was based on a new framework developed by the

National Assessment Governing Board for the 1998 assessment. Chapter 18 discusses the framework and

assessment instruments used in the 1998 assessment.

Chapters 19, 20, and 21 describe analyses of the writing data for national and state assessments.

This analysis included a study of the cognitive variables and student background variables. At grade 8,

background information and data on instructional methods were collected from teachers and the relation

of these variables to writing data was examined. The writing results appear in the NAEP 1998 Writing

Report Card for the Nation and the States (Greenwald et al., 1999).

Chapter 22: The 1998 civics assessment was based on a new framework developed by the

National Assessment Governing Board for the 1998 assessment. Chapter 22 discusses the framework and

assessment instruments used in the 1998 assessment.

Chapters 23 and 24 describe analyses of the civics assessment. This analysis included a study of

the cognitive variables and student background variables. At grades 4 and 8, background information and

data on instructional methods were collected from teachers and the relation of these variables to civics

scale scores was examined. The civics results appear in the NAEP 1998 Civics Report Card for the

Nation (Lutkus et al., 1999).

161

Chapter 10

WEIGHTING PROCEDURES AND ESTIMATION OF

SAMPLING VARIANCE FOR THE NATIONAL ASSESSMENT1

Jiahe Qian, Bruce A. Kaplan, and Eugene G. Johnson

Educational Testing Service

Tom Krenzke and Keith F. Rust

Westat

10.1 INTRODUCTION

As in previous assessments, the 1998 national assessment used a complex sample design with the

goal of securing a sample from which estimates of population and subpopulation characteristics could be

obtained with reasonably high precision (as measured by low sampling variability). At the same time, it

was necessary that the sample be economically and practically feasible to obtain. The resulting sample

had certain properties that had to be taken into account to ensure valid analyses of the data from the

assessment.

The 1998 NAEP sample was obtained through a stratified multistage probability sampling design

that included provisions for sampling certain subpopulations at higher rates (see Chapter 3). To account

for the differential probabilities of selection, and to allow for adjustments for nonresponse, each student

was assigned a sampling weight. Section 10.2 discusses the procedures used to derive these sampling

weights.

Section 10.3 discusses other weighting procedures in the NAEP samples. These procedures

include generating modular weights, which would allow analysts to compare results between sample

types. National linking (NL)2

 weights were generated so that national and state-by-state assessments

could be equated for national and state results to be reported on a common scale. School weights were

created so that school-level data could be analyzed. Also, reporting weights for samples with

accommodations were processed for possible use in 2002 when reporting trend from 1998. Section 10.4

discusses the potential bias due to nonresponse.

Another consequence of the NAEP sample design is its effect on the estimation of sampling

variability. Because of the effects of cluster selection (cluster of elements: students within schools,

schools within primary sampling units) and because of the effects of certain adjustments to the sampling

weights (nonresponse adjustment and poststratification), observations made on different students cannot

be assumed to be independent of one another. In particular, as a result of clustering, ordinary formulas

for the estimation of the variance of sample statistics based on assumptions of independence will tend to

underestimate the true sampling variability. Section 10.5 discusses the jackknife technique used by

NAEP to estimate sampling variability.

 

1 Keith F. Rust and Tom Krenzke were responsible for the design and implementation of the weighting process for the 1998

NAEP national assessment. Jiahe Qian, with the assistance of Bruce Kaplan and in consultation with Eugene G. Johnson, was

responsible for the planning, specification, and coordination of the national weighting at ETS.

2 Note that in previous NAEP state assessments, the weights for national linking samples were called the state aggregate comparison,

or SAC, weights. Many people thought this was easy to confuse with state weights, so the term ‘national linking’ will be used in this

report.
162

10.2 WEIGHTING PROCEDURES FOR ASSESSED AND EXCLUDED

STUDENTS IN THE NATIONAL SAMPLES

Since the sample design determines the derivation of the sampling weights and the estimation of

sampling variability, it will be helpful to note the key features of the 1998 national sample design. A

description of the design appears in the first four sections of this report.

The 1998 sample was a multistage probability sample consisting of four stages. The first stage of

selection, the primary sampling units (PSUs), consisted of counties or groups of counties. The second

stage of selection consisted of elementary and secondary schools. The assignment of sessions and sample

types to sampled schools (see Chapter 3) comprised the third stage of sampling, and the fourth stage

involved the selection of students within schools and their assignment to sessions.

The probabilities of selection of the first-stage sampling units were proportional to measures of

their size, while the probabilities for subsequent stages of selection were such that the overall

probabilities of selection of students were approximately uniform, with exceptions for certain

subpopulations that were oversampled by design. Schools with relatively high concentrations of Black

students, Hispanic students, or both, were deliberately sampled at a higher than normal rate to obtain

larger samples of respondents from those subpopulations, in order to increase the precision in the

estimation of the characteristics of these subpopulations. Nonpublic-school students were sampled at

three times the normal rate, again to increase the precision of estimates for this population subgroup. For

all assessment components, students from schools with smaller numbers of eligible students received

lower probabilities of selection, as a means of enhancing the cost efficiency of the sample.

The 1998 national assessment includes three student cohorts: students in grades 4, 8, and 12. The

national assessment of all grades was conducted in the spring of 1998 to provide a cross-sectional view

of students’ abilities in reading, writing, and civics.

The full 1998 national assessment thus includes a number of different samples from several

populations. Each of these samples has its own set of weights that are to be used to produce estimates of

the characteristics of the population addressed by the sample (the target population). Each sample has an

additional set of weights to accommodate the reporting requirements. The various samples and their

target populations are as follows. The target population for each of these samples (one for each grade)

consisted of all students who were in the specified grade and were deemed assessable by their school.

There were three distinct session types at each grade: writing/civics, reading, and civics special trend.

Each session type was conducted as one or more distinct sessions within a school. Administration of each

session type was always conducted separately from other session types. Within the writing/civics

sessions, students in grade 4 received either a 25-minute writing booklet or a civics booklet, while in

grades 8 and 12 students received a 25-minute writing booklet, a 50-minute writing booklet, or a civics

booklet.

To facilitate analyses, two kinds of weights were produced. “Reporting weights” were produced

separately by grade and assessment type for analyses of the reporting samples that were defined for each

assessment. Several of the reporting samples included students from multiple sample types. “Modular

weights,” as discussed in Section 10.3.1, were produced separately by grade and sample type for the

reading assessment. They are applied for analyses involving any one sample type, or for comparing one

sample type with another. Thus, across grades, session types, and sample types, there were 14 sets of

reporting weights, and there were 6 sets of modular weights for students in reading assessments.
163

10.2.1 Base Weights

As indicated earlier, to enhance the precision of estimates of characteristics of these oversampled

subgroups, NAEP deliberately oversampled certain subpopulations to obtain larger samples of

respondents from those subgroups by using differential sampling rates. Because of the oversampling

public schools with high concentrations of Black and/or Hispanic students and the oversampling of

nonpublic schools, these subpopulations are overrepresented. As a result of oversampling students,

subpopulations to Black and/or Hispanic students from public schools with low concentrations of Black

and/or Hispanics, and corresponding to SD/LEP students in schools assigned reading sessions, are also

overrepresented in the sample. Lower sampling rates were introduced also for very small schools (those

schools with only 1 to 19 eligible students). This reduced level of sampling from small schools was

undertaken in a near optimal manner as a means of reducing variances per unit of cost (since it is

relatively costly to administer assessments in these small schools). Appropriate estimation of population

characteristics must take disproportionate representation into account. This is accomplished by assigning

a weight to each respondent, where the weights approximately account for the sample design and reflect

the appropriate proportional representation of the various types of individuals in the population.

Two sets of weights were computed for the 1998 samples. “Modular weights” were computed for

analyses involving students of reading assessments in one sample type, or for comparing results between

sample types. Each reading assessment type, by grade and sample type, weights up separately to the

target population. “Reporting weights” were computed for analyses of the reporting samples defined in

Table 10-1. The reading reporting samples include students from more than one sample type. For

reporting samples that include only one sample type (i.e., writing/civics and civics special trend), the

reporting weights are identical to the modular weights. The steps for computing these two sets of weights

are identical, up to and including the step of “trimming” the weights. The trimmed weights were

poststratified separately by sample type to create the modular weights. In a parallel procedure, the

trimmed weights were scaled back using a “reporting factor” so that the sample types included in each

reporting sample, when combined, would weight up to the target population. The resulting weights were

poststratified (but not separately by sample type) to create the reporting weights.

Table 10-1

Reporting Samples for 1998 National Assessments

Subject Grade Assessed Reporting Samples*

Civics 4, 8, 12 A3+B3

Civics Special Trend 4, 8, 12 A3+B3

Reading 4, 8, 12 A2+A3+B2

25-Minute Writing 4, 8, 12 A3+B3

* A indicates assessed non SD/LEP students; B indicates assessed SD/LEP

students; and 2 or 3 indicates the sample type.

The weighting procedures for 1998 included computing the student’s base weight, the reciprocal

of the probability that the student was selected for a particular subject type. Such weights are those

appropriate for deriving estimates from probability samples via the standard Horvitz-Thompson estimator

(see Cochran, 1977). These base weights were adjusted for nonresponse and then subjected to a trimming

algorithm to reduce a few excessively large weights. The weights were further adjusted by a student-level

poststratification procedure to reduce the sampling error. The poststratification was performed by

adjusting the weights of the sampled students so that the resulting estimates of the total number of

students in a set of specified subgroups of the population corresponded to population totals, which were

based on information from the Current Population Survey and U.S. Census Bureau estimates of the
164

population. The subpopulations were defined in terms of race, ethnicity, geographic region, grade, and

age relative to grade. The distribution of the various weighting factors is presented in Westat’s report

entitled Sampling Activities and Field Operations for 1998 NAEP (Gray, et al., 2000).

The base weight assigned to a student is the reciprocal of the probability that the student was

selected for a particular assessment. That probability is the product of six factors:

1. The probability that the PSU was selected

2. The probability that a Catholic, religious-affiliated, or other nonpublic school was selected

for the PSS file

 3. The conditional probability, given the PSU, that the school was selected

 4. The conditional probability, given the sample of schools in a PSU, that the school was

allocated to the specified session type

 5. The conditional probability, given the sample of schools in a PSU, that the sample type was

assigned to the school

 6. The conditional probability, given the school, that the student was selected for the specified

subject type

Thus, the base weight for a student may be expressed as the product

WB = PSUWGT_M • QSCHWT • SCH_WT • STYWT • SA_WT • STUSA_WT

where PSUWGT_M, QSCHWT, SCH_WT, STYWT, SA_WT, and STUSA_WT are, respectively,

the reciprocals of the preceding probabilities.

Variations across the various 1998 assessments in probabilities of selection, and consequently of

weights, were introduced by design, either to increase the effectiveness of the sample in achieving its

goals of reporting for various subpopulations, or to achieve increased efficiency per unit of cost.

The PSU weight, PSUWGT_M, is the reciprocal of the probability of selection for the PSU. Of

the 94 PSUs selected, 22 were certainty PSUs and have a PSU weight of 1.0. For the remaining 72 PSUs,

the probability of selection was calculated to account for the initial selection of one PSU per stratum.

The PSS weight, QSCHWT, is the reciprocal of the probability of selection of the Catholic,

religious-affiliated, and other nonpublic schools from the PSS area frame. QSCHWT= 1 for schools on

the PSS list frame. See Section 3.2.4.1 for more information about the PSS list and area frames.

The school weight, SCH_WT, is the reciprocal of the probability of selection of the school

conditional on the PSU.

The session allocation weight, SA_WT, is the reciprocal of the probability that the particular

session was allocated to the school. This is a function of the session type and the number of sessions

allocated to the school. Session allocation weights were calculated separately for each session type. The

values for the session allocation weights are summarized in Table 10-2. The session allocation weights

were adjusted for smaller-than-expected schools to account for one or more session types that were
165

dropped. The adjustment factor was computed as the number of sessions assigned divided by the number

of retained sessions assigned for the session type.

 Table 10-2

Session Allocation Weights Used in the 1998 National Assessment

Writing/Civics Reading Civics Special Trend

Grade

Session

Allocation

Weight

Number of

Sessions

Assigned

Session

Allocation

Weight

Number of

Sessions

Assigned

Session

Allocation

Weight

Number of

Sessions

Assigned

4 18/13 1 18/4 1 18 1

1 2 18/8 2 18/2 2

1 3 18/12 3 18/3 3

1 4 18/16 4 18/4 4

8 47/34 1 47/11 1 47/2 1

1 2 47/22 2 47/4 2

1 3 47/33 3 47/6 3

1 4 47/44 4 47/8 4

1 5 1 5 47/10 5

12 49/34 1 49/13 1 49/2 1

1 2 49/26 2 49/4 2

1 3 49/39 3 49/6 3

1 4 49/45 4 49/8 4

1 5 49/47 5 49/10 5

The sample type weight, STYWT, is the reciprocal of the probability that the sample type was

assigned to the school. For reading, the weight is 2, and for other sessions the weight was set to 1.

Cooperating substitute schools received the values of the following weighting components from

the original sampled school that it replaced: PSUWGT_M, QSCHWT, SCH_WT, SA_WT, STYWT.

For assessed students, the student weight, STUSA_WT, is the reciprocal of the probability that

the student was selected for the particular session to which he or she was assigned. This probability is the

product of the within-school sampling rate; the proportion of the relevant eligible students assigned to the

particular session type within the school, as prescribed by the sampling allocation factor; the proportion

of students in the session given a subject-specific assessment booklet (see Table 10-3 for the subject

factors); and a factor that adjusts for students in year-round schools that are not in school at the time of

assessment. Special attention was given to the writing sample allocation factors for accommodated

SD/LEP students and nonaccommodated students. The SD/LEP students in 50-minute writing that were

accommodated were given 25-minute writing booklets. Therefore, the accommodated students have a

higher chance of being assigned the 25-minute writing booklet than the nonaccommodated students. A

special poststratification procedure was done for the 50-minute writing sample, as described in

Section 10.2.5.1.

Excluded students were weighted with assessed students for each assessment. This was done

because the exclusion criteria did not depend on session type. For excluded students, STUSA_WT is

computed the same way as assessed and absent students.
166

Table 10-3

1998 National Assessment Writing and Civics Sample Allocation

Subject Grade 4 Grade 8 Grade 12

25-Minute Writing Nonaccommodated 13/10 17/10 17/10

25-Minute Writing Accommodated 13/10 17/13 17/13

50-Minute Writing N/A 17/3 17/3

Civics 13/3 17/4 17/4

10.2.2 Adjustment of the Base Weights for Nonresponse

The base weight for a student was adjusted by two nonresponse factors: SF_WT, to adjust for

noncooperating schools and schools that did not conduct all of their assigned sessions (i.e., a session

nonresponse); and STUNRADJ, to adjust for students who were invited to the assessment but did not

appear either in the scheduled or a makeup session. Thus the nonresponse adjusted weight for a student

was of the form:

STUAWT = PSUWGT_M • QSCHWT SCH_WT • SA_WT • STYWT • STUSA_WT •

SF_WT • STUNRADJ

The nonresponse adjustment factors were computed as described below.

10.2.2.1 Session Nonresponse Adjustment (SESNRF)

Sessions were assigned to schools before cooperation status was final. The session nonresponse

adjustment was intended to compensate for session type nonresponse due to refusing schools or

individual session types not conducted. The first three digits of PSU stratum, called subuniverse (formed

by crossing the PSU major stratum and the first socioeconomic characteristic used to define the final

PSU stratum; see Chapter 3 for more detail) were used in calculating nonresponse adjustments. The

adjustment factors were computed separately within classes formed by subuniverse within sample type

for reading, and by subuniverse for the other assessment types. Occasionally, additional collapsing of

classes was necessary to improve the stability of the adjustment factors, especially for the smaller

assessment components. Most classes needing collapsing contained small numbers of cooperating

schools. Occasionally, classes with low-response rates were collapsed.

In subuniverse s in session type h, the session nonresponse adjustment factor SF_WThs was

given by

_ __

_ _ __

hs

hs

i i i hi hi i

B

hs

i i i hi hi i

C

PSUWGT M QSCHWT SCH WT SA WT STYWT G

SF WT

PSUWGT M QSCHWT SCH WT SA WT STYWT G

∑

∑

• • • ••

= • • • ••

where

PSUWGT_Mi = the PSU weight for the PSU containing school i,

QSCHWTi = the PSS school weight for school i,
167

SCH_WTi = the school weight for school i,

SA_WThi = the session allocation weight for session type h in school i,

STYWTi = the sample type weight for school i,

Gi = the estimated number of grade-eligible students in school i (the values of

Gi

 were based on QED or PSS data or updated grade enrollment values

from field operations),

set Bhs = consists of all in-scope originally sampled schools allocated to session

type h in subuniverse s (excluding substitutes), and

set Chs = consists of all schools allocated to session type h in subuniverse s that

ultimately participated (including substitutes).

It should be noted that the nonresponse adjustments assume that nonresponse occurs at random

within the categories within which adjustments are made (see Little & Rubin, 1987). Some degree of bias

could result to the extent that this assumption is false. It should also be noted that the adjustment

accounts for the difference between the substitute’s estimated grade enrollment and its corresponding

original school’s estimated grade enrollment. For the state assessments, a separate weighting factor is

used to account for the difference in estimated grade enrollments (see Section 11.2.4).

10.2.2.2 Student Nonresponse Adjustment (STUNRADJ)

Student nonresponse adjustment factors were computed separately for each subject type. The

adjustment classes were based on sample type (for reading only), subuniverse, modal age status, and race

class (White or Asian/Pacific Islander, other). In some cases, two or more nonresponse classes were

collapsed into one to improve the stability of the adjustment factors. For each class c in subject type k,

the student nonresponse adjustment factor STUNRADJkc is computed by

_ __ _ _

_ __ _ _

kc

kc

j j j hj hj hj kj A

kc

j j j hj hj hj kj B

PSUWGT M QSCHWT SCH WT SA WT STYWT SF WT STUSA WT

STUNRADJ

PSUWGT M QSCHWT SCH WT SA WT STYWT SF WT STUSA WT

∑

∑

• • • •• •

= • • • •• •

where,

PSUWGT_Mj = the PSU weight for the PSU containing student j,

QSCHWTj = the PSS school weight for school containing student j,

SCH_WTj = the school weight for the school containing student j,

SA_WThj = the session allocation weight for the school containing student j in

session type h,

STYWThj = the sample type weight for the school containing student j in

session type h,
168

SF_WThj = the session nonresponse adjustment factor for the school containing

student j in session type h,

STUSA_WThj = the within-school student weight for student j in subject type k,

Set Akc = consists of the students in class c who were sampled for subject

type k and not excluded, and

Set Bkc = consists of the students in class c who were assessed in subject type k.

Excluded students received nonresponse adjustments of 1.0.

10.2.3 Variation in Weights

As mentioned earlier, the basic sampling design was to select students with uniform selection

probability except for planned oversampling in certain types of schools to improve estimates for certain

subgroups. However, additional variation in weights was caused by a number of factors. Variation arose

from undersampling schools with fewer than six expected students eligible for the grade category.

Variation also arose from limiting the number of students selected from large schools. Inaccurate school

measures of size also contributed to variability. When the measures of size were off by more than 20

percent, within-school sampling intervals were changed in order to meet the target sample size in the

school. In these cases the self-weighting sample design was abandoned in order to meet the target sample

size. In addition, the process of session assignment added variability to the weights. The number of

sessions was assigned to the school first, and then specific session types were assigned. Thus, the number

of sessions of any one type assigned to a school was a random variable. More oversampling within

schools, as discussed in Chapter 3, than in 1996 may have caused an increased variation in weights.

Finally, adjustment for nonresponse at the school and student levels added to the variation in weights.

Such variability in weights contributed to the variance of overall estimates from the survey by

approximately a factor of 2

W F = 1+V , where 2WV denotes the coefficient of variation of the student

weights. The calculated factors are displayed in Table 10-4.

By design, the use of poststratification factors, to be discussed in Section 10.2.5, also added to

weight variation. However, poststratification presumably reduced the variance of overall estimates by

reducing the variability in the relative contribution to the overall estimates of subclasses that respond

differently.
169

Table 10-4

Value of Factor F for Sample Subjects

Used in the 1998 National Assessment

Grade Subject F

4 Reading 1.41

25-Minute Writing 1.41

Civics 1.41

Civics Special Trend 1.25

8 Reading 1.42

25-Minute Writing 1.37

50-Minute Writing 1.36

Civics 1.38

Civics Special Trend 1.31

12 Reading 1.45

25-Minute Writing 1.34

50-Minute Writing 1.34

Civics 1.36

Civics Special Trend 1.32

10.2.3.1 Trimming the Weights for Outliers

In a number of cases, students were assigned relatively large weights3

. One cause of large

weights was underestimation of the number of eligible students in some schools, leading to

inappropriately low probabilities of selection for those schools. A second major cause is the presence of

large schools (high schools in particular) in PSUs with small selection probabilities. In such cases, the

maximum permissible within-school sampling rate (determined by the maximum sample size allowed per

school—see Chapter 3) could well be smaller than the desired overall within-PSU sampling rate for

students. Large weights arose also because very small schools were, by design, sampled with low

probabilities. Other large weights arose as the result of high levels of nonresponse coupled with low to

moderate probabilities of selection, and the compounding of nonresponse adjustments at various levels.

Students with notably large weights have an unusually large impact on estimates such as

weighted means. As discussed in the previous section, the variability in weights contributes to the

variance of an overall estimate by an approximate factor(1+V ) W

2 , where VW is the coefficient of

variation of the weights. An occasional unusually large weight is likely to produce large sampling

variances of the statistics of interest, especially when the large weights are associated with students with

atypical performance characteristics.

To reduce the effect of large contributions to variance from a small set of sample schools, the

weights of such schools were reduced, that is, trimmed. The trimming procedure introduces a bias but is

expected to reduce the mean square error of sample estimates.

 

3

 Trimming of small weights was not an issue in national and state NAEP assessments. The distribution of weights for NAEP

assessment samples is usually positively skewed. The size of the student groups with relatively small weights is usually relatively

large. Thus small weights are usually not outliers and would not contribute to a large coefficient of variation of weights.
170

The trimming algorithm was identical to that used since 1996 and had the effect, approximately,

of trimming the weight of any school that contributed more than a specified proportion, , to the

estimated variance of the estimated number of students eligible for assessment. The details of the

algorithm of trimming weights are given in Westat’s Sampling Activities and Field Operations for 1998

NAEP (Gray, et al., 2000).

The trimming procedure was done separately within sample type for reading, and overall for

25-minute writing, 50-minute writing, civics, and civics special trend. The number of schools where

weights were trimmed was no more than 13 in any one assessment. The most extreme trimming factors

applied were of the order of 0.41; trimming affects the weights of only a very small proportion of the

assessed and excluded students.

Table 10-5 shows the distributions of eligible students based on the trimmed weights of assessed

students for the 25-minute writing samples for each grade. The distributions are similar to those before

trimming shown later in the section. To the extent that the characteristics in the table are related to

student performance on the 25-minute writing assessment, there is a small bias introduced in the

assessment by trimming.

Table 10-5

Distribution of Populations of Eligible Students Based on Trimmed Weights

of Assessed Students in Participating Schools, 1998 National 25-Minute Writing Samples

Population Grade 4 Grade 8 Grade 12

Total Population 3,430,090 3,440,089 2,533,413

Age Category

At modal age or younger

Older than modal age

63.8

36.2

59.4

40.6

64.1

35.9

Race/Ethnicity Category

White

Black

Hispanic

Other

58.9

13.8

20.1

7.2

62.1

13.1

18.5

6.4

67.6

11.3

13.7

7.4

Gender*

Male

Female

50.6

49.4

50.0

50.0

47.9

52.0

SD

Yes

No

7.5

92.5

7.0

93.0

4.3

95.7

LEP

Yes

No

3.5

96.5

2.7

97.3

2.2

97.8

SD, LEP

SD yes, LEP yes

SD yes, LEP no

SD no, LEP yes

SD no, LEP no

0.2

7.3

3.3

89.2

0.3

6.8

2.5

90.5

0.1

4.2

2.1

93.6

*

 For a very small percentage of students at grades 4, 8, and 12, gender is unknown.
171

10.2.4 Reporting Factors

Each set of trimmed weights for a given sample type in the reading assessment sums to the target

population. Reporting factors were assigned to students in order to scale back the trimmed weights so

that final student (reporting) weights within each reporting sample (which may combine students from

different sample types) sum to the target population. The reporting factors assigned to students are

specific to the reporting samples defined in Table 10-1. Each assessed and excluded student in the

reporting sample for reading assessment received a reporting factor as shown in Table 10-6. Students that

were assessed or excluded in 25-minute writing, 50-minute writing, civics, and civics special trend, were

assigned a reporting factor equal to 1.0, since all students are part of the reporting sample.

Table 10-6

1998 National Reading Assessment

Reporting Factors for Assessed and Excluded Students

Sample Type

Non SD/LEP

Students

SD/LEP

Students

2 0.5 1

3 0.5 —

10.2.5 Poststratification

As in most sample surveys, the respondent weights are random variables that are subject to

sampling variability. Even if there were no nonresponse, the respondent weights would at best provide

unbiased estimates of the various subgroup proportions. However, since unbiasedness refers to average

performance over a conceptually infinite number of replications of the sampling, it is unlikely that any

given estimate, based on the achieved sample, will exactly equal the population value. Furthermore, the

respondent weights have been adjusted for nonresponse and a few extreme weights have been reduced in

size.

To reduce the mean squared error of estimates using the sampling weights, these weights were

further adjusted so that estimated population totals for a number of specified subgroups of the

population, based on the sum of weights of students of the specified type, were the same as presumably

better estimates based on composites of estimates from the 1995 and 1996 Current Population Survey

and 1997 population projections made by the U.S. Census Bureau. For details of the method used to

derive these independent estimates, see Appendix C in the Sampling Activities and Field Operations for

1998 NAEP (Gray, et al., 2000).

This adjustment, called poststratification, is intended especially to reduce the mean squared error

of estimates relating to student populations that span several subgroups of the population, and thus also

to reduce the variance of measures of changes over time for such student populations.

The poststratification in 1998 was done for all subjects and grades. Within each grade and

assessment type group, poststratification adjustment cells were defined in terms of race, ethnicity, and

Census region as shown in Tables 10-7. Note that NAEP region was used in years prior to 1996 instead

of Census region. This change was made because the data from the Current Population Survey and

Census Projections are more reliable for Census regions than for NAEP regions.

These subgroups were used as adjustment cells at grade 12. For grades 4 and 8, each of the seven

subgroups was further divided into two eligibility classes: of modal age and not of modal age.
172

Table 10-7

Major Subgroups for Poststratification

in the 1998 National Assessment

Race Ethnicity Census Region

Black

Any

Other

White

White

White

White

Not Hispanic

Hispanic

Not Hispanic

Not Hispanic

Not Hispanic

Not Hispanic

Not Hispanic

All

All

All

Northeast

Midwest

South

West

The procedure used at grade 12 was adopted because the independent estimates of the numbers

of students in the population did not provide consistent data on the numbers of twelfth-grade students by

age. Specifically, the counts of twelfth-grade students age 18 and older are not reliable because they

include adult education students. This procedure has been used since 1988. (See Rust, Bethel, Burke, &

Hansen, 1990, and Rust, Burke, & Fahimi, 1992, for further details.)

Thus, there were 7 or 14 cells for poststratification. The poststratified weight for each student

within a particular cell was the student’s base weight, with adjustments for nonresponse and trimming,

and the reporting factor from Section 10.2.4, times a poststratification factor. For each cell, the

poststratification factor is a ratio whose denominator is the sum of the weights (after adjustments for

nonresponse and trimming) of assessed and excluded students, and whose numerator is an adjusted

estimate, based on more reliable data, of the total number of students in the cell. The poststratification

factor for student j in subject type k and poststratification adjustment class c is given by

_ _ _

hc

c

kc

Bjj j j j

C

TOTAL RPTPS AD

W SF WT STUNRADJ TRIMFCTR RPT FCTR =

∑ •• • •

where

WBj = the base weight for student j (see Section 10.2.1);

TOTALc = the total number of grade-eligible students in class c, from the October

1995 and 1996 Current Population Surveys and 1997 population

projections;

SF_WTj = the session nonresponse adjustment factor for the school containing

student j in subject type k;

STUNRADJj = the student nonresponse adjustment for student j;

TRIMFCTRj = the trimming factor for student j;

RPT_FCTRj = the reporting factor for student j;

Set Ckc = consists of the students in class c who were assessed in subject type k,

except those at grade 12 who were age 18 or older.
173

The major subgroups for poststratification in 1998 assessments are shown in Tables 10-7. The

poststratification factors can be found in Westat’s Sampling Activities and Field Operations for 1998

NAEP (Gray, et al., 2000).

10.2.5.1 The 50-Minute Writing Session

The accommodated SD/LEP students sampled in the 50-minute writing session were given a

25-minute writing booklet. Therefore, the set of assessed 50-minute writing students did not contain

accommodated students. To allow for comparisons between nonaccommodated students assessed in

25-minute writing to students (all nonaccommodated) in the 50-minute writing session, a special

poststratification procedure was used for the weighting of students assessed in the 50-minute writing

session. The poststratification adjustment factors for the 50-minute writing session were computed using

the set of accommodated students in 25-minute writing, along with the set of students assessed in the

50-minute writing session. After poststratification, the estimated nonaccommodated universe sizes for

grade 8 25-minute and 50-minute writing sessions were 3,572,375 and 3,570,306, respectively. For grade

12, the estimated nonaccommodated universe sizes for grade 12 25-minute and 50-minute writing

sessions were 3,139,073 and 3,172,348, respectively.

10.2.6 Final Student Reporting Weights

NAEP estimates of student characteristics are based on final student weights, that is, the weight

resulting after adjusting the student base weight for nonresponse, trimming, reporting sample factor, and

poststratification. The student final weight, FSTUWT, is given by

FSTUWT=STUAWT • TRIMFCTR • RPT_FCTR • PSFCTR

where

STUAWT = nonresponse adjusted student base weight, (as defined in Section 10.2.2),

TRIMFCTR = trimming factor (as discussed in Section 10.2.3.1),

RPT_FCTR = reporting sample factor (as defined in Section 10.2.4), and

PSFCTR = poststratification factor (as discussed Section in 10.2.5).

The student full-sample reporting weight, FSTUWT, was used to derive all estimates of population and

subpopulation characteristics that have been presented in the various NAEP reports, including simple

estimates such as the proportion of students of a specified type who would respond in a certain way to an

item and more complex estimates such as mean scale score levels. The distributions of the final student

reporting weights are given in Table 10-8. The sample types contained in each reporting sample of the

assessment can be found in Table 10-1.

As indicated earlier, under some simplifying assumptions the factor 1 + Vw

2

 indicates the

approximate relative increase in variance of estimates resulting from the variability in the weights. The

factor Vw

2

 for each sample is readily derivable from Table 10-8 by squaring the ratio of the standard

deviation to the mean weight. These factors, resulting from the combined effect of the variations in

weights introduced by design and from other causes, are discussed in Section 10.2.3.
174

Table 10-8

Distributions of Final Student Weights for 1998 National Reporting Samples

Grade Subject n Mean

Standard

Deviation Minimum

25th

Percentile Median

75th

Percentile Maximum

4 25-Minute Writing 21,266 186 119 26 102 150 220 1,195

Reading 8,217 480 308 70 269 373 631 2,707

Civics Special Trend 2,264 1,742 867 401 1,098 1,519 2,242 6,585

Civics 6,355 621 399 90 340 489 759 4,140

8 25-Minute Writing 21,463 171 104 17 102 137 207 1,075

Reading 11,674 315 203 29 175 259 388 2,493

Civics Special Trend 2,148 1,710 945 159 1,033 1,388 2,199 5,705

Civics 8,553 430 265 47 254 345 526 2,370

50-Minute Writing 6,275 569 344 61 338 457 698 3,856

12 25-Minute Writing 20,163 158 93 25 94 130 194 1,266

Reading 13,123 241 161 35 129 194 297 1,373

Civics Special Trend 2,296 1,399 790 273 870 1,153 1,693 4,809

Civics 8,010 401 242 64 236 328 501 3,060

50-Minute Writing 6,006 528 309 86 312 432 648 4,972

10.3 OTHER WEIGHTING PROCEDURES IN THE NATIONAL SAMPLES

10.3.1 Modular Weights

As discussed in Section 10.2, modular weights were computed for the reading assessment to

facilitate analyses involving students from a single sample type. The same procedures were used to

derive modular and reporting weights up through the weight trimming step described in Section 10.2.3.1.

After trimming, weighting continued in two parallel processes. Final student reporting weights were the

result of one of these processes, and modular weights were the result of the other.

Modular weights differ from reporting weights for reading in two ways. First, they did not

contain the reporting factor described in Section 10.2.4. The second difference lies in the manner in

which the weights were poststratified. Since the number of students in the reading reporting samples are

nearly twice the number of students in each sample type (type 2 or type 3), the mean of the modular

weights is about twice the mean of reporting weights for reading.

The modular weights were poststratified as described in Section 10.2.5, except that each sample

type within each grade for reading was poststratified separately. The same initial adjustment cells were

used: 7 cells based on race/region for each sample type at grade 12, and 14 cells based on race/region and

eligibility class (of modal age, not of modal age) for each sample type at grades 4 and 8. Some

adjustment factors were quite variable for the same adjustment cell across different sample types for the

same grade and session. This indicates that the individual samples by sample type may not be particularly

stable.

The modular weight is the student’s base weight after the application of the various adjustments

described in Section 10.2, with the exception of applying a reporting factor, and the new

poststratification factor described above. The distributions of the modular weights are given in

Table 10-9. Note that except for the reading subject, modular weights are identical to reporting weights

for a particular grade/subject/sample type combination when that sample type is the only one included in

the reporting sample for that grade.
175

Table 10-9

Distribution of Modular Weights Used in the 1998 National Assessment

Grade Subject n Mean

Standard

Deviation Minimum

25th

Percentile Median

75th

Percentile Maximum

4 Reading/2* 4,593 859 510 127 462 721 1,113 3,460

Reading/3 4,597 858 567 155 481 679 1,034 5,224

8 Reading/2* 6,848 537 344 61 338 457 698 3,856

Reading/3 6,078 604 409 43 336 514 751 5,977

12 Reading/2* 7,048 444 317 45 224 348 594 2,303

Reading/3 7,050 453 313 53 236 373 543 2,615

* 2 refers to sample type 2 and 3 refers to sample type 3.

10.3.2 Linking Weights

Linking (NL) weights were generated so that national NAEP and state-by-state assessments

could be equated for national and state results to be reported on a common scale. Therefore, the results of

each participating jurisdiction would be meaningfully compared with those from the nation samples.

Technical details of the 1996 state assessments can be found in the Technical Report for the NAEP 1996

State Assessment Program in Mathematics (Allen, Jenkins, Kulick, and Zelenak, 1997) and in the

Technical Report for the NAEP 1996 State Assessment Program in Science (Allen, Swinton, Isham, and

Zelenak, 1998).

The fourth-grade reading and eighth-grade reading and writing assessments conducted in

February 1998 in the NAEP 1998 state assessment consisted of identical assessment material to that

administered in the corresponding national sample sessions. The guiding principles in the process of

linking state and national results were similar to those used for the 1996 assessments. (Technical details

of the NAEP 1996 state assessments are given in Allen, Jenkins, Kulick, and Zelenak (1997) and Allen,

Swinton, Isham, and Zelenak (1998).) The national and state-by-state assessments were equated so that

state and national results could be reported on a common scale. The equating was achieved by using from

each assessment that part of the sample representing a common population. For the national samples, this

consisted of those fourth-grade or eighth-grade public-school students from a participating state

(including the District of Columbia) who were assessed in the national reading or (for grade 8) writing

assessment reporting samples.

Although each sample of students received appropriate weights from the weighting procedure

used for the national assessment, in an effort to increase the precision of the equating process, an

additional weighting adjustment was developed and applied to each subsample by grade and subject,

solely for use in equating. For each subsample, the distributions of the national sample reporting weights

for three categorical variables were adjusted to agree closely with those obtained from the weighted

aggregate sample from the state assessments in the participating states. The first two variables were

NAEP region (Northeast, Southeast, Central, and West) and race/ethnicity (White non-Hispanic, Black

non-Hispanic, Hispanic, and other). For fourth- and eighth-grade reading, the third variable was reading

skill (very good, good, other). For eighth-grade writing, the third variable was the student’s writing skill

(“I am good at writing.”). This variable was based on a writing background item that asks how much a

student agrees with the statement “I am good at writing.” The categorical variables and control totals for

each of the assessed grades and subjects are presented in Tables 10-10 and 10-11.
176

Table 10-10

First and Second Categorical Variables Used for Raking*

Raking Dimensions

Fourth Grade

Reading

Control Total

Eighth Grade

Reading

Control Total

Eighth Grade

Writing Control

Total

First Dimension NAEP Region

Northeast 427,412 383,213 400,534

Southeast 731,635 717,450 730,862

Central 478,480 347,368 318,990

West 975,015 960,961 971,641

Total 2,612,532 2,408,992 2,422,027

Second Dimension Race/Ethnicity

White non-Hispanic 1,573,388 1,452,593 1,430,992

Black non-Hispanic 418,533 372,219 375,766

Hispanic 445,567 427,097 454,611

Other 175,043 157,082 160,658

Total 2,612,532 2,408,992 2,422,027

*Due to rounding, the sum of values within categorical variables may not equal the corresponding totals.

Table 10-11

Third Categorical Variable Used for Raking

Grade Skill Control Totals*

4 Reading Skill 1. Very Good 1,105,087

2. Good 965,306

3. Other 542,139

Total 2,612,532

8 Reading Skill 1. Very Good 596,581

2. Good 845,194

3. Other 967,216

Total 2,408,992

8 Writing Skill 1. Agree 1,206,813

(“I am good at writing.”) 2. Undecided 708,624

3. Other 506,590

Total 2,422,027

*Due to rounding, the sum of skill values may not equal the corresponding totals.

The equating of each weight distribution was achieved using a procedure known as iterative

proportional fitting, or raking (described by Little & Rubin, 1987). In raking, the marginal population

totals, Ni. and N.j are known (i.e., age and gender population counts); however, the interior cells of the
177

cross-tabulation Nij (the age by gender cells) are estimated from the sample by ˆNij , where these are the

sum of weights in the cells.

The raking algorithm proceeds by proportionally scaling the ˆNij , such that the following

relations are satisfied:

. ˆij i

j

∑N N=

and

. . ˆij j

i

∑N N=

At the completion of the fitting, adjustment factors were derived. The national sample weights for each

subgroup were multiplied by these adjustment factors to force their distribution to agree with those from

the aggregated state samples for each of these three variables in turn. This process was then repeated, and

the final set of adjusted weights was compared with the state sample weights on all three distributions,

and found to be in very close agreement. Table 10-12 shows the distribution of the adjustment factors for

each of the grades and subjects assessed.

Table 10-12

Percentiles of Raking Adjustments

Distribution

Grade 4

Reading

Grade 8

Reading

Grade 8

Writing

Minimum 0.805 0.885 0.832

10th Percentile 0.816 0.901 0.851

25th Percentile 0.837 0.912 0.899

Median 0.955 1.008 0.987

75th Percentile 1.121 1.026 1.076

90th Percentile 1.150 1.196 1.237

Maximum 1.640 1.523 1.570

10.3.3 School Weights

The sampling procedures used to obtain national probability samples of assessed students also

gave rise indirectly to several national probability samples of schools (from which the students were

subsequently sampled). So that the school samples can be utilized for making national estimates about

schools, appropriate nonresponse adjusted survey weights have been developed.
178

The school weights were computed separately by session within grade. The school weights were

a direct by-product of the student weighting process. The weight for school i in session h is given by

SWhi = PSUWGT_Mi

 • QSCHWTi • SCH_WTi • SA_WThi • STYWThi • SF_WThi

where

PSUWGT_Mi , QSCHWTi , SCH_WTi , SA_WThi , STYWThi , and SF_WThi are defined in

Section 10.2.

The school weights for the reading samples are modular weights. Each sample defined by sample

type weights up separately to the population. Different school weights are required for analyses involving

schools from both sample types. The weights in such cases can be developed by dividing the modular

weights by two.

Twelve samples of schools were weighted to be nationally representative. For each grade, the

samples include writing/civics, civics special trend, reading sample type 2, and reading sample type 3.

10.3.4 Reporting Weights with Accommodations

Reporting weights were generated using accommodated students in the 1998 reading samples as

part of the reporting sample. The weights may be useful in the year 2002 when reporting trend from

1998. These weights will also be used in looking into issues dealing with accommodation. The procedure

began with the trimmed weights (Section 10.2.3.1), and proceeded to the application of the reporting

factors as shown in Table 10-13. The reporting factors relating to the reporting sample with

accommodated students were set to 1.0, while the reporting factors for non-SD/LEP students in the 1998

national reporting sample were 0.5. Thus nonzero weights were produced for the SD/LEP students in

sample type 3, while not including the SD/LEP students in sample type 2.

Table 10-13

Reporting Factors for the Reporting Weights with Accommodations

for the 1998 National Reading Assessment

Sample Type

Non SD/LEP

Students

SD/LEP

Students

2 .5 —

3 .5 1

Poststratification was done on the accommodated reporting weights. The resulting final

accommodated reporting weights are summarized in Table 10-14.
179

Table 10-14

Distribution of Accommodated Reporting Weights

for the 1998 National Reading Assessment

Grade n Mean

Standard

Deviation Minimum

25th

Percentile Median

75th

Percentile Maximum

4 8,205 480.80 306.97 74.22 275.84 366.67 624.37 4,662.20

8 11,561 317.77 223.43 29.09 177.33 260.62 389.67 4,887.60

12 13,087 241.76 162.09 35.34 130.09 191.88 295.97 1,424.57

10.3.5 Jackknife Replicate Weights

In addition to the weights that were used to derive all estimates of population and subpopulation

characteristics, other sets of weights, called jackknife replicate weights, were derived to facilitate the

estimation of sampling variability by the jackknife variance estimation technique. These weights and the

jackknife estimator are discussed in Section 10.5.

10.4 POTENTIAL FOR BIAS DUE TO NONRESPONSE

Although school and student nonresponse adjustments are intended to reduce the potential for

nonparticipation to bias the assessment results, they cannot completely eliminate this potential bias with

certainty. The extent of bias remains unknown, of course, since there are no assessment data for the

nonparticipating schools and students. Recently, some studies related with this issue had been done, such

as on the effects of excluded students in reporting results (see Donoghue, 2000).

Some insight can be gained about the potential for residual nonresponse bias, however, by

examining the weighted school- and student-level distributions of characteristics known for both

participants and nonparticipants, especially for those characteristics known or thought likely to be related

to achievement on the assessment. If the distributions for the full sample of schools (or students) without

the use of nonresponse adjustments are close to those for the participants with nonresponse adjustments

applied, there is reason to be confident that the bias from nonparticipation is small.

There are several school-level characteristics available for both participating and

nonparticipating schools. The tables below show the combined impact of nonresponse and of the

nonresponse adjustments on the distributions of schools (weighted by the estimated number of eligible

students enrolled) and students, by the type of school (public, Catholic, other nonpublic), the size of the

school as measured by the estimated number of eligible students enrolled, and the urban/rural nature of

the place where the school is located. Three size classes have been defined for each grade. The data in

the tables that follow are for the 25-minute writing assessment because it is the largest assessment at each

grade. It is assumed that other large assessments would behave similarly. More of these types of data are

available for other grades and subjects in Appendix A.

Several student-level characteristics are available for both absent and assessed students. The

tables that follow show the impact of school nonresponse and nonresponse adjustments, and student

nonresponse and nonresponse adjustments on the distributions of eligible students for each grade. This

discussion also focuses on the writing/civics session for school-level summaries, and 25-minute writing

assessment for student-level tables. The distributions are presented by age category (at or below modal

age, and above modal age), race category (White, Black, Hispanic, and other), gender, SD, and LEP.
180

Table 10-15 shows the weighted marginal distributions of students for each of the three

classification variables for each grade, using weighted eligible schools. The distributions before school

nonresponse adjustments are based on the full sample of in-scope schools for the writing/civics session—

those participating, plus those refusals for which no substitute participated. The distributions after school

nonresponse adjustments are based only on participating schools for writing/civics, with school

nonresponse adjustments applied to them.

It can be seen from Table 10-15 that even though the level of school nonparticipation is as high

as 18 percent after substitution for grade 12 (see Table 3-7) and somewhat lower for the other grades, for

the most part, the distributions for the three characteristics considered remain similar. Exceptions may be

rural schools in grades 4 and 12, and large grade 12 schools.

 Table 10-15

Distribution of Populations of Eligible Students Based on Full Weighted Sample of Eligible Schools,

Before and After School Nonresponse Adjustments, 1998 National 25-Minute Writing Samples

Grade 4 Grade 8 Grade 12

Population Before After Before After Before After

Total Population 3,775,102 3,775,102 3,714,224 3,714,224 2,856,379 2,856,379

School Type

Catholic

Other Nonpublic

Public*

6.0%

4.5%

89.5%

6.8%

3.7%

89.5%

4.9%

4.4%

90.6%

5.8%

4.3%

89.9%

5.3%

3.8%

90.9%

6.4%

2.7%

90.9%

School Size†

1

2

3

17.8%

43.7%

38.5%

18.1%

42.5%

39.5%

9.7%

53.2%

37.1%

11.1%

52.4%

36.5%

5.3%

67.9%

26.8%

6.1%

69.3%

24.6%

School Location

Large City

Midsize City

Urban Fringe/Large City

Urban Fringe/Midsize City

Large Town

Small Town

Rural

18.5%

19.8%

26.9%

7.8%

1.1%

11.4%

14.5%

17.4%

19.4%

26.6%

8.0%

0.9%

11.2%

16.5%

16.5%

18.5%

27.1%

10.3%

1.7%

12.9%

13.0%

17.2%

17.4%

27.2%

10.5%

1.2%

11.7%

14.7%

14.2%

18.6%

29.1%

9.5%

1.1%

15.4%

12.1%

14.3%

17.3%

28.7%

10.4%

1.0%

13.8%

14.6%

* The term “public schools” extends to state-run, Department of Defense Education Activity (DoDEA), and Bureau of Indian Affairs

(BIA) schools.

† Distributions by school size are only comparable to 1996 assessments, since students were eligible by grade only, instead of by grade

or age before 1996. School size = number of eligible students enrolled:

123

Grade 4 1–49 50–99 100 +

Grade 8 1–49 50–299 300 +

Grade 12 1–49 50–399 400 +

Table 10-16 shows the distributions of the same three classification variables, plus additional

distributions of student-level characteristics, using weighted eligible students. The distributions before

student nonresponse adjustments are based on assessed and absent science students (with base weights

adjusted for school nonparticipation). The distributions after student nonresponse adjustments are based

on assessed science students only, with the student nonresponse adjustments also applied to them.
181

Table 10-16

Distribution of Populations of Eligible Students Before and After Student Nonresponse Adjustments,

1998 National 25-Minute Writing Samples

Grade 4 Grade 8 Grade 12

Population Before After Before After Before After

Total Population 3,447,973 3,447,973 3,477,714 3,477,714 2,598,835 2,598,835

School Type

Catholic

Other Nonpublic

Public*

7.1%

3.8%

89.1%

7.1%

3.9%

89.0%

6.0%

4.2%

89.9%

6.3%

4.3%

89.4%

6.9%

2.7%

90.4%

7.8%

3.2%

88.9%

School Location

Large City

Midsize City

Urban Fringe/Large City

Urban Fringe/Midsize 

City

Large Town

Small Town

Rural

16.6%

19.6%

27.2%

7.7%

0.8%

11.5%

16.7%

16.5%

19.6%

27.3%

7.6%

0.8%

11.5%

16.7%

17.2%

17.0%

28.1%

10.6%

1.1%

11.4%

14.5%

17.0%

16.9%

28.2%

10.7%

1.2%

11.5%

14.5%

14.4%

17.6%

28.9%

10.3%

0.8%

13.7%

14.3%

14.0%

17.3%

28.9%

10.4%

0.8%

14.0%

14.6%

Age Category

At Modal Age or Younger

Older than Modal Age

63.8%

36.2%

63.7%

36.3%

59.2%

40.8%

59.4%

40.6%

63.6%

36.4%

64.0%

36.0%

Race/Ethnicity Category

White

Black

Hispanic

Other

59.2%

14.1%

19.7%

7.0%

59.0%

13.8%

20.0%

7.2%

62.4%

13.2%

18.1%

6.3%

62.3%

13.0%

18.3%

6.4%

68.6%

11.5%

13.2%

6.7%

68.1%

11.1%

13.4%

7.4%

Gender†

Male

Female

50.5%

49.4%

50.6%

49.3%

50.2%

49.8%

50.0%

50.0%

48.4%

51.6%

47.9%

52.0%

SD

Yes

No

7.5%

92.5%

7.5%

92.5%

7.3%

92.7%

7.0%

93.0%

4.7%

95.3%

4.3%

95.7%

LEP

Yes

No

3.5%

96.5%

3.5%

96.5%

2.7%

97.3%

2.7%

97.3%

2.1%

97.9%

2.2%

97.8%

SD, LEP

SD yes, LEP yes

SD yes, LEP no

SD no, LEP yes

SD no, LEP no

0.2%

7.4%

3.3%

89.2%

0.2%

7.4%

3.3%

89.2%

0.3%

7.0%

2.4%

90.3%

0.3%

6.8%

2.5%

90.5%

0.1%

4.6%

2.0%

93.3%

0.1%

4.2%

2.1%

93.6%

* The term “public schools” extends to state-run, Department of Defense Education Activity (DoDEA), and Bureau of Indian Affairs

(BIA) schools.

†

 Gender is unknown for a small percentage of students.

The rates of student nonparticipation for 25-minute writing were 5.1 percent for grade 4, 7.8

percent for grade 8, and 20.3 percent for grade 12 (see Table 3-16). Table 10-17 shows that for the

distributions of type of school attended and place where the school is located, the combined effect of

student nonparticipation and the subsequent nonresponse adjustments have resulted in very little change

in distribution.

When comparing the distributions in Table 10-16 before and after student nonresponse

adjustments, distributions by age category and race/ethnicity are expected to be similar because these

variables were used to determine student nonresponse adjustment classes. However, the distributions by
182

gender, SD, and LEP are also similar. To the extent that nonrespondents would perform like respondents

with the same characteristics (defined by the classification variables in the tables), the bias in the

assessment data is small.

Table 10-17 shows the weighted distributions of eligible students in participating schools, using

the base weights of assessed and absent students unadjusted for school-level nonresponse. Tables 10-16

and 10-17 show that both school and student-level nonresponse and nonresponse adjustments have little

effect on the distributions of eligible students by age, race/ethnicity, gender, SD and LEP. All of the

distributions in the tables are similar.

 Table 10-17

Distribution of Populations of Eligible Students Before School and Student Nonresponse Adjustments,

1998 National 25-Minute Writing Samples

Population Grade 4 Grade 8 Grade 12

Total Population 3,065,866 2,946,000 2,598,835

Age Category

At Modal Age or Younger

Older than Modal Age

64.2%

35.8%

59.3%

40.7%

63.6%

36.4%

Race/Ethnicity Category

White

Black

Hispanic

Other

58.4%

14.5%

20.0%

7.0%

61.9%

13.6%

18.3%

6.2%

68.6%

11.5%

13.2%

6.7%

Gender*

Male

Female

50.5%

49.4%

50.2%

49.8%

48.4%

51.6%

SD

Yes

No

7.6%

92.4%

7.2%

92.8%

4.7%

95.3%

LEP

Yes

No

3.6%

96.4%

2.8%

97.2%

2.1%

97.9%

SD, LEP

SD yes, LEP yes

SD yes, LEP no

SD no, LEP yes

SD no, LEP no

0.2%

7.4%

3.4%

89.0%

0.3%

7.0%

2.5%

90.2%

0.1%

4.6%

2.0%

93.3%

* Gender is unknown for a small percentage of students.

Further information about potential nonresponse bias can be gained by studying the absent

students. NAEP scale score estimates are biased to the extent that assessed and absent students within the

same weighting class differ in their distribution of scale scores. It seems likely that the assumption that

absent students are similar in proficiency to assessed students is reasonable for some absent students

namely, those whose absence can be characterized as random. Conversely, it seems likely that students

with longer and more consistent patterns of absenteeism, such as truants, dropouts, near dropouts, and the

chronically ill, are unlikely to be as proficient as their assessed counterparts.

In the 1998 assessments, schools were asked to classify each absent student into one of nine

categories. The results of this classification for the 25-minute writing assessment are shown in

Table 10-18. The discussion focuses on the 25-minute writing assessment because it is the largest. It is

assumed that the other large assessments would behave similarly.
183

Table 10-18 shows that, as anticipated, the majority of absence from the assessment was the

result of an absence from school of a temporary and unscheduled nature. The table shows that absence

among twelfth-graders occurs at about four times the rate of absence among fourth-graders, and two-anda-half times that of eighth-graders. The proportion of absence classified as temporary differs somewhat

by grade, but is of the same magnitude for grades 8 and 12. These two facts taken together suggest

strongly that a substantial proportion of the temporary absences among twelfth-grade students is not a

result of illness, because such absences are occurring at almost three times the rate that they do among

fourth- or eighth-grade students. Whereas it might be reasonable to regard temporary absence due to

illness as independent of proficiency, for other temporary absences, this appears less tenable. The data in

the table give support to the contention that, at grade 4, student absences are unlikely to introduce any

significant bias into NAEP estimates. The absentee rate is low; most absences are temporary, and a third

of the remaining absences are a result of parental refusal.

Table 10-18

Weighted Distribution of Absent Students by Nature of Absenteeism

for All Grades, 1998 National 25-Minute Writing Samples

Nature of Absenteeism Grade 4 Grade 8 Grade 12

Temporary Absence* 87.4% 74.6% 71.9%

Long-Term Absence† 0.7% 2.2% 0.8%

Chronic Truant 0.2% 1.6% 0.8%

Suspended or Expelled 0.9% 3.7% 0.4%

In School, Did Not Attend 0.2% 1.4% 8.3%

Disruptive Behavior 0.0% 0.4% 0.1%

Parent Refusal 4.1% 9.5% 3.5%

Student Refusal 0.2% 1.7% 7.4%

Missing 0.0% 0.0% 0.0%

Other, Specify on Cover 0.8% 2.0% 5.5%

Incorrectly Coded as Excluded 5.3% 2.8% 1.2%

Total Absentee Sample 1,067 1,731 5,017

Total Sample Size of Invited Students 20,883 22,317 24,522

Overall Absentee Rate, Unweighted 5.1% 7.8% 20.5%

* Absent less than two weeks due to illness, disability, or excused absence.

† Absent more than two weeks due to illness or disability.

At grades 8 and 12, however, a significant component of absenteeism is not temporary or due to

parental refusal. Chronic truants, those suspended, and those in school but did not attend, and disruptive

behavior constitute the obvious candidates for potential bias. These groups comprise 7.1 percent of

absent students at grade 8 (or 0.6% of the total sample) and 9.6 percent of absent students at grade 12 (or

2.0% of the total sample). Thus their potential for introducing significant bias under the current

procedures is minor.
184

10.5 VARIANCE ESTIMATION

A major source of uncertainty in the estimation of the value in the population of a variable of

interest exists because information about the variable is obtained on only a sample from the population.

To reflect this fact, it is important to attach to any statistic (e.g., a mean) an estimate of the sampling

variability to be expected for that statistic. Estimates of sampling variability provide information about

how much the value of a given statistic would be likely to change if the statistic had been based on

another, equivalent, sample of individuals drawn in exactly the same manner as the achieved sample.

Another important source of variability is that due to imprecision in the measurement of

individual scale scores. For the 1998 assessment, scale scores in all subject areas were summarized

through item response theory (IRT) models, but not in the way that these models are used in standard

applications where each person responds to enough items to allow for precise estimation of that person’s

scale score. In NAEP, each individual responds to relatively few items so that individual scale score

values are not well determined. Consequently, the variance of any statistic based on scale score values

has a component due to the imprecision in the measurement of the scale scores of the sampled

individuals in addition to a component measuring sampling variability. The estimation of the component

of variability due to measurement imprecision and its effect on the total variability of statistics based on

scale score values are discussed in Chapter 12.

The estimation of the sampling variability of any statistic must take into account the sample

design. In particular, because of the effects of cluster selection (students within schools, schools within

PSUs) and because of effects of nonresponse and poststratification adjustments, observations made on

different students cannot be assumed to be independent of each other (and are, in fact, generally

positively correlated). Furthermore, to account for the differential probabilities of selection (and the

various adjustments), each student has an associated sampling weight, which should be used in the

computation of any statistic and is itself subject to sampling variability. Ignoring the special

characteristics of the sample design and treating the data as if the observations were independent and

identically distributed, will generally produce underestimates of the true sampling variability, due to the

clustering and unequal sampling weights.

10.5.1 Procedure to Estimate Sampling Variability

The proper estimation of the sampling variability of a statistic based on the NAEP data is

complicated and requires techniques beyond those commonly available in standard statistical packages.

Fortunately, the jackknife procedure (see, e.g., Kish & Frankel, 1974; Rust, 1985; Wolter, 1985) provides

good quality estimates of the sampling variability of most statistics, at the expense of increased

computation, and can be used in concert with standard statistical packages to obtain a proper estimate of

sampling variability.

The jackknife procedure used by NAEP has a number of properties that make it particularly

suited for the analysis of NAEP data. When properly applied, a jackknife estimate of the variability of a

linear estimator (such as a total) will be the same as the standard textbook variance estimate specified for

the sample design (if the first-stage units were sampled with replacement and approximately so

otherwise). Additionally, if the finite sampling corrections for the first-stage units can be ignored, the

jackknife produces asymptotically consistent variance estimates for statistics such as ratios, regression

estimates, or weighted means and for any other nonlinear statistic that can be expressed as a smooth

function of estimated totals of one or more variables (Krewski & Rao, 1981).
185

Through the creation of student replicate weights (defined below), the jackknife procedure

allows the measurement of variability attributable to the use of poststratification and other weight

adjustment factors that are dependent on the observed sample data. Once these replicate weights are

derived, it is a straightforward matter to obtain the jackknife variance estimate of any statistic.

The jackknife procedure in this application is based on the development of a set of jackknife

replicate weights for each assessed student (or school depending on the file involved). The replicate

weights are developed in such a way that, when utilized as described below, approximately unbiased

estimates of the sampling variance of an estimate result, with an adequate number of degrees of freedom

to be useful for purposes of making inferences about the parameter of interest.

The estimated sampling variance of a parameter estimator t is the sum of M squared differences

(where M is the number of replicate weights developed):

2

1

( ) ( ) ˆVar t t t

M

i

= ∑ i − =

where ti denotes the estimator of the parameter of interest, obtained using the i

th set of replicate weights,

SRWTi, in place of the original sample of full sample estimates FSTUWT.

There were 62 replicate weights developed using the procedures outlined below. Full details of

the generation of replicate weights for all samples are given in Sampling Activities and Field Operations

for 1998 NAEP (Gray, et al., 2000).

Of the 62 replicate weights formed for each record from a national assessment sample, 36 act to

reflect the amount of sampling variance contributed by the noncertainty strata of PSUs, with the

remaining 26 replicate weights reflecting the variance contribution of the certainty PSU samples.

The derivation of the 36 replicate weights reflecting the variance of the noncertainty PSUs

involves first defining pairs of PSUs in a manner that models the design as one in which two PSUs are

drawn with replacement per stratum. This definition of pairs is undertaken in a manner closely reflective

of the actual design, in that PSUs are pairs that are drawn from strata within the same subuniverse, and

with similar stratum characteristics. The same definition of pairs was used for each of the age/grade

classes in the national assessment, since all were drawn from the same sample of noncertainty PSUs. The

72 noncertainty PSUs, drawn one from each of 72 strata, were formed into 36 pairs of PSUs, where the

pairs were composed of PSUs from adjacent strata within each subuniverse (thus the strata were

relatively similar on socioeconomic characteristics such as proportion minority population, population

change since 1980, per capita income, civilian unemployment rate, educational attainment, and

unemployment rate). Whereas the actual sample design was to select one PSU with probability

proportional to size from each of 72 strata, for variance estimation purposes the design is regarded as

calling for the selection of two PSUs with probability proportional to size with replacement from each of

36 strata. This procedure likely gives a small positive bias to estimates of sampling error.

The student replicate weight for the ith pair of noncertainty PSUs, for the 36 pairs corresponding

to values of i from 1 to 36, is computed as follows:

1. Let WB be the base weight of a student, as described in Section 10.2, which accounts for the

various components of the selection probability for the student.

2. At random, one PSU in each pair is denoted as PSU number 1, while the other is denoted as

PSU number 2. The ith replicate base weight WBi is given by:
186

0 if the student belongs to PSU number 1 of pa i r 

2 if the student belongs to PSU number 2 of pair 

if the student is from neither PSU in pai r 

Bi B

B

i

W W i

W i

= ×











3. The i

th student replicate weight SRWTi is obtained by applying the various school and

student nonresponse adjustments, the weight trimming, and the poststratification to the i

th

set of replicate base weights, using procedures identical to those used to obtain the final

student weights WT from the set of base weights WB.

In brief, the procedure for deriving the sets of WBi values from the WB values reflects the

sampling of PSUs, schools, sessions, and students. By repeating the various weight adjustment

procedures in each set of replicate base weights, the impact of these procedures on the sampling variance

of the estimator, t, is appropriately reflected in the variance estimator ( ) ˆVar t defined above.

The procedure for obtaining the 26 sets of replicate weights to estimate the sampling variance

from the certainty PSUs is analogous, but somewhat more complex. The first stage of sampling in this

case is at the school level, and the derivation of replicate weights must reflect appropriately the sampling

of schools within certainty PSUs. Since each of the three grade classes in the national assessment

involved different samples of schools, the procedure for forming replicate base weights was

individualized to each of these sample components. In common across these three samples were the 22

certainty PSUs used, and the fact that 26 replicate weights were formed in each case.

For each grade, within the 22 certainty PSUs, a sample of schools was drawn systematically

within each. Using the schools listed in order of sample selection within each of eight “combinations” of

NAEP region and type of school (public, nonpublic), successive schools were grouped (i.e., PAIR). The

number of variance groups within a combination depended on the number of schools in the combination,

or indirectly assigned in proportion to the relative size of the combination. Thus, generally speaking, the

largest combination were assigned the largest numbers of replicates (or pairs). When splitting the

combinations, the schools were split into groups of (as close as possible) equal size, based on the

ordering at the time of sample selection. One group was assigned to each replicate. Within each group in

each combination, schools were alternately numbered 1 or 2 starting randomly. When, however, there

were exactly three schools sampled in the variance group, the schools were randomly numbered 1, 2, or

3. The method of forming replicate base weights in variance groups (i.e., PAIR) where there were not

exactly three schools was the same as for the noncertainty strata. If a variance group (PAIR) contained

three schools, students in these schools had their weights perturbed for two sets of replicates, say i1 and

i2, as follows:

1

0 if the student in school number 1 of a PSU in set 

1.5 if the student in school number 2 or 3 of a PSU in se t 

if the student does not belong to a PSU in set 

Bi B

B

i

WW i

W i

= ×

 











2

1.5 if the student in school number 1 or 2 of a PSU in set 

0 if the student in school number 3 of a PSU i n se t 

if the student does not belong to a PSU in s et 

B

Bi

B

W i

W i

W i

×

=

 










187

The actual pattern of replicate base weight assignment used for each of the samples is given in

Westat’s Sampling Activities and Field Operations for 1998 NAEP (Gray, et al., 2000).

The nonresponse, trimming, and poststratification adjustments were applied to each set of

replicate base weights to derive the final replicate weights in each case, exactly as in the noncertainty

PSUs. In fact, these procedures were applied to the full set of weights from all parts of the given sample

together, just as for the full sample weights. That is, for example, poststratification factors were derived

from the full set of data for each replicate, not separately for certainty and noncertainty PSUs.

This estimation technique was used by NAEP to estimate all sampling errors presented in the

various reports. A further discussion of the variance estimation procedure used by NAEP, including a

discussion of alternative jackknife estimators that were also considered, appears in Johnson (1989).

As stated above, a separate estimate of the contribution to variance due to the imprecision in the

measure of individual proficiencies is made and added to the jackknife estimate of variance. That

variance component could have been approximately reflected in the jackknife variance estimates simply

by separately applying the IRT computations to each jackknife replicate. Because of the heavier IRT

computational load, this was not done. Less work was involved by the simple procedure of making

separate estimates of this component to be added to the jackknife variance estimates. Also, a separate

measure of this component of variance is then available, which would not be so if it were reflected in the

jackknife variance estimate.

10.5.2 Approximating the Sampling Variance Using Design Effects

In practical terms, the major expenditure of resources in the computation of a jackknife variance

estimate occurs in the preparation of estimates for each of the pseudo-replicates. In the 1998 assessment,

this implies that the statistic of interest has to be recomputed up to 63 times, once for the overall estimate

t, and once for each of the up to 62 pseudo-replicates ti. Because this is a considerable increase in the

amount of computation required, relative to a conventional variance estimate, it is of interest to see how

much the jackknife variance estimates differ from their less computationally intensive, simple random

sampling based, analogues.

The comparison of the conventional and the jackknife methods of variance estimation will be in

terms of a statistic called the design effect, which was developed by Kish (1965) and extended by Kish

and Frankel (1974). The design effect for a statistic is the ratio of the actual variance of the statistic

(taking the sample design into account) over the conventional variance estimate based on a simple

random sample with the same number of elements. The design effect is the inflation factor to be applied

to the conventional variance estimate in order to adjust error estimates based on simple random sampling

assumptions to account approximately for the effect of the sample design. The value of the design effect

depends on the type of statistic computed and the variables considered in a particular analysis as well as

the combined clustering, stratification, and weighting effects occurring among sampled elements. While

stratification drives down the sampling variance, the effects of clustering and weighting that drive

variances up are generally sufficient to produce variance estimates that are larger than variances based on

simple random sampling assumptions. Consequently, the design effects will be greater than one. In

NAEP, the underestimates are the result of ignoring the effects of clustering and unequal probabilities of

selection in the variance calculations.

Since most of the analyses conducted by NAEP are based on the results of scaling models that

summarize performance of students across a learning area, design effects are expected for analyses based

on these scale scores. For reasons given in Chapter 12, NAEP provides each individual with a set of

plausible values, each of which is a random draw from the distribution of the potential scale scores for
188

that individual. Since NAEP’s current interest is on the effect of the sampling design on estimation and

inference, attention is restricted to a single measure of an individuals scale score, the first plausible value

of the individuals scale score.

A key statistic of interest is the estimated mean scale score of a subgroup of the population. An

estimate of the subgroup mean scale score is the weighted mean of the first plausible values of scale

score of the sampled individuals who belong to the subpopulation of interest. Let Y be the weighted

mean of the plausible values of the sampled members of the subpopulation. The conventional estimate of

the variance of Y is

Var Y wy Y

N W con

i i i

N

  = −

⋅

=

+

∑ 2

1 ,

where N is the total number of sampled individuals in the subpopulation for which plausible values are

available, wi is the weight of the i

th individual, yi is a plausible value from the distribution of potential

proficiencies for that individual, and W+ is the sum of the weights across the N individuals.

The design effect for the subgroup mean scale score estimate is

deff(Y

_

) = VarJK(Y

_

) / Varcon(Y

_

)

where VarJK( Y ) is the jackknife variance of Y (As has been pointed out previously, VarJK( Y ) as

computed does not measure the variability of Y due to imprecision in the measurement of the

proficiencies of the sampled individuals. The estimation of this very important source of variability is

discussed in Chapter 12.) Of the factors that determine deff( Y ), the effects of stratification are usually

less than one, which means the efficiency of a stratified sampling is better than a simple random

sampling; whereas the clustering effects are always larger than one. The clustering effects can be

approximated by

1 1 + − ( ) m ρ ,

where m is the average cluster size and ρ is the intracluster correlation (Cochran, 1977, p. 209). Therefore,

the large cluster size or large intercluster correlation will inflate the clustering effects.

Values of the design effects for subgroup mean proficiencies are displayed, by grade, in Tables

10-19 through 10-21, for the 1998 national assessments of reading, writing, and civics, respectively.

Design effects are shown for the population as a whole (Total) as well as for a variety of demographic

subgroups: gender; race/ethnicity (White, Black, Hispanic, Asian American, other); type of location

(central city, urban fringe/large town, rural/small town); parental education (did not graduate high school,

graduated high school, post-high school, graduated college, unknown); and type of school (public,

nonpublic). These particular demographic variables were selected because (1) they are major variables in

NAEP reports and (2) they reflect different types of divisions of the population that might have different

levels of sampling variability.

The tables show that the design effects are predominantly larger than 1, indicating that standard

variance estimation formulas will be generally too small, usually markedly so. Although the design

effects appear somewhat different for certain subgroups of the population, they are, perhaps, similar

enough (at least within a subject and grade) to select an overall composite value that is adequate for most

purposes. In choosing a composite design effect, some consideration must be made about the relative
189

consequences of overestimating the variance as opposed to underestimating the variance. For example, if

an overestimate of the variance is viewed as severe an error as an underestimate, the composite design

effect should be near to the center of the distributions of the design effects. Possible composites of this

type are the mean and median design effects across the combined distribution of all design effects. Larger

design effects should be used if it is felt that it is a graver error to underestimate the variability of a

statistic than to overestimate it. For example, Johnson and King (1987) examine estimation of variances

using design effects (among other techniques) under the assumption that the consequences of an

underestimate are three times as severe as those of an overestimate of the same magnitude. Adopting a

loss function that is a weighted sum of absolute values of the deviations of predicted from actual with

underestimates receiving three times the weight of overestimates, produces the upper quartile of the

design effects as the composite value. This assumes that the distribution of design effects is roughly

independent of the jackknife estimates of variance, so that the size of a design effect does not depend on

the size of the variance.

To compare Table 10-21 with Tables 10-19 and 10-20, the design effects for mean civics

proficiencies are smaller than those of reading and writing. The reading reporting samples consist of

non-SD/LEP students in sample types 2 and 3, and SD/LEP students in sample types 2. The intracluster

correlation is larger for reading reporting samples that contain large groups of non-SD/LEP students.

Therefore, the clustering effects for the reading reporting samples become larger than those of civics,

which only used students in sample type 3.

Table 10-19

Design Effects by Demographic Subgroup and Grade

for Mean Reading Scale Scores* 

Grade 4 Grade 8 Grade 12

Total 3.15 5.30 3.98

Male 2.95 3.69 3.86

Female 1.38 3.14 2.09

White 2.55 4.55 2.96

Black 2.31 2.55 3.62

Hispanic 3.01 7.23 3.08

Asian American 1.35 7.62 4.53

Other race/ethnicity 1.50 2.30 1.57

Urban 6.12 7.81 8.11

Suburban 4.72 6.52 3.98

Rural 2.24 4.80 3.70

PARED < HS 1.00 2.22 1.74

PARED = HS 1.41 2.96 1.69

PARED > HS 0.92 2.47 1.77

PARED = College 2.68 2.72 2.15

PARED = Unknown 1.40 2.17 1.51

Public school 2.92 4.64 4.09

Nonpublic school 6.37 6.59 3.68

* Design effects are based on the conventional and jackknife variances of

subgroup means of the first plausible values of scale score.
190

Table 10-20

Design Effects by Demographic Subgroup and Grade for Mean Writing Scale Scores* 

Grade 4 Grade 8 Grade 12

Total 5.42 6.42 6.60

Male 3.48 5.11 4.14

Female 3.11 3.26 3.99

White 3.95 5.57 4.90

Black 1.88 2.53 5.01

Hispanic 5.76 5.45 3.02

Asian American 3.06 9.58 6.89

Other race/ethnicity 2.04 1.66 2.06

Urban 6.90 10.40 10.92

Suburban 5.95 12.95 8.88

Rural 6.48 4.74 2.42

PARED < HS 6.07 3.45 1.87

PARED = HS 1.65 1.40 1.71

PARED > HS 2.12 2.51 2.62

PARED = College 4.21 5.12 3.70

PARED = Unknown 1.45 1.14 1.38

Public school 5.80 5.71 7.09

Nonpublic school 4.59 5.33 5.60

* Design effects are based on the conventional and jackknife variances of

subgroup means of the first plausible values of scale score.

Table 10-21

Design Effects by Demographic Subgroup and Grade for Mean Civics Scale Scores*

Grade 4 Grade 8 Grade 12

Total 2.34 3.23 3.70

Male 1.82 2.57 2.83

Female 1.48 1.95 2.36

White 2.24 3.25 3.39

Black 0.82 1.33 2.95

Hispanic 2.79 1.42 1.54

Asian American 0.94 8.44 6.41

Other race/ethnicity 1.41 1.02 1.78

Urban 2.15 3.67 4.52

Suburban 2.65 3.75 3.74

Rural 4.32 3.88 3.15

PARED < HS 1.35 3.66 1.19

PARED = HS 1.94 1.75 0.97

PARED > HS 1.34 1.84 2.07

PARED = College 1.83 2.16 2.5

PARED = Unknown 1.67 1.67 1.53

Public school 2.13 2.84 3.85

Nonpublic school 4.05 12.31 2.71

* Design effects are based on the conventional and jackknife variances of

subgroup means of the first plausible values of scale score.
191

Table 10-22 gives the composite values of mean, median, and upper quartile of the distribution of

design effects for mean scale score by grade for the reading, writing, and civics assessments, and across

those assessments.

Table 10-22

Within-Grade Mean, Median, and Upper Quartile of the

Distribution of Design Effects for 1998 National Assessments

by Subject Area and Across Subject Areas

Statistic Grade 4 Grade 8 Grade 12

Distribution Across

Demographic Subgroups

Upper Quartile 3.00 6.22 3.95

Mean 2.67 4.40 3.23

 Mean Reading Proficiencies

Median 2.43 4.12 3.35

Upper Quartile 5.79 5.68 6.35

Mean 4.11 5.13 4.60

 Mean Writing Proficiencies

Median 4.08 5.12 4.07

Upper Quartile 2.32 3.67 3.62

Mean 2.07 3.37 2.84

 Mean Civics Proficiencies

Median 1.89 2.71 2.77

Distribution Across

Subject Areas and

Demographic Subgroups

Upper Quartile 4.03 5.42 4.07

Mean 2.95 4.30 3.56

 Across Subject Areas

Median 2.33 3.56 3.12

* Design effects are based on the conventional and jackknife variances of subgroup means of the

first plausible values of scale score.

The Varcon( Y )as defined above is an estimate of S 2 /N where S 2represents the unit variance for a

simple random sample for the population of students from which the sample is also drawn. This is an

appropriate estimate of the increase in variance over simple random sampling from that population due to

the effects of weighting. However, the computer packages used for estimating the variance may not

reflect the weights in estimating the unit variance, as given above, but instead may provide an estimate of

a unit variance of the form

( )

N

2

i

1

y Y

N N-1 i=1

 (- ∑ ) .

In this case, the unweighted estimate of unit variance would be appropriate for the denominator

of a design effect measure of the increase in variance over the unit variance as estimated by the computer

package. If there is no correlation between the wi and yi, there would be little difference between the

two.

193

Chapter 11

STATE WEIGHTING PROCEDURES AND VARIANCE ESTIMATION1

Jiahe Qian, Bruce A. Kaplan, and Eugene G. Johnson

Educational Testing Service

Ibrahim S. Yansaneh and Keith F. Rust

Westat

11.1 OVERVIEW

The 1998 state assessment program included samples of fourth- and eighth-grade students in

public and nonpublic schools. The samples of students were selected using a complex multistage design

involving the sampling of students from participating schools within each state. See Chapter 4 for a

detailed description of the state sample design. Tables providing weighted counts of assessed and

excluded students appear in this chapter. Supplemental data is provided in Appendix B tables.

The weighting process involved the development of survey weights for students, using data from

a periodic assessment of students for each participating school in each of the states, territories, and

military jurisdictions of the U.S. Following the collection of assessment and background data from and

about assessed and excluded students, the processes of deriving sampling weights and associated sets of

replicate weights were carried out. The sampling weights are needed to make valid inferences from the

student samples to the respective populations from which they were drawn. Replicate weights are used in

the estimation of sampling variance, through a procedure known as jackknife repeated replication.

Weights were developed for students sampled at grades 4 and 8 for the state assessment in

reading and at grade 8 for the state assessment in writing. Each student was assigned a weight to be used

for making inferences about each state’s students. This weight is known as the full-sample or overall

sample weight. The full-sample weight contains five components. First, a base weight is established that

is the inverse of the overall probability of selecting the sampled student. The base weight incorporates

the probability of selecting a school and the student within a school. This weight is then adjusted for two

sources of nonparticipation—school level and student level. These weighting adjustments seek to reduce

the potential for bias from such nonparticipation by increasing the weights of students from schools

similar to those schools not participating, and by increasing the weights of students similar to those

students from within participating schools who did not attend the assessment session (or makeup session)

as scheduled. Furthermore, the weights reflect the trimming of extremely large weights at each stage in

the weighting process. For more detail on the implementation of these weighting steps, see Sections 11.2

and 11.3.

Section 11.4 addresses the effectiveness of the adjustments made to the weights using the

procedures described in Section 11.3, examining characteristics of nonresponding schools and students,

and investigating the extent to which nonrespondents differ from respondents in ways not accounted for

 

1 Ibrahim Yansaneh and Keith F. Rust were responsible for the design and implementation of the weighting process for the 1998

NAEP state assessments. Jiahe Qian, with the assistance of Bruce Kaplan and in consultation with Eugene G. Johnson, was

responsible for the planning, specification, and coordination of the state weighting at ETS. The statistical programming for this

chapter was overseen by Bruce Kaplan and provided by Phillip Leung, Michael Narcowich, and Youn-Hee Lim.
194

in the weight adjustment procedures. Section 11.5 considers the distributions of the final student weights

in each jurisdiction, and whether there were outliers that called for further adjustment.

In addition to the full-sample weights, a set of replicate weights was provided for each student.

These replicate weights are used in calculating the sampling errors of estimates obtained from the data,

using the jackknife repeated replication method. Full details of the method of using these replicate

weights to estimate sampling errors are contained in the Technical Report of the NAEP 1994 Trial State

Assessment Program in Reading (Mazzeo, Allen, & Kline, 1995) and in earlier NAEP state technical

reports. Section 11.6 of this report describes how the sets of replicate weights were generated for the

1998 state assessment data. The methods of deriving these weights were aimed at reflecting the features

of the sample design appropriately in each jurisdiction, so that when the jackknife variance estimation

procedure is implemented, approximately unbiased estimates of sampling variance are obtained.

As detailed in Chapter 5, two different sets of administration rules indicated by the sample type

field were used in the 1998 state assessment program for reading. ETS raked the student weights for each

subset to force agreement with the totals estimated using both subsets combined. This raking process is

detailed in Section 11.7. The process of trimming extremely large raked student weights is also

described.

11.2 CALCULATION OF BASE WEIGHTS

11.2.1 Calculation of School Base Weights

Base weights were assigned to schools separately by grade and subject. The base weight assigned

to a school was calculated as the reciprocal of the overall probability of selection of that school. For the

grade 8 samples, the school base weight depended on the assessment subject, because some schools were

so small that students were tested in only one subject. For “new” schools selected using the supplemental

new school sampling procedures (see Chapter 4), the school base weight reflected the combined

probability of selection of the district, and school within district.

Thus the base weight for school i was calculated as

1 f o r originally sampled schools; and { , 1}

1 f o r new schoo l s 

sch

i

Min EHIT

w

DISTPROB TCPNEW





 = 



 ×

where EHIT denotes the expected number of hits during sample selection; DISTPROB denotes the

selection probability assigned to each sampled school district for updating purposes; and TCPNEW

denotes the school probability of selection of new and newly eligible schools.

In each jurisdiction, all schools included in the sample with certainty were assigned school base

weights of unity. Schools sampled with certainty were sometimes selected more than once in the

systematic sampling process. For example, a school that was selected twice was allocated twice the usual

number of students for the assessments, or two sessions; a school that was selected three times was

allocated three times the usual number of students for the assessments, or three sessions. All schools at

grade 8 with less than 20 students were assigned one subject (see Chapter 4). For these schools, the base

weight included a factor of 2. Additional details about the weighting process are given in the sections

below.
195

11.2.2 Weighting New Schools

New public schools were identified and sampled through a two-stage sampling process, involving

the selection of districts, and then of new schools within selected districts. This process is described in

Chapter 4. There were two distinct processes used depending upon the size of the district.

Within each jurisdiction, public school districts were partitioned into “small” districts—those

having at most three schools on the aggregate frame and no more than one fourth-, one eighth-, and one

twelfth-grade school. The remainder of the districts were denoted as “large” districts. For the larger

districts (i.e., those having multiple schools in at least one of grades 4, 8, and 12), a sample of districts

was selected in each jurisdiction. Districts in the sample were asked to identify schools having grade 4 or

grade 8 that were not included on the school frame. A sample of these newly identified schools was then

selected. The base weight for these schools reflected the probability of two factors: (i) that the district

was selected for this updating process; and (ii) that the school was included in the NAEP sample, having

been identified as new by the district. If the school was in grade 8 but was only large enough to assess

one subject, the base weight included a factor of 2, as described in Section 11.2.1. There were no schools

identified in small districts (see Tables 4-8 and 4-9).

11.2.3 Trimming School Base Weights for New Schools

The base weights for new schools were evaluated for possible trimming. The process involved

computing a hypothetical school base weight for the new schools as though they had been selected as

part of the original sample. The hypothetical base weight was then compared to the actual base weight.

Those schools with actual base weights greater than three times the hypothetical base weights had their

base weights trimmed to three times their hypothetical base weights.

The trimming factor was computed as

 

3 for new schools with 3; and

 1 for other new schools and for non-new schools;

i

RSCHBWT

RSCHBWT f

 > 

 = 





where RSCHBWT denotes the ratio of the school base weight to the hypothetical base weight.

The trimmed school base weight, denoted by tsch

wi , was then defined as the product of the school

base weight and the trimming factor. That is,

tsch

wi = i f × schwi .

Two schools had their weights trimmed as a result of this process. One of these schools is in a

state that dropped out of the assessment. The other school has a trimming factor very close to 1, and

therefore is not expected to have a significant impact on the weights.

11.2.4 Treatment of Substitute Schools

A school that replaced a refusing school (i.e., a substitute school) was assigned the weight of the

refusing school. Thus the substitute school was treated as though it were the original school that it
196

replaced, for purposes of obtaining school base weights. The base weight was adjusted by a factor of 2

for grade 8 schools that were only large enough to assess one subject.

11.2.5 Calculation of Student Base Weights

Within the sampled schools, eligible students were sampled for assessment using the procedures

described in Chapter 4. The within-school probability of selection for each subject therefore depended on

the number of grade-eligible students in the school and the number of students selected for the

assessment (usually 30). The within-school weights for sampled schools were adjusted to account for the

fact that some schools operate twelve months per year and have only a proportion of their total

enrollment attending school at any one time. For substitute schools, the within-school weights were

further adjusted to compensate for differences in the grade enrollments of the substitute and the

originally sampled (replaced) schools. In the case of eighth-grade schools, the within-school weight also

incorporated a factor to account for (i) cases in which small schools were assigned at random to do one

subject (reading or writing); and (ii) the random assignment of students to subjects. Thus, in general, the

within-school student weight for the j

th student in school i was equal to:

1 2 within i

ij i i

i

N W = K K

n

⋅ ×

where

Ni = the number of grade-eligible students enrolled in the school, as reported

at the time of student sampling; and

ni = the number of students selected for the given subject.

The factors K1i and K2i in the formula for the within-school student weight generally apply to

only a few schools in each jurisdiction. The factor K1i adjusts the count of grade-eligible students in a

substitute school to be consistent with the corresponding count of the originally sampled (replaced)

school. Specifically, for substitute schools,

1

 i

i

s

i

E

E

K =

with

Ei = the grade enrollment of the originally sampled (replaced) school; and

Ei

s = the grade enrollment of the substitute school.

For nonsubstitute schools, K1i = 1.

The factor K2i, which was applied to schools determined to be year-round schools, is defined as:

2

1 = 1 i

off

K − p

where poff is the percentage of students enrolled in the school who were not scheduled to attend

school at the time of assessment. For schools that are not year-round schools (the great majority), K2i = 1.
197

The overall student base weight for a student j selected for the assessment for a given subject

(reading or writing) in school i was obtained by multiplying the trimmed school base weight by the

within-school student weight and therefore was computed as:

 = base tsch within W WW ij i ij × .

11.3 ADJUSTMENTS FOR NONRESPONSE

As mentioned earlier, the base weight for a student was adjusted by two factors: one to adjust for

nonparticipating schools for which no substitute participated, and another to adjust for students who were

invited to the assessment but did not attend the scheduled sessions (original or makeup).

11.3.1 Defining Initial School-Level Nonresponse Adjustment Classes

School-level nonresponse adjustment classes were created separately for public and nonpublic

schools within each jurisdiction. For each set, these classes were defined as a function of their sampling

strata as follows.

Public Schools. For each jurisdiction, except Virgin Islands, DoDEA/DDESS2

, and

DoDEA/DoDDS3

, the initial school nonresponse adjustment classes were formed by cross classifying the

level of urbanization and minority status (see Chapter 4 for definitions of these characteristics). Where

there was only one minority status category within a particular level of urbanization, a categorized

version of median household income was crossed with the urbanization category. For this purpose within

each level of urbanization, public schools were sorted by the median household income, and then divided

into three groups of about equal size, representing low, middle, and high income areas. In Virgin Islands,

there was no information on minority status or median household income. Thus, for Virgin Islands, at

grade 4 a categorized version of estimated grade enrollment was used, and at grade 8, due to the small

number of schools, all schools were placed in the same initial nonresponse adjustment cell. In all cases,

for schools with SD/LEP students, sample type (whether accommodations were offered or not) was used

in addition to the variables described above.

Department of Defense Education Activity/Department of Defense Domestic Elementary Schools

(DoDEA/DDESS) and Department of Defense Education Activity/Department of Defense Dependents

Schools (DoDEA/DoDDS). For the jurisdictions comprising DoDEA/DDESS and DoDEA/DoDDS

schools, urbanization, median income, and metro status were not available. Therefore, the initial school

nonresponse adjustment classes were defined by the state or district code, except for DoDEA/DDESS

grade 8, which had only one adjustment cell due to the small number of schools. Again, sample type was

used in addition to the variables described above.

Nonpublic Schools. For each jurisdiction (excluding Virgin Islands nonpublic schools), initial

nonresponse adjustment classes were formed by cross classifying school type (Catholic and nonCatholic) and metropolitan status (the urban/rural nature of the place where the school is located). For

Virgin Islands, urban/rural status was not available, so only school type was used. For schools with

SD/LEP students, sample type was used in addition to the variables described above.

 

2 Department of Defense Education Activity/Department of Defense Domestic Elementary and Secondary Schools

3

 Department of Defense Education Activity/Department of Defense Dependents Schools
198

11.3.2 Constructing the Final Nonresponse Adjustment Classes

The objective in forming the nonresponse adjustment classes is to create as many classes as

possible that are internally as homogeneous as possible, but such that the resulting nonresponse

adjustment factors are not subject to large random variation. Consequently, all initial nonresponse

adjustment classes deemed unstable were collapsed with suitable neighboring classes so that: (i) the

combined class contained at least six sessions, and (ii) the resulting nonresponse adjustment factor did

not exceed 1.35. (In a few cases, a factor in excess of 1.35 was permitted). When 100 percent of the

public schools in a jurisdiction responded, no action was taken for a public-school adjustment class that

contained fewer than six sessions. The same approach was used for nonpublic schools where 100 percent

of the schools participated. Although there is clearly no adjustment for school nonresponse in these

cases, this procedure could have an effect on the final definition of the student nonresponse adjustment

classes (see Section 11.3.4).

Public Schools. For public schools, inadequate nonresponse adjustment classes were reinforced

by collapsing adjacent levels of minority status (or median household income level if minority

information was missing). Metropolitan and non-metropolitan schools were combined together in cases

where there were less than six cooperating schools after collapsing across all levels of minority status (or

median household income levels, if minority status information was missing) that were not mixed. No

collapsing was done across sample type.

Nonpublic Schools. For nonpublic schools in all states except Virgin Islands, inadequate classes

were reinforced by collapsing adjacent levels of metropolitan-area status within school type. Catholic and

non-Catholic schools were kept apart to the extent possible, particularly when the only requirement to

combine such schools was as a means of reducing the adjustment factors below 1.35. For nonpublic

schools in Virgin Islands, Catholic and non-Catholic schools were collapsed together in order to form a

stable nonresponse adjustment class.

11.3.3 School Nonresponse Adjustment Factors

The school-level nonresponse adjustment factor for the i

th school in the hth class was computed

as:

(1) = h

h

sch

hi hi

i C

h sch

hi hi hi

i C

W E

F

W E δ

∈

∈

×

× ×

∑

∑

where

Ch = the subset of school records in class h,

sch Whi = the base weight of the ith school in class h,

Ehi = the grade enrollment for the ith school in class h,

hi

i h

δ = 

 if the th school in adjustment class participated in the assessments; and

 otherwise.

1

0










199

Both the numerator and denominator of the nonresponse adjustment factor contained only

schools that were determined to have eligible students enrolled.

In the calculation of the above nonresponse adjustment factors, a school was said to have

participated if:

• it was selected for the sample from the frame or from the lists of new schools

provided by participating school districts, and student assessment data were

obtained from the school; or

• the school participated as a substitute school and student assessment data were

obtained (so that the substitute participated in place of the originally selected

school).

The nonresponse-adjusted weight for the ith school in class h was computed as:

adj (1) sch

hi h hi W W = F × .

11.3.4 Student Nonresponse Adjustment Classes

The initial student nonresponse classes for assessed students were formed based on several

variables. These variables are based on information from the sample design, age of the student, final

collapsed school nonresponse cells, and the actual monitor status (or assigned monitor status, if the actual

monitor status is not available; see Chapter 4) at the session level. The first of these was public/nonpublic

strata and an indicator of whether or not a student was excluded from the assessment. Public/nonpublic

strata were then cross classified by a variable created from combining SD/LEP status and the sample type

for the student.

Within these categories, the initial student nonresponse adjustment classifications were defined

further depending on the SD/LEP status of a student. For all schools except DoDEA/DDESS and

DoDEA/DoDDS, if a student was SD or LEP, then the class was formed by urbanization cross classified

by student age. Age was used to classify students into two groups (for grade 4, those born in September

1987 or earlier and those born in October 1987 or later, and for grade 8, those born in September 1983 or

earlier and those born in October 1983 or later). If a student was neither SD nor LEP, then the initial

nonresponse adjustment class was formed by urbanization cross classified by student age (as defined

above), by the quality control monitoring status (see Chapter 4), then finally by minority status as

collapsed for the school nonresponse. For the DoDEA/DDESS and DoDEA/DoDDS schools, the

nonresponse adjustment classes for SD and LEP students was student age cross classified by the minority

status variable as defined for the school nonresponse adjustment classes.

Following creation of these student nonresponse adjustment classes, all unstable classes were

identified for possible collapsing with other classes. A class was considered to be unstable when either of

the following conditions was true for the given class:

• number of responding eligible students was fewer than 20, or

• nonresponse adjustment factor exceeded 1.5.
200

All classes deemed unstable in the previous step were collapsed with other classes using the

following rules:

• Do not collapse across public and nonpublic.

• Do not collapse across SD/LEP and non-SD–non-LEP.

• If within cells defined by the cross classification of public/nonpublic and SDLEP/non-SD–non-LEP status, and sample type within the SD/LEP categories, all

of the adjustments are one, no adjustments are made.

• Collapse across the last variable of the nonresponse adjustment cell only (i.e.,

collapse across geography for SD/LEP students in Department of Defense

Education Activity (DoDEA) schools).

More collapsing was necessary only if the resulting classes had fewer than 15 responding eligible

students. Collapsing then continued within the successive variables until the class size was no longer

deficient or until a “set” boundary that could not be crossed was reached. In the case of SD or LEP

students, more collapsing was done to eliminate the rare situation in which all students in a class were

nonrespondents.

11.3.5 Student Nonresponse Adjustments

As described above, the student-level nonresponse adjustments for the assessed students were

made within classes defined by the SD/LEP status, sample type, final school-level nonresponse

adjustment classes, monitoring status of the school, and age group of the students. Subsequently, in each

jurisdiction, the final student weight for the j

th student of the ith school in class k was then computed as:

final adj within

kij i ij k W WW = × × F

where

Wi

adj = the nonresponse-adjusted school weight for school i;

Wij

within = the within-school weight for the jth student in school i; and

 = .

ij

j

k

ij kj

j

W

F

W δ

∑

∑

In the above formulation, the summation included all students, j, in the k

th final (collapsed)

nonresponse class. The indicator variable δ kj had a value of 1 when the jth student in adjustment class k

participated in the assessment; otherwise, δ kj = 0.

For excluded students, no nonresponse adjustment procedures were applied because excluded

students were not required to complete an assessment. In effect, all excluded students were considered

respondents. Weights are provided for excluded students so as to estimate the size of this group and its

population characteristics. Tables 11-1 through 11-6 summarize the unweighted and final weighted

counts of assessed and excluded students in public and nonpublic schools for each jurisdiction, grade and

subject.
201

Table 11-1

Unweighted and Final Weighted Counts of Assessed and Excluded Students by Jurisdiction,

Grade 4 Public Schools, 1998 Reading State Samples

Assessed Excluded Assessed and Excluded

Jurisdiction Unweighted Weighted Unweighted Weighted Unweighted Weighted

Total 109,148 2,646,973 9,186 260,558 118,334 2,907,530

Alabama 2,559 56,372 239 4,922 2,798 61,294

Arizona 2,602 55,867 318 6,349 2,920 62,216

Arkansas 2,656 30,773 144 1,613 2,800 32,386

California 1,898 372,225 384 65,127 2,282 437,352

Colorado 2,656 49,221 195 3,309 2,851 52,530

Connecticut 2,607 38,543 379 4,971 2,986 43,514

Delaware 2,483 8,171 127 381 2,610 8,552

District of Columbia 2,464 4,691 284 504 2,748 5,194

DoDEA/DDESS 2,693 2,821 128 128 2,821 2,949

DoDEA/DoDDS 2,670 6,310 105 234 2,775 6,545

Florida 2,658 154,056 224 12,220 2,882 166,276

Georgia 2,733 96,499 179 6,058 2,912 102,557

Hawaii 2,742 13,548 144 676 2,886 14,224

Illinois 2,264 124,291 200 10,148 2,464 134,439

Iowa 2,339 33,263 171 2,324 2,510 35,587

Kansas 1,922 32,925 104 1,657 2,026 34,582

Kentucky 2,508 41,123 233 3,661 2,741 44,784

Louisiana 2,701 51,743 308 5,741 3,009 57,484

Maine 2,464 15,635 231 1,294 2,695 16,929

Maryland 2,344 57,644 204 4,894 2,548 62,538

Massachusetts 2,478 70,290 188 5,222 2,666 75,512

Michigan 2,416 116,655 179 8,068 2,595 124,723

Minnesota 2,425 61,069 94 2,179 2,519 63,248

Mississippi 2,591 36,430 118 1,565 2,709 37,995

Missouri 2,599 60,008 206 4,488 2,805 64,496

Montana 1,936 11,065 67 360 2,003 11,425

Nevada 2,732 20,105 388 2,652 3,120 22,757

New Hampshire 1,908 15,509 91 671 1,999 16,180

New Mexico 2,550 21,238 330 2,521 2,880 23,759

New York 2,318 192,009 196 16,046 2,514 208,055

North Carolina 2,628 87,078 265 8,222 2,893 95,300

Oklahoma 2,647 43,087 303 4,366 2,950 47,453

Oregon 2,550 36,836 192 2,597 2,742 39,433

Rhode Island 2,698 11,139 221 844 2,919 11,983

South Carolina 2,518 43,925 273 4,493 2,791 48,418

Tennessee 2,735 66,272 120 2,737 2,855 69,009

Texas 2,443 249,823 383 37,861 2,826 287,684

(continued)
202

Table 11-1 (continued)

Unweighted and Final Weighted Counts of Assessed and Excluded Students by Jurisdiction,

Grade 4 Public Schools, 1998 Reading State Samples

Assessed Excluded Assessed and Excluded

Jurisdiction Unweighted Weighted Unweighted Weighted Unweighted Weighted

Utah 2,784 31,657 185 1,903 2,969 33,560

Virgin Islands 1,485 1,552 95 95 1,580 1,647

Virginia 2,723 76,981 228 6,123 2,951 83,104

Washington 2,491 67,261 137 3,662 2,628 70,923

West Virginia 2,568 19,137 271 1,868 2,839 21,005

Wisconsin 2,183 55,418 245 5,548 2,428 60,966

Wyoming 2,779 6,708 110 257 2,889 6,965

Table 11-2

Unweighted and Final Weighted Counts of Assessed and Excluded Students by Jurisdiction,

Grade 8 Public Schools, 1998 Reading State Samples

Assessed Excluded Assessed and Excluded

Jurisdiction Unweighted Weighted Unweighted Weighted Unweighted Weighted

Total 93,223 2,441,495 6,068 151,260 99,291 2,592,754

Alabama 2,490 54,366 177 3,718 2,667 58,084

Arizona 2,529 53,001 183 3,376 2,712 56,377

Arkansas 2,489 32,855 170 2,056 2,659 34,911

California 2,182 364,480 159 23,908 2,341 388,388

Colorado 2,673 49,634 133 2,270 2,806 51,904

Connecticut 2,617 35,939 214 2,655 2,831 38,594

Delaware 2,081 8,220 122 399 2,203 8,618

District of Columbia 1,589 3,967 142 306 1,731 4,273

DoDEA/DDESS 630 1,324 28 56 658 1,380

DoDEA/DoDDS 2,221 4,746 61 122 2,282 4,868

Florida 2,545 147,121 145 7,863 2,690 154,984

Georgia 2,600 95,969 146 4,870 2,746 100,839

Hawaii 2,602 12,468 163 715 2,765 13,183

Illinois 2,148 127,567 117 6,459 2,265 134,026

Kansas 1,932 34,261 105 1,574 2,037 35,835

Kentucky 2,342 44,684 105 1,943 2,447 46,627

Louisiana 2,585 50,192 228 3,982 2,813 54,174

Maine 2,474 15,471 164 963 2,638 16,434

Maryland 2,178 54,030 123 2,738 2,301 56,768

Massachusetts 2,306 60,590 148 3,546 2,454 64,136

Minnesota 2,039 63,573 61 1,669 2,100 65,242

Mississippi 2,332 33,909 173 2,363 2,505 36,272

Missouri 2,632 63,890 142 3,288 2,774 67,178

(continued)
203

Table 11-2 (continued)

Unweighted and Final Weighted Counts of Assessed and Excluded Students by Jurisdiction,

Grade 8 Public Schools, 1998 Reading State Samples

Assessed Excluded Assessed and Excluded

Jurisdiction Unweighted Weighted Unweighted Weighted Unweighted Weighted

Montana 1,946 12,021 82 412 2,028 12,433

Nevada 2,564 18,154 200 1,319 2,764 19,473

New Mexico 2,365 21,623 239 1,885 2,604 23,508

New York 1,923 181,223 208 17,019 2,131 198,242

North Carolina 2,595 81,637 222 6,317 2,817 87,954

Oklahoma 2,234 42,355 236 4,081 2,470 46,436

Oregon 2,294 38,419 105 1,498 2,399 39,917

Rhode Island 2,513 10,591 160 596 2,673 11,187

South Carolina 2,509 45,583 169 2,765 2,678 48,348

Tennessee 2,245 58,759 122 2,975 2,367 61,734

Texas 2,500 248,845 175 16,047 2,675 264,892

Utah 2,601 34,340 133 1,548 2,734 35,888

Virgin Islands 643 1,464 54 108 697 1,572

Virginia 2,592 73,995 187 4,824 2,779 78,819

Washington 2,323 69,342 104 2,856 2,427 72,198

West Virginia 2,537 20,565 239 1,756 2,776 22,321

Wisconsin 1,997 62,606 152 4,234 2,149 66,840

Wyoming 2,626 7,716 72 183 2,698 7,899

Table 11-3

Unweighted and Final Weighted Counts of Assessed and Excluded Students by Jurisdiction,

Grade 8 Public Schools, 1998 Writing State Samples

Assessed Excluded Assessed and Excluded

Jurisdiction Unweighted Weighted Unweighted Weighted Unweighted Weighted

Total 91,996 2,429,504 4,872 124,329 96,868 2,553,832

Alabama 2,449 53,997 169 3,521 2,618 57,518

Arizona 2,499 53,315 162 2,992 2,661 56,307

Arkansas 2,462 32,430 162 1,945 2,624 34,375

California 2,157 359,589 155 23,418 2,312 383,007

Colorado 2,697 50,662 117 1,914 2,814 52,576

Connecticut 2,592 36,138 221 2,786 2,813 38,924

Delaware 2,119 8,265 80 269 2,199 8,533

District of Columbia 1,592 4,007 130 276 1,722 4,283

DoDEA/DDESS 650 1,362 19 38 669 1,400

DoDEA/DoDDS 2,182 4,704 34 68 2,216 4,772

Florida 2,574 150,236 130 7,085 2,704 157,321

Georgia 2,605 96,368 138 4,599 2,743 100,967

(continued)
204

Table 11-3 (continued)

Unweighted and Final Weighted Counts of Assessed and Excluded Students by Jurisdiction,

Grade 8 Public Schools, 1998 Writing State Samples

Assessed Excluded Assessed and Excluded

Jurisdiction Unweighted Weighted Unweighted Weighted Unweighted Weighted

Hawaii 2,647 12,619 123 522 2,770 13,141

Illinois 2,145 129,782 95 5,263 2,240 135,045

Kentucky 2,341 44,823 66 1,145 2,407 45,968

Louisiana 2,653 51,962 158 2,882 2,811 54,844

Maine 2,508 15,659 148 860 2,656 16,519

Maryland 2,263 55,675 55 1,216 2,318 56,891

Massachusetts 2,399 62,177 131 3,091 2,530 65,268

Minnesota 1,980 63,353 65 1,884 2,045 65,237

Mississippi 2,401 35,008 130 1,708 2,531 36,716

Missouri 2,621 63,703 79 1,747 2,700 65,450

Montana 2,024 12,492 62 319 2,086 12,811

Nevada 2,553 18,325 181 1,167 2,734 19,492

New Mexico 2,426 22,277 192 1,476 2,618 23,753

New York 1,981 189,995 123 10,306 2,104 200,301

North Carolina 2,669 83,857 127 3,673 2,796 87,530

Oklahoma 2,258 42,418 239 4,054 2,497 46,472

Oregon 2,323 38,838 90 1,251 2,413 40,089

Rhode Island 2,516 10,584 129 488 2,645 11,072

South Carolina 2,469 45,294 160 2,619 2,629 47,913

Tennessee 2,275 59,184 104 2,536 2,379 61,720

Texas 2,530 250,733 169 15,518 2,699 266,251

Utah 2,588 34,091 117 1,355 2,705 35,446

Virgin Islands 614 1,412 59 118 673 1,530

Virginia 2,605 74,518 131 3,392 2,736 77,910

Washington 2,286 68,730 96 2,637 2,382 71,367

West Virginia 2,611 21,219 157 1,127 2,768 22,346

Wisconsin 2,006 62,152 105 2,895 2,111 65,047

Wyoming 2,726 7,551 64 169 2,790 7,720
205

Table 11-4

Unweighted and Final Weighted Counts of Assessed and Excluded Students by Jurisdiction,

Grade 4 Nonpublic Schools, 1998 Reading State Samples

Assessed Excluded Assessed and Excluded

Jurisdiction Unweighted Weighted Unweighted Weighted Unweighted Weighted

Total 8,101 210,902 64 2,131 8,165 213,033

Arkansas 166 2,386 0 0 166 2,386

Colorado 225 4,599 2 54 227 4,653

Connecticut 263 4,214 2 26 265 4,241

Florida 274 20,284 1 67 275 20,351

Georgia 270 6,631 6 113 276 6,744

Hawaii 379 2,000 0 0 379 2,000

Illinois 355 25,870 3 194 358 26,064

Iowa 330 4,257 1 17 331 4,274

Louisiana 425 10,462 4 120 429 10,582

Maine 131 917 0 0 131 917

Maryland 297 8,750 3 115 300 8,865

Massachusetts 284 8,951 5 156 289 9,106

Michigan 265 15,375 3 160 268 15,535

Minnesota 338 8,426 1 22 339 8,448

Mississippi 224 3,763 0 0 224 3,763

Missouri 320 9,621 2 74 322 9,695

Montana 102 466 1 4 103 471

Nebraska 478 3,063 3 21 481 3,083

Nevada 150 962 1 6 151 968

New Mexico 249 2,350 8 83 257 2,433

New York 377 36,271 5 398 382 36,669

North Carolina 236 6,773 0 0 236 6,773

Rhode Island 382 1,506 0 0 382 1,506

South Carolina 227 3,951 2 31 229 3,983

Utah 107 681 0 0 107 681

Virgin Islands 426 461 0 0 426 461

Washington 175 4,965 0 0 175 4,965

West Virginia 125 973 0 0 125 973

Wisconsin 426 11,710 10 463 436 12,173

Wyoming 95 266 1 4 96 271
206

Table 11-5

Unweighted and Final Weighted Counts of Assessed and Excluded Students by Jurisdiction,

Grade 8 Nonpublic Schools, 1998 Reading State Samples

Assessed Excluded Assessed and Excluded

Jurisdiction Unweighted Weighted Unweighted Weighted Unweighted Weighted

Total 5,554 182,810 43 1,000 5,597 183,810

Arkansas 133 1,754 2 33 135 1,787

Arizona 176 6,072 6 223 182 6,294

California 295 44,862 0 0 295 44,862

Colorado 154 2,310 0 0 154 2,310

Connecticut 371 5,143 3 50 374 5,192

Florida 190 14,159 1 45 191 14,204

Georgia 185 7,090 0 0 185 7,090

Illinois 289 20,787 1 78 290 20,865

Louisiana 459 10,267 2 47 461 10,314

Massachusetts 185 5,986 0 0 185 5,986

Maryland 329 8,021 0 0 329 8,021

Maine 78 535 0 0 78 535

Missouri 297 7,199 0 0 297 7,199

Mississippi 0 0 0 0 0 0

Montana 147 646 0 0 147 646

North Carolina 238 5,032 3 75 241 5,107

Nebraska 366 2,950 4 33 370 2,982

New Mexico 170 1,471 9 67 179 1,539

Nevada 130 943 1 11 131 954

New York 351 29,209 3 244 354 29,453

Rhode Island 403 1,507 5 19 408 1,527

Virgin Islands 228 394 0 0 228 394

Washington 230 5,284 3 76 233 5,360

West Virginia 99 1,041 0 0 99 1,041

Wyoming 51 149 0 0 51 149
207

Table 11-6

Unweighted and Final Weighted Counts of Assessed and Excluded Students by Jurisdiction,

Grade 8 Nonpublic Schools, 1998 Writing State Samples

Assessed Excluded Assessed and Excluded

Jurisdiction Unweighted Weighted Unweighted Weighted Unweighted Weighted

Total 5,593 173,497 27 960 5,620 174,457

Arkansas 140 2,143 1 13 141 2,155

Arizona 130 3,234 11 306 141 3,540

California 224 30,585 0 0 224 30,585

Colorado 137 2,916 0 0 137 2,916

Connecticut 240 4,151 2 30 242 4,180

Florida 213 13,409 1 42 214 13,451

Georgia 144 6,246 1 35 145 6,281

Illinois 314 23,623 0 0 314 23,623

Louisiana 580 11,449 0 0 580 11,449

Massachusetts 263 8,395 1 28 264 8,423

Maryland 350 9,168 0 0 350 9,168

Maine 95 831 0 0 95 831

Missouri 303 9,843 0 0 303 9,843

Montana 206 853 1 5 207 858

North Carolina 248 6,142 3 50 251 6,192

Nebraska 354 2,835 0 0 354 2,835

New Mexico 204 1,842 2 12 206 1,854

Nevada 108 730 0 0 108 730

New York 380 27,993 4 439 384 28,432

Rhode Island 434 1,680 0 0 434 1,680

Virgin Islands 193 383 0 0 193 383

Washington 155 3,824 0 0 155 3,824

West Virginia 117 977 0 0 117 977

Wyoming 61 246 0 0 61 246

11.4 CHARACTERISTICS OF NONRESPONDING SCHOOLS AND STUDENTS

In the previous section, procedures were described for adjusting the survey weights so as to

reduce the potential bias of nonparticipation of sampled schools and students. To the extent that the

characteristics of nonresponding schools or students are different from those of respondents in the same

nonresponse adjustment class, potential for nonresponse bias remains. Recently, some studies related

with this issue have been done, such as on the effects of excluded students in reporting results (see

Donoghue, 2000).

This section examines the potential for remaining nonresponse bias in two related ways. First,

weighted distributions for each grade and subject within each jurisdiction of certain characteristics of

schools and students, both for the full sample and for respondents only, are discussed. This analysis is of

necessity limited to those characteristics that are known for both respondents and nonrespondents, and

hence, cannot directly address the question of nonresponse bias. The approach taken does reflect the

reduction in bias obtained through the use of nonresponse weighting adjustments. As such, it is more
208

appropriate than a simple comparison of the characteristics of nonrespondents with those of respondents

for each subject and jurisdiction.

The second approach involves modeling the probability that a school is a respondent, as a

function of the nonresponse adjustment class to which the school belongs, together with other school

characteristics. This was achieved using linear logistic regression models, with school response status as

the dependent variable. By testing to see if the school characteristics add any predictive ability to the

model over using the membership of the nonresponse adjustment class to make this prediction,

researchers can obtain some insight into the remaining potential for nonresponse bias. If these factors are

substantially marginally predictive, there is danger that significant nonresponse bias will remains. See

Section 11.4.2 for details on how this approach was implemented.

11.4.1 Weighted Distributions of Schools Before and After School Nonresponse

To study the potential for nonresponse bias, Westat analysts compared the school characteristics

before and after school nonresponse for public schools. For public schools, the variables for which means

are presented are the percentage of Black students in the school, the percentage of Hispanic students, the

median household income (1989) of the ZIP code area where the school is located, and the type of

location. The first two variables were obtained from the sample frame, and hence from Quality Education

Data, Inc., (QED) as described in Chapter 4. Median income was obtained from the 1990 Donnelly File.

The variable designating type of location was derived for each sampled school using U.S. Bureau of

Census data. The type of location variable has seven possible levels, which are defined in Chapter 4.

Although this variable is not interval-scaled, the mean value does give an indication of the degree of

urbanization of the population represented by the school sample (lower values for type of location

indicate a greater degree of urbanization).

 For public schools, the mean values of the variables, both before and after nonresponse, were

calculated for all jurisdictions in reading grades 4 and 8, and writing grade 8. The means are weighted

appropriately to reflect whether nonresponse adjustments have been applied (i.e., to respondents only) or

not (to the full set of in-scope schools). The tables are presented in Appendix B. For each grade and

subject, two sets of means are presented for these four variables. The first set shows the weighted mean

derived from the full sample of in-scope schools selected for each subject, that is, respondents and

nonrespondents (for which there was no participating substitute). The weight for each sampled school is

the product of the school base weight and the grade enrollment. This weight therefore represents the

number of students in the state represented by the selected school. The second set of means is derived

from responding schools only, after school substitution. In this case the weight for each school is the

product of the nonresponse-adjusted school weight and the grade enrollment of the original school, and

therefore indicates the number of students in the jurisdiction represented by the responding school.

The characteristics of interest for nonpublic schools were the proportion of Catholic schools and

the proportion of schools that are located in urban districts. As was done for public schools, two sets of

means are presented: the means for the full sample and for the responding sample.

For both public and nonpublic schools, the differences between these sets of means give an

indication of the potential for nonresponse bias that has been introduced by nonresponding schools with

no participating substitute. For example, for grade 4 reading in Illinois, the mean percentage Black

enrollment, estimated from the original sample of public schools, is 20.92 percent. The estimate from the

responding schools is 26.33 percent. Thus there may be a slight bias in the results for Illinois because

these two means differ. Note, however, that the differences in the two sets of mean values are generally

very slight, at least in absolute terms, suggesting that it is unlikely that substantial bias has been

introduced by schools that did not participate and for which no substitute participated. Of course in a
209

number of states there was no nonresponse at the school level (weighted participation rate is 100%), so

that these sets of means are identical. Even in those jurisdictions where school nonresponse was

relatively high (such as in New Hampshire grade 4 reading, Minnesota grade 8 writing, and Wisconsin

grade 8 reading and writing), the absolute differences in means are slight. Occasionally the relative

difference is large, for instance, the “Percent Black” in Illinois for both grade 4 and grade 8 reading (for

public schools), or West Virginia grade 4 reading, Wyoming grade 4 reading, and New York grade 8

reading (for nonpublic schools). However, these are for small population subgroups, and thus are very

unlikely to have a large impact on results for the jurisdiction as a whole.

 11.4.2 Characteristics of Schools Related to Response

In an effort to evaluate the possibility that substantial bias remains as a result of school

nonparticipation, following the use of nonresponse adjustments, a series of analyses were conducted on

the response status for public schools. These analyses were restricted to those jurisdictions with a

participation rate of below 90 percent (after substitution), because these are the jurisdictions where the

potential for nonresponse bias was likely to be the greatest. Jurisdictions with an initial public-school

response rate below 70 percent were not included, since NAEP does not report results for these

jurisdictions because of concern about nonresponse bias. Information about this can be found in Chapters

17 and 21. Nonpublic schools were omitted from these analyses as well because of the small sample sizes

involved, meaning that it is difficult to assess whether a potential for bias exists. Table 11-7 gives each

participating states’ participation rate as included in the analysis for each grade and subject.

Table 11-7

Jurisdictions Included in Logistic Regression Analysis

of the NAEP 1998 State Assessment

Grade Subject Jurisdiction

Participation

Rate

4 Reading CA 80%

IL 84%

IA 84%

KS 70%

MD 88%

MA 88%

MN 86%

MT 78%

NH 70%

NY 84%

WA 89%

WI 82%

8 Reading CA 84%

IL 81%

KS 71%

KY 87%

MD 85%

MA 89%

MN 74%

(continued)
210

Table 11-7 (continued)

Jurisdictions Included in Logistic Regression Analysis

of the NAEP 1998 State Assessment

Grade Subject Jurisdiction

Participation

Rate

8 Writing MT 78%

NY 77%

OR 88%

TN 89%

WA 86%

WI 73%

CA 83%

IL 80%

KY 87%

MD 86%

MA 89%

MN 74%

MT 78%

NY 77%

OR 88%

TN 89%

WA 87%

WI 73%

The approach used was to develop a logistic regression model to predict the probability of

participation as a function of the nonresponse adjustment classes and other school characteristics. These

models were developed for public schools in each of the jurisdictions and for each grade and subject

specified in the above table. For the three grade-subject combinations, this resulted in the development of

37 models, which differ only in the number of nonresponse class levels that are included in the model.

The number of final nonresponse adjustment classes varied by state. The logistic regression analysis was

used to determine whether the response rates are significantly related to school characteristics, after

accounting for the effect of the nonresponse class. Thus, “dummy” variables were created to indicate

nonresponse class membership.

If there are k nonresponse classes within a jurisdiction, for nonresponse class i = 1, ..., k-1, let

Xij = 1 if the school j is classified in nonresponse class i,

= 0 otherwise.

Within each jurisdiction, a logistic model was fitted to the data on public-school participation. In

the model, the indicator variables for nonresponse class, and additional variables available for

participating and nonparticipating schools alike were included. These variables are denoted as Yij, for i

from 1 to 4 of school j. They were the percentage of Black students (Y1j), the percentage of Hispanic

students (Y2j), the estimated enrollment for grades 4 and 8 of the school (Y3j), and the median household

income of the ZIP code area in which the school was located (Y4j ).
211

Let Pj denote the probability that school j is a participant, and let Lj denote the logit of Pj. That is,

ln( ) 1

j

j

j

P

L

P = − .

The model fitted in each jurisdiction was the following:

 + + Lj i ij i = A BX CY ∑ ∑ ij ,

where A, Bi

, and Ci are the coefficients of the logistic regression model.

Note that this model cannot be estimated if there are nonresponse classes in which all schools

participated (so that no adjustments for nonresponse were made for those schools). Even though this

analysis was restricted to those jurisdictions with relatively poor response, unestimatable cases occurred

in a number of instances. When this happened, those (responding) schools in such classes were dropped

from the analyses. Tables 11-8, through 11-10 show the proportion of the state public-school student

population that is represented in the sample by schools from classes with less than 100 percent response

for each grade and subject. Thus in grade 4 reading for Illinois, Kansas, and New Hampshire, there was

some nonresponse within every adjustment class, whereas for the other nine states in grade 4, some

portion of the population is not represented because schools were dropped from classes with no

nonresponse. The states in which the entire student population is represented in the sample by schools

from classes with less than 100% response are Illinois, Kansas, Minnesota, New York, and Wisconsin for

grade 8 reading; and Illinois, Minnesota, New York, and Wisconsin for grade 8 writing. For the rest of

the states, in both grades, some portion of the student population is not represented because schools were

dropped from classes with no nonresponse.

The tables show that only three of the 37 models that contained all of the variables were

significant. These were the models for grade 8 reading and writing for Illinois and Minnesota, all with pvalues ranging from 0.0013 to 0.0184. Furthermore, the variables designating median household income

and percent of Hispanic students were not significant for any of the 37 models. For the models for

Minnesota grade 8 reading and writing, the only individual variable that was significant was the

estimated grade enrollment, with p-values of 0.0009 and 0.0007 respectively. The only significant

variable in the model for Illinois grade 8 writing was the percent of Black students, with a p-value of

0.0064. For some states, the overall model was not significant, but had individual variables that were

significant. Examples of such states are Kansas grade 4, where the significant individual variable was the

dummy variable corresponding to nonresponse class 4, which indicates for this state that the nonresponse

classes significantly explain the variation in the response rates. In fact, Kansas was the only state in

which the nonresponse class turned out to be a significant individual variable in the model. There were

two models, for grade 8 reading and writing in the state of Wisconsin, in which the percent of Black

students was significant even though the overall model was not.

As mentioned before, the variable designating the percent of Black students was clearly

significant in the models for Wisconsin grade 8 reading and writing, and for Illinois grade 8 writing. This

variable was used in forming nonresponse adjustment classes in these states. Note that the percent of

Black students in Wisconsin is 7.99 for the grade 8 reading fill sample (see Table B-2 in Appendix B),

and 9.56 for the respondents. This indicates that the final sample is somewhat over-representative of

schools with relatively high proportion of Black students. Similar results hold for Illinois and Wisconsin

grade 8 writing (see Table B-3 in Appendix B).
212

Table 11-8

Results of Logistic Regression Analysis of School Nonresponse - Grade 4, 1998 Reading State Samples

Model with All Variables 

Jurisdiction

School

Participation

Rate (%)

Percent of Population

Covered by Model

Degrees

of

Freedom Significance Significant Variables

De

Fre

California 79.92 92.74 7 p=0.279 none Iowa 83.94 80.13 Illinois 84.13 100.00 12 p=0.309 none Kansas 70.42 100.00 8 p=0.237 nonresponse cell 4, p=0.0390 Massachusetts 88.15 56.93 Maryland 88.42 73.21 Minnesota 85.82 55.45Montana 78.48 91.37 New Hampshire 70.48 100.00 7 p=0.564 none New York 83.92 82.25 Washington 89.25 88.51 Wisconsin 82.04 80.15 
Table 11-9

Results of Logistic Regression Analysis of School Nonresponse – Grade 8, 1998 Reading State Sam

Model with All Variables 

Jurisdiction

School

Participation

Rate (%)

Percent of

Population

Covered

by Model

Degrees

of

Freedom Significance Significant Variables

California 83.74 79.87Illinois 81.12 100.00 9 p=0.001 none Kansas 70.60 100.00 9 p=0.748 none Kentucky 87.32 72.63 Massachusetts 89.20 77.59 Maryland 85.45 81.62 Minnesota 73.73 100.00 7 p=0.009 estimated grade enrollment,

p=0.0009

Montana 77.81 79.74 New York 77.27 100.00 8 p=0.198 none Oregon 87.53 86.66 Tennessee 89.03 60.09 8 p=0.203 none Washington 86.13 95.22 11 p=0.701 none Wisconsin 73.18 100.00 8 p=0.331 percent Black, p=0.0134 
214

Table 11-10

Results of Logistic Regression Analysis of School Nonresponse – Grade 8, 1998 Writing State Samples

Model with All Variables 

Jurisdiction

School

Participation

Rate (%)

Percent of

Population

Covered

by Model

Degrees

of

Freedom Significance Significant Variables

Degr

o

Free

California 83.15 85.83 4Illinois 80.28 100.00 9 p=0.003 Percent of Black students, p=0.0064 4 Kentucky 87.14 73.23 4Massachusetts 89.28 77.42 4Maryland 86.42 81.62 4Minnesota 73.51 100.00 7 p=0.018 Estimated grade enrollment,

p=0.0007

4 

Montana 77.60 82.51 4New York 77.27 100.00 8 p=0.099 none 4 Oregon 87.53 86.66 4Tennessee 89.03 60.07 8 p=0.354 none 4 Washington 86.59 95.16 11 p=0.506 none 4 Wisconsin 72.91 100.00 8 p=0.246 Percent of Black students, p=0.0068 4 
215

The only models in which the estimated grade-specific enrollment is significant are those for grade 8

reading and writing in the state of Minnesota. For public schools, this variable was not used in forming

nonresponse adjustment classes in these states (it was used only for Virgin Islands). This variable is not shown

in Tables B-1 through B-3 in Appendix B. However, the near-zero value of the coefficient for this variable in

the logistic model indicates that small schools have as much chance of participating as larger schools, after

controlling for the other predictor variables.

To determine if the variables other than the nonresponse adjustment class variables added explanatory

power to the model, all variables except the nonresponse adjustment class variables were tested collectively to

see if the estimates of the parameters were equal to zero. This evaluates whether, taken as a group, the Y

variables are significantly related to the response probability, after accounting for nonresponse class. The

results are shown in the last columns of Tables 11-8 through 11-10. Only three of the 37 tests were significant.

The rest of the tests were not significant, which suggests that the variables did not add to the model after

accounting for the nonresponse adjustment classes, even though on occasion an individual variable was

significant. These results hold for Kansas grade 4 reading, where the full model was not significant, but the

dummy variable representing nonresponse class 4 was significant. This seems to indicate for Kansas, the

nonresponse adjustment classes alone explain the significant variations in the probability of participation in the

grade 4 assessments.

The results of the analysis indicate that on occasion there were differences between the originally

sampled schools and those that participated, that were not fully removed by the process of creating nonresponse

adjustments. Although these effects were not dramatic, they were sometimes statistically significant, and in

these instances, this was reflected in noticeable differences in population characteristics between respondents

against those who were originally sampled. However, the evidence presented here does not permit valid

speculation about the likely size or even direction of the bias in achievement results in reading and writing for

the few states where these sample differences are noticeable. The results and details of the logistic regression

analysis are given in Westat’s Sampling Activities and Field Operations for 1998 NAEP (Gray, et al., 2000).

11.4.3 Weighted Distributions of Students Before and After Student Absenteeism

To check the difference between the full sample and the assessed samples, Westat analysts studied

weighted distributions of students before and after student absenteeism. For the public schools in each

jurisdiction, subject, and grade, Westat calculated the weighted sampled percentages of students by gender

(male) and race/ethnicity (White, not Hispanic; Black, not Hispanic; Hispanic), as well as SD/LEP status for the

full sample of students (after student exclusion), and for the assessed sample. See tables in Appendix B. The

mean student age in months is also computed on each basis. In those jurisdictions having adequate school

response rates to permit reporting of combined results for public- and nonpublic-school students, these statistics

were calculated for both grades and subjects for all students, public and nonpublic.

The weight used for the full sample was the adjusted student base weight, defined in Section 11.2.5.

The weight for the assessed students was the final student weight, defined in Section 11.3.5. The difference

between the estimates of the population subgroups is an estimate of the bias in estimating the size of the

subgroup, resulting from student absenteeism.

Care must be taken in interpreting these results. First, note that there is generally little difference in the

proportions estimated from the full sample and those estimated from the assessed students. While this is

encouraging, it does not eliminate the possibility that bias exists within the state as a whole, within the results

for gender and race/ethnicity subgroups, or within other subgroups. Second, when differences do exist, they
216

cannot be used to indicate the likely magnitude or direction of the bias with any reliability. For example, in

Illinois the percentages of White and Black students in the full sample are respectively 56.87 and 22.24 percent.

For assessed students, these percentages are 61.97 for White students and 18.61 for Black students. This

indicates that White students are overrepresented and Black students are underrepresented in the sample of

assessed students. While these differences raise the possibility that some bias exists, it is not appropriate to

speculate on the magnitude of this bias by considering the assessment results for White or Black students in

comparison to other students in the state. The reason is that the overrepresented White students or

underrepresented Black students may not be typical of students that were included in the sample. Similarly,

White students who are disproportionately underrepresented or Black students who are disproportionately

overrepresented may not be typical either, because not all students within the same race/ethnicity group receive

the same student nonresponse adjustment.

One other feature to note is that, for assessed students, information about the student’s gender and

race/ethnicity is provided by the student, whereas for absent students, it is provided by the school. Evidence

from past NAEP assessments (see, for example, Rust & Johnson, 1992) indicates that there can be substantial

discrepancies between those two sources, particularly for grade 4 Hispanic students.

 11.5 VARIATION IN WEIGHTS

After computing the full-sample weights, an analysis was conducted on the distribution of the final

student weights for each grade-subject combination in each jurisdiction. The analysis was intended to (1) check

that the various weight components had been derived properly in each jurisdiction, and (2) examine the impact

of variability in the sample weights on the precision of the sample estimates, both for the jurisdiction as a whole

and for major subgroups within the jurisdiction.

The analysis was conducted by looking at the distribution of the final student weights for the assessed

students in each jurisdiction, grade, and subject separately by public and nonpublic schools. Two key aspects of

the distribution were considered in each case: the coefficient of variation (equivalently, the relative variance) of

the weight distribution, and the presence of outliers—cases whose weights were several standard deviations

away from the median weight.

It was important to examine the coefficient of variation of the weights, because a large coefficient of

variation reduces the effective size of the sample. Assuming that the variables of interest for individual students

are uncorrelated with the weights of the students, the sampling variance of an estimated average or aggregate is

approximately 2 (1+VW )times as great as the corresponding sampling variance based on a self-weighting

sample of the same size, where VW is the coefficient of variation of the weights. Outliers, or cases with extreme

weights, were examined because the presence of such outliers was an indication of the possibility that an error

was made in the weighting procedure, and because it was likely that a few extreme cases would contribute

substantially to the size of the coefficient of variation.

In most jurisdictions, the coefficients of variation were 35 percent or less, both for the whole sample

and for all subgroups. This means that the variation in sampling weights had little impact on the precision of

sample estimates.

A few relatively large student weights were observed in some jurisdictions for reading at both grades 4

and 8. An evaluation was made of the impact of trimming these largest weights back to a level consistent with

the largest remaining weights found in the state and grade. Such a procedure produced an appreciable reduction

in the size of the coefficient of variation for these weights, and hence this trimming was implemented. Westat
217

judged that this procedure had minimal potential to introduce bias, while the reduction in the coefficient of

variation of the weights gave rise to an appreciable decrease in sampling error for all jurisdictions, grades, and

subjects.

 11.6 CALCULATION OF REPLICATE WEIGHTS

A replication method known as jackknife was used to estimate the variance of statistics derived from

the full sample. The process of replication involves repeatedly selecting portions of the sample (replicates) and

calculating the desired statistic (replicate estimates). The variability among the calculated replicate estimates is

then used to obtain the variance of the full-sample estimate.

In each jurisdiction, replicates were formed in two steps. First, each school was assigned to one of a

maximum of 62 replicate groups, each group containing at least one school. In the next step, a random subset of

schools (or, in some cases, students within schools) in each replicate group was excluded. The remaining subset

and all schools in the other replicate groups then constituted one of the 62 replicates. The process of forming

these replicate groups, core to the process of variance estimation, is described below.

11.6.1 Defining Replicate Groups and Forming Replicates for Variance Estimation

Replicate groups were formed separately for public and nonpublic schools. Once replicate groups were

formed for all schools, students were then assigned to their respective school replicate groups. The formation of

replicate groups was done separately for SD/LEP and non-SD/LEP students. For SD/LEP students, there was an

additional set of replicate group assignments for reading at each grade for states with certainty schools.

Different replicate group assignments were needed for SD/LEP students in reading because only SD/LEP

students that were not offered accommodations will be used in reporting for reading. This essentially meant that

certainty schools were treated as noncertainty schools for replication of SD/LEP students in reading.

In general, public schools (except schools in Virgin Islands and DoDEA/DDESS grade 8) were

assigned to replicates as follows: Noncertainty schools were first paired and then each pair was assigned to its

own replicate group. Large certainty schools were assigned to two replicate groups each, and small certainty

schools were assigned to one replicate group each.

For nonpublic schools, the assignment of replicate groups was as follows: If the sample of noncertainty

schools was small, each noncertainty school was randomly assigned to its own replicate group. If the sample of

noncertainty schools was large enough, this procedure was implemented separately for Catholic and nonCatholic noncertainty schools. Then, large certainty schools were assigned to two replicate groups each, and

small certainty schools were assigned to one replicate group each.

Replicate group assignments for schools in Virgin Islands and DoDEA/DDESS grade 8 were handled

differently because of small sample sizes. Nonpublic schools in Virgin Islands were assigned to replicate

groups using the procedure described in the preceding paragraph for nonpublic schools. For public schools in

Virgin Islands and DoDEA/DDESS grade 8, schools were assigned to a number of replicate groups proportional

to the estimated grade-specific enrollment.

The details about the replicate group assignments for all schools are given below.
218

 11.6.1.1 Replicate Group Assignments for Non-SD/LEP Students

All Public Schools, Except Schools in Virgin Islands and DoDEA/DDESS Grade 8. Noncertainty

schools were sorted by jurisdiction according to sample type. Then within sample type, the schools were sorted

by new school status and the order in which they were selected from the sampling frame. The schools were then

grouped in pairs. Where there was an odd number of schools, the last replicate group contained three schools

instead of two. If a jurisdiction had more than 62 pairs, the pair numbering would have gone up to 62 and then

from 62 backwards as needed; however, this did not happen in 1998.

Each of the certainty public schools was assigned to one replicate group or to more replicate groups if

its size was large. If a school was selected three or more times in the sampling process, then it was assigned to

two replicate groups. Here, schools were sorted by the estimated grade enrollment prior to group assignments.

Again, depending on the jurisdiction, a maximum of 62 certainty groups was formed. The group numbering

resumed from the last group number used for the noncertainty schools if the total number of public-school

groups was less than 62. Otherwise, the numbering started from 62 down to the number needed for the last

certainty public school. In jurisdictions where all schools were certainty schools and the total number of public

schools (that is, certainty schools) exceeds 62, the numbering of the groups started at 62 and went downward to

1, and then from 1 up to the number needed for the last certainty school. For instance, in the District of

Columbia grade 4 reading, which had only 114 certainty schools (no noncertainty schools), group numbers

started at 62 and continued down to 1 and then from 1 up to 52. In the District of Columbia grade 8 reading,

which had only 37 certainty schools, the group numbers went from 1 to 55. Eighteen of the 37 certainty schools

in the District of Columbia were selected three or more times and thus were assigned to two replicate groups. A

replicate was formed by randomly deleting one half of the students in a certainty school from the sample. For

certainty schools that were assigned to two replicate groups, the students were split equally between four

“halves,” two halves in each of the two replicate groups. This process was repeated for each certainty school.

The purpose of this scheme was to assign as many replicates to a jurisdiction’s public schools as

permitted by the design, to a maximum of 62. When more than 62 replicates were assigned, the procedure

ensured that no subset of the replicate groups (pairs of noncertainty schools, individual certainty schools, or

groups of these) was substantially larger than the other replicate groups. The aim was to maximize the degrees

of freedom available for estimating variances for public-school data.

A single replicate estimate was formed by dropping one member assigned to a particular replicate

group. This process was repeated successively across replicate groups, giving up to 62 replicate estimates.

Nonpublic Schools. Replicate groups for noncertainty nonpublic schools were formed in one of the

two methods described below. It depends on the number of nonpublic noncertainty schools, such as the number

of available noncertainty Catholic or non-Catholic schools. If any of the following conditions was true for a

given jurisdiction, then the subsequent steps were taken to form replicate groups. Here, the numbering started at

62 down to the last needed number.

Conditions for Method 1:

• fewer than 11 nonpublic noncertainty schools; or

• fewer than 2 Catholic noncertainty schools; or

• fewer than 2 non-Catholic noncertainty schools.
219

Steps for Method 1:

• all schools were grouped into a single replicate group;

• schools were randomly sorted; and

• starting with the second school, replicates were formed by consecutively leaving out one

of the remaining n - 1 schools; each replicate included the first school.

When a given jurisdiction did not match conditions of the first method (i.e., when all of the following

conditions were true), then the preceding steps were repeated separately for two groups, one consisting of

Catholic schools and one consisting of non-Catholic schools.

Conditions for Method 2:

• more than 10 nonpublic noncertainty schools; and

• more than 1 Catholic noncertainty school; and

• more than 1 non-Catholic noncertainty school.

For jurisdictions with certainty nonpublic schools (Hawaii and Virgin Islands for reading at grade 4;

Rhode Island, Virgin Islands, and Wyoming for both reading and writing at grade 8) each school was assigned

to one or more groups. If a school was selected three or more times in the sampling, it was assigned to two

groups. Prior to this assignment, schools were sorted in descending order of the estimated grade enrollment.

The group numbering started at the last number where the noncertainty nonpublic schools ended. A replicate

was formed by randomly deleting one half of the students in a certain school from the sample. For the certainty

schools that were assigned to two replicate groups, the students were split equally between four “halves,” two

halves in each of two replicate groups. This was repeated for each certainty school.

Again, the aim was to maximize the number of degrees of freedom for estimating sampling errors for

nonpublic schools (and indeed for public and nonpublic schools combined) within the constraint of forming 62

replicate groups. Where a jurisdiction had a significant contribution from both Catholic and non-Catholic

schools, Westat ensured that the sampling error estimates reflected the stratification on this characteristic.

Virgin Islands. For Virgin Islands, where all schools were selected with certainty, nonpublic schools

were assigned in the usual way, and public schools were assigned to a number of replicate groups proportional

to their estimated grade enrollment.

DoDEA/DDESS Grade 8. Schools in the DoDEA/DoDDS grade 8 sample were assigned to a number

of replicate groups proportional to their estimated grade enrollment. Schools in all other Department of Defense

Domestic Dependent Elementary and Secondary Schools (DoDEA/DDESS) and DoDEA/DoDDS samples were

assigned to replicate groups following the general rules described above for all public schools. In grade 8

writing, the one noncertainty school was treated like a certainty school.

11.6.1.2 Replicate Group Assignments for SD/LEP Students in Reading

For reading certainty schools with non-SD/LEP students were reassigned to replicate groups. The

replicate group assignments for all other schools remained the same. As mentioned before, there were no

certainty schools for SD/LEP replication for reading (certainty schools were treated as noncertainty schools).

The reassignment of replicate groups for certainty schools was implemented as follows.
220

All Public Schools, Except those in Virgin Islands and DoDEA/DDESS Grade 8. The assignment of

schools to replicate groups was done separately for various subgroups of the reading SD/LEP sample. For

public noncertainty schools, the schools were first sorted by jurisdiction according to sample type. Within each

sample type, the schools were sorted by their new school status and sample selection order. In those

jurisdictions where the number of replicate groups for public schools did not exceed 62, the schools in the

sorted list were assigned group numbers, two to a group, beginning where the previous assignments for the

public non-certainty schools with non-SD/LEP students stopped. If the number of schools was odd, then the last

three schools were assigned to the same replicate group. If the number of public noncertainty schools exceeded

62, then the group numbering started at 62 and proceeded backwards, assigning pairs of schools to the same

replicate group. If the number of public noncertainty schools to be assigned was odd, the last three schools were

assigned to the same replication group. For Arkansas, Illinois, and Mississippi grade 4; and Florida, North

Carolina, and Tennessee grade 8, there was only one public noncertainty school with SD/LEP students assessed

in reading. This school was assigned to the last replicate group used for the public noncertainty schools with

non-SD/LEP students. If there was an odd number of such schools, then the triple was broken up into two

doubles and the school in question was assigned to the last double.

Nonpublic Schools. Nonpublic schools were assigned to replicate groups as follows. For noncertainty

schools, the replicate group assignments were the same for Catholic and non-Catholic schools, and used one of

the two methods described below.

Method 1. If the conditions for Method 1 for non-SD/LEP replication were met, then the first school in

the sorted list was not assigned to any group. The second and subsequent schools were assigned to one replicate

group each, beginning where the numbering for nonpublic noncertainty schools in the non-SD/LEP replication

stopped. The numbering then proceeded backwards.

Method 2. If the conditions for Method 2 for non-SD/LEP replication were met, then the procedure for

Method 1 was implemented for Catholic and non-Catholic schools separately. Catholic schools were assigned

first, starting from where the numbering for nonpublic noncertainty non-Catholic schools in the non-SD/LEP

replication stopped. The numbering for the non-Catholic schools started from where that for the Catholic

schools stopped.

Virgin Islands. In Virgin Islands, nonpublic schools were assigned to replicate groups in the usual

way, and the public schools were assigned in the same way as nonpublic schools.

DoDEA/DDESS Grade 8. In the DoDEA/DDESS grade 8, schools were assigned to replicate groups in

exactly the same way as for nonpublic schools.

 11.6.2 School-Level Replicate Weights

As mentioned above, each replicate sample had to be reweighted to compensate for the dropped unit(s)

defining the replicate. This reweighting was done in two stages. At the first stage, the i

th school included in a

particular replicate r was assigned a replicate-specific school base weight defined as:

sch sch W KW ri ri = × ,

where Wi

sch is the full-sample base weight for school i, and, for public schools,
221

1.5 if school was contained in a "pair" consisting of 3 units

 from which the complimentary member was dropped to form replicate ,

2 if school was contained in a pair consisting of 2 uni r

i

r

i

K = ts 

 from which the complimentary member was dropped to form replicate ,

0 i f s c h o o l w a s d r o p p e d t o form replicate , and 

1 if school was not assi

r

i r

i gned to replicate , or if school was a r i certainty. 



















For nonpublic schools, Method 1:

 if school was not dropped in forming replicate , and 1

0 if school was dropped to form replicate . 

r

n i r

K n

i r



 =  −





For nonpublic schools, Method 2 (with n1 Catholic schools and n2 non-Catholic schools):

1

1

 if school was Catholic not dropped from replicate , 

1

 and replicate was formed by dropping a Catholic school;

 

 1 if school was 

r

n i r

n

r

i

K

−

=

2

2

Catholic and replicate was formed by dropping a non-Catholic school;

 

 if school was non-Catholic not dropped from replicate ,

1

and replicate was formed by dropping a 

r

n i r

n

r − non-Catholic school;

 1 if school was dropped to form replicate .

 

i r









































Using the replicate-specific school base weights, sch Wri , the school-level nonresponse weighting

adjustments were recalculated for each replicate r. That is, the school-level nonresponse adjustment factor for

schools in replicate r and adjustment class k was computed as:

( )

( )

k

k

sch

rki ki

i C

rk sch

rki ki rki

i C

W E

F

W E δ

∈

∈

×

= × ×

∑

∑

where

Ck = the subset of school records in adjustment class k,

sch W rki = the replicate-r base weight of the i

th school in class k, and
222

Eki = the grade enrollment for the i

th school in class k.

In the above formulation, the indicator variable δ rki had a nonzero value only when the i

th school in

replicate r and adjustment class k participated in the assessment. The replicate-specific nonresponse-adjusted

school weight for the i

th school in replicate r in class k was then computed as:

adj sch

rki rki rk rki W W = F × ×δ .

11.6.3 Student-Level Replicate Weights

The replicate-specific adjusted student base weights were calculated by multiplying the replicatespecific adjusted school weights as described above by the corresponding within-school student weights. That

is, the adjusted student base weight for the j

th student in adjustment class k in replicate r was initially computed

as:

adj within WW W rkij rki = × ij

where

adj W rki = the nonresponse-adjusted school weight for school i in

school adjustment class k and replicate r, and

within W ij = the within-school weight for the j

th student in school i.

The final replicate-specific student weights were then obtained by applying the student nonresponse

adjustment procedures to each set of replicate student weights. Let Frk denote the student-level nonresponse

adjustment factor for replicate r and adjustment class k. The final replicate r student weight for student j in

school i in adjustment class k was calculated as:

final adj within W WW rkij rk rki ij =× × F

Finally, estimates of the variance of sample-based estimates were calculated as:

62 2

1

( ˆ) ( ˆ ˆ) JK r rVar x x x == − ∑

where

,

ˆ final

r rkij rkij i j

x = × ∑W x
223

denotes an estimated total based on replicate r (one of 62 replicates), and xˆ denote the corresponding estimate

based on the full sample. The standard error of an estimate xˆ is estimated by taking the square root of the

estimated variance, VarJK( xˆ ).

 11.7 RAKING OF WEIGHTS

Raking (also known as iterative proportional fitting) is done in place of poststratification. Unlike

poststratification, it is performed iteratively to two or more different distributions of a population total (i.e.,

gender and age). It is typically used in situations in which the interior cells of a cross-tabulation are either

unknown, or some sample sizes in the cells are too small for efficient estimation. In raking, the marginal

population totals, Ni. and N.j are known (i.e., age and gender population counts); however, the interior cells of

the cross-tabulation Nij (the age by gender cells) are estimated from the sample by ˆNij , where these are the sum

of weights in the cells.

The raking algorithm proceeds by proportionally scaling the ˆNij , such that the following relations are

satisfied:

. ˆij i

j

∑N = N

and

. . ˆij j

i

∑N = N

The 1998 state NAEP assessment program used two different sets of administration rules indicated by

sample type 2 and sample type 3 (see Chapter 4). To enable ETS to analyze the reading assessment omitting the

SD/LEP students with sample type 3, the SD/LEP student weights were raked separately for the two subsets as

defined by sample type. Note that only the weights of SD/LEP students in public schools were raked.

Agreement was forced with totals estimated using both of the subsets combined for each of the sample types.

The purpose of this was to enhance the reliability (i.e., reduce the sampling error) of estimates produced by

using information about student characteristics from the whole sample to enhance the estimates. Because of

small sample sizes, the weights of nonpublic SD/LEP students were not raked but were assigned a crude raking

factor of 2. Non-SD/LEP students were assigned dummy raking factors of 1.

11.7.1 Raking Dimensions for Full Sample Student Weights

Public Schools. Five variables were used for the raking dimensions. These variables included two

levels of SD (SD/non-SD), two levels of LEP (LEP/non-LEP), two levels of gender, five levels of race (White

and other; Black; Hispanic; Asian or Pacific Islander; and American Indian or Alaskan Native), and two levels

of age. The age variable was defined as follows: for grade 4, those born in August 1987 or earlier and those

born in September 1987 or later; and for grade 8, those born in August 1983 or earlier and those born in

September 1983 or later. Collapsing of levels was done so that no level of a single dimension contained fewer

than 30 students for a state and grade.
224

Control totals were obtained by summing the trimmed nonresponse-adjusted student weights for each

level of the collapsed raking dimension. The final collapsed levels that were used for the raking dimensions, for

each jurisdiction and grade, can be found in Tables B-13 and B-14 in Appendix B. An “X” indicates that the

variable was not collapsed for raking. A dash indicates that all levels were combined, and thus, the variable was

not used as a raking dimension. An asterisk for the race variable indicates that all other levels of the dimension

were combined into one level. For example in fourth grade for Florida, there are three levels of race: White,

Hispanic, and all others combined.

Nonpublic Schools. Because of the small numbers of nonpublic-school students, no raking was carried

out. A factor of 2 was applied to the weights for the SD/LEP students, since only half the SD/LEP sample was

used for analysis.

11.7.2 Raking Student Replicate Weights

The replicate weights for the public SD/LEP students were raked similarly. Control totals for each

replicate were calculated based on the totals for the replicate weights. The levels of the raking dimensions that

were used for the replicates were the same collapsed levels as used for the full sample student weights. For the

nonpublic schools, again a factor of 2 was applied to the replicate weights of the SD/LEP students.

11.8 APPROXIMATING THE SAMPLING VARIANCE USING DESIGN EFFECTS

As in Chapter 10’s discussion of variance estimation (see Section 10.5), design effects (Kish & Frankel,

1974) of mean proficiencies across the state samples were calculated for demographic subgroups for reading

grades 4 and 8, and writing grade 8, respectively. The design effect for a statistic is the ratio of the actual

variance of the statistic (taking the sample design into account) over the conventional variance estimate based

on a simple random sample with the same number of elements. The design effect is the inflation factor to be

applied to the conventional variance estimate in order to adjust error estimates based on simple random

sampling assumptions, thus accounting approximately for the effect of the sample design. Design effects

provide an approximate approach to compute variance from NAEP data for secondary analysis. Moreover, they

provide a measure to analyze the efficiency of a study design.

Since most of the analyses conducted by NAEP are based on the results of scaling models that

summarize performance of students across a learning area, the design effects are based on these scale scores. A

key statistic of interest is the estimated mean scale score of a subgroup of the population. Table 11-11 gives the

average design effects for state-level mean scale score, averaged across all jurisdictions by grade for the 1998

state reading and writing assessments.

The table shows that the design effects are predominantly larger than 1, indicating that standard

variance estimation formulas will be generally too small, usually markedly so. Although the design effects

appear somewhat different for certain subgroups of the population, they are similar enough (at least within a

subject and grade) to select an overall composite value that is adequate for most purposes. In choosing a

composite design effect, some consideration must be made about the relative consequences of overestimating

the variance as opposed to underestimating the variance. (For details, see descriptions in Section 10.5.2.) Table

11-12 gives the composite values of mean, median, and upper quartile of the distribution of design effects for

mean state scale scores by grade for the 1998 state reading and writing assessments.
225

Table 11-11

Average Design Effects by Demographic Subgroup

for 1998 Mean State Reading and Writing Scale Scores

Averaged Across State Samples*

Subgroup

Grade 4

Reading

Grade 8

Reading

Grade 8

Writing

Total 3.81 3.25 3.21

Male 2.54 2.45 2.29

Female 2.49 2.13 2.28

White 2.74 2.44 2.61

Black 1.87 2.17 2.03

Hispanic 2.06 1.70 1.44

Asian/Pacific Islander 1.48 1.42 1.21

Other race/ethnicity 1.47 1.81 1.34

Urban 5.00 4.44 4.37

Suburban 4.07 3.63 3.02

Rural 3.37 3.12 2.75

PARED < HS 1.28 1.52 1.13

PARED = HS 1.39 1.76 1.28

PARED > HS 1.59 1.49 1.59

PARED = College 2.91 2.18 2.40

PARED = Unknown 1.68 1.43 1.11

Public school 3.84 3.13 2.95

* Design effects are based on the conventional and jackknife variances of subgroup

means of the first plausible values of scale score.

Table 11-12

Mean, Median, and Upper Quartile of the 1998 Across-State Average

Design Effects for Mean State Scale Score

(Distribution Across Demographic Subgroups) *

Subgroup

Grade 4

Reading

Grade 8

Reading

Grade 8

Writing

Upper Quartile 3.37 3.12 2.75

Mean 2.56 2.36 2.18

Median 2.49 2.17 2.28

*

 Design effects are based on the conventional and jackknife variances of

subgroup means of the first plausible values of scale score.

227

Chapter 12

SCALING PROCEDURES1

Nancy L. Allen, James E. Carlson, Eugene G. Johnson, and Robert J. Mislevy

Educational Testing Service

12.1 INTRODUCTION

The primary method by which results from the 1998 National Assessment of Educational

Progress (NAEP) were disseminated is scale score reporting. The National Assessment Governing Board

(NAGB) provides achievement levels that are used to give judgmental meaning to the scale. With scaling

methods, the performance of a sample of students in a subject area or subarea can be summarized on a

single scale or series of scales even when different students have been administered different items. This

chapter presents an overview of the scaling methodologies employed in the analyses of the data from

NAEP surveys in general. Details of the scaling procedures specific to the subject areas of reading,

writing, and civics are presented in Chapters 14 through 24.

12.2 BACKGROUND

The basic information from an assessment consists of the responses of students to the items

presented in the assessment. For NAEP, these items are constructed to measure performance on sets of

objectives developed by nationally representative panels of learning-area specialists, educators, and

concerned citizens. Satisfying the objectives of the assessment and ensuring that the tasks selected to

measure each goal cover a range of difficulty levels typically require many items. Depending on the

subject areas, a mixture of multiple-choice, short constructed-response, and extended constructedresponse items were used. To reduce student burden, each assessed student was presented only a fraction

of the full pool of items through multiple matrix sampling procedures.

The most direct manner of presenting the assessment results is to report separate statistics for

each item. However, because of the vast amount of information, having separate results for each of the

items in the assessment pool hinders the comparison of the general performance of subgroups of the

population. Item-by-item reporting masks similarities in trends and subgroup comparisons that are

common across items.

An obvious summary of performance across a collection of items is the average of the separate

item scores. The advantage of averaging is that it tends to cancel out the effects of peculiarities in items

that can affect item difficulty in unpredictable ways. Furthermore, averaging makes it possible to

compare more easily the general performances of subpopulations.

Despite their advantages, there are a number of significant problems with mean item scores.

First, the interpretation of these results depends on the selection of the items; the selection of easy or

difficult items could make student performance appear to be overly high or low. Second, the average

 

1 Nancy L. Allen and James E. Carlson shared responsibility for the psychometric and statistical analysis of the 1998 national and

state NAEP data with John R. Donoghue. Eugene G. Johnson contributed to the design of NAEP and to discussions of sampling

issues. Previously he was responsible for the psychometric and statistical analysis of NAEP data. Robert J. Mislevy is a technical

consultant contributing in the area of item response theory.
228

score is related to the particular items comprising the average, so that direct comparisons in performance

between subpopulations require that those subpopulations have been administered the same set of items.

Third, because this approach limits comparisons to average scores on specific sets of items, it provides

no simple way to report trends over time when the item pool changes. Finally, direct estimates of

parameters or quantities such as the proportion of students who would achieve a certain score across the

items in the pool are not possible when every student is administered only a fraction of the item pool.

While the average score across all items in the pool can be readily obtained (as the average of the

individual item scores), statistics that provide distributional information, such as quantiles of the

distribution of scores across the full set of items, cannot be readily obtained without additional

assumptions.

These limitations can be overcome by the use of response scaling methods. If several items

require similar skills, the regularities observed in response patterns can often be exploited to characterize

both respondents and items in terms of a relatively small number of variables. These variables include a

respondent-specific variable, called scale score, which quantifies a respondent’s tendency to answer

items correctly (or, for multipoint items, to achieve a certain item score) and item-specific variables that

indicate characteristics of the item such as its difficulty, effectiveness in distinguishing between

individuals with different levels of scale score, and the chances of a very low scale score respondent

correctly answering a multiple-choice item. (These variables are discussed in more detail in the next

section.) When combined through appropriate mathematical formulas, these variables capture the

dominant features of the data. Furthermore, all students can be placed on a common scale, even though

none of the respondents takes all of the items within the pool. Using the common scale, it becomes

possible to discuss distributions of scale score in a population or subpopulation and to estimate the

relationships between scale score and background variables.

It is important to point out that any procedure of aggregation, from a simple average to a complex

multidimensional scaling model, highlights certain patterns at the expense of other potentially interesting

patterns that may reside within the data. Every item in a NAEP survey is of interest and can provide

useful information about what United States students know and can do. The choice of an aggregation

procedure must be driven by a conception of just which patterns are salient for a particular purpose.

The scaling for the national main reading, mathematics, science, U.S. history, geography, and

music assessments is carried out separately within purposes of reading, mathematics content strands,

fields of science, themes, or content areas as specified in the framework. Originally, this scaling within

subareas was done because it was anticipated that different patterns of performance or different trends

over time might exist for these essential subdivisions of the subject areas. By creating a separate scale for

each of these content areas, potential differences in subpopulation performance between the content areas

are preserved.

The creation of a series of separate scales to describe performance within a subject area does not

preclude the reporting of a single index of overall performance in the subject area—that is, an overall

subject–area composite. A composite is computed as the weighted average of the content–area scales,

where the weights correspond to the relative importance given to each content area as defined by the

framework. The composite provides a global measure of performance within the subject area, while the

constituent content area scales allow the measurement of important interactions within educationally

relevant subdivisions of the subject area.

For all other national main assessment subjects the framework documents specify a single

(unidimensional) scale. The long-term trend scales for reading, writing, mathematics, and science are

also scaled as if they were unidimensional.
229

12.3 SCALING METHODOLOGY

This section reviews the scaling models employed in the analyses of NAEP data and the multiple

imputation or “plausible values” methodology that allows such models to be used with NAEP’s sparse

item-sampling design. The reader is referred to Mislevy (1991) for an introduction to plausible values

methods and a comparison with standard psychometric analyses to Beaton and Johnson (1992),

Donoghue (1993), and Mislevy, Johnson and Muraki (1992), and for additional information on how the

models are used in NAEP, and to Rubin (1987) for the theoretical underpinnings of the approach. It

should be noted that the imputation procedure used by NAEP is a mechanism for providing plausible

values for the unobserved proficiencies and not for filling in blank responses to background or cognitive

variables.

While the NAEP procedures were developed explicitly to handle the characteristics of NAEP

data, they build on other research, and are paralleled by other researchers. See, for example, Andersen

(1980); Dempster, Laird, and Rubin (1977); Engelen (1987); Hoijtink (1991); Laird (1978); Lindsey,

Clogg, and Grego (1991); Little and Rubin (1983, 1987); Rubin (1987, 1991); Tanner and Wong (1987);

and Zwinderman (1991).

12.3.1 The Scaling Models

Three distinct scaling models, depending on item type and scoring procedure, are used in the

analysis of NAEP data. Each of the models is based on item response theory (IRT; e.g., Lord, 1980).

Each is a “latent variable” model, defined separately for each of the scales, which expresses respondents’

tendencies to achieve certain scores (such as correct/incorrect) on the items contributing to a scale as a

function of a parameter that is not directly observed, called score ( ) on the scale.

A three-parameter logistic (3PL) model is used for the multiple-choice items (which are scored

correct or incorrect). The fundamental equation of the 3PL model defines the probability that a person

whose score on scale k is characterized by the unobservable variable k will respond correctly to item j

as:

( ) ( )

( ) 1( ) 1

1 ,,, , 1 exp 1.7

j

j k jjj j j k

j k j

c

Px abc c P

a b

θ θ

θ

− = =+ ≡ + − −   

(12.1)

where

xj is the response to item j, 1 if correct and 0 if not;

aj where aj >0, is the slope parameter of item j, characterizing its sensitivity to scale

score;

bj is the threshold parameter of item j, characterizing its difficulty; and

cj where 0 cj<1, is the lower asymptote parameter of item j, reflecting the chances of

students of very low scale score selecting the correct option.

Further define the probability of an incorrect response to the item as

P Px abc P j j k jjj j k 0 1 ≡ = =− ( ) 0 ,,, 1 . θ θ( ) (12.2)
230

A two-parameter logistic (2PL) model is used for the short constructed-response items that were

scored correct or incorrect. The form of the 2PL model is the same as Equations (12.1) and (12.2), with

the cj parameter fixed at zero.

In addition to the multiple-choice and other two-category items, a number of extended

constructed-response items are presented in NAEP assessments. The long-term trend and national main

writing assessments include only extended constructed-response items, but most other national main and

state assessments include some extended constructed-response items. Each of these items is scored on a

multipoint scale with potential scores ranging from 0 to 3, from 0 to 4, or from 0 to 5. For some subjects,

short constructed-response items are scored on a three-point scale (0–2) as well as on a two-category

scale. Items that are scored on a multipoint scale are referred to as polytomous items, in contrast with the

multiple-choice and short constructed-response items, which are scored correct or incorrect and referred

to as dichotomous items.

The polytomous items are scaled using a generalized partial credit model (Muraki, 1992). The

fundamental equation of this model is the probability that a person with score k on scale k will have, for

the j

th item, a response xj that is scored in the ith of mj ordered score categories:

( )

( )

( )

( ) 0

1 1 1

0 0

exp 1.7 ,

, , , , ,..., ,

exp 1.7 ,

j j

i

j k j jv

v j k j j j j m ji k m g

j k j jv

g v

a bd

Px i abd d P

a bd

θ

θ θ

θ

= − −

= =

    − +

  == ≡     − +

 

∑

∑ ∑

(12.3)

where

mj is the number of categories in the response to item j;

xj is the response to item j, with possibilities 0, 1, ... , mj – 1;

aj is the slope parameter;

bj is the item location parameter characterizing overall difficulty; and

dj,i is the category i threshold parameter (see below).

Indeterminacies in the parameters of the above model are resolved by setting dj,0 = 0 and setting

1

1

, 0. mj

j i

i

d

−

=

∑ = Muraki (1992) points out that bj - dj,i is the point on the k scale at which the plots of

Pji k , 1− ( ) θ and Pji k ( ) θ intersect and so characterizes the point on the k scale at which the response to

item j has equal probability of falling in response category i-1 and falling in response category i.

When mj = 2, so that there are two score categories (0,1), it can be shown that Pji( k) of Equation

(12.3) for i = 0,1 corresponds respectively to Pj k 0 ( ) θ and Pj k 1 ( ) θ of the 2PL model [(Equations (12.1)

and (12.2) with cj = 0)].

Close examination of the 3PL and generalized partial credit models indicate that both models

have a linear indeterminacy of the theta scale. In other words, if the item parameters are estimated in a
231

different metric, the value of k could be transformed to make Equations (12.1) and (12.3) true. For the

purposes of reporting item parameter estimates and other intermediary estimates, the linear

indeterminacies apparent in Equations (12.1) and (12.3) may be resolved by an arbitrary choice of the

origin and unit size in a given scale. In most cases, a provisional scale standardizing the theta distribution

to have mean 0 and standard deviation 1 is employed. Final results for each content area are linearly

transformed from the scale to a 0-to-500 or a 0-to-300 scale, as described in the subject area chapters in

this report.

A basic assumption of item response theory is the conditional independence of the responses by

an individual to a set of items, given the individual’s scale score. That is, conditional on the

individual’s k, the joint probability of a particular response pattern x = (x1,...,xn) across a set of n items

is simply the product of terms based on Equations (12.1), (12.2), and (12.3):

( ) ( )

1

1 0

, j ji n m u

k ji k

j i

P x item parameters P θ θ −

= =

= ∏ ∏ (12.4)

where Pji( k) is of the form appropriate to the type of item (dichotomous or polytomous), mj is equal to 2

for the dichotomously scored items, and uji is an indicator variable defined by

1 response is in category

0 otherwise

j ji

x i

u

 = 



It is also typically assumed that response probabilities are conditionally independent of

background variables (y), given k, or

P( ) x item parameters y p x item parameters θ θ k k , ,, . = ( ) (12.5)

After x is observed, Equation (12.4) can be viewed as a likelihood function, and provides a basis

for inference about k or about item parameters. Estimates of item parameters were obtained by the

NAEP BILOG/PARSCALE program, which combines Mislevy and Bock’s (1982) BILOG and Muraki

and Bock’s (1991) PARSCALE computer programs2

, and which concurrently estimates parameters for

all items (dichotomous and polytomous). Donoghue (1993) reports on the effect of having both

dichotomous and polytomous items within a scale. The NAEP BILOG/PARSCALE program has also

been adapted to make use of student sampling weights. The item parameters are then treated as known in

subsequent calculations. In NAEP analyses, for subject areas with multiple scales (i.e., national main

reading, mathematics, science, U.S. history, geography, and music), the parameters of the items

constituting each of the separate scales are estimated independently of the parameters of the other scales.

Once items are calibrated in this manner, a likelihood function for the scale score k is induced by a

vector of responses to any subset of calibrated items, thus allowing k-based inferences from matrix

samples. The likelihood function for the scale score k is called the posterior distribution of the thetas for

each student.

In almost all NAEP IRT analyses, missing responses at the end of each block of items a student

was administered are considered “not reached,” and are treated as if they had not been presented to the

respondent. Missing responses to dichotomous items before the last observed response in a block are

considered intentional omissions, and are treated as fractionally correct at the value of the reciprocal of

 

2 See Muraki and Bock (1999) for the current version of PARSCALE.
232

the number of response alternatives, if the item was a multiple-choice item. These conventions are

discussed by Mislevy and Wu (1988). With regard to the handling of not-reached items, Mislevy and Wu

found that ignoring not-reached items introduces slight biases into item parameter estimation when notreached items are present and speed is correlated with ability. With regard to omissions, they found that

the method described above provides consistent limited-information maximum likelihood estimates of

item and ability parameters under the assumption that respondents omit only if they can do no better than

responding randomly.

Missing responses to polytomous items before the last observed response in a block are also

considered intentional omissions and scored so that the response is in the lowest category. Occasionally,

extended constructed-response items are the last item in a block of items. Because considerably more

effort is required of the student to answer these items, nonresponse to an extended constructed-response

item at the end of a block is considered an intentional omission (and scored as the lowest category)

unless the student also did not respond to the item immediately preceding that item. In that case, the

extended constructed-response item is considered not reached and treated as if it had not been presented

to the student. In the case of the main and state writing assessment, there is a single extended

constructed-response item in each separately-timed block. In the writing assessment when a student does

not respond to the item or when the student provides an off-task response, the response is also treated as

if the item had not been administered.

Scaling areas in NAEP are determined a priori by grouping items into content areas for which

overall performance is deemed to be of interest, as defined by the frameworks developed by the National

Assessment Governing Board (NAGB). A scale score k is defined a priori by the collection of items

representing that scale. What is important, therefore, is that the models capture salient information in the

response data to effectively summarize the overall performance on the content area of the populations

and subpopulations being assessed in the content areas.

The local independence assumption embodied in Equation (12.4) implies that item response

probabilities depend only on and the specified item parameters, and not on the position of the item in

the booklet, the content of items around an item of interest, or the test-administration and timing

conditions. However, these effects are certainly present in any application. The practical question is

whether inferences concerning aggregate performance in the scaling area that are based on the IRT

probabilities obtained via Equation (12.4) are robust with respect to the ideal assumptions underlying the

IRT model. Our experience with the 1986 NAEP reading anomaly (Beaton & Zwick, 1990) has shown

that for measuring small changes over time, changes in item context and speededness conditions can lead

to unacceptably large random error components. These can be avoided by presenting items used to

measure change in identical test forms, with identical timings and administration conditions. Thus, we do

not maintain that the item parameter estimates obtained in any particular booklet configuration are

appropriate for other conceivable configurations. Rather, we assume that the parameter estimates are

context-bound. This is the reason that the long-term trend booklets and administration procedures have

not changed since the early 1980s and only a limited number of blocks of items are released after each

national main assessment cycle. It was also the reason we prefer common population equating to

common item equating whenever equivalent random samples are available for linking. In common item

equating, items are assumed to be measuring exactly the same thing for two or more populations, despite

any differences in context or administration. In common population equating, results for two or more

samples from the same population are matched to one another when linking the scales. Therefore, the

data from the state assessment are calibrated separately from the national NAEP data. In this case, the

administration procedures differ somewhat between the state assessment and the national NAEP.

Although the IRT models are employed in NAEP only to summarize performance, a number of

checks are made to detect serious violations of the assumptions underlying the models. Checks are made
233

to detect multidimensionality of the construct being measured and certain condition dependencies. DIF

analyses are used to examine issues of dimensionality, and what are called 2 χ statistics in the IRT

literature are used to flag responses with serious departures from the IRT model. DIF analysis

methodologies are discussed in Chapter 9. The latter statistics might better be called item fit statistics

since they do not really have 2 χ distributions. These checks include comparisons of empirical and

theoretical item response functions to identify items for which the IRT model may provide a poor fit to

the data. When warranted, remedial efforts, such as collapsing categories of polytomous items or

combining items into a single item, are made to mitigate the effects of such violations on inferences.

In practice, PARSCALE item fit statistics are used as a way to identify items that need further

examination. Most of the statistics of this type that are available for use in this setting have distributions

that are unknown. Therefore, they cannot be used for final decisions about the fit of the items to the IRT

model. Because of the lack of statistical tests for IRT model fit, the fit of the IRT models to the observed

data was examined within each scale by comparing the empirical item response functions (IRFs) with the

theoretical curves. The primary means of accomplishing this is to generate plots of empirical versus

theoretical item response curves. The theoretical curves are plots of the response functions based on the

estimates of the item parameters. The empirical proportions are calculated from the posterior

distributions of the thetas for each student who received the item. For dichotomous items, the sum of the

values of the posterior distributions at a point on the theta scale for each student who answered an item

correctly plus the sum of a fractional portion of the values of the posterior distribution at that point on the

theta scale for each student who omitted the item is parallel in meaning to the number of students who

actually answered the item correctly plus a fraction of the number of students who omitted the item. The

sum of the values of the posterior distributions for all students receiving the item at each point on the

theta scale is parallel in meaning to the empirical number of students at that point on the theta scale who

received the item. The plotted values are sums of these individual posteriors at each point on the theta

scale for those who got the item correct plus a fraction of the omitters divided by the sum of the

posteriors of those administered the item, in the case of dichotomous items, and for those who scored in

the category of interest over the sum for those who received the item, in the case of polytomous items.

As an example, Figure 12-1 contains a plot of the empirical and theoretical IRFs for a

dichotomous item from the 1994 NAEP national main reading assessment. In the plot, the horizontal axis

represents the theta (score) scale, the vertical axis represents the probability of a correct response. The

solid curve is the theoretical IRF based on the item parameter estimates and Equation (12.1). The centers

of the diamonds represent the empirical proportions correct as described above. The size of the diamonds

are proportional to the sum of the posteriors at each point on the theta scale for all of those who received

the item; this is related to the number of students contributing to the estimation of that empirical

proportion correct.

Figure 12-2 contains a plot of the empirical and theoretical IRFs for a polytomous item from the

1997 Arts (Theatre) National Assessment. As for the dichotomous item plot in Figure 12-1, the

horizontal axis represents the score scale, but the vertical axis represents the probability of having a

response fall in each category. The solid curves are the theoretical IRFs based on the item parameter

estimates and Equation (12.3). The centers of the diamonds represent the empirical proportions of

students with responses in each category and are proportional to the sum of the posteriors at each point

on the theta scale for the students who received the item.
234

Figure 12-1

Dichotomous Item (R016102) Exhibiting Good Model Fit*

* Diamonds represent 1994 age 13/grade 8 reading assessment data. They indicate estimated

conditional probabilities obtained without assuming a specific model form; the curve indicates the

estimated item response function (IRF) assuming a logistic form.

Figure 12-2

Polytomous Item (HC00004) Exhibiting Good Model Fit*

* Diamonds represent 1997 grade 8 arts assessment data. They indicate estimated conditional

probabilities obtained without assuming a specific model form; the curve indicates the estimated

item category response function (ICRF) using a generalized partial credit model.

PROBABILITY
235

For good fitting items, the empirical and theoretical curves are close together. Therefore, items

for which this is not true are examined carefully. Examples of plots for specific items are provided in the

subject-area chapters. When the same items are presented in two assessment years, the empirical curves

for the two years can be compared. Normally, these curves differ somewhat due to the sampling of

students for each of the two years. Figure 12-3 contains a plot for an item from the NAEP 1996

mathematics national assessment with curves of this type. When the empirical curves differ dramatically,

one cause might be a change in the meaning of the item due to instructional or societal changes across the

years. This type of item is ordinarily treated as two different items—one for each of the assessment years.

Figure 12-4 contains the plot for an item that has been treated in this way.

Figure 12-3

Dichotomous Item (M017901) Exhibiting Good Model Fit Across Assessment Years*

* Circles represent 1996 grade 12 mathematics assessment data; diamonds represent 1992

grade 12 mathematics assessment data. They indicate estimated conditional probabilities

obtained without assuming a specific model form; the curve indicates the estimated item

response function (IRF) assuming a logistic form.
236

Figure 12-4

Dichotomous Item (M018901) Exhibiting Different Empirical Item Functions

for Different Assessment Years*

* Circles represent 1996 grade 8 mathematics assessment data; diamonds represent 1992 grade 8

mathematics assessment data. They indicate estimated conditional probabilities obtained without

assuming a specific model form; the curve indicates the estimated item response function (IRF)

using a generalized partial credit model..

To summarize, using current methodologies in psychometrics, the assumption of conditional

independence and the assumption that the data fit the models in Equations 12.1 and 12.3 are examined

and controlled in NAEP in several ways. They are examined by considering tests of DIF, item fit

statistics, and plots of empirical and theoretical IRFs. They are controlled by treating missing and “not

reached” responses in reasonable ways, maintaining the context and administration of items across

assessments, collapsing categories of polytomous items when appropriate, combining items into a single

item, or making decisions about the inclusion or exclusion of an item in a scale based on data. The

identification and amelioration of violations of IRT assumptions is an area of ongoing research in

educational measurement. For example, recent studies have investigated local item dependence (Yen,
237

1993; Habing & Donoghue, in press), assessing the fit of the item response function (Orlando & Thissen,

2000; Donoghue & Hombo, 1999, Hombo & Donoghue, 2000), item parameter drift (Donoghue & Isham,

1998) and detecting and describing multidimensionality (e.g., Roussos, Stout, & Marden; 1998; Zhang &

Stout, 1999).

12.3.2 An Overview of Plausible Values Methodology

Item response theory was developed in the context of measuring individual examinees’ abilities.

In that setting, each individual is administered enough items (often 60 or more) to permit precise

estimation of his or her , as a maximum likelihood estimate, ˆθ , for example. Because the uncertainty

associated with each is negligible, the distribution of , or the joint distribution of with other

variables, can then be approximated using an individual’s ˆθ values as if they were values.

This approach breaks down in the assessment setting when, in order to provide broader content

coverage in limited testing time, each respondent is administered relatively few items in a subject area

scale. A first problem is that the uncertainty associated with individual s is too large to ignore, and the

features of the ˆθ distribution can be seriously biased as estimates of the distribution. (The failure of

this approach was verified in early analyses of the 1984 NAEP reading survey; see Wingersky, Kaplan,

& Beaton, 1987.) A second problem, occurring even with test lengths of 60, arises when test forms vary

across and within assessments as to the numbers, formats, and content of the test items. The measurement

error distributions thus differ even if underlying distributions do not, causing ˆθ distributions to exhibit

spurious changes and resulting in deceptive comparisons in apparent population distributions—easily

greater than actual differences over time or across groups. Although this latter problem is avoided in

traditional standardized testing by presenting students with parallel test forms, controlled tightly across

time and groups, the same constraints cannot be imposed in the design and data-collection phases of the

present NAEP. Plausible values were developed as a way to estimate key population features

consistently, and approximate others no worse than standard IRT procedures would, even when item

booklet composition, format, and content balances change over time. A detailed development of plausible

values methodology is given in Mislevy (1991). Along with theoretical justifications, that paper presents

comparisons with standard procedures, discussions of biases that arise in some secondary analyses, and

numerical examples. The following provides a brief overview of the plausible values approach, focusing

on its implementation in NAEP analyses.

Let y represent the responses of all sampled examinees to background and attitude questions,

along with variables based on the sampling design such as the school where the student is enrolled, and

let represent the vector of scale score values. If were known for all sampled examinees, it would be

possible to compute a statistic t( ,y), such as a scale or composite subpopulation sample mean, a sample

percentile point, or a sample regression coefficient, to estimate a corresponding population quantity T. A

function U( ,y)for example, a jackknife estimate—would be used to gauge sampling uncertainty, as

the variance of t around T in repeated samples from the population.
238

Because the scaling models are latent variable models, however, values are not observed even

for sampled students. To overcome this problem, we follow Rubin (1987) by considering as “missing

data,” and approximate t( ,y) by its expectation given (x,y), the data that actually were observed, as

follows:

( ) ( )

( ) ( )

*, , ,

, ,.

t xy t y xy

t y p xyd

θ

θθ θ

=Ε    

= ∫ (12.6)

It is possible to approximate t

*

 using random draws from the predictive conditional distribution

of the scale proficiencies given the item responses xi , background variables yi , and model parameters for

sampled student i. These values are referred to as imputations in the sampling literature, and plausible

values in NAEP. The value of for any respondent that would enter into the computation of t is thus

replaced by a randomly selected value from the respondent’s conditional distribution. Rubin (1987)

proposes that this process be carried out several times—multiple imputations—so that the uncertainty

associated with imputation can be quantified. The average of the results of, for example, M estimates of t,

each computed from a different set of plausible values, is a Monte Carlo approximation of

Equation (12.6); the variance among them, B, reflects uncertainty due to not observing , and must be

added to the estimated expectation of U( ,y), which reflects uncertainty due to testing only a sample of

students from the population. Section 12.4 explains how plausible values are used in subsequent

analyses.

It cannot be emphasized too strongly that plausible values are not test scores for individuals in

the usual sense. Plausible values are offered only as intermediary computations for calculating integrals

of the form of Equation (12.6), in order to estimate population characteristics. When the underlying

model is correctly specified, plausible values will provide consistent estimates of population

characteristics, even though they are not generally unbiased estimates of the proficiencies of the

individuals with whom they are associated. The key idea lies in the contrast between plausible values and

the more familiar estimates of scale score (e.g., maximum likelihood estimate or Bayes estimate) that are

in some sense optimal for each examinee: Point estimates that are optimal for individual examinees have

distributions that can produce decidedly nonoptimal (specifically, inconsistent) estimates of population

characteristics (Little & Rubin, 1983). Plausible values, on the other hand, are constructed explicitly to

provide consistent estimates of population effects. For further discussion see Mislevy, Beaton, Kaplan,

and Sheehan (1992).

12.3.3 Computing Plausible Values in IRT-Based Scales

Plausible values for each respondent r are drawn from the predictive conditional distribution

p(r|xr,yr ), where and are regression model parameters defined in this subsection. This

subsection describes how, in IRT-based scales, these conditional distributions are characterized, and how

the draws are taken. An application of Bayes’ theorem with the IRT assumption of conditional

independence produces

p x y Px y p y Px p y ( ) θ ΓΣ θ ΓΣ θ ΓΣ θ θ ΓΣ r r r rr r rr rr rr , ,, , ,, ,, ,, ∝ × =× ( ) ( ) ( ) ( ) (12.7)
239

where, for vector-valued θ θ r r ,P x( r ) is the product over scales of the independent likelihoods induced

by responses to items within each scale, and ( ) , , p y θ ΓΣ r r is the multivariate—and generally

nonindependent—joint density of proficiencies for the scales, conditional on the observed value yr of

background responses and the parameters and. The provisional scales are determined by the item

parameter estimates that constrain the population mean to zero and standard deviation to one. The item

parameter estimates are fixed and regarded as population values in the computation described in this

subsection.

In the analyses of the data from the national main assessments, a normal (Gaussian) form is

assumed for ( ) , , p y θ ΓΣ r r with a common variance-covariance matrix  and with a mean given by a

linear model with slope parameters,  based on the first approximately 200 principal components of

several hundred selected main-effects and two-way interactions of the complete vector of background

variables. The included principal components are referred to as the conditioning variables, and are

denoted yc

. (The complete set of original background variables used in the analyses of each subject area

are listed in Appendix F.) The following model is fit to the data within each subject area:

θΓ ε = + ′yc (12.8)

where ε is multivariately normally distributed with mean zero and variance-covariance matrix Σ.The

number of principal components of the background variables used for each sample is sufficient to

account for 90 percent of the total variance of the full set of background variables (after standardizing

each variable). As in regression analysis, is a matrix, each of whose columns contains the effects for

one scale, and Σ is the matrix variance-covariance of residuals between scales.

A model similar to Equation (12.8) is used for the long-term trend assessments, with the

difference that yc

 consists of main effects and interactions from the smaller set of background variables

(rather than principal components of those variables) available in the long-term trend assessments.

Maximum likelihood estimates of Γ and Σ , denoted by ˆ ˆ Γ Σ and , are obtained with extensions of

Sheehan’s (1985) MGROUP computer program using the EM algorithm described in Mislevy (1985).

The EM algorithm requires the computation of the mean, θ r , and variance-covariance matrix, p

r of the

predictive conditional distribution in Equation (12.7) for respondent r when there are p scales within a

subject area. For subject areas with multiple scales, the CGROUP version of the MGROUP program was

used to compute the moments using higher order asymptotic corrections to a normal approximation

(Thomas, 1993a). For the long-term trend assessments and other assessments with a single scale, the

more precise but computationally intensive BGROUP version of MGROUP (Thomas, 1994) was used.

BGROUP uses numeric quadrature to evaluate the predictive conditional distribution moments required

by the E-step of the EM algorithm for one- and two-dimensional applications (Thomas, 1993a). For

estimation of group means on a single scale, CGROUP (Thomas, 1994) and BGROUP results will be

nearly identical to those from the original MGROUP program. CGROUP and BGROUP yield better

estimates of correlations between scales, and hence better estimates of composite scale means. BGROUP

will, theoretically, yield better estimates than CGROUP, but because of the heavy computational

demands of the methodology used, its function is limited to bivariate scales. Hence CGROUP is used for

assessments involving more than two scales.

After completion of the EM algorithm, the plausible values for all sampled respondents are

drawn in the following three-step process. First, a value of is drawn from a normal distribution with
240

mean being Γˆ and variance being the variance of ˆΓ . Second, conditional on the generated value of 

and the fixed value of Σ = ˆΣ , the predictive conditional distribution mean θ r and the predictive

conditional distribution variance r of respondent r are computed from Equation 12.7 using the EM

algorithm (see Thomas, 1993a). Finally, the θ r are drawn independently from a multivariate normal

distribution with mean θ r and variance Σr approximating the distribution in Equation (12.7). These three

steps are repeated five times producing five sets of imputation values for all sampled respondents.

12.4 INFERENCES ABOUT PROFICIENCIES

When survey variables are observed without error from every respondent, usual variance

estimators quantify the uncertainty associated with sample statistics from the only source of uncertainty,

namely the sampling of respondents. Item-level statistics for NAEP cognitive items meet this

requirement, but scale score values do not. The IRT models used in their construction posit an

unobservable scale score variable to summarize performance on the items in a scale. The fact that 

values are not observed even for the respondents in the sample requires additional statistical analyses to

draw inferences about distributions and to quantify the uncertainty associated with those inferences. As

described above, Rubin’s (1987) multiple imputations procedures were adapted to the context of latent

variable models to produce the plausible values upon which many analyses of the data from NAEP are

based. This section describes how plausible values were employed in subsequent analyses to yield

inferences about population and subpopulation distributions of proficiencies.

12.4.1 Computational Procedures

Even though one does not observe the value of respondent r, one does observe variables that

are related to it: xr, the respondent’s answers to the cognitive items he or she was administered in the area

of interest, and yr, the respondent’s answers to demographic and background variables. Suppose one

wishes to draw inferences about a number T( ,Y) that could be calculated explicitly if the and y values

of each member of the population were known. Suppose further that if values were observable, we

would be able to estimate T from a sample of N pairs of and y values by the statistic t( ,y) [where

( ,y)  ( 1,y1,..., N,yN)], and that we could estimate the variance in t around T due to sampling

respondents by the function U( ,y). Given that observations consist of (xr,yr) rather than ( r,yr), we can

approximate t by its expected value conditional on (x,y), or

( ) ( ) ( ) ( ) *t xy t y xy t y p xyd , ,, , , . = = Εθ θ θ θ     ∫ (12.9)

It is possible to approximate t

*

 with random draws from the conditional distributions p( i|xi,yi),

which are obtained for all respondents by the method described in Section 12.3.3. Let ˆθ m be the m

th such

vector of plausible values, consisting of a multidimensional value for the latent variable of each

respondent. This vector is a plausible representation of what the true vector might have been, had we

been able to observe it.
241

The following steps describe how an estimate of a scalar statistic t( ,y) and its sampling variance

can be obtained from M (>1) such sets of plausible values. (Five sets of plausible values are used in

NAEP analyses.)

1. Using each set of plausible values ˆθ m in turn, evaluate t as if the plausible values

were true values of . Denote the results t

ˆ

m , for m = 1, ... , M.

2. Using the jackknife variance estimator defined in Chapter 10, compute the estimated

sampling variance of t

ˆ

m , denoting the result Um .

3. The final estimate of t is

1

ˆ *

M m

m

t

t

= M = ∑ (12.10)

4. Compute the average sampling variance over the M sets of plausible values, to

approximate uncertainty due to sampling respondents

1

*

M m

m

U U

= M = ∑

(12.11)

5. Compute the variance among the M estimates t

ˆ

m , to approximate the betweenimputation variance

( )

( )

2

1

ˆ *

1

M m

m

t t

B

= M

− = − ∑

(12.12)

6. The final estimate of the variance of t

*

 is the sum of two components

( )1 V U M * 1 B − = ++ (12.13)

In this equation, (1+M-1

)B is the estimate of variance due to the latency of . Due to the

excessive computation that would be required, NAEP analyses do not compute and

average jackknife variances over all five sets of plausible values, but uses that computed

from the first set. Thus, in NAEP reports, U*

 is approximated by U1.

12.4.2 Statistical Tests

The variance described in Section 12.4.1 is used to make statistical tests comparing NAEP

results. This section describes the relationships between these tests and the variance components

described above. Chapter 13 contains details of the hypothesis tests used in this assessment.

If values were observed for all sampled students, the statistic (t - T)/U1/2 would follow a

t-distribution with d degrees of freedom, where d is calculated in the usual way. Then the incompletedata statistic (t*

 - T)/V1/2 is approximately t-distributed, with degrees of freedom (Johnson & Rust, 1993;

Satterthwaite, 1941) given by
242

( ) 2 2

1

1

1

v

f f

M d

=

− + −

(12.14)

where f is the proportion of total variance due to not observing values:

( )1 f 1 / M B V − = + (12.15)

When B is small relative to U*

, the reference distribution for incomplete-data statistics differs

little from the reference distribution for the corresponding complete-data statistics. This is the case with

main NAEP reporting variables. If, in addition, d is large, the normal approximation can be used to flag

“significant” results.

For k-dimensional t, such as the k coefficients in a multiple regression analysis, each Um and U*

 is

a covariance matrix, and B is an average of squares and cross-products rather than simply an average of

squares. In this case, the quantity (T – t*

) V -1 (T – t*), is approximately F distributed, with degrees of

freedom equal to k and with  defined as above but with a matrix generalization of f:

f = (1+ M-1) Trace (BV-1)/k . (12.16)

By the same reasoning as used for the normal approximation for scalar t, a chi-square distribution on

k degrees of freedom often suffices for multivariate t .

12.4.3 Biases in Secondary Analyses

Statistics t

*

 that involve proficiencies in a scaled content area and variables included in the

conditioning variables yc

 are consistent estimates of the corresponding population values T. This includes

interrelationships among scales within a content area that have been treated in the multivariate manner

described above in Section 12.3.3. Statistics involving background variables y that were not conditioned

on, or relationships among scale scores from different purposes, content strands or fields, are subject to

asymptotic biases whose magnitudes depend on the type of statistic and the strength of the relationships

of the nonconditioned background variables to the variables that were conditioned on and to the scale

score of interest. That is, the large sample expectations of certain sample statistics need not equal the true

population parameters.

The direction of the bias is typically to underestimate the effect of nonconditioned variables. For

details and derivations see Beaton and Johnson (1990), Mislevy (1991), and Mislevy and Sheehan (1987,

Section 10.3.5). For a given statistic t

*

 involving one content area and one or more nonconditioned

background variables, the magnitude of the bias is related to the extent to which observed responses x

account for the latent variable , and the degree to which the nonconditioned background variables are

explained by conditioning background variables. The first factor—conceptually related to test

reliability—acts consistently in that greater measurement precision reduces biases in all secondary

analyses. The second factor acts to reduce biases in certain analyses but increase it in others. In

particular:
243

• High shared variance between conditioned and nonconditioned background variables

mitigates biases in analyses that involve only scale score and nonconditioned

variables, such as marginal means or regressions.

• High shared variance exacerbates biases in regression coefficients of conditional

effects for nonconditioned variables, when nonconditioned and conditioned

background variables are analyzed jointly as in multiple regression.

The large number of background variables that have been included in the conditioning vectors for the

1996 assessments allows a large number of secondary analyses to be carried out with little or no bias, and

mitigates biases in analyses of the marginal distributions of in nonconditioned variables. Analysis of

the 1988 NAEP reading data (some results of which are summarized in Mislevy, 1991), which had a

similar design and fewer conditioning variables, indicates that the potential bias for nonconditioned

variables in multiple regression analyses is below 10 percent, and biases in simple regression of such

variables is below 5 percent. Additional research (summarized in Mislevy, 1990) indicates that most of

the bias reduction obtainable from conditioning on a large number of variables can be captured by

instead conditioning on the first several principal components of the matrix of all original conditioning

variables. This procedure was adopted for the 1992, 1994, and 1996 national main assessments by

replacing the conditioning effects by the first K principal components, where K was selected so that 90

percent of the total variance of the full set of conditioning variables (after standardization) was captured.

Mislevy (1990) shows that this puts an upper bound of 10 percent on the average bias for all analyses

involving the original conditioning variables.

12.4.4 A Numerical Example

To illustrate how plausible values are used in subsequent analyses, this subsection gives some of

the steps in the calculation of the 1992 grade 4 reading composite mean and its estimation-error variance.

This illustration is an example of the calculation of NAEP means and variances and can be used to

understand their calculation for any NAEP assessment.

The weighted mean of the first plausible values of the reading composite for the grade 4 students

in the sample is 217.79, and the jackknife variance of these values is 0.833. Were these values true 

values, then 217.79 would be the estimate of the mean and 0.833 would be the estimation-error variance.

The weighted mean of the second plausible values of the same students, however, is 217.62; the third,

fourth, and fifth plausible values give weighted means of 217.74, 218.24, and 218.05. Since all of these

figures are based on precisely the same sample of students, the variation among them is due to

uncertainty about the students’ s, having observed their item responses and background variables.

Consequently, our best estimate of the mean for grade 4 students is the average of the five plausible

values: 217.89. Taking the jackknife variance estimate from the first plausible value, 0.833, as our

estimate U*

 of sampling variance, and the variance among the five weighted means, .063, as our estimate

B of uncertainty due to not observing , we obtain as the final estimate V of total error variance 0.833 +

(1+5-1

) .063 = 0.909.

It is also possible to partition the estimation error variance of a statistic using these same

variance components. The proportion of error variance due to sampling students from the population is

U*

/V, and the proportion due to the latent nature of is (1+M-1)B/V. The results are shown in Table 12-1.

The value of U*

/V roughly corresponds to reliability in classical test theory and indicates the amount of

information about an average individual’s present in the observed responses of the individual. It should

be recalled again that the objective of NAEP is not to estimate and compare values of individual

examinees, the accuracy of which is gauged by reliability coefficients. The objective of NAEP, rather, is
244

to estimate population and subpopulation characteristics, and the marginal estimation methods described

above have been designed to do so consistently regardless of the values of reliability coefficients.

Table 12-1

Estimation Error Variance and Related Coefficients for the 1992 Grade 4 Reading Composite

(Based on Five Plausible Values)

Proportion of Variance Due to...

U* (1+5-1)B V

Student Sampling:

U*/V

Latency of :

(1+5-1)B/V

0.833 0.076 0.908 0.92 0.08

Chapters 16, 17, 20, 21, and 24 and Appendix H provide values of the proportion of variance due

to sampling and due to the latent nature of for all 1996 scales and composites for the populations as a

whole and, in the appendix, for selected subpopulations. It will be seen that the proportion of variance

due to the latency of varies somewhat among subject areas, tending to be largest for the long-term trend

writing assessment, where there is low correlation between tasks and each student responded to only one

or at most two tasks. The proportion of variance due to latency of is smallest for the composites of the

national main assessment subjects with several scales, where the number of items per student is largest.

Essentially, the variance due to the latent nature of is largest when there is less information about a

student’s scale score. (Note the distinction between estimation error variance of a parameter estimate and

the estimate of the variance of the distribution. The former depends on the accuracy of measurement;

the large-sample model-based expected value of the latter does not.) Given fixed assessment time, this

decrease in information will occur whenever the amount of information per unit time decreases as can

happen when many short constructed-response or multiple-choice items are replaced by a few extended

constructed-response items.

12.5 DESCRIBING STUDENT PERFORMANCE

Since its beginning, a goal of NAEP has been to inform the public about what students in United

States schools know and can do. While the NAEP scales provide information about the distributions of

scale scores for the various subpopulations, they do not directly provide information about the meaning

of various points on the scale. Traditionally, meaning has been attached to educational scales by normreferencing—that is, by comparing students at a particular scale level to other students. In contrast,

NAEP achievement levels and scale anchors describe selected points on the scale in terms of the types of

skills that are likely to be exhibited by students scoring at that level. In addition, each NAEP item is

mapped to a point on its corresponding scale, so that the content of each item provides information about

what students at each score level can do in a probabilistic sense. The achievement level process has been

applied to the reading, mathematics, science, U.S. history, and geography composites and to the writing

and civics unidimensional scales. The achievement levels were set for reading in 1992, mathematics in

1990, science in 1996, U.S. history and geography in 1994, and writing and civics in 1998.

12.5.1 Achievement Levels

NAGB has determined that achievement levels shall be the first and primary way of reporting

NAEP results. Setting achievement levels is a method for setting standards on the NAEP assessment that

identifies what students should know and be able to do at various points on the composite. For each grade

of each subject, three levels were defined—basic, proficient, and advanced. Based on initial policy
245

definitions of these levels, panelists were asked to determine operational descriptions of the levels

appropriate with the content and skills assessed in the assessment. With these descriptions in mind, the

panelists were then asked to rate the assessment items in terms of the expected performance of

marginally acceptable examinees at each of these three levels. These ratings were then mapped onto the

NAEP scale to obtain the achievement level cutpoints for reporting. Further details of the achievement

level setting process for subject areas appear in Appendix I for reading and Appendix J for writing and

civics.

12.5.2 Item Mapping Procedures

In order to map items (questions) to particular points on each subject area scale, a response

probability convention had to be adopted that would divide those who had a higher probability of success

from those who had a lower probability. Establishing a response probability convention has an impact on

the mapping of assessment items onto the scales. A lower boundary convention maps the items at lower

points along the scales, and a higher boundary convention maps the same items at higher points along the

scales. The underlying distribution of skills in the population does not change, but the choice of a

response probability convention does have an impact on the proportion of the student population that is

reported as “able to do” the items on the scales.

There is no obvious choice of a point along the probability scale that is clearly superior to any

other point. If the convention were set with a boundary at 50 percent, those above the boundary would be

more likely to get an item right than get it wrong, while those below that boundary would be more likely

to get the item wrong than right. While this convention has some intuitive appeal, it was rejected on the

grounds that having a 50/50 chance of getting the item right shows an insufficient degree of mastery. If

the convention were set with a boundary at 80 percent, students above the criterion would have a high

probability of success with an item. However, many of the students below this criterion show some level

of achievement that would be ignored by such a stringent criterion. In particular, those in the range

between 50 and 80 percent correct would be more likely to get the item right than wrong, yet would not

be in the group described as “able to do” the item.

In a compromise between the 50 percent and the 80 percent conventions, NAEP has adopted two

related response probability conventions: 74 percent for multiple-choice items (to correct for the

possibility of answering correctly by guessing), and 65 percent for constructed-response items (where

guessing is not a factor). These probability conventions were established, in part, based on an intuitive

judgment that they would provide the best picture of students’ knowledge and skills.

Some additional support for the dual conventions adopted by NAEP was provided by Huynh

(1994, 1998). He examined the IRT information provided by items, according to the IRT model used in

scaling NAEP items. Following Bock (1972), Huynh decomposed the item information into that provided

by a correct response [Pji ( ) Ij ( )] and that provided by an incorrect response [(1-P ( )) I ( )]. Huynh

showed that the item information provided by a correct response to a constructed-response item is

maximized at the point along the scale at which two-thirds of the students get the item correct (for

multiple-choice items with four options, information is maximized at the point at which 75 percent get

the item correct). Maximizing the item information, I ( ), rather than the information provided by a

correct response [P ( ) I ( )], would imply an item-mapping criterion closer to 50 percent. Maximizing

just the item information, I ( ), takes into account both responses that are correct and those that are

incorrect, however.
246

For dichotomously scored items the information function as defined by Birnbaum (1968, p. 463)

is defined for the j

th item as

( ) ( ) () ()

( )( )

2 2

0 1

2

1

1.7

1

jj j j k k

j

j j k

a PP c

I

P c

θ θ

θ

θ

 −   = − , (12.17)

where the notation is the same as that used in Equations (12.1) and (12.2). The item information function

was defined by Samejima (1969) in general for polytomously scored items, and has been derived for

items scaled by the generalized partial credit model (Donoghue, 1993; Muraki, 1993) as (in a slightly

different, but equivalent form)

() ( ) ( ) ( )

2 1 1 2 2

0 0

1.7 . m m j j

j j ji ji k k

i i

I a i P iP θ θθ − −

= =

    = −         ∑ ∑ (12.18)

12.6 OVERVIEW OF THE 1998 NAEP SCALES

The following IRT scale score analyses were carried out for each grade in the 1998 NAEP

assessment:

Reading: Three IRT scales linked back to the 1992 and 1994 main assessments of reading.

These three scales, along with a composite scale, are associated with the 1998 main and state

assessments.

Writing: A single newly developed IRT scale for each grade for the main and state

assessments of writing.

Civics: A single newly developed IRT scale for each grade for the main assessment of civics.

Details are in the following chapters.
247

Chapter 13

CONVENTIONS USED IN HYPOTHESIS TESTING

AND REPORTING NAEP RESULTS1

Spencer S. Swinton, David S. Freund, and Nancy L. Allen

Educational Testing Service

13.1 OVERVIEW

Results for the 1998 NAEP assessments were disseminated in several different reports: the NAEP

1998 Reading Report Card for the Nation and the States (Donahue, Voelkl, Campbell, & Mazzeo, 1999),

the NAEP 1998 Writing Report Card for the Nation and the States (Greenwald, Persky, Campbell, &

Mazzeo, 1999), the NAEP 1998 Civics Report Card for the Nation (Lutkus, Weiss, Campbell, Mazzeo,

and Lazer, 1999), and, published only on the web, summary data tables for each report. These reports are

published on the NCES/NAEP web site http://nces.ed.gov/nationsreportcard. Several other reports based

on 1998 NAEP data will be forthcoming.

The NAEP 1998 Reading Report Card for the Nation and the States, the NAEP 1998 Writing

Report Card for the Nation and the States, and the NAEP 1998 Civics Report Card for the Nation

highlight key assessment results for the nation and summarize results across the jurisdictions

participating in the assessments. These reports contain composite scale score results (e.g., scale score

means) for the nation, for each of the four regions of the country, and for public-school students within

each jurisdiction participating in the state assessments of reading and writing, both overall and by

primary reporting variables. The seven key reporting variables (referred to here as primary reporting

variables) are gender, race/ethnicity, level of parents’ education, Title I participation, eligibility for free

or reduced cost school lunch, type of location, and type of school (public, Catholic schools, other

religious schools, and other private schools). For public-school students, scale score means were reported

for a variety of other subpopulations defined by responses to items from the student, teacher, and school

questionnaires and by school and location demographic variables provided by Westat2

. Upcoming reports

will include estimates of scale score means and selected percentiles for specific subgroups of students of

interest in each report.

The second type of summary report is an electronically delivered collection of summary data

tables (available on the NCES/NAEP web site) that contain detailed breakdowns of the scale score data

for each sample according to the responses to the student, teacher, and school questionnaires for the

public-school, nonpublic-school, and combined populations as a whole and for important subgroups of

the public-school population, as defined by the primary reporting variables. There are six sections in each

collection of summary data tables:

 

1 Spencer S. Swinton played a role in making decisions about hypothesis-testing methods and procedures and worked with David

S. Freund, who implemented many of the methods and procedures in computer programs. Nancy L. Allen contributed to the

current version of this chapter.

2 Some of these variables were used by Westat, in developing the sampling frame for the assessment and in drawing the sample

of participating schools.
248

Student Summary Data Tables break down the composite scale score data according to

the students’ responses to questions in the three student questionnaires (common core,

subject-specific background, and motivational section) included in the assessment

booklets.

Teacher Summary Data Tables break down the composite scale score data according to

the teachers’ responses to questions in teacher questionnaires, where they are available.

School Summary Data Tables break down the composite scale score data according to

the principals’ (or other administrators’) responses to questions in the school

characteristics and policies questionnaire.

Question Summary Data Tables provide the response data (percent of students choosing

each option) for each cognitive item in the assessment.

Achievement-Level Summary Data Tables provide estimates of the percentage of

students at or above each achievement level as well as the percentage of students below

the Basic level.

Percentile Summary Data Tables provide selected composite-scale and subscale

percentiles for the public-school, nonpublic-school, and total populations and for the

major demographic subgroups of the national school population.

The production of the Report Cards and the summary data tables required many decisions about

a variety of data analysis and statistical issues. For example, certain categories of the reporting variables

contained limited numbers of examinees. A decision was needed as to what constituted a sufficient

sample size to permit the reliable reporting of subgroup results, and which, if any, estimates were

sufficiently unreliable to need to be “flagged” as a caution to readers. As a second example, the

performance for subgroups of students were compared. A number of inferential rules, based on logical

and statistical considerations, had to be developed to ensure that conclusions are adequately supported by

the data from the assessment. Practical comparison procedures were required to control for Type I errors

without paying too large a penalty with respect to the statistical power for detecting real and

substantively interesting differences. Prior to 1998, the Bonferroni procedure (Hochberg, 1988) was the

principal method used by NAEP to protect against Type I error. Currently, a new multiple comparison

criterion, false discovery rate or FDR (Benjamini & Hochberg, 1994), is used. FDR controls the rate of

false rejections (e.g., 5 false rejections per 100 rejections), rather than controlling the probability of one

such error (familywise error rate, or FWE), as the Bonferroni procedure does. To implement the use of

the FDR, the 1994 procedure of Benjamini and Hochberg was selected.

The purpose of this chapter is to document the major conventions and statistical procedures used

in generating the Report Cards and the summary data tables. Additional details about procedures relevant

to the Report Cards can be found in the text and technical appendices of those reports. Information is

available on the Internet, describing procedures used in creating the summary data tables.

13.2 MINIMUM SCHOOL AND STUDENT SAMPLE SIZES FOR

REPORTING SUBGROUP RESULTS

In all of the reports, estimates of quantities such as composite and scale score means and

percentages of students indicating particular levels of background variables (as measured in the student,

teacher, and school questionnaires) are reported for the population of students in each grade. These

estimates are also reported for certain key subgroups of interest as defined by primary NAEP reporting
249

variables. Where possible, NAEP reports results for gender, for five racial/ethnic subgroups (White,

Black, Hispanic, Asian American/Pacific Islander, and American Indian/Alaskan Native), three types of

locations (central cities, urban fringes/large towns, rural/small town areas), four levels of parents’

education (did not finish high school, high school graduate, some college, college graduate), Title 1

participation, eligibility for the free or reduced-cost school lunch component of the National School

Lunch Program, and type of school. However, for some regions of the country and sometimes for the

nation as a whole, school and/or student sample sizes were too small for one or more of the categories of

these variables to permit accurate reporting.

A consideration in deciding whether to report an estimated quantity is whether the sampling error

is too large to permit effective use of the estimates. A second, and equally important, consideration is

whether the standard error estimate that accompanies a statistic is itself sufficiently accurate to inform

potential readers about the reliability of the statistic. The precision of a sample estimate (be it sample

mean or standard error estimate) for a population subgroup from a three-stage sample design (the one

used to select samples for the national assessments) is a function of the sample size of the subgroup and

of the distribution of that sample across first-stage sampling units (i.e., PSUs in the case of the national

assessments). Hence, both of these factors were used in establishing minimum sample sizes for reporting.

Here a decision was reached to report subgroup results only if the student sample size exceeded

61.3 A design effect of two was assumed for this decision, implying a sample design-based variance twice

that of simple random sampling. This assumption is consistent with previous NAEP experience (Johnson

& Rust, 1992). In carrying out the statistical power calculations when comparing a subgroup to the total

group, it was assumed that the total population sample size is large enough to contribute negligibly to

standard errors. Furthermore, it was required that the students within a subgroup be adequately

distributed across PSUs to allow for reasonably accurate estimation of standard errors. In consultation

with Westat, a decision was reached to publish only those statistics that had standard error estimates

based on five or more degrees of freedom. The same minimum student and PSU sample size restrictions

were applied to proportions and to comparisons of percentages or proportions as well as average scale

scores and comparisons of average scale scores.

13.3 IDENTIFYING ESTIMATES OF STANDARD ERRORS WITH LARGE MEAN

SQUARED ERRORS

As noted above, standard errors of average scale scores, proportions, and percentiles play an

important role in interpreting subgroup results and in comparing the performances of two or more

subgroups. The jackknife standard errors reported by NAEP are statistics whose quality depends on

certain features of the sample from which the estimate is obtained. In certain cases, the mean squared

error4

 associated with the estimated standard errors may be quite large. This result typically occurred

when the number of students upon which the standard error is based is small or when this group of

students comes from a small number of participating PSUs. The minimum PSU and student sample sizes

that were imposed in most instances suppressed statistics where such problems existed. However, the

possibility remained that some statistics based on sample sizes that exceed the minimum requirements

had standard errors that were not well estimated. Therefore, in the reports, estimated standard errors for

published statistics that are themselves subject to large mean squared errors are followed by the symbol

“!”.

 

3 This number was obtained by determining the sample size necessary to detect an effect size of 0.5 with a probability of 0.8 or

greater.

4 The mean squared error of the estimated standard error is defined as      ˆ σ where ˆ is the estimated standard

error,σ is the “true” standard error, and  is the expectation, or expected value operator.
250

The magnitude of the mean squared error associated with an estimated standard error for the

mean or proportion of a group depends on the coefficient of variation (CV) of the estimated size of the

population group, denoted as Nˆ (Cochran, 1977, Section 6.3). The coefficient of variation is estimated

by:

where Nˆ is a point estimate of N and ˆ SE(N) is the jackknife standard error (described in Chapter 10 of

this report) of N.ˆ

Experience with previous NAEP assessments suggests that when this coefficient exceeds 0.2, the

mean squared error of the estimated standard errors of means and proportions based on samples of this

size may be quite large. (Further discussion of this issue can be found in Johnson & Rust, 1992.)

Therefore, the standard errors of means and proportions for all subgroups for which the coefficient of

variation of the population size exceeds 0.2 are marked as described above. In the Report Cards and the

summary data tables, statistical tests involving one or more quantities that have standard errors,

confidence intervals, or significance tests so flagged should be interpreted with caution.

13.4 TREATMENT OF MISSING DATA FROM THE STUDENT, TEACHER,

AND SCHOOL QUESTIONNAIRES

As previously described, responses to the student, teacher, and school questionnaires played a

prominent role in all reports. Although the return rate on all three types of questionnaire was high,5 there

were missing data for each type of questionnaire.

The reported estimated percentages of students in the various categories of background

variables, and the estimates of the average scale score of such groups, were based on only those students

for whom data on the background variable were available. In the terminology of Little and Rubin (1987),

the analyses pertaining to a particular background variable presented in the reports are contingent on the

assumption that the data are missing completely at random.6

The estimates of proportions and proficiencies based on “missing completely at random”

assumptions are subject to potential nonresponse bias if, as may be the case, the assumptions are not

correct. The amount of missing data was small (usually, less than 2%) for most of the variables obtained

from the student, school, and teacher questionnaires. For analyses based on these variables, reported

results are subject to little, if any, nonresponse bias. However, for particular background items in these

questionnaires, the level of nonresponse was somewhat higher, and so the potential for nonresponse bias

is also somewhat greater. Results for background questions for which more than 10 percent of the

responses were missing should be interpreted with caution.

To analyze the relationships among teachers’ questionnaire responses and their students’

achievement, each teacher’s questionnaire had to be matched to the students who were taught by that

teacher. If a student could not be matched to a teacher, all teacher questionnaire responses are missing for

that student. Lower percentages of students with teacher questionnaire data indicate that there is less

 

5 Information about survey participation rates (both school and student), as well as proportions of students excluded by each

jurisdiction from the assessment, is given in Appendix A. Sampling adjustments intended to account for school and student

nonresponse are described in Chapters 10 and 11.

6 The term "missing completely at random" means that the mechanism generating the missing data is independent of the response

to the particular background items and the scale score.

N

SE(N) CV(N)= ˆ

ˆ ˆ
251

certainty about results for variables from the teacher questionnaire. Note that these match rates do not

reflect the additional missing data due to item-level nonresponse. The amount of additional item-level

nonresponse in the returned teacher questionnaires can be found in the summary data tables.

13.5 HYPOTHESIS-TESTING CONVENTIONS

13.5.1 Comparing Means and Proportions for Different Groups of Students

Many of the group comparisons explicitly commented on in the reports involved mutually

exclusive sets of students. Examples include comparisons of the average scale score for male and female

students, White and Hispanic students, students attending schools in central city and urban fringe or

large-town locations, students who reported watching six or more hours of television each night, and

students who report watching less than one hour of television each night.

The text in the reports indicate that means or proportions from two groups were different only

when the difference in the point estimates for the groups being compared was statistically significant at

an approximate simultaneous  level of .05. An approximate procedure was used for determining

statistical significance NAEP staff judged to be statistically defensible, as well as being computationally

tractable. Although all pairs of levels within a variable were tested and reported in the summary data

tables, some text within the reports was developed for only a subset of these comparisons, although the

family size was maintained at that of the original tests. For example, text was included in the reports to

compare the majority ethnic group and each minority group, but text for all possible comparisons of

groups may not have been included. The procedure used to make statistical tests is described in the

following paragraphs.

Let Ai be the statistic in question (e.g., a mean for group i) and let Ai S be the jackknife standard

error of the statistic. The text in the reports identified the means or proportions for groups i and j as being

different if:

( ) ( )

2 .05

2c A A i A A i j j

i j

2

| A - | A

 T

S + S

≥

where T is the (1 - ) percentile of the t distribution with degrees of freedom, df, as estimated below,

and c is the number of related comparisons being tested. See the following section (Section 13.5.2) for a

more specific description of multiple comparisons. In cases where group comparisons were treated as

individual units, the value of c was taken as 1, and the test statistic was equivalent to a standard two-tailed ttest for independent samples. When c is greater than 1, this test is based on the Benjamini and Hochberg

(1995) procedure of controlling the FDR, described below.

The procedures in this section assume that the data being compared are from independent

samples. Because of the sampling design in which PSUs, schools, and students within school are

randomly sampled, the data from mutually exclusive sets of students may not be strictly independent.

Therefore, the significance tests employed are, in many cases, only approximate. Another procedure, one

that does not assume independence, could have been conducted. However, that procedure is

computationally burdensome. A comparison of the standard errors using the independence assumption

and the correlated group assumption was made using NAEP data. The estimated standard error of the

difference based on independence assumptions was approximately 10 percent larger than the more

complicated estimate based on correlated groups. In almost every case, the correlation of NAEP data

across groups was positive. Because, in NAEP, significance tests based on assumptions of independent
252

samples are only somewhat conservative, the approximate (assuming independence) procedure was used

for most comparisons.

Because of clustering and differential weighting in the sample, the degrees of freedom are less than

for a simple random sample of the same size. The degrees of freedom of this t-test is defined by a

Satterthwaite (Johnson & Rust, 1992) approximation as follows:

where N is the number of subgroups involved, and d Ak f is as follows:

where m is the number of jackknife replicates (usually 62 in NAEP), tj is the j

th replicated estimate for

the mean of a subgroup, and tk is the estimate of the subgroup mean using the overall weights and the

first plausible value.

The number of degrees of freedom for the variance equals the number of independent pieces of

information used to generate the variance. In the case of data from NAEP, the 62 pieces of information

are the squared differences (tjk  tk)

2

, each supplying at most one degree of freedom (regardless of how

many individuals were sampled within PSUs). If some of the squared differences (tjk  tk)

2

 are much larger

than others, the variance estimate of mk is predominantly estimating the sum of these larger components,

which dominate the remaining terms. The effective degrees of freedom of Ak S in this case will be nearer

to the number of dominant terms. The estimate Ak df reflects these relationships.

The two formulae above show us that when Ak df is small, the degrees of freedom for the t-test, df,

will also be small. This will tend to be the case when only a few PSU pairs have information about

subgroup differences relevant to a t-test. It will also be the case when a few PSU pairs have subgroup

differences much larger than other PSU pairs.

The procedures described above were used for testing differences of both means and nonextreme

percentages. The approximation for the test for percentages works best when sample sizes are large, and

the percentages being tested have magnitude relatively close to 50 percent. Statements about group

differences should be interpreted with caution if at least one of the groups being compared is small in

size or if “extreme” percentages are being compared.

( )

( )

2

2

1

4

1

2.77 3.16

k

k

m

j k

j

Ak m

j k

j

t t

df = m t t

=

=

        −

      −         −    

∑

∑

k

k

k

N

2 2

A

k=1

N 4

A

k=1 A

( S )

df = 

S

df

∑

∑
253

Differences in percentages were treated as involving “extreme” percentages if for either

percentage, P:

200 ,

2 lim

EFF

P P

N

< =

+

where the effective sample size is

2

(100 ) , ( ) EFF

JK

P P N

SE

− = and SEJK

is the jackknife standard error of P. Similarly, at the other end of the 0 – 100 scale, a percentage is

deemed extreme if 100 – P < Plim. In either extreme case, the normal approximation to the distribution is

a poor approximation, and the value of P was reported, but no standard error was estimated and hence no

significance tests were conducted.

13.5.2 Multiple Comparison Procedures

Frequently, groups (or families) of comparisons were made and were presented as a single set.

The appropriate text, usually a set of sentences or a paragraph, was selected for inclusion in a report

based on the results for the entire set of comparisons. For example, some reports contain a section that

compared average scale scores for a predetermined group, generally the majority group (in the case of

race/ethnicity, for example, White students) to those obtained by other minority groups. The entire set of

tests was presented in the summary data tables. The procedures described above and the certainty

ascribed to intervals (e.g., a 95 % confidence interval) are based on statistical theory that assumes that

only one confidence interval or test of statistical significance is being performed. However, in some

sections of a report, many different groups are compared (i.e., multiple sets of confidence intervals are

being analyzed). In sets of confidence intervals, statistical theory indicates that certainty associated with

the entire set of intervals is less than that attributable to each individual comparison from the set. To hold

the significance level for the set of comparisons at a particular level (e.g., .05), adjustments—called

“multiple comparison procedures”—must be made to the methods described in the previous section. One

such procedure, the false discovery rate (FDR) procedure (Benjamini & Hochberg, 1995) was used to

control the certainty level.

Unlike the other multiple comparison procedures (e.g., the Bonferroni procedure) that control the

familywise error rate (i.e., the probability of making even one false rejection in the set of comparisons),

the FDR procedure controls the expected proportion of falsely rejected hypotheses. Furthermore,

familywise procedures are considered conservative for large families of comparisons (Williams, Jones, &

Tukey, 1999). Therefore, the FDR procedure is more suitable for multiple comparisons in NAEP than

other procedures.

The 1998 assessment is the first time NAEP has used the Benjamini-Hochberg procedure to

maintain FDR for all multiple comparisons. Prior to the 1996 assessment, the Bonferroni procedure was

used for multiple comparisons. In 1996, either the Bonferroni or Benjamini-Hochberg FDR procedure

was used, depending on the testing situation. The Benjamini-Hochberg FDR procedure was used for

large numbers of comparisons (i.e., any comparisons involving all of the states): (a) all pairwise

comparisons of the states; (b) all comparisons of individual states to the national average; and (c) the

trend for each state, which compared the current mean for the state to the state’s mean in the previous
254

assessment. All other multiple comparisons for the 1996 assessment used the Bonferroni procedure. The

1994 NAEP reading assessments used the Bonferroni procedure exclusively for multiple comparisons.

The Benjamini and Hochberg application of the false discovery rate (FDR) criterion can be

described as follows. Let q be the number of significance tests made and let P(1) ≤ P(2) ≤ . . . ≤ P(q) be

the ordered significance levels of the q	




 	


significance level desired, usually .05 for one-tailed tests (or .025 for two-tailed tests). The procedure

compares P(q
 q
 q-1)/q, . . ., P(j
 j/q, stopping the comparisons with the first j

such that P(j) ≤ j/q. All tests associated with P(1) , . . ., P(j) are declared significant; all tests associated

with P(j+1) , . . . , Pq are declared nonsignificant.

13.5.3 Comparing Proportions Within a Group

Certain analyses involved the comparison of proportions. One example was the comparison of

the proportion of students who reported that a parent graduated from college to the proportion of students

who indicated that their parents did not finish high school to determine which proportion was larger.

There are other such proportions of interest in this example, such as the proportion of students with at

least one parent graduating from high school but neither parent graduating from college. For these types

of analyses, NAEP staff determined that the dependencies in the data could not be ignored.

Unlike the case for analyses of the type described in Section 13.5.1, the correlation between the

proportion of students reporting a parent graduated from college and the proportion reporting that their

parents did not finish high school is likely to be negative and large. For a particular sample of students, it

is likely that the higher the proportion of students reporting “at least one parent graduated from college”

is, the lower the proportion of students reporting “neither parent graduated from high school” will be. A

negative dependence will result in underestimates of the standard error if the estimation is based on

independence assumptions (as is the case for the procedures described in Section 13.5.1). Such

underestimation can result in an unacceptably large number of “nonsignificant” differences being

identified as significant.

The procedures of Section 13.5.1 were modified for analyses that involved comparisons of

proportions within a group. The modification involved using a jackknife method for obtaining the

standard error of the difference in dependent proportions. The standard error of the difference in

proportions was obtained by first obtaining a separate estimate of the difference in question for each

jackknife replicate (using the first plausible value only) then taking the standard deviation of the set of

replicate estimates as the estimate. The procedures used for proportions within a group differed from the

procedures of Section 13.5.1 only with respect to estimating the standard error of the difference; all other

aspects of the procedures were identical.
255

Chapter 14

ASSESSMENT FRAMEWORKS AND INSTRUMENTS

FOR THE 1998 NATIONAL AND STATE READING ASSESSMENTS1

Patricia L. Donahue and Terry L. Schoeps

Educational Testing Service

14.1 INTRODUCTION

The reading framework was originally developed through a broad-based consensus process

conducted by the Council of Chief State School Officers (CCSSO) working under contract to the

National Assessment Governing Board (NAGB). The development process involved a steering

committee, a planning committee, and CCSSO project staff. Educators, scholars, and citizens,

representative of many diverse constituencies and points of view, participated in the national consensus

process to design objectives for the reading assessment. The framework that was used for the 1998

NAEP reading assessment was also used for the 1992 and 1994 assessments.

The instrument used in the 1998 reading assessment was composed of a combination of reading

passages and questions from the 1992 and 1994 assessments and a set of passages and questions newly

developed for 1998. A total of twenty-three unique blocks (a block is a reading passage with a set of

questions) were administrated in 1998. Three of these blocks were developed for 1998 and the remaining

twenty were carried over from the 1992 and 1994 assessments. Administering the same blocks across

assessment years allows for the reporting of trends in reading performance. At the same time, developing

new sets of passages and questions made it possible to release three blocks for public use. The

framework for the reading assessment is available on the National Assessment Governing Board (NAGB)

web site at http://www.nagb.org.

Sections 14.3 through 14.5 include a detailed description of the framework and the development

of reading questions, or items, for the 1998 NAEP reading assessment. Section 14.8 also describes the

student background questionnaires and the reading teacher questionnaire. Additional information on the

structure and content of assessment booklets can be found in Section 14.9. The list of committee

members who participated in the 1998 development process is provided in Appendix K.

Samples of assessment instruments and student responses are published in the NAEP 1998

Reading Report Card for the Nation and the States: Findings from the National Assessment of

Educational Progress (Donahue, Voelkl, Campbell, & Mazzeo, 1999).

14.2 DEVELOPING THE READING ASSESSMENT FRAMEWORK

NAGB is responsible for setting policy for NAEP; this policymaking role includes the

development of assessment frameworks and test specifications. Appointed by the Secretary of Education

from lists of nominees proposed by the Board itself in various statutory categories, the 24-member board

is composed of state, local, and federal officials, as well as educators and members of the public.

 

1 Patricia L. Donahue manages the item development process for NAEP reading assessments. Terry L. Schoeps

coordinates the production of NAEP technical reports.
256

NAGB began the development process for the 1992 reading objectives (which also served as the

objectives for the 1994 and 1998 assessments) by conducting a widespread mail review of the objectives

for the 1990 reading assessment and by holding a series of public hearings throughout the country. The

contract for managing the remainder of the consensus process was awarded to the CCSSO. The

development process included the following activities:

• A Steering Committee consisting of members recommended by each of 16

national organizations was established to provide guidance for the consensus

process. The committee monitored the progress of the project and offered advice.

Drafts of each version of the document were sent to members of the committee

for review and reaction.

• A Planning Committee was established to identify the objectives to be assessed in

reading and prepare the framework document. The members of this committee

consisted of experts in reading, including college professors, an academic dean, a

classroom teacher, a school administrator, state level assessment and reading

specialists, and a representative of the business community. This committee met

with the Steering Committee and as a separate group. A subgroup also met to

develop item specifications. Between meetings, members of the committee

provided information and reactions to drafts of the framework.

• The project staff at CCSSO met regularly with staff from NAGB and NCES to

discuss progress made by the Steering and Planning committees.

During this development process, input and reactions were continually sought from a wide range

of members of the reading field, experts in assessment, school administrators, and state staff in reading

assessment. In particular, innovative state assessment efforts and work being done by the Center for the

Learning and Teaching of Literature (Langer, 1989, 1990).

For more detail on the development and specifications of the reading framework, refer to the

Reading Framework and Specifications for the 1998 National Assessment of Educational Progress,

1992–1998 (NAGB, 1990).

14.3 READING FRAMEWORK AND ASSESSMENT DESIGN PRINCIPLES

The reading objectives framework was designed to focus on reading processes and outcomes,

rather than reflect a particular instructional or theoretical approach. It was stated that the framework

should focus not on the specific reading skills that lead to outcomes, but rather on the quality of the

outcomes themselves. The framework was intended to embody a broad view of reading by addressing the

increasing level of literacy needed for employability, personal development, and citizenship. The

framework also specified a reliance on contemporary reading research and the use of nontraditional

assessment formats that more closely resemble desired classroom activities.

The objectives development was guided by the consideration that the assessment should reflect

many of the curricular emphases and objectives in various states, localities, and school districts in

addition to what various scholars, practitioners, and interested citizens believed should be included in the

curriculum. Accordingly, the committee gave attention to several frames of reference:

• The purpose of the NAEP reading assessment is to provide information about the

progress and achievement of students in general rather than to test individual
257

students ability. NAEP is designed to inform policymakers and the public about

reading ability in the United States.

• The term reading literacy should be used in the broad sense of knowing when to

read, how to read, and how to reflect on what has been read. It represents a

complex, interactive process that goes beyond basic or functional literacy.

• The reading assessment should use valid and authentic tasks that are both broad

and complete in their coverage of important reading behaviors so that the test will

be useful and valid, and will demonstrate a close link to desired classroom

instruction.

• Every effort should be made to make the best use of available methodology and

resources in driving assessment capabilities forward. New types of items and new

methods of analysis were recommended for NAEP reading assessments.

• Every effort must be made in developing the assessment to represent a variety of

opinions, perspectives, and emphases among professionals, as well as state and

local school districts.

14.4 FRAMEWORK FOR THE 1998 READING ASSESSMENT

The framework adopted for the 1998 reading assessment, which also served as the framework for

the 1992 and 1994 assessments, was organized according to a four-by-three matrix of reading stances by

reading purposes. The stances include:

• Initial Understanding;

• Developing an Interpretation;

• Personal Reflection and Response; and

• Demonstrating a Critical Stance.

These stances were assessed across three global purposes defined as:

• Reading for Literary Experience;

• Reading to Gain Information; and

• Reading to Perform a Task.

Different types of texts were used to assess the various purposes for reading. Students reading

abilities were evaluated in terms of a single purpose for each type of text. At grade 4, only Reading for

Literary Experience and Reading to Gain Information were assessed, while all three global purposes were

assessed at grades 8 and 12. Figure 14-1 and 14-2 describe the four reading stances and three reading

purposes that guided the development of NAEPs 1992, 1994, and 1998 reading assessments.

The Planning Committee was interested in creating an assessment that would be forwardthinking and reflect quality instruction. In recognition that the demands made of readers change as they

mature and move through school, it was recommended that the proportion of items have some relation to

reading purpose (i.e., for literary experience, to gain information, to perform a task). The distribution of

items by reading purpose across grade levels recommended in the assessment framework is provided in

Table 14-1.
258

Readers use a range of cognitive abilities and assume various stances that should be assessed

within each of the reading purposes. While reading, students form an initial understanding of the text and

connect ideas within the text to generate interpretations. In addition, they extend and elaborate their

understanding by responding to the text personally and critically and by relating ideas in the text to prior

knowledge.

For more detail on the development and specifications of the Reading Framework, refer to

Reading Framework for the National Assessment of Educational Progress, 1992-1998 (NAGB, 1990).

Figure 14-1

Description of Reading Stances

Readers interact with text in various ways as they use background knowledge and understanding of

text to construct, extend, and examine meaning. The NAEP reading assessment framework

specified four reading stances to be assessed that represent various interactions between readers

and texts. These stances are not meant to describe a hierarchy of skills or abilities. Rather, they are

intended to describe behaviors that readers at all developmental levels should exhibit.

Initial Understanding

Initial understanding requires a broad, preliminary construction of an understanding of the text.

Questions testing this aspect ask the reader to provide an initial impression or unreflected

understanding of what was read. The first question following a passage was usually one testing

initial understanding.

Developing an Interpretation

Developing an interpretation requires the reader to go beyond the initial impression to develop a

more complete understanding of what was read. Questions testing this aspect require a more

specific understanding of the text and involve linking information across parts of the text as well as

focusing on specific information.

Personal Reflection and Response

Personal reflection and response requires the reader to connect knowledge from the text more

extensively with his or her own personal background knowledge and experience. The focus is on

how the text relates to personal experience; questions on this aspect ask the readers to reflect and

respond from a personal perspective. Personal reflection and response questions were typically

formatted as constructed-response items to allow for individual possibilities and varied responses.

Demonstrating a Critical Stance

Demonstrating a critical stance requires the reader to stand apart from the text, consider it, and

judge it objectively. Questions on this aspect require the reader to perform a variety of tasks such

as critical evaluation, comparing and contrasting, application to practical tasks, and understanding

the impact of such text features as irony, humor, and organization. These questions focus on the

reader as critic and require reflection on and judgments about how the text is written.
259

Figure 14-2

Description of Purposes for Reading

Reading involves an interaction between a specific type of text or written material and a reader,

who typically has a purpose for reading that is related to the type of text and the context of the

reading situation. The reading assessment presented three types of text to students representing

each of three reading purposes: literary text for literary experience, informational text to gain

information, and documents to perform a task. Students reading skills were evaluated in terms of a

single purpose for each type of text.

Reading for Literary Experience

Reading for literary experience involves reading literary text to explore the human condition, to

relate narrative events with personal experiences, and to consider the interplay in the selection

among emotions, events, and possibilities. Students in the NAEP reading assessment were

provided with a wide variety of literary text, such as short stories, poems, fables, historical fiction,

science fiction, and mysteries.

Reading to Gain Information

Reading to gain information involves reading informative passages in order to obtain some general

or specific information. This often requires a more utilitarian approach to reading that requires the

use of certain reading/thinking strategies different from those used for other purposes. In addition,

reading to gain information often involves reading and interpreting adjunct aids such as charts,

graphs, maps, and tables that provide supplemental or tangential data. Informational passages in

the NAEP reading assessment included biographies, science articles, encyclopedia entries, primary

and secondary historical accounts, and newspaper editorials.

Reading to Perform a Task

Reading to perform a task involves reading various types of materials for the purpose of applying

the information or directions in completing a specific task. The readers purpose for gaining

meaning extends beyond understanding the text to include the accomplishment of a certain activity.

Documents requiring students in the NAEP reading assessment to perform a task included

directions for creating a time capsule, a bus schedule, a tax form, and instructions on how to write

a letter to a senator. Reading to perform a task was assessed only at grades 8 and 12.
260

Table 14-1

Percentage Distribution of Items by Reading Purpose

as Specified in the NAEP Reading Framework

Purpose for Reading

Grade

Reading for

Literary Experience

Reading to

Gain Information

Reading to

Perform a Task

4 55% 45% (Not Assessed)

8 40% 40% 20%

12 35% 45% 20%

Table 14-2 shows the distribution of items by reading stance, as specified in the reading

framework, for all three grade levels.

Table 14-2

Percentage Distribution of Items by Reading Stance

as Specified in the NAEP Reading Framework

Reading Stance Grades 4, 8, and 12

Initial Understanding/Developing an Interpretation 33%

Personal Reflection and Response 33%

Demonstrating a Critical Stance 33%

14.5 DEVELOPING THE READING COGNITIVE ITEMS

In developing the new portion of the 1998 NAEP reading assessment, the same framework and

procedures used in 1992, and again in 1994, were followed. After careful review of the objectives,

reading materials were selected and questions were developed that were appropriate to the objectives. All

questions were extensively reviewed by specialists in reading, measurement, and bias/sensitivity, as well

as by state representatives.

The development of cognitive items began with a careful selection of grade-appropriate passages

for the assessment. Passages were selected from a pool of reading selections contributed by teachers from

across the country. The framework states that the assessment passages should represent authentic,

naturally occurring reading material that students may encounter in and out of school. Furthermore, these

passages were to be reproduced in test booklets as they had appeared in their original publications. In

some cases, materials (such as bus schedules) were provided to students separate from the printed

assessment booklet. Final passage selections were made by the Reading Instrument Development

Committee. In order to guide the development of items, passages were outlined or mapped to identify

essential elements of the text.

The assessment included constructed-response (short and extended) and multiple-choice items.

The decision to use a specific item type was based on a consideration of the most appropriate format for

assessing the particular objective. Both types of constructed-response items were designed to provide an

in-depth view of students ability to read thoughtfully and to respond appropriately to what they read.

Short constructed-response questions were used when students needed to respond in only one or two

sentences in order to demonstrate full comprehension. Extended constructed-response questions were

used when the task required more thoughtful consideration of the text and engagement in more complex
261

reading processes. Multiple-choice items were used whenever a reading outcome could be measured

through use of these items.

A carefully developed and proven series of steps was used to create the assessment items. These

steps are described in Chapter 2.

The assessment included 25-minute and 50-minute blocks," each consisting of one or more

passages and a set of multiple-choice and constructed-response items to assess students comprehension

of the written material. At grade 8 and 12 students were asked to respond to either two 25-minute blocks

or one 50-minute block. The grade-4 assessment included eight 25-minute blocks (four blocks measuring

each of the two global purposes for reading assessed at this grade). The instruments at grades 8 and 12

each included nine 25-minute blocks (three blocks measuring each of the global purposes for reading). In

addition, the grade 8 assessment included one 50-minute block and the grade-12 assessment included two

50-minute blocks.

14.6 DEVELOPING THE READING OPERATIONAL FORMS

A reading field test was conducted in March 1997 to test new reading questions that were

developed to replace the few 1994 items that had been publicly released and were, therefore, no longer

able to be used in an operational assessment. The field test was given to national samples of fourth-,

eighth-, and twelfth-grade students. The field test data were collected, scored, and analyzed in

preparation for meetings with the Reading Instrument Development Committee. Using item analysis,

which provided the mean percentage of correct responses, the polyserial correlations, and the difficulty

level for each item in the field test, committee members, ETS test development staff, and NAEP/ETS

staff reviewed the materials. The objectives that guided these reviews included:

• determining which items were most related to overall student achievement,

• determining the need for revisions of items that lacked clarity or had ineffective

item formats,

• prioritizing items to be included in the assessment, and

• determining appropriate timing for assessment items.

Once the committees had selected the items, all items were rechecked for content, measurement,

and sensitivity concerns. The federal clearance process was initiated in June 1997 with the submission of

draft materials to NCES. The package containing the final set of cognitive items assembled into blocks

and questionnaires was submitted in June 1997. Throughout the clearance process, revisions were made

in accordance with changes required by the government. Upon approval, the blocks (assembled into

booklets) and questionnaires were prepared for printing.

14.7 DISTRIBUTION OF READING ASSESSMENT ITEMS

Figure 14-3 lists the total number of items at each grade level in the 1998 assessment. Of the total

of 247 items, there are 93 unique multiple-choice items and 154 unique constructed-response questions

that make up the 1998 reading assessment. Some of these items are used at more than one grade level. As

a result, the sum of the items that appear at each grade level is greater than the total number of unique

items.
262

Figure 14-3

Distribution of Items for the 1998 Reading Assessment

In the development process, every effort was made to meet the content and process targets

specified in the assessment framework. Table 14-3 shows the approximate percentage of aggregate

assessment time devoted to each purpose for reading at each grade level. Percentages are based on the

classifications agreed upon by NAEPs 1998 Instrument Development Committee. Note that the numbers

presented in Table 14-3 differ from Table 14-1 in that Table 14-1 shows the distribution of assessment

items as specified in the reading framework.

Table 14-3

Percentage Distribution of Assessment Time by Grade and Reading Purpose

for the NAEP 1998 Reading Assessment

Reading Purpose Grade 4 Grade 8 Grade 12

Reading for Literary Experience 50% 38% 33%

Reading to Gain Information 50% 38% 47%

Reading to Perform a Task N/A 23% 20%

Table 14-4 shows the approximate percentage of assessment time devoted to each reading stance.

Unlike the purposes for reading, in which individual students did not receive questions in all areas, every

student completed tasks involving each of the reading stances. It is recognized that making discrete

classifications is difficult for these categories and that independent efforts to classify NAEP questions

have led to different results (National Academy of Education, 1992). Also, it has been found that

developing personal response questions that are considered equitable across students different

backgrounds and experiences is difficult. Note that the numbers presented in Table 14-4 differ from

Table 14-2, in that Table 14-2 shows the distribution of items as specified in the reading framework.

Grade 4

36 Multiple-Choice

38 Short Constructed-Response

8 Extended Constructed-Response

Grade 8

39 Multiple-Choice

56 Short Constructed-Response

12 Extended Constructed-Response

Grade 12

33 Multiple-Choice

63 Short Constructed-Response

13 Extended Constructed-Response

25 Multiple-Choice

30 Short Constructed-Response

6 Extended Constructed-Response

12 Multiple-Choice

26 Short Constructed-Response

5 Extended Constructed-Response

27 Multiple-Choice

41 Short Constructed-Response

8 Extended Constructed-Response

11 Multiple-Choice

8 Short Constructed-Response

2 Extended Constructed-Response

16 Multiple-Choice

22 Short Constructed-Response

5 Extended Constructed-Response
263

Table 14-4

Percentage Distribution of Assessment Time by Grade

and Reading Stance for the NAEP 1998 Reading Assessment

Reading Stance Grade 4 Grade 8 Grade 12

Initial Understanding/ Developing an Interpretation 56% 49% 52%

Personal Reflection and Response 21% 19% 16%

Demonstrating a Critical Stance 23% 32% 32%

14.8 BACKGROUND QUESTIONNAIRES FOR THE 1998 READING ASSESSMENT

Research indicates that school, home, and attitudinal variables affect students reading

comprehension and literacy. Therefore, in addition to assessing how well students read, it is important to

understand the instructional context in which reading takes place, students home support for literacy, and

their reading habits and attitudes. To gather contextual information, NAEP assessments include

background questions designed to provide insight into the factors that may influence reading scale scores

in the literary, informational, and document categories assessed.

NAEP includes both general background questionnaires given to participants in all subjects and

subject-specific questionnaires for both students and their teachers. The development of the general

background questionnaires is discussed below. It is worth noting that members of the Reading Instrument

Development Committee were consulted on the appropriateness of the issues addressed in all

questionnaires that may relate to reading instruction and achievement. Like the cognitive items, all

background questions were submitted for extensive review and field testing. Recognizing the reliability

problems inherent in self-reported data, particular attention was given to developing questions that were

meaningful and unambiguous and that would encourage accurate reporting.

In addition to the cognitive questions, the 1998 assessment included one five-minute set each of

general and reading background questions designed to gather contextual information about students, their

instructional and recreational experiences in reading, and their attitudes toward reading. Students in the

fourth grade were given additional time because the items in the general questionnaire were read aloud

for them. A one-minute questionnaire was also given to students at the end of each booklet to measure

students motivation in completing the assessment and their familiarity with assessment tasks.

14.8.1 Student Reading Questionnaires

Three sets of multiple-choice background questions were included as separate sections in each

student booklet:

General Background: The general background questions collected demographic information

about race/ethnicity, language spoken at home, mother’s and father’s level of education, reading

materials in the home, homework, school attendance, which parents live at home, and which

parents work outside the home.

Reading Background: Students were asked to report their instructional experiences

related to reading in the classroom, including group work, special projects, and writing in

response to reading. In addition, they were asked about the instructional practices of their

reading teachers and the extent to which the students themselves discussed what they

read in class and demonstrated use of skills and strategies.
264

Motivation: Students were asked five questions about their attitudes and perceptions

about reading and self-evaluation of their performance on the NAEP assessment.

Table 14-5 shows the number of questions per background section and the placement of each

within student booklets.

Table 14-5

NAEP 1998 Background Sections of Student Reading Booklets

Number of Questions Placement in Student Booklet

Grade 4

General Background 21 Section 1

Reading Background 22 Section 4

Motivation 5 Section 5

Grade 8

General Background 17 Section 1

Reading Background 24 Section 4

Motivation 5 Section 5

Grade 12

General Background 18 Section 1

Reading Background 25 Section 4

Motivation 5 Section 5

14.8.2 Language Arts Teacher Questionnaire

To supplement the information on instruction reported by students, the reading teachers of the

fourth and eighth graders participating in the NAEP reading assessment were asked to complete a

questionnaire about their educational background, content-area preparation, and classroom practices. The

teacher questionnaire contained two parts. The first part pertained to the teachers background and

general training. The second part pertained to specific training in teaching reading and the procedures the

teacher used for each class containing an assessed student.

The Teacher Questionnaire, Part I: Background, Education, and Resources (49 questions at

grade 4 and 48 questions at grade 8) included questions pertaining to:

• gender;

• race/ethnicity;

• years of teaching experience;

• certification, degrees, major and minor fields of study;

• coursework in education;

• coursework in specific subject areas;

• amount of in-service training;

• extent of control over instructional issues; and

• availability of resources for their classroom.
265

The Teacher Questionnaire, Part IIA: Reading/Writing Preparation (12 questions at grade 4

and 12 at grade 8) included questions on the teachers professional development in reading theory and

instruction.

The Teacher Questionnaire, Part IIB: Reading/Writing Instructional Information (84

questions at grade 4 and 85 questions at grade 8) included questions pertaining to:

• ability level of students in the class;

• whether students were assigned to the class by ability level;

• time on task;

• homework assignments;

• frequency of instructional activities used in class;

• methods of assessing student progress in reading;

• instructional emphasis given to the reading abilities covered in the assessment; and

• use of particular resources.

14.9 STUDENT BOOKLETS FOR THE 1998 READING ASSESSMENT

The assembly of reading blocks into booklets and their subsequent assignment to sampled

students was determined by a partially balanced incomplete block (PBIB) design with spiraled

administration. The 25-minute blocks were assembled into 52 booklets such that two different blocks

were assigned to each booklet and each block appeared in four booklets. Each 25-minute block was

paired with another block measuring the same purpose for reading (i.e., reading for literary experience,

reading to gain information, reading to perform a task) approximately 75 percent of the time at grade 4

and approximately 50 percent of the time at grades 8 and 12. This was the partially balanced part of the

PBIB design.

The focused PBIB design also balances the order of presentation of the blocks—every block

appears as the first cognitive block in two booklets and as the second cognitive block in two other

booklets. This design allows for some control of context and fatigue effects.

At grade 4, the blocks were assembled into 16 booklets. At grade 8, the 25-minute blocks were

assembled into 18 booklets, and the 50-minute block appeared in a single booklet. At grade 12, the 25-

minute blocks were assembled into 18 booklets, and each 50-minute block appeared in a separate

booklet. The assessment booklets were then spiraled and bundled. Spiraling involves interweaving the

booklets in a systematic sequence so that each booklet appears an appropriate number of times in the

sample. The bundles were designed so that each booklet would appear equally often in a position in a

bundle.

As in the other subjects, the final step in the BIB or PBIB spiraling procedure was the assigning

of booklets to the assessed students. The students in the assessment session were assigned booklets in the

order in which the booklets were bundled. Thus, most students in an assessment session received

different booklets. Tables 14-6, 14-7, and 14-8 detail the configuration of booklets administered in the

1998 national and state reading assessment.
266

Table 14-6

NAEP 1998 Reading Grade 4 Booklet Configuration

Booklet

Number

Common Core

Background

Question

Block 1

Question

Block 2

Reading

Background Motivation

1 CR R4 R3 RB RA

2 CR R3 R5 RB RA

3 CR R5 R9 RB RA

4 CR R9 R4 RB RA

5 CR R4 R5 RB RA

6 CR R3 R9 RB RA

7 CR R6 R10 RB RA

8 CR R10 R7 RB RA

9 CR R7 R8 RB RA

10 CR R8 R6 RB RA

11 CR R6 R7 RB RA

12 CR R10 R8 RB RA

13 CR R7 R4 RB RA

14 CR R8 R3 RB RA

15 CR R5 R6 RB RA

16 CR R9 R10 RB RA
267

Table 14-7

NAEP 1998 Reading Grade 8 Booklet Configuration

Booklet

Number

Common Core

Background

Question

Block 1

Question

Block 2

Reading

Background Motivation

1 CR R3 R4 RB RA

2 CR R4 R5 RB RA

3 CR R5 R3 RB RA

4 CR R6 R8 RB RA

5 CR R8 R7 RB RA

6 CR R7 R6 RB RA

7 CR R10 R9 RB RA

8 CR R9 R11 RB RA

9 CR R11 R10 RB RA

10 CR R3 R8 RB RA

11 CR R7 R4 RB RA

12 CR R5 R6 RB RA

13 CR R6 R9 RB RA

14 CR R8 R11 RB RA

15 CR R10 R7 RB RA

16 CR R4 R10 RB RA

17 CR R9 R5 RB RA

18 CR R11 R3 RB RA

21 CR ———— R13*———— RB RA

* Block R13 contained one 50-minute task.
268

Table 14-8

NAEP 1998 Reading Grade 12 Booklet Configuration

Booklet

Number

Common Core

Background

Question

Block 1

Question

Block 2

Reading

Background Motivation

1 CR R3 R4 RB RA

2 CR R4 R5 RB RA

3 CR R5 R3 RB RA

4 CR R6 R7 RB RA

5 CR R7 R8 RB RA

6 CR R8 R6 RB RA

7 CR R10 R9 RB RA

8 CR R9 R11 RB RA

9 CR R11 R10 RB RA

10 CR R3 R7 RB RA

11 CR R8 R4 RB RA

12 CR R5 R6 RB RA

13 CR R6 R9 RB RA

14 CR R7 R11 RB RA

15 CR R10 R8 RB RA

16 CR R4 R10 RB RA

17 CR R9 R5 RB RA

18 CR R11 R3 RB RA

21 CR RB RA

22 CR

———— R13*————

———— R14*———— RB RA

* Blocks R13 and R14 contained one 50-minute task each.
269

Chapter 15

INTRODUCTION TO THE DATA ANALYSIS

FOR THE NATIONAL AND STATE READING ASSESSMENTS1

Jinming Zhang, Jiahe Qian, and Steven P. Isham

Educational Testing Service

15.1 INTRODUCTION

This chapter introduces the analyses performed on the responses to the cognitive and background

items in the 1998 assessment of reading. The results of these analyses are presented in the NAEP 1998

Reading: A Report Card for the Nation and the States (Donahue et al., 1999). The emphasis of this

chapter is on the description of student samples, items, assessment booklets, administrative procedures,

scoring constructed-response items, and student weights, and on the methods and results of DIF analyses.

The major analysis components are discussed in Chapter 16 for the national assessment and Chapter 17

for the state assessment.

The objectives of the reading analyses were to:

• prepare scale values and estimate subgroup scale score distributions for national

and state samples of students who were administered reading items from the main

assessment,

• link the 1998 main focused PBIB samples to the 1994 reading scale,

• perform all analyses necessary to produce a short-term trend report in reading

(The reading short-term trend results include the years 1992, 1994 and 1998),

• link the 1998 state assessment scales to the corresponding scales from the 1998

national assessment.

15.2 DESCRIPTION OF STUDENT SAMPLES, ITEMS, ASSESSMENT BOOKLETS,

AND ADMINISTRATIVE PROCEDURES

The student samples that were administered reading items in the 1998 assessment are shown in

Table 15-1. The data from the national main focused PBIB assessment of reading (4 [Reading–Main],

8 [Reading–Main], and 12 [Reading–Main]) were used for national main analyses comparing the levels

of reading achievement for various subgroups of the 1998 target populations. Chapters 1 and 3 contain

descriptions of the target populations and the sample design used for the assessment. The target

populations were grade 4, grade 8, and grade 12 students in the United States. Unlike previous reading

NAEP assessments, only grade-defined cohorts were assessed in the 1998 NAEP. The sampled students

in these three cohorts were assessed in the winter (January to March with final makeup sessions held

 

1

 Jinming Zhang was the primary person responsible for the planning, specification, and coordination of the national reading

analyses. Jiahe Qian was the primary person responsible for the planning, specification, and coordination of the state reading

analyses. Computing activities for all reading scaling and data analyses were directed by Steven P. Isham and completed by Lois H.

Worthington. Others contributing to the analysis of reading data were David S. Freund, Bruce A. Kaplan, and Katharine E. Pashley.
270

from March 30 to April 3). As described in Chapter 3, the reporting sample for the national reading

assessment consisted of students in the S2 sample and the S3 sample, excluding the SD/LEP students.

Table 15-1

NAEP 1998 Reading Student Samples

Sample

Booklet

ID Number

Cohort

Assessed Time of Testing*

Reporting

Sample Size

 4 [Reading–Main] R1–R16 Grade 4 1/5/98 – 3/27/98 7,672

 8 [Reading–Main] R1–R18, R21 Grade 8 1/5/98 – 3/27/98 11,051

12 [Reading–Main] R1–R18, R21–R22 Grade 12 1/5/98 – 3/27/98 12,675

 4 [Reading–State] R1–R16 Grade 4 1/5/98 – 3/27/98 112,138

 8 [Reading–State] R1–R18, R21 Grade 8 1/5/98 – 3/27/98 94,429

* Final makeup sessions were held March 30–April 3, 1998.

LEGEND: Main NAEP national main assessment

State NAEP state assessment 

The data from the state focused PBIB assessment of reading (4[Reading–State] and

8[Reading–State]) were used for the state analyses. The 1998 state reading assessment included the

assessment of both public- and nonpublic-school students for many jurisdictions. The state results

reported in the NAEP 1998 Reading: Report Card for the Nation and the States (Donahue et al., 1999)

are based on public-school students. The state results for both public and nonpublic schools are presented

separately in Chapter 17. The procedures used were similar to those of previous state assessments.

The items in the assessment were based on the curriculum framework described in Reading

Framework for the National Assessment of Educational Progress, 1992–1998 (NAGB, 1990). The 1998

reading assessment is based on the same objectives as the 1994 reading assessment. Compared to earlier

NAEP assessments, the current assessment contains longer reading passages that are intended to be more

authentic examples of the reading tasks encountered in and out of school. As described in the reading

framework, these blocks are organized into three subscales, corresponding to three purposes for reading:

reading for literary experience, reading to gain information, and reading to perform a task. At grade 4,

only the first two purposes are represented. Scales were produced for each of the purposes of reading. In

addition, a composite scale for reading was created as a weighted sum of the purposes-for-reading scales

(see Table 14-1).

In the main samples, each student was administered a booklet containing either two separately

timed 25-minute blocks of cognitive reading items or one 50-minute reading block (in lieu of the two

25-minute blocks). In addition, each student was administered a block of background questions, a block

of reading-related background questions, and a block of questions concerning the students motivation

and his or her perception of the difficulty of the cognitive items. The background and motivational blocks

were common to all reading booklets for a particular grade level. Eight (grade 4) or nine (grade 8 and

grade 12) 25-minute blocks of reading items were administered at each grade level. As described in

Chapter 2, the 25-minute blocks were combined into booklets according to a partially balanced

incomplete block (PBIB) design. See Chapter 14 for more information about the blocks and booklets.

Fifty-minute reading blocks were presented to the older students, one at grade 8 and two at grade 12. The
271

50-minute blocks were closely examined to ensure the appropriateness of including them with the shorter

blocks in the scaling.2

For each grade, more than 80 percent of the items in the main assessment were identical to items

in the 1994 main assessment. These items occurred in intact blocks, and provided the common

information needed to establish the short-term trend. Table 15-2 gives the blocks and numbers of items

common across assessment years.

Table 15-2

1998 Reading Blocks and Items Common to the 1992 and 1994 Assessments

Sample

New

Blocks

Common Blocks to 1994

(Number of Common Items)

Common Blocks to 1992 and 1994

(Number of Common Items)

4 [Reading–Main] and

4 [Reading–State]

R3 R4, R5, R6, R7,

R8, R9, R10; (73)

R4, R5, R6,

R7, R10; (55)

8 [Reading–Main] and

8 [Reading–State]

R3, R8 R4, R5, R6, R7, R9,

R10, R11, R13*

; (90)

R5, R6, R7,

R10, R11; (60)

12 [Reading–Main] R3 R4, R5, R6, R7, R8, R9,

R10, R11, R13*

, R14*; (111)

R4, R6, R7, R10,

R11, R13*; (78)

* 50-minute block

The total number of scaled items was 82, 110, and 118, respectively, for grades 4, 8, and 12.

Note that some items overlap across grade. Table 15-3 shows the numbers of items within reading

purpose subscales for each grade. The numbers presented in Table 15-3 show item counts both for the

original item pool, and after the necessary adjustments were made during scaling (see Section 16.3.2.1).

Table 15-3

Number of Items in Subscales in the Reading Main Assessment, by Reading Purposes

Grade

Literary

Experience

Gain

Information

Perform

a Task Total

 4 Prescaling

Postscaling

41

41

41

41

—

—

82

82

 8 Prescaling

Postscaling

29

29

48

48

33

33

110

110

12 Prescaling

Postscaling

27

27

56

55

36

36

119

118

The composition of each block of items by item type is given in Tables 15-4, 15-6, and 15-8.

Common labeling of these blocks across grade levels does not necessarily denote common items (e.g.,

Block R4 at grade 4 does not contain the same items as Block R4 at grade 12). During scaling, some

items received specific treatment (for details see Section 16.3). As a result, the composition of each block

 

2

 These analyses were identical to those described in Assessing Some of the Properties of Longer Blocks in the 1992 NAEP

Reading Assessment (Donoghue & Mazzeo, 1995). Additional comparisons based on bootstrap comparisons (Donoghue, 1995)

further supported the comparability of the 25- and 50-minute reading blocks.
272

of items by item type might changed. Tables 15-5, 15-7, and 15-9 present the final block composition by

item type as defined after scaling.

Table 15-4

1998 NAEP Reading Block Composition by Purpose for Reading and Item Type

As Defined Before Scaling, Grade 4

Constructed-Response Items

Block

Purpose for

Reading

MultipleChoice Items 2-category* 3-category 4-categoryTotal

Items

Total 36 27 11 8 82

R3 Literary 3 3 2 1 9

R4 Literary 5 6 0 1 12

R5 Literary 7 3 0 1 11

R6 Information 5 4 0 1 10

R7 Information 4 5 0 1 10

R8 Information 3 0 5 1 9

R9 Literary 3 1 4 1 9

R10 Information 6 5 0 1 12

* For a small number of constructed-response items, adjacent categories were combined.

Table 15-5

1998 NAEP Reading Block Composition by Purpose for Reading and Item Type

As Defined After Scaling, Grade 4

Constructed-Response Items

Block

Purpose for

Reading

MultipleChoice Items 2-category* 3-category 4-categoryTotal

Items

Total 36 27 13 6 82

R3 Literary 3 3 2 1 9

R4 Literary 5 6 1 0 12

R5 Literary 7 3 0 1 11

R6 Information 5 4 0 1 10

R7 Information 4 5 0 1 10

R8 Information 3 0 6 0 9

R9 Literary 3 1 4 1 9

R10 Information 6 5 0 1 12

* For a small number of constructed-response items, adjacent categories were combined.
273

Table 15-6

1998 NAEP Reading Block Composition by Purpose for Reading and Item Type

As Defined Before Scaling, Grade 8

Constructed-Response Items

Block

Purpose for

Reading

MultipleChoice Items 2-category* 3-category 4-categoryTotal

Items

Total 41 32 25 12 110

R3 Literary 3 2 4 1 10

R4 Literary 1 1 5 1 8

R5 Literary 7 3 0 1 11

R6 Information 5 5 0 2 12

R7 Information 6 6 0 1 13

R8 Information 4 1 4 1 10

R9 Task 4 0 5 0 9

R10 Task 4 6 0 2 12

R11 Task 3 8 0 1 12

R13 Information 4 0 7 2 13

* For a small number of constructed-response items, adjacent categories were combined.

Table 15-7

1998 NAEP Reading Block Composition by Purpose for Reading and Item Type

As Defined After Scaling, Grade 8

Constructed-Response Items

Block

Purpose for

Reading

MultipleChoice Items 2-category* 3-category 4-categoryTotal

Items

Total 41 35 25 9 110

R3 Literary 3 3 3 1 10

R4 Literary 1 1 5 1 8

R5 Literary 7 3 0 1 11

R6 Information 5 5 0 2 12

R7 Information 6 6 0 1 13

R8 Information 4 1 4 1 10

R9 Task 4 1 4 0 9

R10 Task 4 7 1 0 12

R11 Task 3 8 1 0 12

R13 Information 4 0 7 2 13

* For a small number of constructed-response items, adjacent categories were combined.
274

Table 15-8

1998 NAEP Reading Block Composition by Purpose for Reading and Item Type

As Defined Before Scaling, Grade 12

Constructed-Response Items

Block

Purpose for

Reading

MultipleChoice Items 2-category* 3-category 4-categoryTotal

Items

Total 43 35 28 13 119

R3 Literary 3 2 4 1 10

R4 Literary 3 5 0 1 9

R5 Literary 1 0 6 1 8

R6 Information 5 5 0 2 12

R7 Information 5 6 0 1 12

R8 Information 1 0 6 1 8

R9 Task 4 0 5 0 9

R10 Task 4 6 0 2 12

R11 Task 7 7 0 1 15

R13 Information 10 4 0 2 16

R14 Information 0 0 7 1 8

* For a small number of constructed-response items, adjacent categories were combined.

Table 15-9

1998 NAEP Reading Block Composition by Purpose for Reading and Item Type

As Defined After Scaling, Grade 12

Constructed-Response Items

Block

Purpose for

Reading

MultipleChoice Items 2-category* 3-category 4-categoryTotal

Items

Total 43 39 28 8 118

R3 Literary 3 3 3 1 10

R4 Literary 3 5 1 0 9

R5 Literary 1 0 6 1 8

R6 Information 5 5 0 2 12

R7 Information 5 7 0 0 12

R8 Information 1 0 6 1 8

R9 Task 4 1 4 0 9

R10 Task 4 7 1 0 12

R11 Task 7 7 1 0 15

R13 Information 10 4 0 2 16

R14 Information 0 0 6 1 7

* For a small number of constructed-response items, adjacent categories were combined.
275

To ensure the quality of the administration in the state assessment, the sampling contractor

Westat monitored some of the sampled schools. As described in Chapter 5, a randomly selected portion

of the administration sessions within each jurisdiction were observed by Westat-trained quality control

monitors. Thus, within and across jurisdictions, randomly equivalent samples of students received each

block of items under monitored and unmonitored administration conditions. For most jurisdictions the

monitored rate was about 25 percent of the schools. Since Kansas was new to the state assessment, 50

percent of the sessions were monitored.

15.3 SCORING CONSTRUCTED-RESPONSE ITEMS

A block consisted of one or two reading passages, each followed by several items. In addition to

multiple-choice items, each block contained a number of constructed-response items, accounting for well

over half of the testing time. Constructed-response items were scored by specially trained readers

(described in Chapter 7). Some of the constructed-response items required only a few sentences or a

paragraph response. These short constructed-response items were scored dichotomously as correct or

incorrect. Other constructed-response items required somewhat more elaborated responses, and were

scored polytomously on a 3-point (0–2) scale:

0 = Unsatisfactory (and omit)

1 = Partial

2 = Complete

In addition, most blocks (except one) contained at least one constructed-response item that

required a more in-depth, elaborated response. These items were scored polytomously on a 4-point (0-3)

scale:

0 = Unsatisfactory (and omit)

1 = Partial

2 = Essential

3 = Extensive, which demonstrates more in-depth understanding

Originally, the scoring guides for 3-point constructed-response items and 4-point constructedresponse items separated the unsatisfactory from the omit responses, with omits and off-task

responses forming a category below the unsatisfactory responses (the treatment of items that were not

reached is discussed below in Section 16.2.1). During the 1992 scaling process, it was discovered that

this scoring rule resulted in unexpectedly poor fit to the IRT model. After much investigation, the 0

category (omitted and off-task responses) was recoded. Off-task responses were treated as not

administered for each of the items, and omitted responses were combined with the next lowest category,

unsatisfactory. For new items (administered for the first time in 1998), decisions concerning the

treatment of omit and off-task responses were reexamined and found to be appropriate for these new

items.

In addition, adjacent categories of a small number of constructed-response items were combined

(collapsed). These changes were made so that the scaling model used for these items fit the data more

closely, and are described more fully in Section 16.3.2.2. Some of the short-term trend items had been

collapsed in the original 1994 scaling. These items were collapsed in an identical manner for the 1998

assessment. New items (unique to 1998) were also examined, and where necessary, adjacent categories

were collapsed.
276

Reliability of constructed-response scoring was calculated within year (1998) and across years

(1994 and 1998). Interrater and trend scoring reliability information is provided in Appendix C.

15.4 DIF ANALYSIS

A differential item functioning (DIF) analysis of new items (administered for the first time in

1998) was done to identify potentially biased items that were differentially difficult for members of

various subgroups with comparable overall scores. Sample sizes were large enough to compare male and

female students, White and Black students, and White and Hispanic students. Appendix A specifies the

sample size for each of these groups (see Table A-7). The purpose of these analyses was to identify items

that should be examined more closely by a committee of trained test developers and subject-matter

specialists for possible bias and consequent exclusion from the assessment. The presence of DIF in an

item means that the item is differentially harder for one group of students than another, while controlling

for the ability level of the students. DIF analyses were conducted separately by grade for national

samples.

A similar DIF analysis was not conducted on the state data, since the results of the national DIF

analysis were assumed to hold for the state sample. However, DIF analyses were carried out on 1998

state reading samples at both grade 4 and grade 8 to check items that were not differentially difficult for

students between public and nonpublic schools with comparable overall scores. (The nonpublic-school

population that was sampled included students from Catholic schools, private religious schools, and

private nonreligious schools [all referred to by the term “nonpublic schools”].) Since the participation of

nonpublic schools was less than public schools, the data included in the scaling process were only those

from public schools. The results of DIF analyses were used to examine the appropriateness of the

parameters of IRT models, based on public-school data, for the nonpublic-school data.

For dichotomous items, the Mantel-Haenszel procedure as adapted by Holland and Thayer

(1988) was used as a test of DIF (this is described in Chapter 9). The Mantel procedure (Mantel, 1963) as

described by Zwick, Donoghue, and Grima (1993) was used for detection of DIF in polytomous items.

This procedure assumes that item scores are appropriately treated as ordered categories. SIBTEST

(Shealy & Stout, 1993) was also used in the DIF analyses for the first time in NAEP.

For dichotomous items, the DIF index generated by the Mantel-Haenszel procedure is used to

place items into one of three categories: A, B, or C. A items exhibit little or no evidence of DIF,

while C items exhibit a strong indication of DIF and should be examined more closely. Positive values

of the index indicate items that are differentially easier for the focal group (female, Black, or Hispanic

students) than for the reference group (male or White students). Similarly, negative values indicate

items that are differentially harder for the focal group than for the reference group. An item that was

classified as a C item in any analysis was considered to be a C item. For details, see Section 9.3.4.

For polytomous items (regular constructed-response items and extended constructed-response

items), the Mantel statistic provides a statistical test of the hypothesis of no DIF. A categorization similar

to that described for dichotomous items was developed to classify items (this is discussed in detail in

Donoghue, 2000). Polytomous items were placed into one of three categories: “AA”, “BB”, or “CC”

similar to dichotomous items. “AA” items exhibit no DIF, while CC items exhibit a strong indication of

DIF and should be examined more closely. The classification criterion for polytomous items is presented

in Donoghue (2000). As with dichotomous items, positive values of the index indicate items that are

differentially easier for the focal group (female, Black, or Hispanic students) than for the reference

group (male or White students). Similarly, negative values indicate items that are differentially harder for

the focal group than for the reference group. An item that was classified as a CC item in any analysis

was considered to be a CC item.
277

For the national samples, Table 15-10 summarizes the results of DIF analyses for dichotomously

scored items in the new blocks. One C item as showing significant DIF in favor of male students was

identified in grade 8 by the Mantel-Haenszel procedure.

Table 15-10

DIF Category for National Samples by Grade for Dichotomous Items

DIF Analysis

Grade Category* Male/Female White/Black White/Hispanic

 4 CBAA+

B+

C+

 0

0

5

1

0

0

0

0

4

1

1

0

0

0

4

1

1

0

 8 CBAA+

B+

C+

1

0

5

4

0

0

0

0

5

5

0

0

0

0

6

4

0

0

12 CBAA+

B+

C+

0

0

5

0

0

0

0

1

1

2

1

0

0

0

1

4

0

0

* Positive values of the index indicate items that are differentially easier for the focal group (female,

Black, or Hispanic students) than for the reference groups (male or White students). “A+” or “A-”

means no indication of DIF, “B+” means a weak indication of DIF in favor of the focal group, “B-”

means a weak indication of DIF in favor of the reference group, and “C+” or “C-” means a strong

indication of DIF.

Table 15-11 summarizes the results of DIF analyses for polytomously scored items. No CC

item was identified in the new blocks by the Mantel procedure. The only item that SIBTEST flagged as

showing significant DIF is exactly the C item identified by the MH procedure. An independent

reviewer examined the C item whose DIF statistics indicate that it favors males. The reviewer found no

reason for its being biased for or against any group. Therefore, this item was not removed from scaling

due to DIF.

In the analysis of DIF between public and nonpublic schools for the state assessment, Table 15-

12 summarizes the results for dichotomous items. The focal group consists of students from nonpublic

schools. Positive values indicate items that were differentially easier for the focal group. Table 15-13

summarizes the results for polytomous items. As for dichotomous items, the focal group consists of

students from nonpublic schools and positive values indicate that the item was differentially easier for

the focal group. To aid in interpreting the results for polytomous items, the standardized mean difference

between focal and reference groups was produced. This statistic was rescaled by dividing the

standardized mean differences by the standard deviation of the respective item. The description of this

procedure can be found in Chapter 12. For polytomous items, a standardized mean difference ratio of .25

or greater (coupled with a significant Mantel statistic) was considered a strong indication of DIF. It can
278

be shown that standardized mean difference ratios of .25 are at least as extreme as Mantel-Haenszel

statistics corresponding to “C” items (Donoghue, 1998a).

Table 15-11

DIF Category for National Samples by Grade for Polytomous Items

DIF Analysis

Grade Category* Male/Female White/Black White/Hispanic

4 CCBBAAAA+

BB+

CC+

0

0

2

1

0

0

0

0

2

1

0

0

0

0

0

3

0

0

 8 CCBBAAAA+

BB+

CC+

0

0

5

5

0

0

0

0

3

6

1

0

0

1

2

7

0

0

12 CCBBAAAA+

BB+

CC+

0

0

2

3

0

0

0

0

3

1

1

0

0

1

2

2

0

0

* Positive values of the index indicate items that are differentially easier for the focal group

(female, Black, or Hispanic students) than for the reference groups (male or White students).

“AA+” or “AA-” means no indication of DIF, “BB+” means a weak indication of DIF in favor of

the focal group, “BB-” means a weak indication of DIF in favor of the reference group, and

“CC+” or “CC-” means a strong indication of DIF.

For the dichotomous items, at grade 4, there were 82 items analyzed from two scales and, at

grade 8, there were 110 items from three scales. Table 15-12 gives the number of items in each of six

categories (C+, B+, A+, A-, B-, C-) for the comparison. No dichotomous items were classified as “C”

items for any of the analyses for both fourth- and eighth-grade state reading assessment data. All the

dichotomous items were classified as A+ or A- in the comparisons.
279

Table 15-12

The Category of DIF between Public and Nonpublic Schools

for State Samples, by Grade for Dichotomous Items

DIF Analysis

Grade Category* Public/Nonpublic

 4 CBAA+

B+

C+

0

0

33

30

0

0

 8 CBAA+

B+

C+

0

0

33

40

0

0

* Positive values of the index indicate items that are differentially easier for

the focal group (nonpublic) than for the reference groups (public). “A+” or

“A-” means no indication of DIF, “B+” means a weak indication of DIF in

favor of the focal group, “B-” means a weak indication of DIF in favor of the

reference group, and “C+” or “C-” means a strong indication of DIF.

For the polytomous items, there were 19 polytomous from grade 4 and 37 items from grade 8.

Table 15-13 is in a format similar to that of Table 15-12, showing items in six categories (CC+, BB+,

AA+, AA-, BB-, CC-). All the polytomous items were classified as “AA” for the analyses for both

fourth- and eighth-grade state reading assessment data; no polytomous items were classified as “BB” or

“CC” items.

Because no DIF items were found in the public and nonpublic comparisons for both fourth- and

eighth-grade data, the results of IRT scaling, based on public-school data, were applied to nonpublicschool data.
280

Table 15-13

The Category of DIF between Public and Nonpublic Schools

for State Samples, by Grade for Polytomous Items

DIF Analysis

Grade Category* Public/Nonpublic

 4 CCBBAAAA+

BB+

CC+

0

0

9

10

0

0

 8 CCBBAAAA+

BB+

CC+

0

0

25

12

0

0

* Positive values of the index indicate items that are differentially easier for the

focal group (nonpublic) than for the reference groups (public). “AA+” or “AA-

” means no indication of DIF, “BB+” means a weak indication of DIF in favor

of the focal group, “BB-“ means a weak indication of DIF in favor of the

reference group, and “CC+” or “CC-” means a strong indication of DIF.

15.5 THE WEIGHT FILES

For the 1998 reading assessments, Westat produced files of final student and school weights and

corresponding replicate weights for both national and state samples. Information for the creation of the

weight files was supplied by National Computer Systems (NCS) under the direction of Educational

Testing Service (ETS). Because both the national and state samples were split into two subsamples, one

using the revised inclusion rules for SD/LEP students (S2) and one using the revised inclusion rules and

accommodations for SD/LEP students (S3), the weighting process was more complex than in previous

assessments. Westat provided student files and school files to ETS for the assessments.

The student weight files contained one record for every student who was not classified as SD or

LEP; the weight files contained two records for every student who was classified as SD or LEP. Each

record had a full set of weights, including replicate weights. The first set of weights for the SD and LEP

students is to be used when estimating results for either S2 or S3 alone. The second set of weights

provided for those students is to be used when estimating results for students from both S2 and S3

together. (See Chapters 3 and 10 for more information about the sampling and weighting procedures for

the S2 and S3 samples.)

From the student weight files, ETS constructed three sets of student weights, called modular

weights, reporting weights, and all-inclusive weights. The modular weights were used when examining

S2 and S3 separately, or for comparing S2 to S3. The reporting weights, used for most reports, were used

when reporting results for the students in reading who were not classified as being SD or LEP in both S2

and S3 and the students classified as SD or LEP from S2 only. The reporting sample was formed so that

unbiased estimation and valid comparisons with previous NAEP assessments could be made. The

SD/LEP students were divided into two types, those who were assessed and those who could not be

assessed (called excluded students). The all-inclusive weights were used for estimating results for both

S2 and S3 together.
281

The reporting weights were formed from the student weight files by taking the records for

students not classified as SD or LEP, the first record in the weight file for students in S2 classified as SD

or LEP, and a record containing a missing value code for the students in S3 classified as SD or LEP. In

this way, the old inclusion rules used with the students classified as SD or LEP in S3 would not affect the

reading results of the 1998 state assessment. For the modular weights, all students approximately from

that sample (S2 or S3) not classified as SD or LEP had their final and replicate weights proportionally

increased (doubled), while the first record in the weight file for each SD/LEP student from the

appropriate sample (S2 or S3) was selected directly from the student weight files. It is important to note

that the samples should be separated into the S2 and S3 subsamples when using weights generated in this

way. To analyze data from S2 and S3 together, the all-inclusive weights should be used. They were

created from the student weight files by taking the records for the students not classified as SD or LEP,

and the second records for all students classified as SD or LEP.

For the reporting sample for the state assessments, two other weights were created. These are

called “house weights” and “senate weights.” As with the respective branches of Congress, these weights

represent jurisdictions in two different ways. The house weights weight the student records within a

jurisdiction so that the sum of the weights for each jurisdiction is proportional to the fraction of the

national in-grade enrollment in that jurisdiction. The senate weights weight the student records within a

jurisdiction so that the sums of the weights for each jurisdiction are approximately equal to each other. In

other words, a jurisdiction like California, with many eighth-grade students, and a jurisdiction like Rhode

Island, with fewer eighth-grade students, would have equal weight when all of the state assessment data

are combined. Both of these sets of weights are constructed only for the reporting sample. The reporting

sample and either the house or senate weights are used during scaling, conditioning, and all major

reporting.

The house weight is the student’s reporting weight times a factor, which is the number of publicschool students sampled over the sum of the reporting weights of the public-school students in all the

jurisdictions. The senate weight is calculated for each jurisdiction separately. Within each jurisdiction a

factor, which is 2,500 divided by the sum of the reporting weights of the jurisdiction’s public-school

students, is computed. (In previous state assessments, 2,000 was used.) The reporting weights for

students in both public and nonpublic schools are multiplied by this factor to create the senate weights.

For DoDEA/DDESS3

 and DoDEA/DoDDS4 jurisdictions, all schools were considered public in the

calculation of these factors.

Accordingly, there are three sets of weights (modular, reporting, and all-inclusive weights) for

the national assessments and, for the state assessments, there are five sets of weights (modular, reporting,

house, senate, and all-inclusive weights). Each set of weights has replicate weights associated with it.

Replicate weights are used to estimate jackknife standard errors for each statistic estimated.

In addition to student weights, school weights are available for use in school-level analyses.

These weights are modular weights for use when examining S2 and S3 separately or for comparing S2 to

S3. No other school weights are available. School-level statistics should be calculated on the basis of S2

or S3 subsamples, as opposed to the reporting sample. If school-level statistics are calculated for the

reporting sample, biases might occur.

 

3 Department of Defense Education Activity /Department of Defense Elementary and Secondary Schools (DoDEA/DDESS)

comprise the NAEP jurisdiction for domestic Department of Defense schools.

4 Department of Defense Education Activity /Department of Defense Dependents Schools (DoDEA/DoDDS) comprise the NAEP

jurisdiction for overseas Department of Defense schools.

283

Chapter 16

DATA ANALYSIS OF THE NATIONAL READING ASSESSMENT1

Jinming Zhang, Steven P. Isham, and Lois H. Worthington

Educational Testing Service

16.1 INTRODUCTION

This chapter describes the analyses performed on the responses to the cognitive and background

items in the 1998 national assessment of reading. These analyses led to the results presented in Chapters

1 through 4 of the NAEP 1998 Reading: Report Card for the Nation and the States (Donahue et al.,

1999). The emphasis of this chapter is on the methods and results of procedures used to develop the IRTbased scale scores that formed the basis of these chapters in that report. However, some attention is given

to the analysis of constructed-response items as reported in the NAEP 1998 Reading: Report Card for the

Nation and the States. The theoretical underpinnings of the IRT and plausible values methodology

described in this chapter are given in Chapter 12, and several of the statistics are described in Chapter 9.

The major analysis components are discussed in turn. Some aspects of the analysis, such as

procedures for item analysis, scoring of constructed-response items, and methods of scaling, are

described in previous chapters and are therefore not detailed here. There were five major steps in the

analysis of the reading data, each of which is described in a separate section:

1. Conventional item and test analyses (Section 16.2.1)

2. Item response theory (IRT) scaling (Section 16.3)

3. Estimation of national and subgroup scale score distributions based on the

“plausible values” methodology (Section 16.4)

4. Transformation of the purposes-for-reading scales to the 1994 scale score

metric (Section 16.5)

5. Creation of the reading composite scale (Section 16.5.2)

Section 16.6 describes the results of partitioning the error variance; 16.7 discusses the matching

of student responses to those of their teachers.

16.2 NATIONAL ITEM ANALYSES

16.2.1 Conventional Item and Test Analyses

This section contains a detailed description of the conventional item analysis performed on the

national reading data. This analysis was done within block so that a student’s score is the sum of item

scores in a block. In forming the block total score, dichotomous items (multiple-choice and 2-category

constructed-response items) were scored as right or wrong; polytomous items were not scored as right or

wrong but were scored with three or more categories reflecting several degrees of knowledge.

 

1

 Jinming Zhang was the primary person responsible for the planning, specification, and coordination of the national reading

analyses. Computing activities for all reading scaling and data analyses were directed by Steven P. Isham and completed by Lois

H. Worthington. Others contributing to the analysis of reading data were David S. Freund, Bruce A. Kaplan, Norma A. Norris,

and Katharine E. Pashley.
284

Tables 16-1, 16-2, and 16-3 show the number of items in the block, the average weighted item

score, average weighted polyserial correlation, and the weighted alpha reliability for each block

administered. These statistics are described in Chapter 9. These values were calculated for the items

within each block used in the scaling process. The tables also give the number of students who were

administered the block and the percentage of students not reaching the last item in the block. These

numbers include only those students who contributed to the summary statistics provided in the NAEP

1998 Reading: Report Card for the Nation and the States, Chapter 1 through Chapter 4. Student weights

were used for all statistics, except for the sample sizes. The results for the blocks administered to each

grade level indicate that the blocks differ in number of items, average difficulty, reliability, and percent

not reaching the last item, and so are not parallel to each other. Preliminary item analyses for all items

within a block were completed before scaling; however, the results shown here indicate the

characteristics of the items that contributed to the final scale, and reflect decisions made in scaling to

combine adjacent categories (collapse) for a small number of items.

As described in Chapter 12, in NAEP analyses (both conventional and IRT-based), a distinction

is made between missing responses at the end of each block (not reached) and missing responses prior to

the last observed response (omitted). Items that were not reached were treated as if they had not been

presented to the examinee, while omitted items were regarded as incorrect. The proportion of students

attempting the last item of a block (or, equivalently, one minus the proportion not reaching the last item)

is often used as an index of the degree of speededness of the block of items.

Standard practice at ETS is to treat all nonrespondents to the last item as if they had not reached

the item. For multiple-choice items, short constructed-response items, and regular constructed-response

items (3-category), this convention produced a reasonable pattern of results, in that the proportion

reaching the last item does not differ markedly from the proportion attempting the next-to-last item.

However, for the blocks that ended with extended constructed-response items (4-category), this

convention resulted in an implausibly large drop in the number of students attempting the final item.

Therefore, for blocks that ended with an extended constructed-response item, students who attempted the

next-to-last item but did not respond to the last item were classified as having intentionally omitted that

item. Therefore, this item was regarded as incorrect.

The results in Tables 16-1 to 16-3 indicate that the difficulty and internal consistency of the

blocks varied. Such variability is expected, because the blocks were not constructed to be parallel. Based

on the proportion of students attempting the last item, all of the blocks appear to be somewhat speeded.

This effect is larger for grade 4 than for the other grades.

Small but consistent differences were noted based on whether a block appeared first or second

within a booklet. When the block appeared first in the booklet, the average item score tended to be higher

and the average polyserial correlation tended to be lower. The largest differences were noted in the

proportion of students not attempting the last item in the block; more students attempted the last item

when the block appeared in the second position. It appears that students learned to pace themselves

through the second block, based on their experience with the first block. Recall that the design of the

reading assessment is not completely balanced. Thus, when these serial position effects were first

noticed, it was feared that they might adversely affect the results of the IRT scaling. As part of the

analysis of the 1992 reading assessment, a special study was completed to examine the effects of the

serial position differences. The serial position effects were found to have minimal results on the scaling,

most likely due to the balance of the partial BIB design of the booklets. The effects portrayed in Tables

16-1 through 16-3 are similar in size to the effects observed in the 1992 reading assessment, and were

therefore unlikely to produce adverse effects on the final IRT scaling.
Table 16-1

Descriptive Statistics for Item Blocks by Position Within Test Booklet and Overall

Occurrences for the National Main Reading Sample, Grade 4, As Defined After Scaling

Statistic Position R3 R4 R5 R6 R7 R8 R9 Number of Scaled Items 9 12 11 10 10 9 9 Unweighted Sample Size First

SecondBoth

952971

1,923

949945

1,894

960929

1,889

961959

1,920

942933

1,875

962944

1,906

964942

1,906

Weighted Average Item Score First

SecondBoth

.49.47.48

.64.63.64

.48.43.45

.59.57.58

.45.41.43

.52.49.51

.62.61.61

Weighted Average R-Polyserial First

SecondBoth

.64.65.64

.68.68.68

.63.63.63

.60.62.61

.68.69.68

.63.65.64

.62.67.64

Weighted Alpha Reliability First

SecondBoth

.69.69.69

.80.79.80

.76.73.75

.71.71.71

.74.74.74

.72.74.73

.76.76.76

Weighted Proportion ofStudents Attempting Last Item

FirstSecondBoth

.67.82.75

.61.73.67

.76.82.79

.72.84.78

.60.75.67

.71.79.75

.65.82.74
286

Table 16-2

Descriptive Statistics for Item Blocks by Position Within Test Booklet and Overall

 Occurrences for the National Main Reading Sample, Grade 8, As Defined After Scaling

Statistic Position R3 R4 R5 R6 R7 R8 R9 R10 

Number of Scaled Items 10 8 11 12 13 10 9 12 Unweighted Sample Size First

SecondBoth

986999

1,985

968

1,0061,974

1,0351,0002,035

1,034

994

2,028

996

1,0042,000

1,016

991

2,007

989

1,0372,026

1,01

96

1,977

Weighted Average Item Score First

SecondBoth

.43.41.42

.45.41.43

.67.67.67

.57.54.55

.69.66.68

.49.47.48

.61.60.61

.61.59.60

Weighted Average R-Polyserial First

SecondBoth

.68.69.68

.61.64.63

.73.70.71

.65.64.65

.70.72.71

.59

65

.62

.69.69.69

.61.62.62

Weighted Alpha Reliability First

SecondBoth

.76.76.76

.67.71.70

.77.75.76

.72.72.72

.79.80.79

.66.74.70

.70.73.72

.73.71.72

Weighted Proportion ofStudents Attempting Last Item

FirstSecondBoth

.79.83.81

.65.72.68

.94.95.95

.85.87.86

.85.87.86

.84.89.87

.94.94.94

.79.86.82

* A 50-minute block that comprised an entire booklet.
Table 16-3

Descriptive Statistics for Item Blocks by Position Within Test Booklet and Overall

Occurrences for the National Main Reading Sample, Grade 12, As Defined After Scaling

Statistic Position R3 R4 R5 R6 R7 R8 R9 R10 Number of Scaled Items 10 9 8 12 12 8 9 12 Unweighted Sample Size First

SecondBoth

967961

1,928

943940

1,883

940949

1,889

965949

1,914

993918

1,911

949973

1,922

965986

1,951

997953

1,950

Weighted Average Item Score First

SecondBoth

.58.56.57

.54.51.52

.46.43.44

.68.67.68

.52.52.52

.59.56.58

.75.74.75

.72.71.72

Weighted Average R-Polyserial First

SecondBoth

.69.70.70

.67.69.68

.63.66.64

.66.70.68

.54.59.57

.61.63.62

.73.76.74

.63.66.64

Weighted Alpha Reliability First

SecondBoth

.76.78.77

.66.67.66

.69.72.71

.66.69.67

.54.62.58

.69.70.70

.66.72.69

.71.73.72

Weighted Proportion ofStudents Attempting Last Item

FirstSecondBoth

.86.90.88

.65.74.70

.81.83.81

.92.91.91

.79.86.82

.87.91.89

.96.95.96

.82.89.85

* A 50-minute block that comprised an entire booklet.
288

16.2.2 Scoring the Constructed-Response Items

As indicated earlier, the reading assessment included constructed-response items. Responses to

these items were included in the scaling process. In addition, detailed analyses of the constructedresponse items were also conducted, and are summarized in the NAEP 1998 Reading: Report Card for

the Nation and the States. Chapter 7 provides the ranges for percent agreement between raters for the

items as they were originally scored. The percent agreement for the raters and Cohen’s (1968) Kappa are

given in Appendix C.

16.3 NATIONAL IRT SCALING

16.3.1 Overview of Item Parameter Estimation

In 1992, separate IRT-based scales were developed for each of the purposes for reading

identified in the reading framework. As described in Chapter 12, multiple-choice items were fit using a

3PL model. Short constructed-response items were fit using a 2PL model. Regular and extended

constructed-response items were fit using a generalized partial-credit model.

For calibration, all items that were not reached were treated as if they had not been presented to

the examinees.2 Recall that responses to regular and extended constructed-response items that were offtask were also treated as if they had not been presented. The treatment of omitted responses differed

according to the item type. Omitted responses to multiple-choice items were treated as fractionally

correct (see Chapter 9 and Mislevy & Wu, 1988, for a discussion of these conversions). Omitted

responses to short constructed-response items were treated as incorrect, and omitted responses to regular

and extended constructed-response items were assigned to the lowest category.

For each purpose of reading, three separate scalings, one for each grade sample, were conducted.

The analyses were conducted on the following samples:

• The 1998 grade 4 national main sample with the 1994 grade 4 only national sample

• The 1998 grade 8 national main sample with the 1994 grade 8 only national sample

• The 1998 grade 12 national main sample with the 1994 grade 12 only national sample

That is, item parameters were estimated using combined data from both assessment years. Items that were

administered for more than one assessment (trend items) were constrained to have equal item response

functions across assessment years. However, some items exhibited clear evidence of functioning

differently across assessment years (see discussion in Section 16.3.2.3). These items were treated as

separate items for each assessment year.

The calibration was performed using all the available examinees in the reporting sample. Student

sampling weights were used for the analysis. For scaling, sampling weights were restandardized to ensure

that each assessment year had a similar sum of weights, and so had approximately equal influence in the

calibration. Each assessment years data were treated as a sample from a separate subpopulation. Thus,

separate scale score distributions were estimated for each assessment year.

Item responses were calibrated using the BILOG/PARSCALE program. Starting values were

computed from item statistics based on the entire data set. BILOG/PARSCALE calibrations were done in

 

2 An exception to this rule was the treatment of extended constructed-response items at the end of the block. See Section 16.2.1

for a discussion.
289

two stages. At stage one, the scale score distribution of each assessment year was constrained to be

normally distributed, although the means and variances differed across assessments. The values of the

item parameters from this normal solution were then used as starting values for a second-stage estimation

run in which the scale score distribution (modeled as a separate multinomial distribution for each

assessment) was estimated concurrently with item parameters. Calibration was concluded when changes

in item parameter estimates became negligibly small.

A complexity introduced by the 50-minute blocks in reading is that those blocks of items must be

linked in some way to the shorter blocks. This is complicated by the fact that no students received the

shorter blocks in addition to the 50-minute blocks. Because the samples of students receiving each

booklet are representative of the population as a whole, it was assumed that the distribution of student

scale score was the same for the students receiving the 50-minute blocks as for the students receiving the

booklets containing the shorter blocks.

16.3.2 Evaluation of Model Fit

During and subsequent to item parameter estimation, evaluations of the fit of the IRT models

were carried out for each of the items. These evaluations were based primarily on graphical analysis.

First, model fit was evaluated by examining plots of nonmodel-based estimates of the expected

proportion correct (conditional on scale score) versus the proportion correct predicted by the estimated

item response function (see Chapter 12 and Mislevy & Sheehan, 1987, p. 302). Figure 16-1 gives an

example plot of a multiple-choice item that demonstrates good model fit, R017002, from the Reading for

Literary Experience scale at grade 4. For regular and extended constructed-response items, similar plots

were produced for each item category response function (see Chapter 12). Figure 16-2 gives an example

plot of a regular constructed-response item that demonstrates good model fit, R017104, from the Reading

for Literary Experience scale at grade 8. Items that did not fit the model received some treatment (e.g.,

recoding), or were excluded from the final scales (see the next three subsections for details). Note that

the remaining item plots in this section (Figures 16-3 through 16-7) were obtained from preliminary item

parameter calibrations. They are presented to reflect the information used to make the decisions

discussed in the text. Plots produced from the final item parameters (listed in Appendix E) were very

similar to those presented and supported the decisions made.

16.3.2.1 Items Deleted from the Final Scale

In making decisions about excluding items from the final scales, a balance was sought between

being too stringent, hence, deleting too many items and possibly damaging the content representativeness

of the pool of scaled items, and being too lenient, hence including items with model fit poor enough to

endanger the types of model-based inferences made from NAEP results. For the majority of the items, the

model fit was extremely good. Items that clearly did not fit the model were not included in the final

scales; however, a certain degree of misfit was tolerated for a number of items included in the final

scales.

At grade 12, one item from the Reading to Gain Information scale, R016603, was dropped from

the final scales due to poor fit to the IRT model in the 1994 reading assessment (See Chapter 12, The

NAEP 1994 Technical Report, Allen, Kline, & Zelenak, 1997). In the 1998 data analysis, this item was

reused to check whether it fitted a model or not, using the 1998 data. Figure 16-3 gives an IRT plot of

this item. Category 1 provides virtually no discrimination; the empirical item category response function

is essentially flat. Thus, the item was also deleted from the final scales in this analysis. As shown in

Table 16-4, this is the only item that was deleted from the final scales in the 1998 reading national data

analysis.
290

Figure 16-1

Dichotomous Item (R017002) Exhibiting Good Model Fit*

* Diamonds represent 1998 grade 4 reading assessment data. They indicate estimated

conditional probabilities obtained without assuming a specific model form; the curve indicates

the estimated item response function (IRF) assuming a logistic form.
291

Figure 16-2

Polytomous Item (R017104) Exhibiting Good Model Fit*

* Diamonds represent 1998 grade 8 reading assessment data. They indicate estimated

conditional probabilities obtained without assuming a specific model form; the curve indicates

the estimated item category response function (ICRF) using a generalized partial credit model.
292

Figure 16-3

Polytomous Item (R016603) Exhibiting Unacceptably Poor Model Fit*

* Diamonds represent 1998 grade 12 reading assessment data. They indicate estimated

conditional probabilities obtained without assuming a specific model form; the curve indicates the

estimated item category response function (ICRF) using a generalized partial credit model.
293

Figure 16-4

Polytomous Item (R017110) Exhibiting Poor Model Fit*

* Diamonds represent 1998 grade 12 reading assessment data They indicate estimated conditional

probabilities obtained without assuming a specific model form; the curve indicates the estimated item

category response function (ICRF) using a generalized partial credit model.
294

Figure 16-5

Dichotomous Item (R017110) After Collapsing Categories 1 and 2*

* Diamonds represent 1998 grade 12 reading assessment data. They indicate estimated conditional

probabilities obtained without assuming a specific model form; the curve indicates the estimated item

response function (IRF) assuming a logistic form.
295

Figure 16-6

Short-Term Trend Polytomous Item (R016210)

Demonstrating Differential Item Functioning Across Assessment Years 1994 and 1998*

* Diamonds represent 1998 grade 8 reading assessment data; circles represent 1994 grade 8

reading assessment data. They indicate estimated conditional probabilities obtained without

assuming a specific model form; the curve indicates the estimated item category response

function (ICRF) using a generalized partial credit model.
296

Figure 16-7a

Short-Term Trend Polytomous Item (R016210)

Fitting Separate Item Response Functions for Each Assessment Year*

* Diamonds represent 1998 grade 8 reading assessment data. They indicate estimated

conditional probabilities obtained without assuming a specific model form; the curve indicates

the estimated item category response function (ICRF) using a generalized partial credit model.
297

Figure 16-7b

Short-Term Trend Polytomous Item (R016210)

Fitting Separate Item Response Functions for Each Assessment Year*

* Circles represent 1994 grade 8 reading assessment data. They indicate estimated conditional

probabilities obtained without assuming a specific model form; the curve indicates the estimated

item category response function (ICRF) using a generalized partial credit model.
298

Table 16-4

Items Deleted from the Final Scaling

Scale NAEP ID Block Grade Affected Reason for Decision

Reading to Gain Information R016603 R14 12 Poor fit in 1994 and 1998

16.3.2.2 Recoded Polytomous Items

Polytomous items received special treatment (i.e., recoding) for one of two reasons. First, some

of the short-term trend items were recoded in the original 1994 scaling. These items were recoded again

for the 1998 assessment. Second, two of the new (unique to 1998) polytomous items received this

treatment in the scaling. Figure 16-4 shows one such item, R017110, from the Reading for Literary

Experience scale at grade 12.

There is a lack of fit for both the unsatisfactory and partial categories for low scale score ( < -

1.0) values. There is also a marked misfit for categories 1 and 2 in high scale score ( > 1.0) values.

Categories 1 and 2 of this item were collapsed:

0 = Unsatisfactory

1 = Partial

2 = Complete

Figure 16-5 shows the recoded version of R017110 from the final scaling. The fit is substantially

improved.

Table 16-5 lists polytomous items that were recoded for scaling in 1998.

Table 16-5

Recoding of Polytomous Items for Scaling

Scale NAEP ID Block

Grade(s)

Affected Reason for Decision Disposition

Reading for Literary Experience R012111 R4 4 Recoded in 1992 and 1994 Combine categories 0 + 1

R013506 R4 12 Recoded in 1992 and 1994 Combine categories 0 + 1

R017110 R3 8, 12 Poor fit in 1998 Combine categories 1 + 2

(dichotomize)

Reading to Gain Information R015707 R8 4 Recoded in 1994 Combine categories 2 + 3

R013706 R7 12 Recoded in 1992 and 1994 Combine categories

0 + 1, 2 + 3 (dichotomize)

Reading to Perform a Task R013004 R11 8 Recoded in 1992 and 1994 Combine categories 0 + 1

R013403 R10 8, 12 Recoded in 1992 and 1994 Combine categories 0 + 1

R013406 R10 8, 12 Recoded in 1992 and 1994 Combine categories 0 + 1,

2 + 3 (dichotomize)

R013915 R11 12 Poor fit in 1998 Combine categories 0 + 1

R016104 R9 8, 12 Recoded in 1994 Combine categories 1 + 2

(dichotomize)
299

16.3.2.3 Item Category Response Functions (ICRFs) Common Across Assessment Years

The adequacy of the assumption of a common item (category) response function across

assessment years was also evaluated. For dichotomous items, this was evaluated by comparing the

nonmodel-based expected proportions for each assessment year to the single, model-based item response

function fit by BILOG/PARSCALE. For polytomously scored items, similar plots were produced for

each item category response function (ICRF, see Chapter 12). Plots showing each assessment year’s data

separately and the common item (category) response function were then examined. Items that showed

clear evidence of functioning differently across assessments were treated as separate items for each

assessment year. As was the case with deleting items, in making decisions about scaling items separately

by assessment year, a balance was sought between being too stringent, hence, splitting too many items

and possibly damaging the common item link between the assessment years, and being too lenient, hence,

including items with model fit poor enough to endanger the model-based trend inferences.

For each short-term trend constructed-response item, a sample of approximately 600–1,000 of the

1994 responses was rescored in 1998. Most items showed an acceptably high level of exact agreement.

However, several items showed a clear trend in the disagreements. Special attention was paid to these

items in the process of scaling.

Figure 16-6 gives an example plot for an item that was split early in the process, R016210 at

grade 8. The circles represent data from the 1994 assessment, and the diamonds represent the data from

the 1998 assessment. There is a marked separation between the two sets of symbols that indicate that the

item functioned substantially differently across assessment years.

Figures 16-7a and 16-7b show the result of splitting this item. Figure 16-7a gives the ICRF fit

using only the 1998 data, and Figure 16-7b gives the ICRF fit to the 1994 data. Within each assessment

year, there is good or acceptable agreement between the curve and the plotted points.

At each grade, several items were calibrated separately for each assessment year, because these

items functioned differently across assessment years according to item plots. In addition, these items are

constructed-response items that either have relatively low rater agreement across assessment years (as

revealed in rescoring) or have relatively low rater reliabilities in the 1998 scoring. Tables 16-6 through

16-8 list the short-term trend items that were calibrated separately across assessment years. A list of the

items scaled for each of the grades, along with their final item parameter estimates, appears in

Appendix E.

Table 16-6

Grade 4 Items Scaled Separately by Assessment Years

Scale Block NAEP ID Type

Reading for Literary Experience R9 R015802

R015803

R015807

Short constructed-response

Regular constructed-response

Regular constructed-response

Reading to Gain Information R8 R015702 Regular constructed-response
300

Table 16-7

Grade 8 Items Scaled Separately by Assessment Years

Scale Block NAEP ID Type

Reading for Literary Experience R5 R012607

R012611

Extended constructed-response

Short constructed-response

Reading to Gain Information R6

R7

R13

R013212

R012711

R016210

Extended constructed-response

Short constructed-response

Extended constructed-response

Reading to Perform a Task R11 R013004 Extended constructed-response

Table 16-8

Grade 12 Items Scaled Separately by Assessment Years

Scale Block NAEP ID Type

Reading for Literary Experience R5 R016301 Regular constructed-response

R016302 Regular constructed-response

R016305 Regular constructed-response

Reading to Gain Information R6 R013207 Short constructed-response

R013211 Short constructed-response

R7 R013704 Short constructed-response

R8 R016401 Regular constructed-response

R016402 Regular constructed-response

R016405 Regular constructed-response

R13 R015514 Extended constructed-response

R14 R016602 Regular constructed-response

Reading to Perform a Task R11 R013913 Short constructed-response

16.4 GENERATION OF PLAUSIBLE VALUES

Multivariate plausible values were generated for each grade group separately using the CGROUP

program. Final student weights were used in this analysis. Reporting plans required analyses that

examined the relationships between proficiencies and a large number of background variables. The

background variables included student demographic characteristics (e.g., race/ethnicity of the student,

highest level of education attained by parents), students perceptions about reading, student behavior both

in and out of school (e.g., amount of television watched daily, amount of homework done each day), and

a variety of other aspects of the educational, social, and financial environment of the schools they

attended. For grade 4 and grade 8, information was also collected from students teachers concerning

teachers’ background, education, and instructional practices in the classroom (see Section 3.4.9).

To avoid bias in reporting results and to minimize biases in secondary analyses, it was desirable

to incorporate a large number of independent variables in the conditioning model. When expressed in

terms of contrast-coded main effects and interactions, the number of variables to be included totaled
301

1,081 for grade 4, 1,059 for age grade 8, and 568 for grade 12. The much larger numbers for grade 4 and

grade 8 reflect the number of contrasts from the teacher questionnaires.

Some of these contrasts involved relatively small numbers of individuals and some were highly

correlated with other contrasts or sets of contrasts. Given the large number of contrasts, an effort was

made to reduce the dimensionality of the predictor variables. Consistent with what was done for the 1994

reading assessment, the original background variable contrasts were standardized and transformed into a

set of linearly independent variables by extracting separate sets of principal components at each grade

level. The principal components, rather than the original variables, were used as the independent

variables in the conditioning model. The number of principal components was the number required to

account for at least 90 percent of the variance in the original contrast variables. Research based on data

from the 1990 trial state assessment in mathematics suggests that results obtained using such a subset of

components will differ only slightly from those obtained using the full set (Mazzeo, Johnson, Bowker, &

Fong, 1992). Table 16-9 contains a list of the number of principal components included in conditioning,

as well as the proportion of variance accounted for by the conditioning model for each grade.

Table 16-9

Proportion of Scale Score Variance Accounted for by the Conditioning Model

for the National Main Reading Assessment

Proportion of Scale Score Variance

Grade

Number of

Conditioning

Contrasts*

Number of

Principal

Components*

Reading for

Literary

Experience

Reading to

Gain

Information

Reading to

Perform

a Task

 4 1,081 381 .600 .610 NA

 8 1,059 380 .599 .608 .662

12 568 235 .600 .565 .589

* Excluding the constant term

For each grade, Table 16-10 provides an estimated residual variance for each purpose-for-reading

scale and the residual correlation matrix between the reading scales. The values, taken directly from the

output of the CGROUP program, are estimates of relationships between the subscales conditional on the

set of principal components included in the conditioning model. The marginal correlations between the

purpose-for-reading scales are presented in Table 16-11.
302

Table 16-10

Conditional Correlations and Variances from Conditioning (CGROUP)

Grade Scale

Reading

for Literary

Experience

Reading

to Gain

Information

Reading

to Perform

a Task

4 Reading for Literary Experience

Reading to Gain Information

1.000

0.853

—

1.000

NA

NA

Residual Variance 0.327 0.337 NA

8 Reading for Literary Experience

Reading to Gain Information

Reading to Perform a Task

1.000

0.863

0.827

—

1.000

0.868

—

—

1.000

Residual Variance 0.353 0.357 0.341

12 Reading for Literary Experience

Reading to Gain Information

Reading to Perform a Task

1.000

0.807

0.688

—

1.000

0.758

—

—

1.000

Residual Variance 0.404 0.428 0.393

Table 16-11

Marginal Correlations of Reading Scales*

Grade Scale

Reading for

Literary

Experience

Reading

to Gain

Information

Reading

to Perform

a Task

4 Reading for Literary Experience

Reading to Gain Information

1.000

0.851

—

1.000

NA

NA

8 Reading for Literary Experience

Reading to Gain Information

Reading to Perform a Task

1.000

0.858

0.837

—

1.000

0.866

—

—

1.000

12 Reading for Literary Experience

Reading to Gain Information

Reading to Perform a Task

1.000

0.861

0.797

—

1.000

0.827

—

—

1.000

* Tabled values were obtained by computing a separate Pearson correlation coefficient for each plausible value,

computing Fishers z-transformation for each value, computing the average of the transformed values, and

computing the inverse transformation of the average.

16.5 THE FINAL READING SCALES

16.5.1 Purpose-for-Reading Scales

The linear indeterminacy of the reading scale was resolved by linking the 1998 reading shortterm trend scales to previous scales. For each grade, the item parameters from the joint calibration based

on data from 1994 and 1998 were used with the 1994 data to find plausible values for the 1994 data. The

mean and standard deviation of all of the plausible values were calculated and matched to the mean and

standard deviation of all of the plausible values based on the original analysis of the 1994 data, as given
303

in earlier reports. This linking was performed separately for each of the purpose-for-reading scales using

the transformation:

scale score = A  calibrated + B

where scale score denotes values on the final transformed scale and calibrated denotes values on the original

calibration scale from BILOG/PARSCALE. The constants for the linear transformation for each scale are

given in Table 16-12.

Table 16-12

Coefficients of Linear Transformations of the Purpose-for-Reading Scales

from the Calibrating Scale Units to the Units of the Reporting Scale

Grade Scale A B

4 Reading for Literary Experience 43.17 217.25

Reading to Gain Information 42.23 213.71

8 Reading for Literary Experience 36.27 260.82

Reading to Gain Information 38.05 261.17

Reading to Perform a Task 41.37 262.68

12 Reading for Literary Experience 48.04 285.44

Reading to Gain Information 33.81 291.87

Reading to Perform a Task 39.65 286.17

16.5.2 The Composite Reading Scale

For the national assessment, a composite scale was created as an overall measure of reading

proficiency. The composite was a weighted average of plausible values on the purpose-for-reading scales

(Reading for Literary Experience, Reading to Gain Information, and, at grade 8 and grade 12, Reading to

Perform a Task). The weights for the scales were proportional to the importance assigned to each reading

purpose contained in the assessment specifications given in the Reading Framework. The percentages of

assessed time are given in Table 16-13. Weights for each reading purpose are similar to the actual

proportion of assessment time devoted to that purpose. In developing the composite scale, the weights

were applied to the plausible values for each reading purpose as expressed in terms of the final scale (i.e.,

after transformation from the provisional θ scales). Overall summary statistics for the composite scale are

given in Tables 16-14.

Table 16-13

Weighting of the Purpose-for-Reading Scales

on the Reading Composite Scale

Grade

Reading for

Literary Experience

Reading to

Gain Information

Reading to

Perform a Task

4 55% 45% Not assessed

8 40% 40% 20%

12 35% 45% 20%
304

Table 16-14

Means and Standard Deviations

on the Reading Composite Scale*

Grade Year Mean S. D.

4 1998

1994

1992

217.32

214.26

216.74

37.61

40.58

35.57

8 1998

1994

1992

263.63

259.64

260.04

34.65

36.75

35.89

12 1998

1994

1992

290.79

287.35

292.15

37.63

36.66

32.81

* Tabled values were computed separately for each plausible

value. The mean is the mean of the individual means. The

standard deviation is computed as the square root of the

average of the individual variances.

16.6 PARTITIONING OF THE ESTIMATION ERROR VARIANCE

For each grade, the variance of the final, transformed scale mean was partitioned into two parts.

This analysis yielded estimates of the proportion of error variance due to sampling students and the

proportion due to the latent nature of  These estimates are given in Table 16-15 for each purpose-forreading scale and the composite scale (for stability, the estimates of the between-imputation variance B

in Equation 12.12 are based on 100 plausible values). Additional results, including those by gender and

race/ethnicity, are presented in Appendix H.

Table 16-15

Estimation Error Variance and Related Coefficients for the National Main Reading Assessment

Proportion of Variance Due to ...

Grade Scale

Total

Estimation

Error

Variance

Student

Sampling

Latency

of 

4 Reading for Literary Experience

Reading to Gain Information

0.72

0.88

0.84

0.85

0.16

0.15

Composite 0.64 0.89 0.11

8 Reading for Literary Experience

Reading to Gain Information

Reading to Perform a Task

0.75

0.77

0.89

0.85

0.91

0.87

0.15

0.09

0.13

Composite 0.62 0.93 0.07

12 Reading for Literary Experience

Reading to Gain Information

Reading to Perform a Task

1.07

0.44

0.62

0.79

0.80

0.75

0.21

0.20

0.25

Composite 0.51 0.88 0.12
305

16.7 READING TEACHER QUESTIONNAIRES

Teachers of fourth- and eighth-grade students were surveyed about their educational background

and teaching practices. Each student’s records were matched first with his or her reading teacher, and

then with the specific classroom period. Variables derived from the questionnaire were used in the

conditioning models. An additional conditioning variable was included that indicated whether the student

had been matched with a teacher record. This contrast controlled estimates of subgroup means for

differences that exist between matched and nonmatched students. Of the 7,672 fourth-grade students in

the sample, 6,741 (88%, unweighted) were matched with teachers who answered both parts of the teacher

questionnaire, and 334 (4%, unweighted) of the students had teachers who answered only the teacher

background section of the questionnaire. For the eighth-grade sample, 8,935 of the 11,051 students (81%,

unweighted) were matched to both sections of the teacher questionnaire. An additional 935 students (8%,

unweighted) were matched with the first part of the teacher questionnaire, but could not be matched to

the appropriate classroom period. Thus, 92 percent of the fourth-graders and 89 percent of the eighthgraders were matched with at least the background information about their reading teacher.

307

Chapter 17

DATA ANALYSIS OF THE STATE READING ASSESSMENT1

Jiahe Qian, Steven P. Isham, Lois H. Worthington, and Jo-Lin Liang

Educational Testing Service

17.1 INTRODUCTION

This chapter describes the analyses used in developing the reading scales for the 1998 state

assessment of reading that was carried out at grades 4 and 8. The procedures used were similar to those

employed in the analysis of the 1992 and 1994 state assessments in reading (Allen, Mazzeo, Ip, Swinton,

Isham, & Worthington, 1995; Allen, Mazzeo, Isham, Fong, & Bowker, 1994) and are based on the

philosophical and theoretical rationale given in the previous chapter. For 1998, the NAEP reading

assessment framework incorporated a balance of knowledge and skills based on current reform reports,

exemplary curriculum guides, and research on the teaching and learning of reading. The 1998 state

assessment included the assessment of both public- and nonpublic-school students for most jurisdictions.

The NAEP report card for state assessments only presents average scale scores and achievement-level

results for public-school students selected using the 1996 inclusion rules and provided no

accommodations. The inclusion rules used are discussed in more detail in Section 1.1.

There were five major steps in the analysis of the state assessment reading data, each of which is

described in a separate section:

• Conventional item and test analyses (Section 17.2)

• Item response theory (IRT) scaling (Section 17.3)

• Estimation of state and subgroup scale score distributions based on the “plausible

values” methodology (Section 17.4)

• Linking of the 1998 state assessment scales to the corresponding scales from the

1998 national assessment (Section 17.5)

• Creation of the state assessment reading composite scale (Section 17.5)

For the context of the assessment instruments and administration procedures of the reading

assessments, see Chapters 5 and 14.

17.2 STATE ITEM AND TEST ANALYSES

For grades 4 and 8, Tables 17-1 through 17-4 contain summary statistics for each block of items

for public- and nonpublic-school sessions, respectively. (The nonpublic-school population that was

sampled included students from Catholic schools, private religious schools, and private nonreligious

schools [all referred to by the term “nonpublic schools”].) Block-level statistics are provided both overall

and by serial position of the block within booklet. To produce the tables for grade 4, data from all 44

 

1 Jiahe Qian was the primary person responsible for the planning, specification, and coordination of the state reading analyses.

Computing activities for all reading scaling and data analyses were directed by Steven P. Isham and completed by Lois H.

Worthington. Others contributing to the analysis of reading data were David S. Freund, Bruce A. Kaplan, Jo-Lin Liang, and

Katharine E. Pashley.
308

jurisdictions were aggregated and statistics were calculated using rescaled versions of the final (reporting

sample) sampling weights provided by Westat. The same processes employed the data from all 41

jurisdictions in the grade 8 assessment. The senate weights were used in item analysis and scaling

procedure (see Section 15.5). Use of the senate weights does nothing to alter the value of statistics

calculated separately within each jurisdiction. However, for statistics obtained from samples that

combine students from different jurisdictions, use of the senate weights results in a roughly equal

contribution of each jurisdiction’s data to the final value of the estimate. As discussed in Mazzeo (1991),

equal contribution of each jurisdiction’s data to the results of the IRT scaling was viewed as a desirable

outcome and the same rescaled weights were only adjusted slightly in carrying out the scaling. Hence, the

item analysis statistics for each grade shown in Tables 17-1 through 17-4 are approximately consistent

with the weighting used in scaling.

Table 17-1

Descriptive Statistics for Each Block of Items by Position Within Test Booklet and Overall*

Public Schools, Grade 4

Statistic Position R3 R4 R5 R6 R7 R8 R9 R10

Unweighted Sample

Size

First

Second

Both

12,349

12,414

24,763

12,296

12,390

24,686

12,136

12,158

24,294

12,233

12,265

24,498

12,272

12,228

24,500

12,440

12,227

24,667

12,307

12,224

24,531

12,335

12,283

24,618

First .49 .65 .46 .59 .43 .53 .62 .67

Second .47 .63 .44 .56 .42 .50 .60 .64

Average Item Score

Both .48 .64 .45 .58 .42 .51 .61 .65

First .68 .79 .73 .71 .73 .71 .75 .78

Second .70 .80 .73 .70 .74 .73 .75 .77

Weighted Alpha

Reliability

Both .69 .79 .72 .70 .73 .72 .75 .77

First .63 .67 .61 .60 .67 .61 .60 .65

Second .66 .70 .63 .62 .70 .64 .65 .67

Average R-Polyserial

Both .65 .68 .62 .61 .68 .63 .62 .66

First .70 .60 .71 .67 .59 .69 .63 .79

Second .82 .74 .84 .84 .74 .82 .78 .88

Proportion of Students

Attempting Last Item

Both .76 .67 .78 .75 .66 .75 .71 .85

* The number and types of items contained in each block are shown in Table 15-4.

Tables 17-1 through 17-4 show the number of students assigned each block of items, the average

item score, the weighted alpha reliability, the average polyserial correlation, and the proportion of

students attempting the last item in the block for each grade. The average item score for the block is the

average, over items, of the score means for each of the individual items in the block. For binary-scored

multiple-choice and constructed-response items, these score means correspond to the proportion of

students who correctly answered each item. For the extended constructed-response items, the score

means were calculated as item score mean divided by the maximum number of points possible.

 In NAEP analyses (both conventional and IRT-based), a distinction is made between missing

responses at the end of each block (i.e., missing responses subsequent to the last item the student

answered) and missing responses prior to the last observed response. Missing responses before the last

observed response are considered intentional omissions. Intentional omissions were considered “omitted”

and were treated as incorrect responses. In calculating the average score for each item, only students

classified as having been presented the item were included in the denominator of the statistic. Missing

responses at the end of the block are considered “not-reached,” and treated as if they had not been
309

presented to the student. The proportion of students attempting the last item of a block (or, equivalently,

one minus the proportion of students not reaching the last item) is often used as an index of the degree of

speededness associated with the administration of that block of items. Mislevy and Wu (1988) discussed

these conversions.

Table 17-2

Descriptive Statistics for Each Block of Items by Position Within Test Booklet and Overall*

Nonpublic Schools, Grade 4

Statistic Position R3 R4 R5 R6 R7 R8 R9 R10

Unweighted Sample Size First

Second

Both

942

965

1,907

945

954

1,899

950

941

1,891

958

951

1,909

973

965

1,938

974

968

1,942

946

944

1,890

969

957

1,926

First .57 .73 .53 .67 .52 .59 .68 .74

Second .56 .71 .54 .64 .52 .58 .66 .72

Average Item Score

Both .56 .72 .53 .66 .52 .58 .67 .73

First .57 .69 .72 .65 .71 .64 .70 .69

Second .62 .69 .69 .64 .72 .67 .67 .72

Weighted Alpha

Reliability

Both .59 .69 .70 .64 .71 .65 .68 .70

First .57 .63 .60 .56 .65 .57 .54 .60

Second .60 .64 .61 .59 .67 .61 .61 .66

Average R-Polyserial

Both .59 .64 .60 .57 .66 .59 .58 .63

First .81 .70 .80 .78 .66 .77 .73 .89

Second .88 .83 .92 .90 .83 .88 .86 .92

Proportion of Students

Attempting Last Item

Both .84 .77 .86 .84 .74 .82 .80 .90 

* The number and types of items contained in each block are shown in Table 15-4.

The average polyserial correlation is the average, over items, of the item-level polyserial

correlations (r-biserial for dichotomous items) between the item and the number-correct block score. For

each item-level r-polyserial, total block number-correct score (including the item in question, and with

students receiving zero points for all not-reached items) was used as the criterion variable for the

correlation. The number-correct score was the sum of the item scores where correct dichotomous items

are assigned 1 and correct polytomous (or multiple-category) items are assigned the score category for

the response. Data from students classified as not reaching the item were omitted from the calculation of

the statistic. As is evident from Tables 17-1 through 17-4, the difficulty and the average item-to-total

correlations of the blocks varied somewhat for each grade. Such variability was expected, since these

blocks were not created to be parallel in either difficulty or content. In general, the proportion of

nonpublic-school students reaching the last item in blocks was higher. For public-school students, only

67 percent of the fourth-graders and 69 percent of the eighth-graders receiving block R4 reached the last

item in the block. For nonpublic-school students, 77 percent of fourth-graders and 82 percent of eighthgraders receiving block R4 reached the last item in the block.
310

Table 17-3

Descriptive Statistics for Each Block of Items by Position Within Test Booklet and Overall*

Public Schools, Grade 8

Statistic Position R3 R4 R5 R6 R7 R8 R9 R10 R11

Unweighted Sample Size First

Second

Both

7,781

7,864

15,645

7,882

7,586

15,468

7,836

7,788

15,624

7,741

7,942

15,683

7,792

7,796

15,588

7,683

7,860

15,543

7,850

7,638

15,488

7,760

7,833

15,593

7,917

7,726

15,643

First .42 .44 .68 .57 .70 .49 .61 .60 .68

Second .40 .42 .66 .55 .67 .47 .60 .61 .67

Average Item Score

Both .41 .43 .67 .56 .69 .48 .60 .60 .68

First .77 .67 .74 .68 .77 .66 .69 .70 .79

Second .77 .70 .77 .71 .79 .69 .70 .72 .79

Weighted Alpha

Reliability

Both .77 .69 .75 .70 .78 .68 .70 .71 .79

First .69 .61 .69 .61 .70 .59 .68 .59 .70

Second .70 .64 .72 .64 .71 .61 .68 .61 .71

Average R-Polyserial

Both .70 .63 .71 .63 .70 .60 .68 .60 .70

First .79 .67 .95 .86 .83 .85 .95 .77 .81

Second .85 .72 .95 .86 .88 .90 .95 .84 .90

Proportion of Students

Attempting Last Item

Both .82 .69 .95 .86 .85 .88 .95 .81 .86

* The number and types of items contained in each block are shown in Table 15-6.

 Block R13 did not appear with any other cognitive block, so no information on positions is available.

These tables also indicate that there was little variability in average item scores or average

polyserial correlations for each block by serial position within the assessment booklet. The differences in

item statistics were small for items appearing in blocks in the first position and in the second position.

However, differences were consistent in their direction. Average item scores were almost always highest

when each block was presented in the first position. Average polyserial correlations were usually higher

when each block was presented in the second position. An aspect of block-level performance that did

differ noticeably by block position was the proportion of students attempting the last item in the block.

As shown in Tables 17-1 through 17-4, the percentage of the students attempting the last item increased

in the second block position. Students may have learned to pace themselves through the later block after

they had experienced the format of the first block they received. This was similar to what occurred in the

previous state reading assessments. For the 1992 state assessment, a study was completed to examine the

effect of the block position differences on scaling. Due to the partial BIB design of the booklets, those

effects were minimal.

As mentioned earlier, in an attempt to maintain rigorous standardized administration procedures

across the jurisdictions, a randomly selected 50 percent of all sessions within each jurisdiction that had

never participated in a state assessment were observed by a Westat-trained quality control monitor. In the

1998 state reading assessment, Kansas was the only new participant, and 50 percent of those sessions

were monitored. A randomly selected 25 percent of the sessions within other jurisdictions were

monitored. Observations from the monitored sessions provided information about the quality of

administration procedures and the frequency of departures from standardized procedures in the

monitored sessions (see Chapter 5 for a discussion of the substance of these observations).
311

Table 17-4

Descriptive Statistics for Each Block of Items*

by Position Within Test Booklet and Overall

Nonpublic Schools, Grade 8

Statistic Position R3 R4 R5 R6 R7 R8 R9 R10 R11

Unweighted Sample Size First

Second

Both

482

473

955

491

471

962

466

486

952

461

493

954

482

483

965

458

468

926

479

463

942

483

479

962

484

459

943

First .51 .50 .75 .65 .80 .57 .72 .69 .80

Second .50 .50 .76 .64 .79 .55 .71 .70 .79

Average Item Score

Both .51 .50 .75 .65 .79 .56 .71 .70 .79

First .71 .60 .75 .58 .65 .55 .62 .63 .71

Second .75 .60 .68 .55 .71 .59 .62 .60 .63

Weighted Alpha

Reliability

Both .73 .60 .72 .56 .68 .58 .62 .62 .67

First .64 .59 .74 .56 .68 .55 .64 .55 .66

Second .68 .58 .70 .55 .73 .57 .65 .54 .66

Average R-Polyserial

Both .66 .58 .72 .55 .70 .56 .65 .54 .66

First .83 .78 .96 .94 .92 .91 .97 .80 .90

Second .89 .85 .98 .94 .96 .94 .96 .88 .92

Proportion of Students

Attempting Last Item

Both .86 .82 .97 .94 .94 .92 .96 .84 .91 

* The number and types of items contained in each block are shown in Table15-6.

 Block R13 did not appear with any other cognitive block, so no information on positions is available.

Tables 17-5 through 17-8 provide the block-level descriptive statistics for the monitored and

unmonitored sessions. When results were aggregated over all participating jurisdictions, there was little

difference between the performance of students who attended monitored or unmonitored sessions. When

data were classified by school type, there was also little difference between the performance of students

who attended monitored or unmonitored sessions. For grade 4, the average item score over all 8 blocks

and over all 44 participating jurisdictions was 0.54 for both monitored and unmonitored public-school

sessions. The average item score was 0.62 for monitored nonpublic-school sessions and 0.62 for

unmonitored nonpublic-school sessions. For grade 8, the average item score over all 10 blocks and over

all 41 participating jurisdictions was 0.577 and 0.582 for monitored and unmonitored public-school

sessions, respectively. The average item score was 0.67 for both monitored and unmonitored nonpublicschool sessions.
312

Table 17-5

Block-Level* Descriptive Statistics for Monitored and Unmonitored Public-School Sessions, Grade 4

Statistic R3 R4 R5 R6 R7 R8 R9 R10

Unweighted Sample Size

 Unmonitored

 Monitored

18,540

6,223

18,473

6,213

18,159

6,135

18,322

6,176

18,359

6,141

18,500

6,167

18,325

6,206

18,386

6,232

Average Item Score

Unmonitored .48 .64 .45 .58 .42 .51 .61 .66

Monitored .48 .64 .45 .57 .42 .51 .61 .65

Weighted Alpha Reliability

Unmonitored .69 .79 .73 .70 .73 .72 .75 .77

Monitored .68 .80 .74 .70 .73 .73 .75 .78

Average R-Polyserial

Unmonitored .65 .68 .62 .61 .69 .63 .62 .66

Monitored .64 .69 .63 .62 .68 .63 .62 .66

Proportion of Students

Attempting Last Item

Unmonitored .77 .67 .78 .76 .67 .76 .71 .84

Monitored .74 .66 .77 .75 .65 .74 .69 .83

* The number and types of items contained in each block are shown in Table 15-4.

Table 17-6

Block-Level* Descriptive Statistics for Monitored and Unmonitored

Nonpublic-School Sessions, Grade 4

Statistic R3 R4 R5 R6 R7 R8 R9 R10

Unweighted Sample Size

 Unmonitored

 Monitored

1,372

535

1,361

538

1,345

546

1,365

544

1,382

556

1,381

561

1,342

548

1,370

556

Average Item Score 

Unmonitored .57 .72 .54 .66 .52 .58 .67 .73

Monitored .56 .72 .51 .65 .52 .59 .68 .74

Weighted Alpha Reliability

Unmonitored .59 .68 .70 .64 .70 .65 .67 .70

Monitored .60 .71 .71 .63 .75 .64 .70 .70

Average R-Polyserial

Unmonitored .58 .64 .60 .57 .64 .59 .58 .64

Monitored .60 .63 .62 .57 .70 .59 .58 .63

Proportion of Students

Attempting Last Item

Unmonitored .82 .78 .87 .84 .75 .82 .81 .91

Monitored .84 .74 .83 .82 .73 .84 .76 .90

* The number and types of items contained in each block are shown in Table 15-4.
313

Table 17-7

Block-Level* Descriptive Statistics for Monitored and Unmonitored

Public-School Sessions, Grade 8

Statistic R3 R4 R5 R6 R7 R8 R9 R10 R11 R13

Unweighted Sample Size

 Unmonitored

 Monitored

11,803

3,842

11,618

3,850

11,732

3,892

11,798

3,885

11,681

3,907

11,691

3,852

11,609

3,879

11,695

3,898

11,720

3,923

11,823

3,914

Average Item Score

Unmonitored .41 .43 .67 .55 .69 .48 .60 .60 .67 .67

Monitored .42 .43 .67 .56 .69 .49 .61 .61 .69 .67

Weighted Alpha

Reliability

Unmonitored .77 .69 .76 .70 .78 .68 .70 .71 .79 .74

Monitored .77 .67 .75 .70 .78 .67 .69 .71 .78 .73

Average R-Polyserial

Unmonitored .70 .63 .71 .63 .71 .60 .68 .60 .70 .62

Monitored .71 .62 .71 .63 .70 .60 .68 .60 .69 .60

Proportion of Students

Attempting Last Item

Unmonitored .82 .69 .95 .86 .85 .87 .94 .81 .86 .95

Monitored .83 .70 .95 .86 .86 .88 .96 .81 .85 .95

* The number and types of items contained in each block are shown in Table 15-6.

Table 17-8

Block-Level* Descriptive Statistics for Monitored and Unmonitored Nonpublic-School Sessions

Grade 8

Statistic R3 R4 R5 R6 R7 R8 R9 R10 R11 R13

Unweighted Sample Size

 Unmonitored

 Monitored

645

310

651

311

649

303

655

299

652

313

631

295

637

305

646

316

641

302

673

299

Average Item Score

Unmonitored .51 .49 .75 .64 .79 .56 .72 .70 .80 .73

Monitored .50 .52 .76 .66 .80 .58 .69 .70 .79 .74

Weighted Alpha

Reliability

Unmonitored .74 .60 .72 .57 .70 .58 .64 .62 .65 .57

Monitored .70 .59 .72 .54 .63 .55 .59 .62 .72 .53

Average R-Polyserial

Unmonitored .67 .59 .71 .56 .73 .56 .65 .55 .65 .53

Monitored .63 .56 .76 .54 .64 .55 .65 .54 .67 .46

Proportion of Students

Attempting Last Item

Unmonitored .87 .81 .97 .94 .95 .92 .96 .82 .92 .97

Monitored .83 .83 .97 .94 .92 .93 .98 .87 .89 .94 

* The number and types of items contained in each block are shown in Table 15-6.
314

Table 17-9 for grade 4 and Table 17-10 for grade 8 summarize the differences between

monitored and unmonitored average item scores for the jurisdictions. These are mean differences within

a jurisdiction averaged over all items in all blocks. The results in the tables are from combined samples

of public- and nonpublic-school data. The mean difference and median difference were close to zero. For

grade 4, 26 jurisdictions had negative differences (i.e., students from unmonitored sessions scored higher

than students from monitored sessions). None was larger in absolute magnitude than 0.029. For grade 8,

17 jurisdictions had negative differences. The largest in absolute magnitude is 0.052. The results indicate

that across jurisdictions, the differences between monitored and unmonitored sessions are relatively

small for both grades. While these tables list differences, no significance tests were done. This is true for

all the descriptive statistics in Tables 17-5 to 17-12.

As has been the case since the 1994 trial state assessment in reading, the 1998 state assessment in

reading included students sampled from nonpublic schools. Tables 17-11 and 17-12 show the difference

between public and nonpublic schools with respect to sample size, average item scores, alpha reliability,

average r-polyserial correlation, and proportion of students attempting the last item in a block. As with

the monitored/unmonitored comparisons, results were aggregated over all participating jurisdictions. For

grade 4, 43 of the 44 jurisdictions that participated in the state assessment in reading had public-school

samples and 29 of the 44 jurisdictions had nonpublic-school samples that met reporting requirements. For

grade 8, 40 of the 41 jurisdictions had public-school samples and 23 of the 41 jurisdictions had

nonpublic-school samples that met reporting requirements.

Consistent differences are evident between the public- and nonpublic-school groups. Table 17-

11, for grade 4, indicates that the difference in average item score between public- and nonpublic-school

students (i.e., public block mean minus nonpublic block mean) ranged from -.095 to -.061, with an

average of -.079, indicating that public-school students were generally lower in average item score.
315

Table 17-9

Effect of Monitoring Sessions by Jurisdiction:

Average Jurisdiction Item Scores for Monitored and Unmonitored Sessions, Grade 4

Monitored Unmonitored Monitored – Unmonitored

 Alabama 0.506 0.489 0.017

 Arizona 0.467 0.494 -0.027

 Arkansas 0.512 0.491 0.022

 California 0.459 0.473 -0.014

 Colorado 0.548 0.553 -0.005

 Connecticut 0.609 0.592 0.017

 Delaware 0.490 0.500 -0.009

 Florida 0.517 0.493 0.024

 Georgia 0.495 0.501 -0.006

 Hawaii 0.483 0.473 0.010

 Iowa 0.553 0.557 -0.004

 Kansas 0.549 0.548 0.001

 Kentucky 0.519 0.527 -0.008

 Louisiana 0.490 0.488 0.002

 Maine 0.571 0.561 0.010

 Maryland 0.539 0.538 0.001

 Massachusetts 0.584 0.569 0.015

 Michigan 0.541 0.535 0.006

 Minnesota 0.560 0.558 0.002

 Mississippi 0.468 0.473 -0.005

 Missouri 0.554 0.525 0.029

 Montana 0.550 0.571 -0.021

 Nebraska 0.561 0.608 -0.047

 Nevada 0.493 0.489 0.004

 New Hampshire 0.538 0.575 -0.036

 New Mexico 0.475 0.488 -0.013

 New York 0.523 0.533 -0.010

 North Carolina 0.505 0.535 -0.030

 Oklahoma 0.520 0.533 -0.013

 Oregon 0.517 0.515 0.002

 Rhode Island 0.546 0.545 0.001

 South Carolina 0.499 0.502 -0.002

 Tennessee 0.499 0.503 -0.004

 Texas 0.538 0.525 0.013

 Utah 0.515 0.518 -0.002

 Virginia 0.525 0.532 -0.007

 Washington 0.525 0.544 -0.019

 West Virginia 0.511 0.530 -0.019

 Wisconsin 0.551 0.566 -0.014

 Wyoming 0.529 0.539 -0.010

 District of Columbia 0.365 0.373 -0.008

 DoDEA/DDESS 0.538 0.535 0.002

 DoDEA/DoDDS 0.539 0.554 -0.016

 Virgin Islands 0.348 0.399 -0.051 

Mean -0.005

Median -0.005

Minimum -0.051

1st Quartile -0.013

3rd Quartile 0.003

Maximum 0.029 
316

Table 17-10

Effect of Monitoring Sessions by Jurisdiction:

Average Jurisdiction Item Scores for Monitored and Unmonitored Sessions, Grade 8

Monitored Unmonitored Monitored - Unmonitored

 Alabama 0.499 0.514 -0.014

 Arizona 0.545 0.541 0.004

 Arkansas 0.533 0.516 0.017

 California 0.527 0.514 0.012

 Colorado 0.567 0.559 0.008

 Connecticut 0.606 0.600 0.006

 Delaware 0.559 0.507 0.052

 Florida 0.540 0.513 0.027

 Georgia 0.533 0.534 -0.002

 Hawaii 0.510 0.480 0.031

 Kansas 0.590 0.569 0.021

 Kentucky 0.568 0.546 0.022

 Louisiana 0.513 0.521 -0.008

 Maine 0.601 0.607 -0.006

 Maryland 0.555 0.569 -0.014

 Massachusetts 0.594 0.583 0.010

 Minnesota 0.596 0.576 0.020

 Mississippi 0.509 0.487 0.022

 Missouri 0.558 0.560 -0.002

 Montana 0.584 0.594 -0.010

 Nebraska 0.640 0.627 0.014

 Nevada 0.532 0.527 0.005

 New Mexico 0.535 0.532 0.004

 New York 0.573 0.582 -0.009

 North Carolina 0.567 0.559 0.008

 Oklahoma 0.564 0.560 0.004

 Oregon 0.559 0.572 -0.012

 Rhode Island 0.588 0.560 0.028

 South Carolina 0.508 0.510 -0.002

 Tennessee 0.522 0.537 -0.014

 Texas 0.533 0.547 -0.015

 Utah 0.576 0.553 0.023

 Virginia 0.588 0.564 0.024

 Washington 0.565 0.566 -0.002

 West Virginia 0.548 0.545 0.003

 Wisconsin 0.580 0.566 0.014

 Wyoming 0.517 0.559 -0.043

 District of Columbia 0.414 0.436 -0.022

 DoDEA/DDESS 0.607 0.562 0.045

 DoDEA/DoDDS 0.567 0.583 -0.016

 Virgin Islands 0.436 0.447 -0.011 

Mean 0.005

Median 0.004

Minimum -0.043

1st Quartile -0.009

3rd Quartile 0.020

Maximum 0.052 
317

The public/nonpublic difference in average item-to-total block correlation (the average r-polyserial)

ranged from 0.017 to 0.059, with an average of 0.037, indicating that public-school students generally

had a somewhat higher item-to-total correlation. As for the proportion of students attempting the last

item, public minus nonpublic differences ranged from -.097 to -.06, with an average of -.080, indicating

that somewhat fewer students in public schools attempted the last item.

Table 17-11

Block-Level Descriptive Statistics for Overall Public- and Nonpublic-School Sessions

Grade 4

Statistic R3 R4 R5 R6 R7 R8 R9 R10

Unweighted Sample Size

Public

Nonpublic

24,763

1,907

24,686

1,899

24,294

1,891

24,498

1,909

24,500

1,938

24,667

1,942

24,531

1,890

24,618

1,926

Weighted Average Item Score

Public .48 .64 .45 .58 .42 .51 .61 .65

Nonpublic .56 .72 .53 .66 .52 .58 .67 .73

Weighted Alpha Reliability

Public .69 .79 .72 .70 .73 .72 .75 .77

Nonpublic .59 .69 .70 .64 .71 .65 .68 .70

Weighted Average R-Polyserial

Public .65 .68 .62 .61 .68 .63 .62 .66

Nonpublic .59 .64 .60 .57 .66 .59 .58 .63

Weighted Proportion of

Students Attempting Last Item

Public .76 .67 .78 .75 .66 .75 .71 .85

Nonpublic .84 .77 .86 .84 .74 .82 .80 .90 

Table 17-12

Block-Level Descriptive Statistics for Overall Public- and Nonpublic-School Sessions

Grade 8

Statistic R3 R4 R5 R6 R7 R8 R9 R10 R11 R13

Unweighted Sample Size

Public

Nonpublic

15,645

955

15,468

962

15,624

952

15,683

954

15,588

965

15,543

926

15,488

.942

15,593

962

15,643

943

15,737

972

Weighted Average Item Score

Public .41 .43 .67 .56 .69 .48 .60 .60 .68 .67

Nonpublic .51 .50 .75 .65 .79 .56 .71 .70 .79 .74

Weighted Alpha Reliability

Public .77 .69 .75 .70 .78 .68 .70 .71 .79 .74

Nonpublic .73 .60 .72 .56 .68 .58 .62 .62 .67 .56

Weighted Average R-Polyserial

Public .70 .63 .71 .63 .70 .60 .68 .60 .70 .61

Nonpublic .51 .50 .75 .65 .79 .56 .71 .70 .79 .51

Weighted Proportion of

Students Attempting Last Item

Public .82 .69 .95 .86 .85 .88 .95 .81 .86 .95

Nonpublic .86 .82 .97 .94 .93 .92 .96 .84 .91 .96
318

17.3 STATE IRT SCALING

As described in Chapter 12, separate IRT-based scales were developed using the scaling models.

For grade 4, two scales were produced by separately calibrating the sets of items classified in each of the

two content areas. For grade 8, three scales were produced in each of the three content areas.

For the reasons discussed in Mazzeo (1991), for each scale, a single set of item parameters for

each item was estimated and used for all jurisdictions. Item-parameter estimation was carried out using a

25 percent systematic random sample of the students participating in the 1998 state assessment and

included equal numbers of students from each participating jurisdiction, half from monitored sessions

and half from unmonitored sessions whenever possible. All students in the scaling sample were publicschool students. The grade 4 sample consisted of 98,873 students, with 590 students being sampled from

each of the 42 participating jurisdictions (excluding DoDEA/DDESS2 and DoDEA/DoDDS3

 schools). Of

the 590 records sampled from each jurisdiction, 295 were drawn from the monitored sessions and 295

were drawn from the unmonitored sessions. The grade 8 sample consisted of 86,210 students, with 554

students being sampled from each of the 39 participating jurisdictions. Of the 554 records sampled from

each jurisdiction, 277 were drawn from the monitored sessions and 277 were drawn from the

unmonitored sessions. In grade 8, there were less than 277 monitored students in the District of Columbia

and Virgin Islands; therefore, all the monitored students in these two jurisdictions were included. The

rescaled weights for the 25 percent sample of students used in item calibration were adjusted slightly to

ensure that (1) each jurisdiction’s data contributed equally to the estimation process, and (2) data from

monitored and unmonitored sessions contributed equally. All calibrations were carried out using the

rescaled sampling weights described in Section 11.2 in an effort to ensure that each jurisdiction’s data

contributed equally to the determination of the item-parameter estimates.

To the extent that items may have functioned differently in monitored and unmonitored sessions,

the single set of item parameters obtained defines a set of item characteristic curves “averaged over” the

two types of sessions. Tables 17-5 through 17-8 (shown earlier) presented block-level item statistics that

suggested little, if any, difference in item functioning by session type.

Only public-school data were used in the scaling models for the state assessments, since no DIF

items were found in the public versus nonpublic comparisons for both fourth- and eighth-grade data. For

details on DIF analysis, see Chapter 15, Section 15.4.

17.3.1 Item Parameter Estimation

For each content-area scale, item parameter estimates were obtained using the NAEP

BILOG/PARSCALE program, which combines Mislevy and Bock’s (1982) BILOG and Muraki and

Bock’s (1991) PARSCALE computer programs. The program uses marginal maximum likelihood

estimation procedures to estimate the parameters of the one-, two-, and three-parameter logistic models,

and the generalized partial-credit model described by Muraki (1992).

Multiple-choice items were dichotomously scored and were scaled using the three-parameter

logistic model. Omitted responses to multiple-choice items were treated as fractionally correct, with the

fraction being set to 1 over the number of response options. Short constructed-response items that were

also in the 1992 assessment were dichotomously scored and scaled using the two-parameter logistic

model. New short (regular) constructed-response items were scored on a three-point generalized partial-

 

2

 DoDEA/DDESS is the Department of Defense Education Activity Department of Defense Domestic Dependent Elementary

and Secondary Schools.

3

 DoDEA/DoDDS is the Department of Defense Education Activity Department of Defense Dependents Schools.
319

credit scale. These items appear in block 3 for grade 4, and in blocks 3 and 8 for grade 8. Omitted

responses to short constructed-response items were treated as incorrect.

There were a total of eight extended constructed-response items. Each of these items was also

scaled using the generalized partial-credit model. Four scoring levels were defined:

0 = Unsatisfactory response or omitted

1 = Partial response

2 = Essential response

3 = Extensive response

Note that omitted responses were treated as the lowest possible score level. As stated earlier, not-reached

and off-task responses were treated as if the item were not administered to the student. Table 17-13

provides a listing of the blocks, positions within the block, content-area classifications, and NAEP

identification numbers for all extended constructed-response items included in the 1998 assessment for

grade 4 and grade 8 data.

Table 17-13

Extended Constructed-Response Items, 1998 State Assessment in Reading

Grade Block

Position

in Block

Content Area

Classifications NAEP ID

4 R3 6 Literary Experience R017007

 R4 11 Literary Experience R012111

 R5 7 Literary Experience R012607

 R6 4 Gain Information R012204

 R7 8 Gain Information R012708

 R8 7 Gain Information R015707

 R9 4 Literary Experience R015804

 R10 12 Gain Information R012512

8 R3 5 Literary Experience R017105

 R4 6 Literary Experience R015906

 R5 7 Literary Experience R012607

 R6 1 Gain Information R013201

 R6 12 Gain Information R013212

 R7 8 Gain Information R012708

 R8 5 Gain Information R017205

 R13 4 Gain Information R016204 

Empirical Bayes modal estimates of all item parameters were obtained from the

BILOG/PARSCALE program. Prior distributions were imposed on item parameters with the following

starting values: thresholds, normal [0,2]; slopes, log-normal [0,.5]; and asymptotes, two-parameter beta

with parameter values determined as functions of the number of response options for an item and a

weight factor of 50. The locations (but not the dispersions) were updated at each program-estimation

cycle in accordance with provisional estimates of the item parameters.

Item parameter estimation proceeded in two phases. First, the subject ability distribution was

assumed fixed (normal [0,1]) and a stable solution was obtained. Starting values for the item parameters

were provided by item analysis routines. The parameter estimates from this initial solution were then
320

used as starting values for a subsequent set of runs in which the subject ability distribution was freed and

estimated concurrently with item parameter estimates. After each estimation cycle, the subject ability

distribution was standardized to have a mean of zero and standard deviation of one. Correspondingly,

parameter estimates for that cycle were also linearly standardized.

During and subsequent to item parameter estimation, evaluations of the fit of the IRT models

were carried out for each of the items in the item pool. These evaluations were conducted to determine

the final composition of the item pool making up the scales by identifying misfitting items that should not

be included. Evaluations of model fit were based primarily on graphical analyses. For dichotomously

scored multiple-choice and two-category response items, model fit was evaluated by examining plots of

estimates of the expected conditional (on theta) probability of a correct response that do not assume a

two-parameter or three-parameter logistic model versus the probability predicted by the estimated itemcharacteristic curve (see Mislevy & Sheehan, 1987, p. 302). For the extended constructed-response items,

similar plots were produced for each item-category characteristic curve.

As with most procedures that involve evaluating plots of data versus model predictions, a certain

degree of subjectivity is involved in determining the degree of fit necessary to justify use of the model.

There are a number of reasons why evaluation of model fit relied primarily on analyses of plots rather

than seemingly more objective procedures based on goodness-of-fit indices such as the “pseudo chisquares” produced in BILOG (Mislevy & Bock, 1982). First, when the model fits, the exact sampling

distributions of these indices are not well understood, even for fairly long tests. Mislevy and Stocking

(1989) point out that the usefulness of these indices appears particularly limited in situations like NAEP,

where examinees have been administered relatively short tests. A study by Stone, Mislevy, and Mazzeo

(1994) using simulated data suggests that the correct reference chi-square distributions for these indices

have considerably fewer degrees of freedom than the value indicated by the BILOG/PARSCALE

program, and require additional adjustments of scale. However, it is not yet clear how to estimate the

correct number of degrees of freedom and necessary scale factor adjustment factors. Consequently,

pseudo chi-square goodness-of-fit indices are used only as rough guides in interpreting the severity of

model departures.

Second, as discussed in Chapter 12, it is almost certainly the case that, for most items, item

response models hold only to a certain degree of approximation. Given the large sample sizes used in

NAEP and the state assessment, there will be sets of items for which one is almost certain to reject the

hypothesis that the model fits the data, even though departures are minimal in nature or involve kinds of

misfit unlikely to impact on important model-based inferences. In practice, one is almost always forced to

temper statistical decisions with judgments about the severity of model misfit and the potential impact of

such misfit on final results.

To maximize the agreement between the state analysis and national analysis, the 1998 state

assessment incorporated most adjustments and deletions resulting from the analysis of the 1998 national

assessment in reading.

For the large majority of the items for grade 4 and grade 8 data, the fit of the model was

extremely good. Figure 17-1 provides typical examples of what the plots look like for this class of items.

Item R012106 for grade 4 is a binary-scored constructed-response item. Item R012711 for grade 8, at the

top of Figure 17-1 (continued), is a multiple-choice item; item R013405 for grade 8, at the bottom of

Figure 17-1 (continued), is a binary-scored constructed-response item. In each plot, the x-axis indicates

scale score level (theta) and the y-axis indicates the probability of a correct response. The diamonds show

estimates of the conditional (on theta) probability of a correct response that do not assume a logistic form

(referred to subsequently as nonlogistic-based estimates). The sizes of the diamonds are proportional to

the number of students categorized as having thetas at or close to the indicated value. The solid curve

shows the estimated item response function. The item response function provides estimates of the
321

conditional probability of a correct response based on an assumed logistic form. The vertical dashed line

indicates the estimated location parameter (b) for the item and the horizontal dashed line (e.g., item

R012711) indicates the estimated lower asymptote (c). Also shown in the plot are the values of the item

parameter estimates. As is evident from the plots, the nonlogistic-based estimates of conditional

(diamonds) probabilities are in extremely close agreement with those given by the estimated item

response function (the solid curves).

Figure 17-1

Dichotomous Items (R012106, R012711, and R013405) Exhibiting Good Model Fit*

* Diamonds represent 1998 grade 4 reading assessment data. They indicate estimated

conditional probabilities obtained without assuming a specific model form; the curve indicates

the estimated item response function (IRF) assuming a logistic form.

 (continued)
322

Figure 17-1 (continued)

Dichotomous Items (R012106, R012711, and R013405) Exhibiting Good Model Fit*

* Diamonds represent 1998 grade 8 reading assessment data.

They indicate estimated conditional probabilities obtained

without assuming a specific model form; the curve indicates

the estimated item response function (IRF) assuming a

logistic form.
323

Figure 17-2 provides an example of a plot for a four-category extended constructed-response

item (R013201, grade 8) exhibiting good model fit. Like the plots for the binary items, this plot shows

two estimates of each item category characteristic curve, one set that does not assume the partial-credit

model (shown as diamonds) and one that does (the solid curves). The estimates for all parameters for the

item in question are also indicated on the plot. As shown by the figure, there is strong agreement and

only slight differences between the item category characteristic curve and the curve of diamonds at the

high categories. Although few student responses were scored in the highest category, there were adequate

data to calculate the model-based estimates for those categories (the solid curves). Such results were

typical for the extended constructed-response items.

Figure 17-2

Polytomous Item (R013201) Exhibiting Good Model Fit*

* Diamonds represent 1998 grade 8 reading assessment data. They indicate estimated

conditional probabilities obtained without assuming a specific model form; the curve indicates

the estimated item category response function (ICRF) using a generalized partial credit model.
324

17.3.2 Recoded Extended Constructed-Response Items

As discussed above, some of the items retained for the final scales display some degree of model

misfit. In general, good agreement between nonlogistic and logistic estimates of conditional probabilities

was found in the regions of the theta scale that includes most of the examinees. Misfit was confined to

conditional probabilities associated with theta values in the tails of the subject ability distributions.

For grade 4 data, item R012111, an item of Literary Experience in the eleventh position in block

R4, received special treatment in the scaling process in the 1992, 1994, and 1998 assessments. Figure 17-3

shows the plot of item R012111 before collapsing unsatisfactory and partial-response categories using

1998 assessment data.

Figure 17-3

Polytomous Item (R012111) Before Collapsing Unsatisfactory and Partial-Response Categories*

* Diamonds represent 1998 grade 4 reading assessment data. They indicate estimated conditional

probabilities obtained without assuming a specific model form; the curve indicates the estimated

item category response function (ICRF) using a generalized partial credit model.
325

To obtain a good fit of the generalized partial-credit model to the extended constructed-response

items in 1998 assessment, the categories 0 and 1 were combined and the other categories were relabeled as

in previous assessments. Therefore, the codings for the three scoring levels were defined:

0 = Unsatisfactory, partial response, or omitted

1 = Essential response

2 = Extensive response

The plot for this item for the 1998 data after collapsing the unsatisfactory and partial-response

categories is given in Figure 17-4. The figure shows good model fit, except that the nonlogistic-based

estimates tend to be somewhat different from the model-based estimates for theta values greater than 1.

Note that this item is functioning essentially as a dichotomous item due to the small frequencies in the

top category. There were enough data, however, to calculate the model-based estimates of the categorycharacteristic curve for this category (shown as the rightmost solid curve in both figures).

Another fourth-grade item, R015707, an item of Gain Information in the seventh position in

block R8, also received special treatment in the 1994 and 1998 assessments. As with item R012111, the

general partial-credit model did not fit the response to the extended constructed-response item R015707

well. This Reading to Gain Information item was treated the same way as was item R012111, and good

model-data fit was obtained.

To be consistent with the scaling of the 1998 national reading assessment for grade 8 data, item

R017110, an item of Literary Experience in the tenth position in block R3, received special treatment.

The categories 0 and 1 were combined as 0 and the other categories were relabeled as 1. Therefore

R017110 was defined as a dichotomous item. A plot for this item after collapsing the categories is

displayed in Figure 17-5.

To be consistent with the previous assessments, for grade 8 data, item R017102, an item of

Literary Experience in the second position in block R3, received special treatment. It was recoded as a

dichotomous item: the categories 0 and 1 were combined as 0 and the other categories were relabeled as

1. Item R016212, an item of Gain Information in the twelfth position in block R13, was recoded in the

state assessment as it was recoded in the national assessment: The categories 0 and 1 were combined as 0

and the other categories were relabeled as 1. A plot for this item after collapsing the categories is

displayed in Figure 17-6.

The IRT parameters for the items included in the state assessment are listed in Appendix E.
326

Figure 17-4

Polytomous Item (R012111) After Collapsing Unsatisfactory and Partial-Response Categories*

* Diamonds represent 1998 grade 4 reading assessment data. They indicate estimated

conditional probabilities obtained without assuming a specific model form; the curve indicates

the estimated item category response function (ICRF) using a generalized partial credit model.
327

Figure 17-5

Polytomous Item (R017110) After Collapsing Unsatisfactory and Partial-Response Categories*

* Diamonds represent 1998 grade 8 reading assessment data. They indicate estimated

conditional probabilities obtained without assuming a specific model form; the curve indicates

the estimated item category response function (ICRF) using a generalized partial credit model.
328

Figure 17-6

Polytomous Item (R016212) After Collapsing Unsatisfactory and Partial-Response Categories*

* Diamonds represent 1998 grade 8 reading assessment data. They indicate estimated conditional

probabilities obtained without assuming a specific model form; the curve indicates the estimated

item category response function (ICRF) using a generalized partial credit model.
329

17.4 GENERATION OF PLAUSIBLE VALUES

The scale score distributions for each jurisdiction (and for subgroups of interest within each

jurisdiction) were estimated using the multivariate plausible values methodology and the corresponding

CGROUP computer program. As described in Chapter 12, the CGROUP program estimates scale score

distributions using information from student item responses, measures of student background variables,

and the item parameter estimates obtained from the BILOG/PARSCALE program.

Results from Mazzeo’s research (1991) suggested that separate conditioning models be estimated

for each jurisdiction because the parameters estimated by the conditioning model differed across

jurisdictions. If a jurisdiction had a nonpublic-school sample, students from that sample were included in

this part of the analysis, and a conditioning variable differentiating between public- and nonpublic-school

students was included. This resulted in the estimation of 44 distinct conditioning models for grade 4, and

41 distinct conditioning models for grade 8.

Reporting each jurisdiction’s results required analyses describing the relationships between scale

scores and a large number of background variables. The background variables included in each

jurisdiction’s model were principal component scores derived from the within-jurisdiction correlation

matrix of selected main-effects and two-way interactions associated with a wide range of student,

teacher, school, and community variables. The background variables included student demographic

characteristics (e.g., the race/ethnicity of the student, highest level of education attained by parents),

students’ perceptions about reading, student behavior both in and out of school (e.g., amount of TV

watched daily, amount of reading homework done each day), the type of reading class being taken, and a

variety of other aspects of the students’ background and preparation, and the educational, social, and

financial environment of the schools they attended. Information was also collected from students’

teachers about their teaching practices, such as the amount of classroom emphasis on various topics

included in the assessment, and their educational background and professional preparation.

As described in the previous chapter, to avoid biases in reporting results and to minimize biases

in secondary analyses, it is desirable to incorporate measures of a large number of independent variables

in the conditioning model. For grade 4, when expressed in terms of contrast-coded main effects and

interactions, the number of variables to be included totaled 1,086; for grade 8, the number of variables to

be included totaled 1,064. Appendix F provides a listing of the full set of contrasts defined. These

contrasts were the common starting point in the development of the conditioning models for each of the

participating jurisdictions.

Because of the large number of these contrasts and the fact that, within each jurisdiction, some

contrasts had zero variance, some involved relatively small numbers of individuals, and some were

highly correlated with other contrasts or sets of contrasts, an effort was made to reduce the

dimensionality of the predictor variables in each jurisdiction’s CGROUP models. As was done for the

1990 and 1992 state assessments in mathematics and the 1992 and 1994 state assessments in reading, the

original background variable contrasts were standardized and transformed into a set of linearly

independent variables by extracting separate sets of principal components (one set for each of the 44

jurisdictions) from the within-jurisdiction correlation matrices of the original contrast variables. The

principal components, rather than the original variables, were used as the independent variables in the

conditioning model. As was done for the previous assessments, the number of principal components

included for each jurisdiction was the number required to account for approximately 90 percent of the

variance in the original contrast variables. Research based on data from the 1990 state assessment in

mathematics suggested that results obtained using such a subset of the components will differ only

slightly from those obtained using the full set (Mazzeo et al., 1992).
330

Table 17-14

Summary Statistics for State Assessment Conditioning Models, Grade 4

Jurisdiction

Number of

Principal

Components

Proportion* of Scale

Score Variance in the

Reading Assessment for

Literary Experience

Scale Accounted for by

the Conditioning Model

Proportion* of Scale

Score Variance in the

Reading Assessment to

Gain Information Scale

Accounted for by the

Conditioning Model

Conditional

Correlation Between

Literary Experience

and Gain

Information

Alabama 240 0.68 0.69 0.86

Arizona 242 0.71 0.72 0.89

Arkansas 253 0.68 0.69 0.86

California 195 0.70 0.71 0.89

Colorado 236 0.61 0.65 0.86

Connecticut 262 0.71 0.69 0.78

Delaware 231 0.77 0.75 0.85

District of Columbia 186 0.64 0.69 0.87

Florida 278 0.69 0.67 0.90

Georgia 275 0.74 0.75 0.84

Hawaii 260 0.62 0.56 0.84

Iowa 202 0.66 0.65 0.77

Kansas 191 0.69 0.74 0.85

Kentucky 221 0.70 0.67 0.87

Louisiana 256 0.56 0.61 0.86

Maine 230 0.73 0.76 0.80

Maryland 218 0.58 0.48 0.91

Massachusetts 235 0.68 0.72 0.89

Michigan 229 0.69 0.71 0.86

Minnesota 243 0.72 0.66 0.89

Mississippi 247 0.54 0.70 0.90

Missouri 241 0.66 0.63 0.89

Montana 180 0.80 0.75 0.80

Nebraska 110 0.93 0.89 0.91

Nevada 256 0.56 0.71 0.92

New Hampshire 209 0.84 0.80 0.86

New Mexico 238 0.65 0.67 0.91

New York 238 0.67 0.68 0.75

North Carolina 258 0.58 0.59 0.84

Oklahoma 234 0.66 0.72 0.89

Oregon 226 0.70 0.72 0.84

Rhode Island 253 0.68 0.68 0.76

South Carolina 254 0.67 0.66 0.88

Tennessee 253 0.68 0.61 0.85

Texas 235 0.75 0.73 0.90

Utah 238 0.64 0.64 0.88

Virginia 259 0.71 0.67 0.93

Virgin Islands 160 0.49 0.62 0.90

Washington 233 0.55 0.58 0.91

West Virginia 217 0.64 0.66 0.80

Wisconsin 219 0.87 0.82 0.90

Wyoming 206 0.80 0.78 0.86

DoDEA/DDESS 184 0.65 0.69 0.90

DoDEA/DoDDS 207 0.88 0.86 0.77

* (Total Variance – Residual Variance)/Total Variance, where Total Variance consists of both sampling and measurement error variance.
Table 17-15

Summary Statistics for State Assessment Conditioning Models, Grade 8

Jurisdiction

Number of

Principal

Components

Proportion* of Scale

Score Variance in the

Reading for Literary

Experience Scale

Accounted for by the

Conditioning Model

Proportion* of Scale

Score Variance in

the Reading to Gain

Information Scale

Accounted for by the

Conditioning Model

Proportion* of Scale

Score Variance in the

Reading to Perform a

Task Scale Accounted

for by the

Conditioning Model

Conditional

Correlation

Between Literary

Experience and

Gain Information

Co

Co

Betw

Exp

Perf

 Alabama 229 0.70 0.66 0.74 0.90  Arizona 244 0.69 0.72 0.82 0.87  Arkansas 233 0.72 0.68 0.76 0.79  California 245 0.76 0.72 0.82 0.82  Colorado 233 0.69 0.71 0.73 0.83  Connecticut 264 0.73 0.78 0.81 0.92  Delaware 179 0.78 0.72 0.84 0.92  District of Columbia 148 0.77 0.72 0.78 0.91  Florida 267 0.76 0.60 0.79 0.79  Georgia 283 0.77 0.78 0.83 0.89  Hawaii 194 0.58 0.59 0.70 0.82  Kansas 191 0.81 0.71 0.74 0.92  Kentucky 222 0.70 0.63 0.72 0.92  Louisiana 255 0.75 0.74 0.77 0.78  Maine 210 0.75 0.77 0.83 0.87  Maryland 234 0.66 0.67 0.67 0.86  Massachusetts 232 0.75 0.74 0.85 0.91  Minnesota 197 0.81 0.69 0.80 0.83  Mississippi 223 0.72 0.57 0.67 0.88  Missouri 236 0.67 0.69 0.75 0.85  Montana 172 0.88 0.76 0.89 0.91  Nebraska 99 1.00 0.96 1.00 0.55 * (Total Variance – Residual Variance)/Total Variance, where Total Variance consists of both sampling and measurement error variance.

(continued)
332

Table 17-15 (continued)

Summary Statistics for State Assessment Conditioning Models, Grade 8

Jurisdiction

Number of

Principal

Components

Proportion* of Scale

Score Variance in the

Reading for Literary

Experience Scale

Accounted for by the

Conditioning Model

Proportion* of Scale

Score Variance in the

Reading to Gain

Information Scale

Accounted for by the

Conditioning Model

Proportion* of Scale

Score Variance in the

Reading to Perform a

Task Scale Accounted

for by the

Conditioning Model

Conditional

Correlation

Between Literary

Experience and

Gain Information

Co

Co

Betw

Exp

Perf

 Nevada 213 0.75 0.64 0.79 0.91  New Mexico 234 0.73 0.69 0.84 0.71  New York 221 0.78 0.75 0.77 0.83  North Carolina 271 0.64 0.60 0.71 0.81  Oklahoma 219 0.69 0.74 0.85 0.90  Oregon 225 0.82 0.76 0.82 0.87  Rhode Island 206 0.74 0.70 0.79 0.85  South Carolina 279 0.77 0.75 0.78 0.90  Tennessee 222 0.62 0.70 0.82 0.89  Texas 249 0.79 0.71 0.78 0.85  Utah 241 0.72 0.70 0.76 0.77  Virginia 273 0.78 0.72 0.81 0.82  Virgin Islands 129 0.75 0.64 0.81 0.96  Washington 247 0.74 0.70 0.75 0.91  West Virginia 229 0.78 0.76 0.77 0.92  Wisconsin 195 0.84 0.83 0.90 0.91  Wyoming 181 0.88 0.85 0.92 0.79  DoDEA/DDESS 130 0.98 0.92 0.97 0.87  DoDEA/DoDDS 160 0.89 0.86 0.90 0.83 

* (Total Variance – Residual Variance)/Total Variance, where Total Variance consists of both sampling and measurement error variance
333

Tables 17-14 for grade 4 and 17-15 for grade 8 list the number of principal components included

in and the proportion of scale score variance accounted for by the conditioning model for each

participating jurisdiction. It is important to note that the proportion of variance accounted for by the

conditioning model differs across scales within a jurisdiction, and across jurisdictions within a scale.

Such variability is not unexpected for at least two reasons. First, there is no reason to expect the strength

of the relationship between scale score and demographics to be identical across all jurisdictions. In fact,

one of the reasons for fitting separate conditioning models is that the strength and nature of this

relationship may differ across jurisdictions. Second, the homogeneity of the demographic profile also

differs across jurisdictions. As with any correlation analysis, restriction of the range in the predictor

variables will attenuate the relationship.

Table 17-16 provides a matrix of estimated within-state correlations among the three purpose for

reading scales averaged over the 40 jurisdictions for grade 8. In parentheses are the lowest and the

highest estimated correlation among the 40 jurisdictions. The listed values, taken directly from the

CGROUP program, are estimates of the within-state correlations conditional on the set of principal

components included in the conditioning model. For grade 4, the average correlation between Literary

Experience and Gain Information is 0.86, with a range of (0.75, 0.93).

Table 17-16

Average Correlations and Ranges of Scale

Correlations Among the Reading Scales for 40 Jurisdictions*

 for Grade 8

 Literary Experience Perform A Task

Literary Experience 1.0 (1.0) 0.83 (0.66 - 0.95)

Gain Information 0.86 (0.71 - 0.96) 0.88 (0.81 - 0.94)

* Since Nebraska only had private schools participating, it was not included in the

calculation of the average correlation.

As discussed in Chapter 12, NAEP scales are viewed as summaries of consistencies and

regularities that are present in item-level data. Such summaries should agree with other reasonable

summaries of the item-level data. In order to evaluate the reasonableness of the scaling and estimation

results, a variety of analyses were conducted to compare state-level and subgroup-level performance in

terms of the content-area scale scores and in terms of the average proportion correct for the set of items

in a content area. High agreement was found in all of these analyses. One set of such analyses is

presented in Figures 17-7 and 17-8. The figures contain scatterplots of the state scale score mean (mean

scale score) versus the state item score means, for each of the two reading content areas and the

composite scale for grade 4 and the three reading content areas and the composite scale for grade 8. As is

evident from the figures, there is an extremely strong relationship between the estimates of state-level

performance in the scale score and item score metrics for both figures.
334

Figure 17-7

Plot of Mean Scale Score Versus Mean Item Score by Jurisdiction, Grade 4

Reading for Literary Experience

Grade 4

100

125

150

175

200

225

250

0.300 0.350 0.400 0.450 0.500 0.550 0.600 0.650

Mean Item Score



























	






Reading to Gain Information

Grade 4

100

125

150

175

200

225

250

0.300 0.350 0.400 0.450 0.500 0.550 0.600 0.650

Mean Item Score



























	






Reading Composite Scale

Grade 4

100

125

150

175

200

225

250

0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65

Mean Item Score



























	




335

Figure 17-8

Plot of Mean Scale Score Versus Mean Item Score by Jurisdiction, Grade 8

(continued)

Reading for Literary Experience

Grade 8

150

175

200

225

250

275

300

0.300 0.350 0.400 0.450 0.500 0.550

Mean Item Score

Reading to Perform a Task

Grade 8

150

175

200

225

250

275

300

0.300 0.400 0.500 0.600 0.700

Mean Item Score

Reading to Gain Information

Grade 8

150

175

200

225

250

275

300

0.300 0.400 0.500 0.600 0.700

Mean Item Score
336

Figure 17-8 (continued)

Plot of Mean Scale Score Versus Mean Item Score by Jurisdiction, Grade 8

17.5 THE FINAL SCORE SCALES

17.5.1 Linking State and National Scales

A major purpose of the state assessment program was to allow each participating jurisdiction to

compare its 1998 results with the nation as a whole and with the region of the country in which that

jurisdiction is located. Although the students in the 1998 state reading assessment were administered the

same test booklets as the fourth- and eighth-graders in the national assessment, separate state and

national scalings were carried out (for reasons explained in Mazzeo, 1991, and Yamamoto & Mazzeo,

1992). Again, to ensure a similar scale unit system for the state and national metrics, the scales had to be

linked.

For meaningful comparisons to be made between each of the state assessment jurisdictions and

the relevant national samples, results from these two assessments had to be expressed in terms of a

similar system of scale units. The purpose of this section is to describe the procedures used to align the

1998 state assessment scales with their 1998 national counterparts. The procedures that were used

represent an extension of the common population equating procedures employed to link the previous

national and state scales (Mazzeo, 1991; Yamamoto & Mazzeo, 1992).

Using the house sampling weights provided by Westat (see Section 15.5), the combined sample

of students from all participating jurisdictions was used to estimate the distribution of scale scores for the

population of students enrolled in public schools that participated in the state assessment.4

 The total

sample sizes were 104,129 for the fourth-graders, and 94,429 for the eighth-graders. A subsample of the

fourth- grade national sample, consisting of grade-eligible public-school students from any of the 44

jurisdictions that participated in the 1998 state assessment, was used to obtain estimates of the

distribution of scale scores for the same target population. A subsample of the eighth-grade national

sample, consisting of the students from any of the 41 jurisdictions that participated in the 1998 state

assessment, was used to obtain estimates of the distribution of scale scores for the same target

population. This subsample of national data is referred to as the national linking sample (NL).5

 Again,

 

4 Students from Virgin Islands, DoDEA/DDESS, and DoDEA/DoDDS schools were excluded from the state aggregate sample

for purposes of linking.

5 Note that in previous state assessments, the national linking sample was called the state aggregate comparison, or SAC, sample.

Many people thought this was easy to confuse with state data, so the term “national linking” is used in this report.

Reading Composite Scale

Grade 8

150

175

200

225

250

275

300

0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65

Mean Item Score
337

appropriate weights provided by Westat were used. Thus, for each scale, two sets of scale score

distributions were obtained and used in the linking process. One set, based on the sample of combined

data from the state assessment (referred to as the state aggregate, or SA) and using item parameter

estimates and conditioning results from that assessment, was in the metric of the 1998 state assessment.

The other, based on the NL sample from the 1998 national assessment and obtained using item

parameters and conditioning results from the national assessment, was in the reporting metric of the 1998

national assessment. The state assessment and national scales, two for grade 4 and three for grade 8, were

made comparable by constraining the mean and standard deviation of the two sets of estimates to be

equal.

More specifically, the following steps were followed to linearly link the scales of the two

assessments:

1) For each scale, estimates of the scale score distribution for the SA sample was

obtained using the full set of plausible values generated by the CGROUP program.

The weights used were the final (reporting sample) sampling weights provided by

Westat (see Section 15.5). For each scale, the arithmetic mean of the five sets of

plausible values was taken as the overall estimated mean and the arithmetic average

of the standard deviations of the five sets of plausible values was taken as the overall

estimated standard deviation.

2) For each scale, the estimated scale score distribution of the NL sample was obtained,

again using the full set of plausible values generated by the CGROUP program. The

weights used were specially provided by Westat to allow for the estimation of scale

score distributions for the same target population of students estimated by the

jurisdiction data. The means and standard deviations of the distributions (in the 1998

national reporting metric) for each scale were obtained for this sample in the same

manner as described in Step 1.

3) For each scale, a set of linear transformation coefficients was obtained to link the

state scale to the corresponding national scale. The linking was of the form

θ*= A • θ + B

where

θ = a scale score level in terms of the system of units of the provisional

BILOG/PARSCALE scale of the state assessment scaling

θ* = a scale score level in terms of the system of units comparable to

those used for reporting the 1998 national reading results

A = [Standard DeviationNL]/[Standard DeviationSA]

B = MeanNL - A• [MeanSA]

where the subscripts refer to the NL sample and to the SA sample.

The final conversion parameters for transforming plausible values from the provisional

BILOG/PARSCALE scales to the final state assessment reporting scales are given in Table 17-17. All

state assessment results are reported in terms of the Y* metric.
338

Table 17-17

Coefficients of Linear Transformations

for the 1998 State Reading Assessment

Grade Field of Reading Scale A B

4 Literary Experience 39.66 216.15

Gain Information 38.88 211.09

8 Literary Experience 31.55 260.11

Gain Information 35.89 259.25

Perform a Task 38.33 261.11

As is evident from the discussion above, a linear method was used to link the scales from the

state and national assessments. While these linear methods ensure equality of means and standard

deviations for the SA (after transformation) and the NL samples, they do not guarantee the shapes of the

estimated scale score distributions for the two samples will be the same. As these two samples are both

from a common target population, estimates of the scale score distribution of that target population based

on each of the samples should be quite similar in shape in order to justify strong claims of comparability

for the state and national scales. Substantial differences in the shapes of the two estimated distributions

would result in differing estimates of the percentages of students above achievement levels or of

percentile locations depending on whether state or national scales were useda clearly unacceptable

result given claims about the comparability of the scales. In the face of such results, nonlinear linking

methods would be required.

Analyses were carried out to verify the degree to which the linear linking process described

above produced comparable scales for state and national results. Comparisons were made between two

estimated scale score distributions, one based on the SA sample and one based on the NL sample, for

each of the three fields of reading scales. The comparisons were carried out using slightly modified

versions of what Wainer (1974) refers to as suspended rootograms. The final reporting scales for the state

and national assessments were each divided into 10-point intervals. Two sets of estimates of the

percentage of students in each interval were obtained, one based on the SA sample and one based on the

NL sample. Following Tukey (1977), the square roots of these estimated percentages were compared.6

The comparisons are shown in Figures 17-9 through 17-13. The height of each of the unshaded

bars corresponds to the square root of the percentage of students from the state assessment aggregate

sample in each 10-point interval on the final reporting scale. The shaded bars show the differences in root

percents between the SA and NL estimates. Positive differences indicate intervals in which the estimated

percentages from the NL sample are lower than those obtained from the SA. Conversely, negative

differences indicate intervals in which the estimated percentages from the NL sample are higher. For all

three scales, differences in root percents are quite small, suggesting that the shapes of the two estimated

distributions are quite similar (i.e., unimodal with small positive coefficient of skewness). There is some

evidence that the estimates produced using the NL data are slightly heavier in the extreme upper tails

(above 400 for Literary reading and Information reading for grade 4; above 350 for Literary reading,

above 380 for Information reading, and above 400 for Perform a Task for grade 8). However, even these

differences at the extremes are small in magnitude (0.2 in the root percent metric and 0.09 in the percent

metric) and have little impact on estimates of reported statistics such as percentages of students above the

achievement levels.

 

6 The square root transformation allows for more effective comparisons for counts (or equivalently, percentages) when the expected

number of counts in each interval is likely to vary greatly over the range of intervals, as is the case for the NAEP scales where the

expected counts of individuals in intervals near the extremes of the scale (e.g., below 150 and above 350) are dramatically smaller

than the counts obtained near the middle of the scale.
339

Figure 17-9

Rootogram Comparing Scale Score Distributions

for the State Assessment Aggregate Sample

and the National Linking Sample

for the Reading for Literary Experience Scale, Grade 4

Figure 17-10

Rootogram Comparing Scale Score Distributions

for the State Assessment Aggregate Sample

and the National Linking Sample

for the Reading to Gain Information Scale, Grade 4

-1 01234

50 100 150 200 250 300 350 400 450

Root Percent

-1 0 1 2 3 4

50 100 150 200 250 300 350 400 450

Root Percent

State

Diff

State

Diff
340

Figure 17-11

Rootogram Comparing Scale Score Distributions

for the State Assessment Aggregate Sample

and the National Linking Sample

for the Reading for Literary Experience Scale, Grade 8

Figure 17-12

Rootogram Comparing Scale Score Distributions

for the State Assessment Aggregate Sample

and the National Linking Sample

for the Reading to Gain Information Scale, Grade 8

Figure 17-13

Rootogram Comparing Scale Score Distributions

for the State Assessment Aggregate Sample

and the National Linking Sample

for the Reading to Perform a Task Scale, Grade 8

-1 0 1 2 3 4

50 100 150 200 250 300 350 400 450

Root Percent

-1 0 1 2 3 4

50 100 150 200 250 300 350 400 450

Root Percent

-1 0 1 2 3 4

50 100 150 200 250 300 350 400 450

Root Percent

State

Diff

State

Diff

State

Diff
341

17.5.2 Producing a Reading Composite Scale

For the national assessment, a composite scale was created for the fourth, eighth, and twelfth

grades as an overall measure of reading scale scores for students at that grade. The composite was a

weighted average of plausible values on the purpose-for-reading scales (Reading for Literary Experience,

Reading to Gain Information, and at grades 8 and 12, Reading to Perform a Task). The weights for the

national fields of reading scale scales were proportional to the relative importance assigned to each field

of reading scale in each grade in the assessment specifications developed by the Reading Objectives

Panel. Consequently, the weights for each of the fields of reading scales are similar to the actual

proportion of items from that field of reading scale.

State assessment composite scales for grades 4 and 8 were developed using weights identical to

those used to produce the composites for the 1998 national reading assessment. The weights are given in

Table 16-14. In developing the state assessment composite, the weights were applied to the plausible

values for each field of reading scale as expressed in terms of the final state assessment scales (i.e., after

transformation from the provisional BILOG/PARSCALE scales.)

Figures 17-14 and 17-15 provide rootograms comparing the estimated scale score distributions

based on the SA and NL samples for the grade 4 and grade 8 composites. Consistent with the results

presented separately by scale, there is some evidence that the estimates produced using the NL are

slightly heavier in the upper tails than the corresponding estimate based on the SA samples. Again

however, these differences in root relative percents are small in magnitude.

Figure 17-14

Rootogram Comparing Scale Score Distributions

for the State Assessment Aggregate Sample

and the National Linking Sample

for the Reading Composite Scale, Grade 4

-1 0 1 2 3 4

50 100 150 200 250 300 350 400 450

Root Percent

State

Diff
342

Figure 17-15

Rootogram Comparing Scale Score Distributions

for the State Assessment Aggregate Sample

and the National Linking Sample

for the Reading Composite Scale, Grade 8

17.6 PARTITIONING OF THE ESTIMATION ERROR VARIANCE

For each grade in state reading assessments, the error variance of the final transformed scale score

mean was partitioned as described in Chapter 12. The partition of error variance consists of two parts: the

proportion of error variance due to sampling students (sampling variance) and the proportion of error

variance due to the fact that scale score, , is a latent variable that is estimated rather than observed. For

grades 4 and 8, Tables 17-18 and 17-19 contain estimates of the total error variance, the proportion of

error variance due to sampling students, and the proportion of error variance due to the latent nature of

θ . Instead of using 100 plausible values as in national assessment, the calculations for the state samples

are based on 5 plausible values. More detailed information is available for gender and race/ethnicity

subgroups in Appendix H.

17.7 READING TEACHER QUESTIONNAIRES

Teachers of fourth- and eighth-grade students were surveyed about their educational background

and teaching practices. The students were matched first with their reading teacher, and then the specific

classroom period. Variables derived from the questionnaire were used in the conditioning models. An

additional conditioning variable was included that indicated whether the student had been matched with a

teacher record. This contrast controlled estimates of subgroup means for differences that exist between

matched and nonmatched students. Of the 112,138 fourth-grade students in the sample, 105,026 (93.7%,

unweighted) were matched with teachers who answered both parts of the teacher questionnaire, and 13 of

the students had teachers who answered only the teacher background section of the questionnaire. For the

eighth-grade sample, 82,118 of the 94,429 students (87%, unweighted) were matched to both sections of

the teacher questionnaire. There were 6,575 students (7%, unweighted) who were matched with the first

part of the teacher questionnaire, but could not be matched to the appropriate classroom period. Thus,

93.7 percent of the fourth-graders and 94 percent of the eighth-graders were matched with at least the

background information about their reading teacher. -1 0 1 2 3 4

50 100 150 200 250 300 350 400 450

Root Percent

State

Diff
343

Table 17-18

Estimation Error Variance and Related Coefficients

for the Reading State Assessment, Grade 4

Proportion of Variance due to ...

State

Total Estimation

Error Variance Student Sampling Latency of θ

Alabama 3.197 0.94 0.06

Arizona 4.062 0.97 0.03

Arkansas 2.208 0.93 0.07

California 10.325 0.96 0.04

Colorado 1.721 0.94 0.06

Connecticut 3.425 0.93 0.07

Delaware 1.637 0.57 0.43

Florida 2.128 0.96 0.04

Georgia 2.519 0.95 0.05

Hawaii 3.085 0.66 0.34

Iowa 1.397 0.97 0.03

Kansas 2.173 0.89 0.11

Kentucky 2.218 0.81 0.19

Louisiana 2.254 0.98 0.02

Maine 1.529 0.72 0.28

Maryland 2.656 0.97 0.03

Massachusetts 1.965 0.89 0.11

Michigan 2.755 0.94 0.06

Minnesota 2.195 0.89 0.11

Mississippi 2.123 0.98 0.02

Missouri 2.762 0.96 0.04

Montana 2.774 0.59 0.41

Nevada 1.855 0.93 0.07

New Hampshire 1.783 0.76 0.24

New Mexico 4.089 0.79 0.21

New York 2.639 0.89 0.11

North Carolina 1.804 0.89 0.11

Oklahoma 1.286 0.92 0.08

Oregon 2.644 0.94 0.06

Rhode Island 3.018 0.84 0.16

South Carolina 1.648 0.91 0.09

Tennessee 2.224 0.95 0.05

Texas 4.493 0.97 0.03

Utah 1.775 0.86 0.14

Virginia 1.777 0.97 0.03

Washington 1.791 0.97 0.03

West Virginia 2.205 0.96 0.04

Wisconsin 1.322 0.95 0.05

Wyoming 2.624 0.47 0.53

District of Columbia 1.971 0.38 0.62

DoDEA/DDESS 1.702 0.32 0.68

DoDEA/DoDDS 1.208 0.57 0.43

Virgin Islands 3.779 0.39 0.61
344

Table 17-19

Estimation Error Variance and Related Coefficients

for the Reading State Assessment, Grade 8

Proportion of Variance due to ...

State

Total Estimation

Error Variance Student Sampling Latency of θ

Alabama 1.822 0.97 0.03

Arizona 1.394 0.95 0.05

Arkansas 1.753 0.79 0.21

California 2.726 0.96 0.04

Colorado 1.196 0.98 0.02

Connecticut 1.159 0.89 0.11

Delaware 1.626 0.72 0.28

Florida 2.890 0.91 0.09

Georgia 2.052 0.95 0.05

Hawaii 1.745 0.39 0.61

Kansas 1.437 0.94 0.06

Kentucky 1.664 0.98 0.02

Louisiana 2.157 0.95 0.05

Maine 1.389 0.92 0.08

Maryland 3.376 0.82 0.18

Massachusetts 2.435 0.92 0.08

Minnesota 1.672 0.93 0.07

Mississippi 2.054 0.79 0.21

Missouri 1.728 0.85 0.15

Montana 1.291 0.72 0.28

Nevada 1.301 0.95 0.05

New Mexico 1.524 0.79 0.21

New York 2.531 0.91 0.09

North Carolina 1.301 0.85 0.15

Oklahoma 1.631 0.71 0.29

Oregon 2.087 0.91 0.09

Rhode Island 0.925 0.89 0.11

South Carolina 1.756 0.93 0.07

Tennessee 1.679 0.91 0.09

Texas 2.142 0.99 0.01

Utah 1.123 0.78 0.22

Virginia 1.232 0.90 0.10

Washington 1.639 0.88 0.12

West Virginia 1.417 0.88 0.12

Wisconsin 2.466 0.91 0.09

Wyoming 1.734 0.58 0.42

District of Columbia 3.846 0.30 0.70

DoDEA/DDESS 10.719 0.24 0.76

DoDEA/DoDDS 1.054 0.44 0.56

Virgin Islands 8.264 0.26 0.74
345

Chapter 18

ASSESSMENT FRAMEWORKS AND INSTRUMENTS

FOR THE 1998 NATIONAL AND STATE WRITING ASSESSMENTS1

Elissa A. Greenwald and Terry L. Schoeps

Educational Testing Service

18.1 INTRODUCTION

The framework that was used for the 1998 NAEP writing assessment detailed the structure of the

assessment to be given at grades 4, 8, and 12 at the national level and at grade 8 at the state level. The

framework was developed under contract by the Center for Research on Evaluation, Standards, and

Student Testing (CRESST) and American College Testing (ACT) for the National Assessment

Governing Board (NAGB) in 1996. The framework for the writing assessment is available on the

National Assessment Governing Board (NAGB) web site at http://www.nagb.org.

Sections 18.2 through 18.5 explain the development of the framework, objectives, and items for

the 1998 NAEP writing assessment. Section 18.8 also describes the student background questionnaires

and the writing teacher questionnaire. Additional information on the structure and content of assessment

booklets can be found in Section 18.9. Various committees worked on the development of the

framework, objectives, and items for the writing assessment. The list of committee members and

consultants who participated in the 1998 development process is provided in Appendix K.

Samples of assessment instruments and student responses are published in the NAEP 1998

Writing Report Card for the Nation and the States (Greenwald, Persky, Campbell, & Mazzeo, 1999).

18.2 DEVELOPING THE WRITING ASSESSMENT FRAMEWORK

NAGB is responsible for setting policy for NAEP; this policy-making role includes the

development of assessment frameworks and test specifications. Appointed by the Secretary of Education

from lists of nominees proposed by the board itself in various statutory categories, the 24-member board

is composed of state, local, and federal officials, as well as educators and members of the public.

NAGB began the development process for the 1998 writing objectives by convening a writing

framework panel. The panel solicited recommendations from members of the academic and business

communities, from state and local government representatives, from members of the press, and from the

general public. After reviewing the responses, the panel designed the framework.

For more detail on the development and specifications of the writing framework, refer to the

Writing Framework and Specifications for the 1998 National Assessment of Educational Progress,

1992–1998 (NAGB, 1996b).

 

1 Elissa A. Greenwald managed the item-development process for the 1998 NAEP writing assessment. Terry L. Schoeps

coordinates the production of NAEP technical reports.
346

18.3 WRITING FRAMEWORK AND ASSESSMENT DESIGN PRINCIPLES

The writing framework was designed to focus on writing processes and outcomes, rather than to

reflect a particular instructional or theoretical approach. The framework focuses not on the specific

writing skills that lead to outcomes, but rather on the quality of the outcomes themselves. The framework

was intended to embody a broad view of writing by addressing the increasingly higher level of literacy

needed for employment, personal development, and good citizenship. The people who designed the

framework also relied on contemporary writing research and sought to use nontraditional assessment

formats that resemble desired classroom activities to the extent possible within the constraints of a timed

assessment.

The development of the framework objectives was guided by the consideration that the

assessment should reflect many of the curricular emphases and objectives in various states, localities, and

school districts, as well as what various scholars, practitioners, and interested citizens believed should be

included in the assessment. Under contract to NAGB, ACT developed the test specifications to address

overarching objectives of the 1998 writing assessment framework:

• Write for a variety of purposes—narrative, informative, and persuasive

• Write on a variety of tasks and for many different audiences

• Write from a variety of stimulus materials and within various time constraints

• Generate, draft, revise, and edit ideas and forms of expression in their writing

• Display effective choices in the organization of their writing

• Value writing as a communicative activity

18.4 FRAMEWORK FOR THE 1998 WRITING ASSESSMENT

The 1998 writing assessment framework was organized according to three purposes for writing:

• Narrative

• Informative

• Persuasive

Narrative writing tasks require students to produce a story or personal essay. Informative writing

tasks focus primarily on the subject-matter element in communication. Informative writing is used to

share knowledge and to convey messages, instructions, and ideas. In persuasive writing, the primary aim

is to influence others to take some action or to bring about change. This type of writing involves a clear

awareness of what arguments might most affect the audience being addressed. Further explanation of the

purposes is contained in Figure 18-1.

The cognitive portion of the writing assessment included only constructed-response exercises.

These tasks were designed to measure students’ abilities to write for a variety of purposes and to a

diverse set of audiences. To accomplish these goals, a wide variety of stimulus materials were used in the

assessment. The first step in the development effort was the identification of appropriate stimulus

materials that would allow the construction of tasks that would, in aggregate, measure the range of

writing outcomes described in the framework.
347

Figure 18-1

Description of NAEP 1998 Writing Purposes*

Narrative

Narrative writing involves the production of stories or personal essays. Practice with these forms helps

writers to develop an ear for language. Also, informative and persuasive writing can benefit from many of the

strategies used in narrative writing. For example, there must be an effective ordering of events when relating

an incident as part of a report. Sometimes narrative writing contributes to an awareness of the world as the

writer creates, manipulates, and interprets reality. Such writing—whether fact or fiction, poem, play, or

personal essay—requires close observation of people, objects, and places. Further, this type of writing fosters

creativity, imagination, and speculation by allowing the writer to express thoughts and then stand back, as a

more detached observer might, and grasp more fully what is being felt and why. Thus, narrative writing

offers a special opportunity to analyze and understand emotions and actions.

Informative

Informative writing focuses primarily on the subject-matter element in communication. This type of writing

is used to share knowledge and to convey messages, instructions, and ideas. Like all writing, informative

writing may be filtered through the writer’s impressions, understanding, and feelings. Used as a means of

exploration, informative writing helps both the writer and the reader to learn new ideas and to reexamine old

conclusions. Informative writing may also involve reporting on events or experiences, or analyzing concepts

and relationships, including developing hypotheses and generalizations. Any of these types of informative

writing can be based on the writer’s personal knowledge and experience or on information newly presented

to the writer that must be understood in order to complete a task. Usually, informative writing involves a mix

of the familiar and the new, and both are clarified in the process or writing. Depending on the task, writing

based on either personal experience or secondary information may span the range of thinking skills from

recall to analysis to evaluation.

Persuasive

Persuasive writing emphasizes the reader. Its primary aim is to influence others to take some action or bring

about change. Persuasive writing may contain great amounts of information—facts, details, examples,

comparisons, statistics, or anecdotes—but its main purpose is not simply to inform but to persuade. This type

of writing involves a clear awareness of what arguments might most affect the audience being addressed.

Writing persuasively also requires use of critical thinking skills such as analysis, inference, synthesis, and

evaluation.

Persuasive writing is called for in a variety of situations. It may involve responding to a request for advice by

giving an opinion and providing sound reasons to support it. It may also involve presenting an argument in

such a way that a particular audience will find it convincing. When there is opposition, persuasive writing

may entail refuting arguments that are contrary to the writer’s point of view.

In all persuasive writing, authors must choose the approach they will use. They may, for instance, use

emotional or logical appeals or an accommodating or demanding tone. Regardless of the situation or

approach, persuasive writers must be concerned with having a particular desired effect on their readers,

beyond merely adding to knowledge of the topic presented.

* The text in Figure 18-1 is from the Writing Framework and Specifications for the 1998 National Assessment of Educational

Progress, 1992–1998 (NAGB, 1996b), developed under contract by the Center for Research on Evaluation, Standards, and

Student Testing (CRESST) and American College Testing (ACT) for the National Assessment Governing Board (NAGB) in

1996.
348

A carefully developed and proven series of steps was used to create the assessment items. These

steps are described in Chapter 2.

The distribution of items by writing purpose across grade levels recommended in the assessment

framework is provided in Table 18-1.

Table 18-1

Percentage Distribution of Items by Purpose for Writing

as Specified in the NAEP Writing Framework

Purposes for Writing

Grade Narrative Informative Persuasive

4 40% 35% 25%

8* 33% 33% 33%

12 25% 35% 40%

* The grade 8 percentages shown in this table do not total 100% because the numbers have been rounded.

The writing framework also discusses the ways in which the assessment tasks should be scored.

Students’ responses to each writing task were evaluated by trained raters who used scoring guides that

emphasized development, organization, and control of language.

18.5 DEVELOPING THE WRITING COGNITIVE ITEMS

The assessment included 25-minute and 50-minute writing tasks (referred to as "blocks" in test

development). Students were asked to respond to either two 25-minute writing tasks or one 50-minute

writing task (for some students at grades 8 and 12). In accordance with the framework objective to

include writing on a variety of tasks and for many different audiences, students were asked to write in a

variety of forms. Some of the forms in which students were asked to write (across the tasks in the

assessment) are listed in Figure 18-2.

Figure 18-2

NAEP 1998 Forms of Writing

Story

Essay

Letter to Authority

Letter to a Friend

Article

Report

Speech

18.6 DEVELOPING THE WRITING OPERATIONAL FORMS

Writing field tests were conducted in October and November of 1997 and involved national

samples of fourth-, eighth-, and twelfth-grade students. More than 100 items were field tested across the

three grades.
349

The field-test data were collected, scored, and analyzed in preparation for meetings with the

Writing Instrument Development Committee. Committee members, ETS test-development staff, and

NAEP/ETS staff reviewed the materials and chose the 66 writing tasks used in the operational

assessment. The objectives that guided these reviews included determining

• which tasks were most related to overall student achievement;

• the need for revisions of tasks that lacked clarity or had ineffective formats; and

• which tasks could be scored with the highest levels of interrater reliability.

The tasks were chosen according to the distributions of narrative, informative, and persuasive

writing tasks specified in the framework. Once the committees had selected the tasks, all tasks were

rechecked for content, measurement, and sensitivity concerns. Finally, a clearance package was

submitted to NCES. Throughout the clearance process, revisions were made in accordance with changes

required by the government. Upon approval, the tasks (assembled into booklets) and questionnaires were

ready for printing.

The 50-minute tasks that were administered at grades 8 and 12 were not administered as part of

the state assessment.

18.7 DISTRIBUTION OF WRITING ASSESSMENT ITEMS

At grade 4, all tasks were 25-minute writing tasks; eight measured narrative writing, seven

measured informative writing, and six measured persuasive writing. Of the 25-minute tasks administered

at grade 8, seven measured narrative writing, seven measured informative writing, and six measured

persuasive writing. At grade 12, of the 25-minute tasks, five measured narrative writing, seven measured

informative writing, and eight measured persuasive writing. At grades 8 and 12, three 50-minute tasks

were given—one for each writing purpose. The 50-minute tasks were administered in the national

assessment but were not given in the state assessment.

Tables 18-2 through 18-4 provide the title and writing purpose of each writing task administered.
350

Table 18-2

NAEP 1998 Writing Grade 4 Blocks by Title and Writing Purpose

Writing Block Title Block Purpose

Aunt Dot W3 Narrative

Cartoon Story W4 Narrative

Very Unusual Day W5 Narrative

Castle W6 Narrative

Casey and Duke W7 Narrative

Old Tree W8 Narrative

Secret Door W9 Narrative

Mr. Tooms W10 Narrative

Letter from TX8 W11 Informative

Letter from MZ3 W12 Informative

Letter from Lilex W13 Informative

Animal Lesson W14 Informative

City Scenes W15 Informative

Unusual Animal W16 Informative

Favorite Object * W17 Informative

Invisible Friend W18 Persuasive

Day Trip* W19 Persuasive

Class Pet W20 Persuasive

Library Book W21 Persuasive

Child or Adult W22 Persuasive

*

 This block appeared in booklets administered to students

requiring accommodations.
351

Table 18-3

NAEP 1998 Writing Grade 8 Blocks by Title and Writing Purpose

Writing Block Title Block Purpose

Cartoon Story W3 Narrative

President for a Day W4 Narrative

Plums W5 Narrative

Tower W6 Narrative

Principal for a Day* W7 Narrative

Pioneer Journal W8 Narrative

Space Visitor W9 Narrative

Ancient Tree† W10 Narrative

Performance Review W11 Informative

New Park W12 Informative

Dream Weekend W13 Informative

Backpack W14 Informative

Designing a TV Show W15 Informative

Save a Book W16 Informative

Life’s Lessons W17 Informative

Vandalism† W18 Informative

Lengthening the School Year* W19 Persuasive

School Schedule W20 Persuasive

Fast Food W21 Persuasive

Class Trip W22 Persuasive

Driving Age W23 Persuasive

Teens in Malls W24 Persuasive

Student of the Year† W25 Persuasive

*

 This block appeared in booklets administered to students

requiring accommodations.

† This was a 50-minute block and was not part of the main

national reporting sample.
352

Table 18-4

NAEP 1998 Writing Grade 12 Blocks by Title and Writing Purpose

Writing Block Title Block Purpose

Tall Tale W3 Narrative

Plums W4 Narrative

Special Object W5 Narrative

The Arch W6 Narrative

Pioneer Journal W7 Narrative

Ancient Tree* W8 Narrative

Cafeteria W9 Informative

Writing Mentor W10 Informative

Movie Review W11 Informative

Technology W12 Informative

Handbook W13 Informative

Save a Book W14 Informative

Life’s Lessons W15 Informative

Vandalism† W16 Informative

Summer Job W17 Persuasive

Big or Small Inventions W18 Persuasive

Work Less/Study More W19 Persuasive

Heroes W20 Persuasive

One Vote* W21 Persuasive

Teens in Malls W22 Persuasive

Driving Age W23 Persuasive

Person of the Year W24 Persuasive

Campaign Speech* W25 Persuasive

*

 This was a 50-minute block and was not part of the main

reporting sample.

†

 This block appeared in booklets administered to students

requiring accommodations.

Each student received an assessment booklet containing a either 25-minute exercises or one 50-

minute exercise. Following the exercise or exercises in each booklet were a set of general background

questions, a set of subject-specific background questions, and a set of questions about his or her

motivation and familiarity with the assessment materials.

In the development process, every effort was made to meet the content targets specified in the

assessment framework. Table 18-5 shows the approximate percentage of aggregate assessment time

devoted to each purpose for writing, at each grade level. Percentages are based on the classifications

agreed on by the Writing Instrument Development Committee. Note that the numbers presented in Table

18-5 differ slightly from those in Table 18-1 in that Table 18-1 (at grade 8 only) shows the distribution of

assessment items as specified in the writing framework.
353

Table 18-5

Percentage Distribution of Assessment Time by Grade

and Purpose for Writing for the NAEP 1998 Writing Assessment*

Purposes for Writing

Grade Narrative Informative Persuasive

4 40% 35% 25%

8 35% 35% 30%

12 25% 35% 40%

18.8 BACKGROUND QUESTIONNAIRES FOR THE 1998 WRITING ASSESSMENT

In addition to assessing how well students read, it is important to understand the instructional

context in which writing takes place, students’ home support for literacy, and students’ writing habits and

attitudes. To gather contextual information, NAEP assessments include background questions designed

to provide insight into factors that may influence writing performance.

NAEP includes both general background questionnaires given to participants in all subjects and

subject-specific questionnaires for both students and their teachers. The development of the general

background questionnaires is discussed below. Members of the Writing Instrument Development

Committee were consulted on the appropriateness of the issues addressed in all questionnaires that relate

to writing instruction and achievement. Like the writing tasks, all background questions were submitted

for extensive review and field testing. Recognizing the validity problems inherent in self-reported data,

particular attention was given to developing questions that were meaningful and unambiguous and that

would encourage accurate reporting.

In addition to the cognitive questions, the 1998 assessment included one five-minute set of

general and one five-minute set of subject-specific background questions designed to gather contextual

information about students, their instructional and recreational experiences in writing, and their attitudes

toward writing. Students in the fourth grade were given additional time because the items in the general

questionnaire were read aloud for them. A one-minute questionnaire was also given to students at the end

of each booklet to determine students motivation in completing the assessment and their familiarity with

assessment tasks.

18.8.1 Student Writing Questionnaires

Three sets of multiple-choice background questions were included as separate sections in each

student booklet:

General Background: The general background questions collected demographic information

about race/ethnicity, language spoken at home, mother’s and father’s level of education, reading

materials in the home, homework, school attendance, which parents live at home, and which

parents work outside the home.

Writing Background: Students were asked to report their instructional experiences related to

writing in the classroom, including how often their teachers asked them to write more than one

draft of a paper and whether or not they or their teachers saved their written work in a folder or

portfolio.
354

Motivation: Students were asked five questions about how hard they tried on the test

and about friends’ attitudes toward writing.

Table 18-6 gives the number of questions per background section and notes the placement of

each within student booklets.

Table 18-6

NAEP 1998 Background Sections of Student Writing Booklets

Number of Questions Placement in Student Booklet

Grade 4

General Background 21 Section 3

Writing Background 17 Section 4

Motivation 5 Section 5

Grade 8

General Background 22 Section 3

Writing Background 28 Section 4

Motivation 5 Section 5

Grade 12

General Background 24 Section 3

Writing Background 28 Section 4

Motivation 5 Section 5

18.8.2 Language Arts Teacher Questionnaire

To supplement the information on instruction reported by students, writing teachers of the

fourth- and eighth-graders participating in the NAEP writing assessment were asked to complete a

questionnaire about characteristics such as their gender, teaching backgrounds, and instructional

practices. The teacher questionnaire contained two parts. The first part pertained to the teachers

background and general training. The second part pertained to specific training in teaching writing and

the procedures the teacher used for each class containing an assessed student.

The Teacher Questionnaire, Part I: Background, Education, and Resources (49 questions at

grade 4 and 48 at grade 8) included questions pertaining to:

• gender;

• race/ethnicity;

• years of teaching experience;

• certification, degrees, major and minor fields of study;

• coursework in education;

• coursework in specific subject areas;

• amount of in-service training;

• extent of control over instructional issues; and

• availability of resources for their classroom.

This component of the questionnaire was completed by teachers whose students participated in

any subject assessed in NAEP.
355

The Teacher Questionnaire, Part IIA: Reading/Writing Preparation (12 questions at grade 4

and 12 at grade 8) included questions on the teachers’ exposure to various issues related to writing

instruction through college or university courses or professional-development workshops.

The Teacher Questionnaire, Part IIB: Reading/Writing Instructional Information

(84 questions at grades 4 and 85 questions at grade 8) included questions pertaining to:

• the ability level of students in the class;

• whether students were assigned to the class by ability level;

• time spent weekly on teaching writing and helping students with their writing;

• writing homework assignments;

• frequency of various instructional activities in class;

• methods of assessing student progress in writing;

• instructional emphasis given to the writing abilities covered in the assessment; and

• use of particular resources.

18.9 STUDENT BOOKLETS FOR THE 1998 WRITING ASSESSMENT

At each grade in the assessment, the 25-minute tasks were assembled into 18 booklets. At grades

8 and 12, there were 3 additional booklets containing 50-minute tasks. The assessment booklets were

then spiraled and bundled. Spiraling involves interweaving the booklets in a systematic sequence so that

each booklet appears an appropriate number of times in the sample. The bundles were designed so that

each booklet would appear equally often in a position in a bundle.

The assembly of writing blocks (with one task per block) into booklets and their subsequent

assignment to sampled students was determined by a partially balanced incomplete block (PBIB) design

with spiraled administration (see Section 1.5). At each grade, the 25-minute tasks were assembled into 40

booklets such that two different blocks were assigned to each booklet and each block appeared in four

booklets. Tables 18-6 through 18-8 show this configuration. At all grades, every 25-minute task appears

in four booklets. This is the partially balanced part of the balanced incomplete block design. Every 50-

minute task appears only in one booklet (although booklets containing the 50-minute tasks are included

in the main national assessment, they cannot be assembled in the PBIB fashion).

The focused PBIB design also balances the order of presentation of the 25-minute blocks—every

25-minute block appears as the first cognitive task in two booklets and as the second cognitive task in

two other booklets. This design allows for some control of context and fatigue effects.

As in the other subjects, the final step in the PBIB-spiraling procedure was the assigning of

booklets to the assessed students. The students in the assessment session were assigned booklets in the

order in which the booklets were bundled. Thus, most students in an assessment session received

different booklets. Tables 18-7, 18-8, and 18-9 detail the configuration of booklets administered in the

1998 writing assessment.

18.10 WRITING CLASSROOM-BASED STUDY IN 1998

In 1998, NAEP conducted a special study designed to explore methods of assessing students’

writing abilities by using written assignments that students had completed as part of their school

curriculum. A full report on this study is due to be published in the year 2000.
356

Table 18-7

NAEP 1998 National and State Writing Grade 4 Booklet Configuration

Booklet

Number

Question

Block 1

Question

Block 2

Common Core

Background

Writing

Background Motivation

201 W4 W16 CW WB WA

202 W16 W11 CW WB WA

203 W11 W3 CW WB WA

204 W3 W18 CW WB WA

205 W18 W19 CW WB WA

206 W19 W20 CW WB WA

207 W20 W12 CW WB WA

208 W12 W7 CW WB WA

209 W7 W21 CW WB WA

210 W21 W22 CW WB WA

211 W22 W18 CW WB WA

212 W18 W14 CW WB WA

213 W14 W5 CW WB WA

214 W5 W19 CW WB WA

 215* W19 W17 CW WB WA

216 W17 W6 CW WB WA

217 W6 W20 CW WB WA

218 W20 W21 CW WB WA

219 W21 W15 CW WB WA

220 W15 W8 CW WB WA

221 W8 W22 CW WB WA

222 W22 W13 CW WB WA

223 W13 W9 CW WB WA

224 W9 W4 CW WB WA

225 W4 W3 CW WB WA

226 W3 W5 CW WB WA

227 W5 W6 CW WB WA

228 W6 W7 CW WB WA

229 W7 W8 CW WB WA

230 W8 W9 CW WB WA

231 W9 W10 CW WB WA

232 W10 W11 CW WB WA

233 W11 W14 CW WB WA

234 W14 W17 CW WB WA

235 W17 W12 CW WB WA

236 W12 W15 CW WB WA

237 W15 W13 CW WB WA

238 W13 W16 CW WB WA

239 W16 W10 CW WB WA

240 W10 W4 CW WB WA

*Booklet number 215 was an accommodations booklet. Accommodations booklets contain type that is larger

than the type used in other booklets; they are given to participating students who have a visual disability.
357

Table 18-8

NAEP 1998 National and State Writing Grade 8 Booklet Configuration

Booklet

Number

Question

Block 1

Question

Block 2

Common Core

Background

Writing

Background Motivation

201 W3 W4 CW WB WA

202 W4 W5 CW WB WA

203 W5 W6 CW WB WA

204 W6 W7 CW WB WA

205 W7 W8 CW WB WA

206 W8 W9 CW WB WA

207 W9 W13 CW WB WA

208 W13 W19 CW WB WA

 209* W19 W7 CW WB WA

210 W7 W14 CW WB WA

211 W14 W21 CW WB WA

212 W21 W5 CW WB WA

213 W5 W12 CW WB WA

214 W12 W17 CW WB WA

215 W17 W23 CW WB WA

216 W23 W20 CW WB WA

217 W20 W21 CW WB WA

218 W21 W22 CW WB WA

219 W22 W19 CW WB WA

220 W19 W24 CW WB WA

221 W24 W8 CW WB WA

222 W8 W15 CW WB WA

223 W15 W22 CW WB WA

224 W22 W6 CW WB WA

225 W6 W16 CW WB WA

226 W16 W20 CW WB WA

227 W20 W4 CW WB WA

228 W4 W11 CW WB WA

229 W11 W12 CW WB WA

230 W12 W16 CW WB WA

231 W16 W14 CW WB WA

232 W14 W15 CW WB WA

233 W15 W13 CW WB WA

234 W13 W17 CW WB WA

235 W17 W11 CW WB WA

236 W11 W9 CW WB WA

237 W9 W3 CW WB WA

238 W3 W24 CW WB WA

239 W24 W23 CW WB WA

240 W23 W3 CW WB WA

241 –––––––––––––W10†

––––––––––– CW WB WA

242 –––––——––––W18†

———–—— CW WB WA

243 —————–––W25†

——–––—— CW WB WA

* Booklet number 209 was an accommodations booklet. Accommodations booklets contain type that is larger than the type

used in other booklets; they are given to participating students who have a visual disability.

†

 Booklets containing blocks W10, W18, and W25 were booklets that contained 50-minute tasks.
358

Table 18-9

NAEP 1998 National and State Writing Grade 12 Booklet Configuration

Booklet

Number

Question

Block 1

Question

Block 2

Common Core

Background

Writing

Background Motivation

201 W3 W4 CW WB WA

202 W4 W5 CW WB WA

203 W5 W6 CW WB WA

204 W6 W7 CW WB WA

205 W7 W23 CW WB WA

206 W23 W15 CW WB WA

207 W15 W9 CW WB WA

208 W9 W10 CW WB WA

209 W10 W11 CW WB WA

210 W11 W12 CW WB WA

211 W12 W13 CW WB WA

212 W13 W14 CW WB WA

213 W14 W15 CW WB WA

214 W15 W17 CW WB WA

215 W17 W18 CW WB WA

216 W18 W19 CW WB WA

217 W19 W20 CW WB WA

218 W20 W21 CW WB WA

219 W21 W22 CW WB WA

220 W22 W23 CW WB WA

221 W23 W24 CW WB WA

222 W24 W9 CW WB WA

223 W9 W17 CW WB WA

224 W17 W24 CW WB WA

225 W24 W18 CW WB WA

226 W18 W10 CW WB WA

227 W10 W3 CW WB WA

228 W3 W19 CW WB WA

229 W19 W11 CW WB WA

230 W11 W4 CW WB WA

231 W4 W20 CW WB WA

232 W20 W12 CW WB WA

233 W12 W5 CW WB WA

 234* W5 W21 CW WB WA

235 W21 W13 CW WB WA

236 W13 W6 CW WB WA

237 W6 W22 CW WB WA

238 W22 W14 CW WB WA

239 W14 W7 CW WB WA

240 W7 W3 CW WB WA

241 –––––––––––––W8†

–––––––––––– CW WB WA

242 –––––––––––––W16†

––––––––––– CW WB WA

243 –––––––––––––W25†

––––––––––– CW WB WA

* Booklet number 234 was an accommodations booklet. Accommodations booklets contain type that is larger than the type

used in other booklets; they are given to participating students who have a visual disability.

† Booklets containing blocks W8, W16, and W25 were booklets that contained 50-minute tasks.
359

Chapter 19

INTRODUCTION TO THE DATA ANALYSIS FOR THE

NATIONAL AND STATE WRITING SAMPLES1

Frank Jenkins, Jiahe Qian, Hua-Hua Chang, and Bruce A. Kaplan

Educational Testing Service

19.1 INTRODUCTION

This chapter gives an introduction to the analyses performed on the responses to the cognitive

and background items in the 1998 assessment of writing. These analyses led to the results presented in

the NAEP 1998 Writing Report Card for the Nation and the States (Greenwald et al., 1999). The topics

discussed in this chapter center on issues such as the description of student samples, student weights,

items, assessment booklet, administrative procedures, scoring of the constructed-response items and

student weights. Reasons why a formal analysis of differential item functioning (DIF) were not attempted

will be presented. The major analysis components are discussed in Chapter 20 for the national

assessment and Chapter 21 for the state assessment.

The objectives of the writing analyses were to prepare scale values, estimate subgroup scale

score distributions for pertinent populations of students, and estimate the percent of students performing

at or above various achievement-level cut points. The 1998 state assessment scales were linked to the

corresponding scales from the 1998 national assessment. All analyses used data from students

participating in the 1998 national and state writing assessments.

19.2 DESCRIPTION OF STUDENT SAMPLES, ITEMS, ASSESSMENT

BOOKLETS, AND ADMINISTRATIVE PROCEDURES

The student samples that were administered writing items in the 1998 assessment are shown in

Table 19-1. The data from the national main focused partially balanced incomplete block (PBIB)

assessment of writing (4 [Writing–Main], 8 [Writing–Main], and 12 [Writing–Main]) were used for

national main analyses comparing the levels of writing achievement for various subgroups of the 1998

target populations. See Section 1.5 for an explanation of the focused partially balanced incomplete block

(PBIB). Chapters 3 and 4 contain descriptions of the target populations and the sample design used for

the assessment. The target populations were grade 4, grade 8, and grade 12 students in the United States.

Unlike previous writing NAEP assessments, only grade-defined cohorts were assessed in the 1998

NAEP. The students were sampled in the winter (January to March with final makeup sessions held from

March 30 to April 3). As described in Chapter 3, the reporting sample for the national writing assessment

has students with disabilities (SD) and limited English proficient students (LEP) who were included

under new inclusion rules and who were given appropriate accommodations as available.

The sample designated as 8 [Writing–State] was used for the grade 8 state writing analysis. This

sample included the assessment of both public- and nonpublic-school students for most jurisdictions. The

procedures used were similar to those of previous state assessments.

 

1 Frank Jenkins was the primary person responsible for coordinating the national writing analysis. Hua-Hua Chang and Jiahe Qian

were responsible for coordinating the state writing analysis. Computing activities for all writing analyses were directed by Bruce

A. Kaplan and assisted by Youn-Hee Lim. Others contributing to the analysis were David S. Freund and Katherine Pashley.
360

Table 19-1

NAEP 1998 Writing Student Samples*

Sample

Booklet

Number

Cohort

Assessed Time of Testing†

Reporting

Sample Size

 4 [Writing–Main] W201–W240 Grade 4 1/5/98 – 3/27/98 19,816

 8 [Writing–Main] W201–W240 Grade 8 1/5/98 – 3/27/98 20,586

12 [Writing–Main] W201–W237 Grade 12 1/5/98 – 3/27/98 19,505

 8 [Writing–50 Min] W241–W243 Grade 8 1/5/98 – 3/27/98 6,009

12 [Writing–50 Min] W241–W243 Grade 12 1/5/98 – 3/27/98 5,804

 8 [Writing–State] W201–W240 Grade 8 1/5/98 – 3/27/98 97,589

* All sessions were administered in a printed format.

† Final makeup sessions were held March 30–April 3, 1998.

The major analysis components are discussed below. Some aspects of the analysis, such as

procedures for item analysis, scoring of constructed-response items, and methods of scaling, are

described in Chapters 9 and 12 and are therefore not detailed here. There were four major steps in the

analysis of the writing data, each of which is described in a separate section:

• Conventional item and test analyses (Section 20.2)

• Item response theory (IRT) scaling (Section 20.3)

• Estimation of subgroup scale score distributions based on the plausible values

methodology (Section 20.4)

• Transforming the 1998 assessment scales to the final reporting metric (Section 20.5)

Section 20.6 describes the results of partitioning the error variance, 20.7 discusses the matching

of student responses to those of their teachers, and 19.6 provides a brief explanation of sampling weights.

Analysis of the state writing assessment consisted of similar steps and is detailed in Chapter 21.

To set the context within which to describe the methods and results of scaling procedures, a brief

review of the assessment instruments and administration procedures is provided.

The 1998 NAEP national main writing assessment differed from the long-term trend assessment

in the sample age definition, the time of testing, the objectives that define the emphasis of the

assessment, and the items used. It also differed from the 1992 national main NAEP writing assessment in

that (1) the framework was revised, (2) most of the prompts (the exercises administered to the students)

were new, and (3) for those prompts that were also administered in 1992, different rubrics (the rules for

assigning scores to responses) were used to score responses. Because of these differences, equating or

linking to the earlier main and the long-term trend assessments was not appropriate. The 1998 national

main writing assessment can be used to start a new baseline for measuring trends in the nation.

The prompts used in the 1998 writing assessment consisted of two types of six-point constructedresponse items: those allowing for a 25-minute response and those allowing for a 50-minute response.

The items in the assessment were based on the curriculum framework described in Writing Framework

and Specifications for the 1998 National Assessment of Educational Progress (NAGB, 1996b). The 1998

framework resulted from augmenting the 1992 framework with new exercise specifications. This lead to

the development of new writing prompts and scoring guides. As described in the writing framework, the

prompts represented three purposes of writing: narrative, informative, and persuasive. All three item

types were used to measure a single scale of writing performance. Table 19-2 gives the number of

25-minute writing prompts in each grade that were used in the national main assessment. There were a
361

total of 20 25-minute prompts per grade in the main assessment. In grade 4, there was an emphasis on

narrative items (8 of 20), whereas at grade 12 the emphasis was on persuasive prompts (8 of 20).

Table 19-2

Number of 25-Minute Items in the National Main Writing Assessment

Within the Three Purposes of Writing

Grade Narrative Informative Persuasive Total

 4 8 7 5 20

 8 7 7 6 20

12 5 7 8 20

Three 50-minute prompts were administered at grades 8 and 12, one for each purpose of writing,

as shown in Table 19-3. Administering these items provided an opportunity to study how students

responded to longer writing exercises that were more like regular classroom assignments. These items

were not included as part of the main writing scale, however, because only one such prompt was

administered per person. It was thought that a single item per person yielded too unreliable a measure of

writing skill. Therefore, only 25-minute prompts were used in calculating scale score results. Data from

the 50-minute prompts were not included.

Table 19-3

Number of 50-Minute Items in the National Writing Assessment

Within the Three Purposes of Writing

Grade Narrative Informative Persuasive Total

 81 1 1 3

12 1 1 1 3

In the main samples, each student was administered a booklet containing two separately timed

25-minute blocks. Each block contained a single writing prompt. In addition, each student was

administered a block of background questions, a block of writing-related background questions, and a

block of questions concerning the student’s motivation and his or her perception of the difficulty of the

NAEP writing items. The background and motivational blocks were common to all writing booklets for a

particular grade level. Twenty 25-minute blocks of writing prompts were administered at each grade

level. As described in Chapter 18, the 25-minute blocks were combined into booklets according to a

partially balanced incomplete block (PBIB) design. See Chapter 18 for more information about the

blocks and booklets. In addition, the 50-minute writing prompts were given to some students at grades 8

and 12 in lieu of two 25-minute prompts. In these cases, the single prompt given a student composed the

block and the book. As mentioned before, these prompts were not included in the writing scale.

Tables 19-4 through 19-6 give the correspondence between writing prompts and the respective

blocks they define. As mentioned above, the 50-minute prompts were the only writing task in a book. The

25-minute prompts, however, are arranged into 40 books. Tables 19-7 through 19-9 gives the

correspondence between prompts (which are also blocks) and books. It also indicates in which books a

block (or item) was ordered first and in which book a block (or item) was ordered second.
362

Table 19-4

Grade 4: Prompt, Block, and Purpose Correspondence

Prompt Description Block Purpose

W004002 Aunt Dot W3 Narrative

W004102 Cartoon Story W4 Narrative

W004202 Very Unusual Day W5 Narrative

W004302 Castle W6 Narrative

W004402 Casey and Duke W7 Narrative

W004502 Old Tree W8 Narrative

W004602 Secret Door W9 Narrative

W004702 Mr. Tooms W10 Narrative

W004802 Letter from TX8 W11 Informative

W004902 Letter from MZ3 W12 Informative

W005002 Letter from Lilex W13 Informative

W005102 Animal Lesson W14 Informative

W005202 City Scenes W15 Informative

W005302 Unusual Animal W16 Informative

W005402 Favorite Object W17* Informative

W005502 Invisible Friend W18 Persuasive

W005602 Day Trip W19* Persuasive

W005702 Class Pet W20 Persuasive

W005802 Library Book W21 Persuasive

W005902 Child or Adult W22 Persuasive

*

 This block appears in booklets administered to students requiring accommodations.
363

Table 19-5

Grade 8: Prompt, Block, and Purpose Correspondence

Prompt Description Block Purpose

W006002 Cartoon Story W3 Narrative

W006102 President for a Day W4 Narrative

W006202 Plums W5 Narrative

W006302 Tower W6 Narrative

W006402 Principal for a Day W7* Narrative

W006502 Pioneer Journal W8 Narrative

W006602 Space Visitor W9 Narrative

W006702 Ancient Tree W10† Narrative

W006802 Performance Review W11 Informative

W006902 New Park W12 Informative

W007002 Dream Weekend W13 Informative

W007102 Backpack W14 Informative

W007202 Designing a TV Show W15 Informative

W007302 Save a Book W16 Informative

W007402 Life’s Lessons W17 Informative

W007502 Vandalism W18† Informative

W007602 Lengthening the School Year W19* Persuasive

W007702 School Schedule W20 Persuasive

W007802 Fast Food W21 Persuasive

W007902 Class Trip W22 Persuasive

W008002 Driving Age W23 Persuasive

W008102 Teens in Malls W24 Persuasive

W008202 Student of the Year W25† Persuasive

* This block appeared in booklets administered to students requiring accommodations.

† This was a 50-minute block and was not part of the main spiral.
364

Table 19-6

Grade 12: Prompt, Block, and Purpose Correspondence

Prompt Description Block Purpose

W008302 Tall Tale W3 Narrative

W008402 Plums W4 Narrative

W008502 Special Object W5* Narrative

W008602 The Arch W6 Narrative

W008702 Pioneer Journal W7 Narrative

W008802 Ancient Tree W8† Narrative

W008902 Cafeteria W9 Informative

W009002 Writing Mentor W10 Informative

W009102 Movie Review W11 Informative

W009202 Technology W12 Informative

W009302 Handbook W13 Informative

W009402 Save a Book W14 Informative

W009502 Life’s Lessons W15 Informative

W009602 Vandalism W16† Informative

W009702 Summer Job W17 Persuasive

W009802 Big or Small Inventions W18 Persuasive

W009902 Work Less/Study More W19 Persuasive

W010002 Heroes W20 Persuasive

W010102 One Vote W21* Persuasive

W010202 Teens in Malls W22 Persuasive

W010302 Driving Age W23 Persuasive

W010402 Person of the Year W24 Persuasive

W010502 Campaign Speech W25† Persuasive

* This block appeared in booklets administered to students requiring accommodations.

† This was a 50-minute block and was not part of the main spiral.
365

Table 19-7

Correspondence of Prompts, Blocks, and Books: Grade 4

Item Block

Books Where Item Occurs

in 1st Position

Books Where Item Occurs

in 2nd Position

W004002 W3 204 226 203 225

W004102 W4 201 225 224 240

W004202 W5 214 227 213 226

W004302 W6 217 228 216 227

W004402 W7 209 229 208 228

W004502 W8 221 230 220 229

W004602 W9 224 231 223 230

W004702 W10 232 240 231 239

W004802 W11 203 233 202 232

W004902 W12 208 236 207 235

W005002 W13 223 238 222 237

W005102 W14 213 234 212 233

W005202 W15 220 237 219 236

W005302 W16 202 239 201 238

W005402 W17 216 235 215 234

W005502 W18 205 212 204 211

W005602 W19 206 215 205 214

W005702 W20 207 218 206 217

W005802 W21 210 219 209 218

W005902 W22 211 222 210 221
366

Table 19-8

Correspondence of Prompts, Blocks, and Books: Grade 8

Item Block

Books Where Item Occurs

in 1st Position

Books Where Item Occurs

in 2nd Position

W006002 W3 201 238 237 240

W006102 W4 202 228 201 227

W006202 W5 203 213 202 212

W006302 W6 204 225 203 224

W006402 W7 205 210 204 209

W006502 W8 206 222 205 221

W006602 W9 207 237 206 236

W006702 W10* 241 — — —

W006802 W11 229 236 228 235

W006902 W12 214 230 213 229

W007002 W13 208 234 207 233

W007102 W14 211 232 210 231

W007202 W15 223 233 222 232

W007302 W16 226 231 225 230

W007402 W17 215 235 214 234

W007502 W18* 242 — — —

W007602 W19 209 220 208 219

W007702 W20 217 227 216 226

W007802 W21 212 218 211 217

W007902 W22 219 224 218 223

W008002 W23 216 240 215 239

W008102 W24 221 239 220 238

W008202 W25* 243 — — —

* Booklets containing 50-minute blocks included only one block.
367

Table 19-9

Correspondence of Prompts, Blocks, and Books: Grade 12

Item Block

Books Where Item Occurs

in 1st Position

Books Where Item Occurs

in 2nd Position

W008302 W1 201 228 227 240

W008402 W2 202 231 201 230

W008502 W3 203 234 202 233

W008602 W4 204 237 203 236

W008702 W5 205 240 204 239

W008802 W6* 241 — — —

W008902 W7 208 223 207 222

W009002 W8 209 227 208 226

W009102 W9 210 230 209 229

W009202 W10 211 233 210 232

W009302 W11 212 236 211 235

W009402 W12 213 239 212 238

W009502 W13 207 214 206 213

W009602 W14* 242 — — —

W009702 W15 215 224 214 223

W009802 W16 216 226 215 225

W009902 W17 217 229 216 228

W010002 W18 218 232 217 231

W010102 W19 219 235 218 234

W010202 W20 220 238 219 237

W010302 W21 206 221 205 220

W010402 W22 222 225 221 224

W010502 W23* 243 — — —

* Booklets containing 50-minute blocks included only one block.

Some writing prompts were common with the 1992 assessment. However, because the scoring

rubrics differed from those used in the 1992 assessment, all items were treated as if they were new. As a

result, there was no trend with the 1992 assessment. Also, there was no overlap of items across grades.

Thus, a separate writing scale was defined for each grade.

19.3 SCORING CONSTRUCTED-RESPONSE ITEMS

Responses to each writing prompt were scored holistically using a six-category rubric. The six

categories defined six levels of partial credit and are referred to by the following descriptors:

0 = Unsatisfactory

1 = Insufficient Response

2 = Uneven Response

3 = Sufficient Response

4 = Skillful Response

5 = Excellent Response
368

“Missing” responses (students did not write a response to the task, or provided an off-task

response) were treated as if the item had not been presented to the student (see Section 12.3.1 or Mislevy

& Wu [1988]). 

Teams of trained raters scored the written student responses according to scoring guides that

defined particular features for the score points appropriate to the grade and purpose of writing. This

means that there were nine scoring guides: one for narrative, informative, and persuasive purposes for

each grade. See the upcoming NAEP 1998 Writing Report Card for the Nation and the States (Greenwald

et al., 1999) for details of the scoring rubrics.

In order to determine interrater reliability of scoring, a percentage of responses was scored twice:

for the 25-minute prompts, 25 percent of the responses at grades 4 and 12, and 10 percent of the

responses at grade 8 (the only grade at which the state-by-state assessment was given) were scored by

two raters. In addition, 25 percent of responses to the 50-minute prompts were scored by a second rater.

For the national and state writing assessments, approximately 370,000 responses to writing

prompts were scored. This number includes rescoring to monitor interrater reliability. The average

within-year percentages of agreement on the six-level scale for the 1998 reliability samples were 77

percent at grade 4, 71 percent at grade 8, and 74 percent at grade 12. The reliabilities for each writing

prompt can be found in Appendix C.

 19.4 DIFFERENTIAL ITEM FUNCTIONING

A differential item functioning (DIF) analysis is customarily done to identify potentially biased

items. In standard DIF analyses such as Mantel-Haenszel and SIBTEST, it is well established that a

moderately long matching test is required for the procedures to be valid (i.e., identify DIF in items

unconfounded by other irrelevant factors [e.g., Donoghue, Holland, & Thayer, 1993]). In the 1998 NAEP

writing assessment, the booklets contain two 25-minute blocks, with one writing prompt per block. Thus,

each examinee has (at most) two responses on six-category prompts. This is too little information for the

test statistics associated with Mantel (1963) or SIBTEST (Shealy & Stout, 1993) procedures to function

effectively. Thus, standard DIF approaches based on statistical tests of items are likely to function

poorly, and so were not used in the 1998 writing assessment.

In the writing assessment the standardization method of Dorans and Kulick (1986) was used to

produce descriptive statistics. The matching variable was the total score on the booklet (see Section

9.3.4). As in other NAEP DIF analyses, the statistics were computed based on pooled booklet matching;

the results are accumulated over the booklets in which a given item appears (e.g., Allen & Donoghue,

1996). This analysis was accomplished using the standard NAEP DIF program NDIF. The statistic of

interest appears under the label SMD for "standardized mean DIF." (First, differences in the item score

between the two comparison groups are calculated for each level of the booklet score. Then, the

standardized mean DIF for the item is the average of these differences divided by their standard

deviation.

Significance testing was not performed, due to the low reliability of the matching variable.

Instead, the standardized mean difference values were used descriptively, to identify those items that

demonstrate the most evidence of DIF. A rough criterion used in the past to describe DIF for polytomous

items has been to create the ratio of the SMD to the item’s standard deviation and flag any item with a

ratio of at least .25. In the writing data no items approached that level. If, as a rule of thumb we use as a

criterion for flagging DIF, that the absolute SMD was at least .1, six prompts are flagged. These are listed

in Table 19-10. This ad hoc descriptive analysis of DIF did not lead to the rejection of any items as

biased.
369

Table 19-10

Items With Absolute SMD (Standardized Mean DIF) > .10

Group Grade SMD ID

NonAcc/Acc 4 -.106 W005402

B/W 4 -.108 W005302

B/W 12 -.129 W009802

B/W 12 .127 W010402

H/W 4 -.101 W004602

H/W 12 -.112 W009202

LEGEND

NonAcc/Acc Nonaccommodated versus accommodated students

B/W Black versus White students

H/W Hispanic versus White students

Tables A-6 and A-8 in Appendix A provide sample sizes for each of the race/ethnicity and 

accommodated/nonaccommodated groups noted in the table above.

ETS NAEP staff examined these items, although no formal DIF committee for writing was

convened. As a result of this informal analysis of DIF it was decided that there was insufficient evidence

of DIF to delete any items. It should be noted that this descriptive procedure was not a formal DIF

analysis. Since there were only two items per book, standard DIF procedures wear not appropriate. The

descriptive procedure used (standardized mean DIF) did not rule out the possibility of DIF in writing

items.

19.5 50-MINUTE WRITING STUDY

It was previously mentioned that there were three 50-minute writing prompts at grade 8 as well

as grade 12. For those assigned such prompts, the writing portion of the book consisted of the single

50-minute prompt. Response to these items were not put on the main writing scale. The single response

per student was thought to yield inadequate information about students’ writing abilities to put their

scores on the writing scale. The 50-minute prompts were administered in order to provide a writing

experience that more closely reflects actual classroom assignment. It was also an attempt to see if

students would do more pre-writing (e.g., outlining) if given more time. Indeed, as the result of an

analysis of pre-writing behavior, it was determined that there was more pre-writing with the 50-minute

prompts. Details of the responses to 50-minute prompts will be given in the item release materials.

19.6 THE WEIGHT FILES

The sampling contractor Westat produced the final student and school weights and the

corresponding replicate weights for the 1998 writing assessment. Information for the creation of the

weight files was supplied by NCS under the direction of ETS. Details of the general weighting scheme

for the 1998 assessments is given in Chapters 10 and 11. Some features of the weighting procedure

peculiar to the 1998 writing assessment will be discussed here.

Students designated as SD or LEP were included in the assessment under new inclusion rules.

SD and LEP students who customarily received accommodations were offered those same

accommodations in NAEP (i.e., writing used an S3 sample only). At each grade, all accommodated
370

students took the same booklet, which consisted of two 25-minute blocks. The weighting of

accommodated students was handled somewhat differently in different phases of the analysis.

The first stage of a NAEP analysis is an item analysis (IA), which yields information such as

item-level frequencies, item means, and item-to-block score correlations. For the IA, the weights were

normalized so that the sum of the weights equaled the sample size of the reporting sample (all students

taking 25-minute items).

In order to understand the effect that the accommodated students had on the responses for the

two items in the “accommodation” book, the item analysis was run three ways:

1. With accommodated students deleted. In this way the responses to items in

the “accommodated” book were directly comparable with the responses to

other items.

2. With the accommodated students included and using the weights provided

by Westat. When compared with the first IA analysis, this showed the full

effect that accommodated students had on item responses.

3. Finally, IA was run with accommodated students included, but weighted

down by a factor of 4/40. This showed the effect accommodated students

would have on items, if the responses for those items were a representative

sample from the population. The 4/40 factor was derived from the fact that

there are 40 booklets and each item appears in 4 booklets. If evenly

distributed, only 4/40s of the entire sample takes each item.

The two items in the accommodated book are “downweighted” in the final IA analysis because

there were more accommodated students taking these items than would be expected from a simple

random sample. This is because all accommodated students initially assigned to other books were

reassigned to the accommodated book. The 4/40 factor comes from the fact that there are 40 books

funneling accommodated students into this one book, but an item occurs in 4 books. So we downweight

by 1/40 and weight up by 4, which is the same as weighting by 4/40.

The “downweighting” of the accommodated students was also used in the IRT scaling analysis.

For estimation of imputed values (using NSWEEP and CGROUP, see Section 20.4), the

accommodated students were not downweighted and the weights were used as they were provided by

Westat, as they were in the second IA analysis mentioned above. This was done to assure that statistics

based on weighted proficiencies would be representative of the entire population.
371

Chapter 20

DATA ANALYSIS FOR THE NATIONAL WRITING SAMPLES1

Frank Jenkins, Bruce A. Kaplan, and Youn-Hee Lim

Educational Testing Service

20.1 INTRODUCTION

The purpose of the national writing analysis was to produce estimates of subgroup means and

standard deviations on the 1998 writing achievement scale and to estimate the percentage of students

scoring within each of the achievement level ranges (basic, proficient and advanced) as defined by the

National Assessment Governing Board (NAGB) achievement level cut points. To accomplish these goals,

data from the 1998 national writing assessment was analyzed through the stages detailed in the following

sections. Standard item analyses (e.g., estimation of item means) were performed. Next, an IRT scaling

was done to create a writing achievement scale at each grade. Third, estimated (plausible) values on a

latent writing trait were estimated in order to get unbiased estimates of subgroup achievement

distributions, and finally estimates were put in a convenient metric to facilitate interpretation and prevent

confusion with other assessments.

20.2 NATIONAL ITEM ANALYSIS

This section contains a detailed description of the conventional item analysis performed on the

writing data. Since there was only one item per block, this analysis could not be done within block as is

usual in NAEP assessments. Item to total correlations are meaningless with one item per block. Instead,

item analysis was run within grade as if all twenty 25-minute blocks (items) came from one large block.

Frequencies of responses at each score point and item averages were the only meaningful statistics that

could be reported. Tables 20-1 through 20-3 give the item statistics for the 25-minute items in the three

grades. These tables show the number of students taking each item, the percentage of those taking the

item that scored in each category, the overall average item score, the average score for the item when it

appeared first in a booklet and the average item score when it appeared second in a booklet. The means

by block order show a small but consistent order effect advantaging the item when it is in the first

position. Fortunately, order effects were balanced over all subsamples through the partially balanced

incomplete block (PBIB) design for assigning blocks to books. Books were then assigned to students

through a spiral procedure, which results in an equivalent sample of students being assigned to each book

(see Chapter 9, Section 9.2). The item means do not vary greatly, ranging from 3.3 to 4.0 at grade 4, 3.4

to 3.9 at grade 8, and 3.3 to 4.2 at grade 12. The reader is cautioned that average item means cannot be

compared across grades since there is not a cross-grade scale.

 

1 Frank Jenkins was the primary person responsible for the coordination of the National writing analysis. Computing activities

for all writing analyses were directed by Bruce A. Kaplan and assisted by Youn-Hee Lim. Others contributing to the analysis

were David S. Freund and Katherine E. Pashley.
372

Table 20-1

Descriptive Statistics for 25-Minute Writing Prompts: Grade 4

Percentage of Students in Each Category

Item ID Description n Missing 0 1 2 3 4 5

Total Item

Mean

1st Positio

Item Mea

W004002 Aunt Dot 1,680 8.6 1.3 8.8 34.5 40.4 10.0 5.1 3.64 3.67 W004102 Cartoon Story 1,805 5.6 3.0 17.0 42.9 24.1 10.8 2.2 3.29 3.36 W004202 Very Unusual Day 1,698 10.8 4.9 12.8 36.2 28.3 13.9 3.9 3.45 3.49 W004302 Castle 1,730 8.5 2.0 12.1 30.7 38.4 14.0 2.8 3.59 3.65 W004402 Casey And Duke 1,831 3.2 1.9 6.7 22.8 43.2 20.9 4.4 3.88 3.96 W004502 Old Tree 1,740 8.3 2.4 7.8 21.3 47.6 16.9 4.0 3.81 3.82 W004602 Secret Door 1,733 8.3 1.1 6.0 19.4 44.0 23.0 6.5 4.01 4.05 W004702 Mr. Tooms 1,740 8.3 3.3 7.1 22.7 41.8 20.6 4.5 3.83 3.87 W004802 Letter from TX8 1,791 3.5 6.4 11.6 36.2 29.8 12.6 3.3 3.40 3.42 W004902 Letter from MZ3 1,841 4.2 4.4 8.3 45.5 32.6 7.9 1.4 3.36 3.38 W005002 Letter from Lilex 1,846 3.3 4.1 14.7 43.2 29.2 7.9 1.0 3.25 3.30 W005102 Animal Lesson 1,893 2.2 1.4 7.9 31.1 47.4 10.5 1.7 3.63 3.68 W005202 City Scenes 1,747 7.5 4.4 13.7 36.9 35.9 7.8 1.4 3.33 3.39 W005302 Unusual Animal 1,848 2.9 1.7 5.3 38.3 42.7 9.3 2.8 3.61 3.65 W005402 Favorite Object 1,827 7.9 1.7 8.7 37.5 41.0 9.4 1.7 3.53 3.59 W005502 Invisible Friend 1,746 6.3 1.8 8.1 25.2 46.9 15.2 2.8 3.74 3.80 W005602 Day Trip 1,790 6.9 5.5 13.5 28.3 39.0 11.4 2.3 3.44 3.59 W005702 Class Pet 1,712 8.4 4.6 9.9 30.0 43.6 9.1 2.7 3.51 3.53 W005802 Library Book 1,721 7.8 2.8 7.9 31.7 48.2 7.6 1.7 3.55 3.60 W005902 Child or Adult 1,721 8.6 4.6 7.5 33.7 44.1 9.0 1.1 3.49 3.54 

Average 1,772 3.57 3.62 

LEGENDn = Unweighted sample size 3 = Sufficient0 = Unsatisfactory 4 = Skilled1 = Insufficient 5 = Excellent2 = Uneven
Table 20-2

Descriptive Statistics for 25-Minute Writing Prompts: Grade 8

Percentage of Students in Each Category

Item ID Description n Missing 0 1 2 3 4 5

Total Item

Mean

1st Posit

Item M

W006002 Cartoon Story 1,940 3.3 1.4 13.4 29.7 33.6 16.1 5.9 3.67 3.78 W006102 President For a Day 1,943 2.3 1.2 12.6 31.0 37.6 12.7 4.8 3.62 3.73 W006202 Plums 1,988 2.3 2.0 16.2 34.1 32.6 11.7 3.3 3.46 3.57 W006302 Tower 1,932 1.5 6.0 6.4 21.2 39.3 23.1 4.0 3.79 3.84 W006402 Principal For a Day 1,921 2.6 3.3 9.3 20.5 39.4 20.4 7.2 3.86 3.97 W006502 Pioneer Journal 1,935 2.5 1.4 6.9 21.3 46.6 21.5 2.4 3.87 3.96 W006602 Space Visitor 1,928 3.0 1.5 11.0 20.8 46.2 15.2 5.4 3.79 3.91 W006802 Performance Review 1,927 2.3 1.4 8.5 30.9 42.4 13.8 3.1 3.68 3.77 W006902 New Park 1,971 2.1 1.8 8.6 28.1 51.2 8.7 1.6 3.62 3.68 W007002 Dream Weekend 1,950 1.7 1.8 7.5 26.6 50.3 10.4 3.4 3.70 3.81 W007102 Backpack 1,936 1.6 2.7 6.4 24.5 49.1 15.2 2.1 3.74 3.79 W007202 Designing a TV Show 1,929 2.3 3.2 12.7 39.9 33.8 8.5 1.8 3.37 3.44 W007302 Save a Book 1,915 3.7 4.0 9.4 29.4 47.3 7.1 2.8 3.53 3.68 W007402 Life’s Lessons 1,964 2.2 3.2 8.1 25.8 43.6 15.5 3.9 3.72 3.88 W007602 Lengthening School Year 1,949 1.8 4.0 9.6 34.1 35.2 14.0 3.0 3.55 3.64 W007702 School Schedule 1,921 2.3 3.6 11.6 33.8 40.2 9.5 1.3 3.44 3.54 W007802 Fast Food 1,976 1.2 5.2 9.4 28.3 38.5 15.3 3.3 3.59 3.71 W007902 Class Trip 1,940 1.9 2.4 8.6 35.9 43.8 6.7 2.5 3.51 3.59 W008002 Driving Age 1,969 2.5 1.8 11.2 34.2 40.8 10.4 1.7 3.52 3.59 W008102 Teens in Malls 1,966 1.8 4.7 10.6 24.3 42.4 15.3 2.7 3.61 3.69 

Average 1,945 3.63 3.73 

LEGENDn = Unweighted sample size 3 = Sufficient0 = Unsatisfactory 4 = Skilled1 = Insufficient 5 = Excellent2 = Uneven
374

Table 20-3

Descriptive Statistics for 25-Minute Writing Prompts: Grade 12

Percentage of Students in Each Category

Item ID Description n Missing 0 1 2 3 4 5

Total Item

Mean

1st Positi

Item Me

W008302 Tall Tale 1,838 3.0 6.7 3.6 17.3 49.1 21.6 1.8 3.81 3.87 W008402 Plums 1,863 3.5 3.1 3.9 11.9 44.5 34.8 1.8 4.10 4.15 W008502 Special Object 1,889 3.2 1.2 4.3 14.2 36.7 42.0 1.7 4.19 4.28 W008602 The Arch 1,945 1.8 0.3 4.0 18.0 49.2 26.6 2.0 4.04 4.11 W008702 Pioneer Journal 1,932 2.0 0.8 6.7 21.5 45.7 21.0 4.3 3.92 4.03 W008902 Cafeteria 1,878 2.7 0.5 4.3 13.8 46.6 29.9 4.9 4.16 4.23 W009002 Writing Mentor 1,841 2.4 3.0 4.3 21.4 40.3 25.5 5.4 3.97 4.13 W009102 Movie Review 1,761 5.1 2.1 7.0 19.7 53.3 13.1 4.8 3.83 3.92 W009202 Technology 1,815 3.1 3.2 7.8 18.1 38.8 30.3 1.8 3.90 3.98 W009302 Handbook 1,850 2.5 2.0 5.9 14.5 39.4 26.8 11.5 4.17 4.31 W009402 Save a Book 1,826 3.1 4.3 8.9 19.6 39.7 25.8 1.6 3.79 3.92 W009502 Life’s Lessons 1,805 5.2 3.5 6.2 14.3 44.8 27.2 4.1 3.98 4.07 W009702 Summer Job 1,892 2.3 3.2 8.5 28.1 39.3 16.3 4.5 3.70 3.76 W009802 Big or Small Inventions 1,874 2.9 2.9 8.5 18.3 48.5 15.8 5.9 3.84 3.88 W009902 Work Less/Study More 1,842 2.0 3.7 9.9 26.1 43.6 10.9 5.7 3.65 3.73 W010002 Heroes 1,884 2.3 2.3 8.5 17.2 45.7 21.4 4.9 3.90 4.00 W010102 One Vote 1,892 2.2 4.2 21.3 30.1 31.8 10.1 2.6 3.30 3.40 W010202 Teens in Malls 1,876 2.5 3.3 9.7 23.6 41.0 18.0 4.4 3.74 3.84 W010302 Driving Age 1,907 2.6 3.4 11.9 24.6 36.9 18.1 5.1 3.70 3.82 W010402 Person of the Year 1,882 2.5 2.3 7.0 21.7 37.1 22.3 9.6 3.99 4.11 Average 1,865 3.88 3.98 

LEGENDn = Unweighted sample size 3 = Sufficient0 = Unsatisfactory 4 = Skilled1 = Insufficient 5 = Excellent2 = Uneven
375

A few details about the tables need to be explained. Item means were calculated using weights.

The denominator for calculating means and percents in responses 1 through 6 were the weighted total

number giving legitimate responses (1 through 6). “Missing” responses (i.e., students did not write a

response to the task, or provided an off-task response) were treated as “not presented,” (i.e., were not

given a score and were not used in IRT calibration [see Section 12.3.1 or Mislevy & Wu, 1988]). The

denominator for calculating percent missing was the sum of total missing and legitimate responses for the

item. The column labeled “n” in the tables shows the unweighted number of students presented with the

item who gave a legitimate response. In order to facilitate comparisons among items, the accommodated

students were not included in these item analysis tables. At each grade, accommodated students were

given the same two items and including this data would make the responses on these two items

noncomparable to responses of other items.

20.3 ITEM RESPONSE THEORY (IRT) SCALING

In 1993, the National Assessment Governing Board (NAGB) determined that future NAEP

assessments should be developed using within-grade frameworks. Within-grade scaling removes the

constraint that the trait being measured is cumulative across the grade levels of the assessment. It also

means that there is no need for overlap items across grades. Consistent with this view, NAGB also

declared that scaling be performed within-grade. Any items that happened to be the same across grades in

the assessment were scaled separately for each grade, thus making it possible for common items to

function differently in the separate grades. Therefore, the writing framework specifies that the 1998

writing assessment be developed within-grade. Likewise, all IRT scaling was performed within-grade.

Within each grade, a single writing scale was defined that summarizes student performance on the

25-minute items.

20.3.1 Item Parameter Estimation

Item parameter estimates were obtained for the univariate writing achievement scale by using the

NAEP BILOG/PARSCALE program, which combines Mislevy and Bock’s (1982) BILOG and Muraki

and Bock’s (1991) PARSCALE computer programs. The program uses marginal estimation procedures to

estimate the parameters of the one-, two-, and three-parameter logistic models, and the generalized

partial-credit model described by Muraki (1992) (see Chapter 12). In the writing assessment, only the

partial-credit model was used. Although only two prompts are present in any booklet, each booklet is

administered to a randomly equivalent sample of students by employing a spiral procedure of assigning

books to students (see Section 20.2).

The accommodated students were weighted down in the scaling analysis. This is because all

accommodated students were assigned to the same book. With 40 books and each item occurring in 4

books, this implies that accommodated students were oversampled for these items by a factor of 40/4,

(i.e., there were 10 times too many accommodated students). As a result, the accommodated students

were weighted down by a factor of 4/40 (1/10) to make their influence on the items the same as would

occur in a representative sample. As with the item analysis, weights were normalized (multiplied by a

constant) so that the sum of the weights was equal to the sample size.

BILOG/PARSCALE was run with model assumptions to more accurately account for the

influence of accommodated students. Two subgroups were defined, one for accommodated and the other

for nonaccommodated students. Separate prior achievement scale distributions were estimated for the

two subgroups. The subgroup priors were defined as normal with combined mean equal to zero and the

combined standard deviation equal to one. The means and standard deviations of the subsamples were
376

free to vary. As it turned out, the accommodated group mean was always lower than the

nonaccommodated group, and the subgroup variances were less than one. The scale was transformed to

the reporting metric with an overall mean of 150 and overall standard deviation of 35, in a later stage of

the analysis (see Section 20.5).

As with the item analysis, “missing” responses (i.e., students did not reach the task, or provided

an off-task response) were treated as “not presented,” (i.e., were not given a score and were not used in

IRT calibration).

Empirical Bayes modal estimates of all item parameters were obtained from the

BILOG/PARSCALE program. Prior distributions were imposed on item parameters with the following

starting values: thresholds (normal [0,2]); slopes (log-normal [0,.5]); and asymptotes (two-parameter beta

with parameter values determined as functions of the number of response options for an item and a

weight factor of 50). The locations (but not the dispersions) of the item parameter prior distributions

were updated at each program-estimation cycle in accordance with provisional estimates of the item

parameters. Starting values were computed from item statistics. Item parameters are listed in Appendix E.

20.3.2 Evaluation of Model Fit

During and subsequent to item parameter estimation, an evaluation of the fit of the IRT models

was carried out for each of the items in the item pool. These evaluations were conducted to determine if

any items had to be dropped or have categories collapsed. Evaluations of model fit were based primarily

on graphical analyses. The 6-category polytomous items are depicted by graphs that display response

curves for each item category (see Chapter 12). The model-based (theoretical) item category curves were

compared with empirical response plots derived from the observed responses. An item’s fit was assessed

by comparing the theoretical curves with the empirical ones. The closer they coincide, the better the fit.

As with most procedures that involve evaluating plots of data versus model predictions, a certain

degree of subjectivity was involved in determining the degree of fit necessary to justify use of the model.

The seemingly objective procedures of assessing model fit based on goodness-of-fit indices such as the

"pseudo chi-squares" produced in BILOG (Mislevy & Bock, 1982) cannot be used as an absolute gauge

of fit. The exact sampling distributions of these indices when the model fits are not well understood, even

for fairly long tests. Mislevy and Stocking (1989) point out that the usefulness of these indices appears

particularly limited in situations like NAEP, where examinees have been administered relatively short

tests. A study by Stone, Mislevy, and Mazzeo (1994) using simulated data suggests that the correct

reference chi-square distributions for these indices have considerably fewer degrees of freedom than the

value indicated by the BILOG/PARSCALE program and require additional adjustments of scale.

However, it is not yet clear how to estimate the correct number of degrees of freedom and necessary

scale factor adjustment factors. Consequently, pseudo chi-square goodness-of-fit indices were used only

as rough guides in interpreting the severity of model departures.

In the case of the writing assessment, there was not much information with which to evaluate

model fit. Since there were only, at most, two items administered to each respondent, about half of the

achievement scale was determined by the item being evaluated for fit. The IRT model fits well if higher

levels of the scale are associated with higher score levels on an item. Since much of a person’s scale

score was determined by the item in question, items almost always fit. Without an independent measure

of achievement, with only two items per person, item fit will usually be (trivially) good.

As expected, the fit of the model to the item responses was good for all items. Figure 20-1

provides an example of a particularly good-fitting item. In the plot, the y-axis indicates the probability of

a correct response and the x-axis indicates scale score level (theta). The diamonds show empirical
377

estimates of item category responses. The sizes of the diamonds are proportional to the estimated sample

size at the indicated value. The solid curve shows the estimated theoretical item response function. The


	


	




 

when a logistic response function is assumed.2

 Also shown in the plot are the values of the item

parameter estimates (in the box on the left side). As is evident from the plot, the empirical item category

traces are in extremely close agreement with the model-based item response function curves.

Figure 20-1

Polytomous Item (W010002) Exhibiting Good Model Fit*

* Diamonds represent 1998 grade 12 writing assessment data. They indicate estimated

conditional probabilities obtained without assuming a specific model form; the curve indicates

the estimated item category response function (ICRF) using a generalized partial credit model.

Figure 20-2 shows an item with poorer fit. This is especially true for the lower end of the

achievement distribution, where the empirical plots for two category functions (diamonds) are quite far

from the theoretical item category function (solid line). Fortunately, this misfit represents a very small

portion of the respondents, as is evidenced by the small size of the diamonds. This is the poorest fitting

item even though the figure shows quite good fit. As a result, it was not necessary to delete or collapse

categories for any items to improve the fit of the model.

20.4 GENERATION OF PLAUSIBLE VALUES

20.4.1 Principal Components (NSWEEP Program)

Univariate plausible values were generated for each sample using the univariate conditioning

program BGROUP as written by Thomas (1993b). This procedure employed student weights. Prior to the

1990 assessment, selected background variables were used for conditioning. However, from 1990 to the

present, principal components of the background variables have been used as conditioning variables.

Almost all of the background variables were coded as 0-1 contrasts, so no standardization took place.

 

2 Note that in the generalized partial-credit model, the displayed theoretical curves are not logistic. Rather, logistic curves

represent the conditional probabilities given adjacent values, so that 

( ) ( ( 1) , ) ( ( 1) ) ( )Px k

P x k x k or x k

Px k Px k

θ

θ

θ θ

= = =− = = =− + = is logistic.
378

Principal components of these contrasts were employed to remedy problems of extreme collinearity

among some of the original conditioning variables. The principal components used accounted for at least

90 percent of the variance of the original conditioning variables.

Figure 20-2

Polytomous Item (W008402) Exhibiting Less Than Optimal Model Fit*

* Diamonds represent 1998 grade 12 writing assessment data. They indicate estimated conditional

probabilities obtained without assuming a specific model form; the curve indicates the estimated item

category response function (ICRF) using a generalized partial credit model.

Results from research on the 1990 trial state assessment in mathematics suggests that using a

large subset of principal components will yield estimates that differ only slightly from those obtained

using the full set (Mazzeo et al., 1992). Table 20-4 contains a list of the number of principal components

included in conditioning, as well as the proportion of variance accounted for by the conditioning model

for each grade.

Table 20-4

Proportion of Scale Score Variance Accounted for by the Conditioning Model

for the 1998 National Main Writing Assessment

Grade

Number of

Conditioning Contrasts*

Number of

Principal Components*

Proportion of

Scale Score Variance

Accounted For

4 1,095 416 .53

8 1,123 405 .62

12 633 255 .59

*

 Excluding the constant term
379

20.4.2 Conditioning (BGROUP Program)

The codings of the original writing-specific conditioning variables, before principal components

were calculated, are presented in Appendix F. NAEP BGROUP (described in Chapter 12) creates

posterior distributions of scale scores by combining information from item responses of individuals and

information from linear regression of scale score on conditioning variables. For each individual, five

plausible values were randomly drawn from their posterior scale distribution.

The values of the conditioning effects were expressed in the metrics of the original calibration

scale. Definitions of derived conditioning variables are given in Appendix G.

20.5 FINAL REPORTING SCALES

Like all IRT scales, the writing scales have a linear indeterminacy that may be resolved by an

arbitrary choice of origin and unit size. The 1998 writing assessment was developed using a new

definition of the content domain of the items (see Section 18.2). Because it was not appropriate to

compare results from the 1998 assessment with those of previous NAEP writing assessments, no attempt

was made to link or align scores on the new assessment to those of previous assessments. Therefore, it

was necessary to establish a new scale for reporting. The NAGB has decided that all NAEP scales will be

defined within-grade. As a result, the univariate writing achievement scales at each grade were

transformed to a reporting metric with scale points ranging from 0 to 300, with an overall mean of 150

and with a standard deviation of 35. Because of the arbitrary nature of the metric, cross-grade

comparisons are meaningless.

At each grade the writing scale was transformed from the original scaling metric (mean 0, SD=1)

to the reporting metric (mean 150, SD=35) using the transformation:

reporting = A • scaling + B.

with scaling being the scale score in the scaling metric (approximately mean=0, SD=1), and reporting being

the scale the scale score in the reporting metric (mean=150, SD=35). Calculation of the constants for this

linear transformation, "A" and "B", is described in Chapter 9. These linear transformation constants are

given for each grade in Table 20-5. As previously mentioned, the scaling metric is roughly standardized

with mean about 0 and standard deviation about 1 and the scale score metric has mean 150 and standard

deviation 35. As a result, one would expect all A’s to be 35 and all B’s to be 150. As Table 20-5 shows,

this is not the case. The reason is that accommodated students were weighted differently in the scaling

and conditioning phases of analysis.

Table 20-5

Coefficients of Linear Transformations of the Writing Scales

from the Scaling Metric to the Reporting Metric

Sample A B

Grade 4 34.01 152.24

Grade 8 34.06 151.50

Grade 12 34.54 151.11
380

20.6 PARTITIONING OF THE ESTIMATION ERROR VARIANCE

For each grade, the error variance of the final, transformed scale mean was partitioned as

described in Chapter 12. The variance was partitioned into two parts: the proportion of error variance due

to sampling students (sampling variance) and the proportion of variance due to the fact that the scale

score, , is a latent variable that was estimated rather than observed. Table 20-6 contains estimates of the

total error variance, the proportion due to sampling of students, and the proportion due to the latent

nature of scale scores. To get greater stability of the variance estimates, they are based on drawing 100

imputations from the posterior achievement distribution of each student. More detailed information of

proportion of variance by gender and race/ethnicity is presented in Appendix H.

Table 20-6

Estimation Error Variance and Related Coefficients

for the National Main Writing Assessment

Proportion of Variance Due to . . .

Grade Student Sampling Latency of 

4 .90 .10

8 .94 .06

12 .93 .07

20.7 WRITING TEACHER QUESTIONNAIRES

Teachers of fourth- and eighth-grade students were surveyed about their educational background

and teaching practices. Each student’s records were matched with his or her teacher’s survey

information. Variables derived from the questionnaire were used in the conditioning models, along with a

variable that indicated whether a student record had been matched with a teacher record, which controls

estimates of subgroup means for differences that exist between the matching and nonmatching students.

Of the 19,816 fourth-grade students in the sample, 89 percent were matched with both parts of the

teacher questionnaire and 4 percent were matched with only the first, teacher background, part of the

questionnaire. Of the 20,586 eighth-grade students sampled, 72 percent were matched with both parts of

the teacher questionnaire and 8 percent were matched with only the first part (the demographic

background section) of the questionnaire. The lower match rate for both parts of the questionnaire for

eighth-grade students was due in part to the fact that in grade 8 students were matched to the particular

class that the teacher taught. Class membership information was often missing or ambiguous. For grade

4, students only had to be matched to the main teacher, resulting in higher match rates. Thus, 93 percent

of the fourth-graders and 79 percent of the eighth-graders were matched with at least the background

information about their writing teachers.
381

Chapter 21

DATA ANALYSIS OF THE STATE WRITING ASSESSMENT1

Jiahe Qian, Hua-Hua Chang, Bruce A. Kaplan, Jo-Lin Liang, and Youn-Hee Lim

Educational Testing Service

21.1 INTRODUCTION

This chapter describes the analyses used in developing the 1998 state assessment writing scale.

The 1998 state writing assessment was administered to eighth-grade public- and nonpublic-school

students for 40 jurisdictions. This was the first state assessment in writing. The procedures used were

similar to those employed in the analysis of the 1990, 1992, and 1996 state assessments in mathematics

(Jenkins, Kulick, Kaplan, Wang, Qian, Wang, 1997; Mazzeo, 1991; Mazzeo, Chang, Kulick, Fong, &

Grima, 1993), the 1992 and 1994 state assessments in reading (Allen, Mazzeo, Ip, Swinton, Isham, &

Worthington, 1995; Allen, Mazzeo, Isham, Fong, & Bowker, 1994), and are based on the philosophical

and theoretical rationale given in Chapter 12. For 1998, the NAEP writing assessment framework

incorporated a balance of knowledge and skills based on current reform reports, exemplary curriculum

guides, and research on the teaching and learning of writing. The NAEP report card for state assessments

presents average scale scores and achievement-level results for public-school students. In the 1998 state

assessment, an attempt was made to include more students with disabilities (SD) and students with

limited English proficiency (LEP) by liberalizing inclusion rules allowing for accommodations. Although

the 1998 state writing analysis is the first state writing assessment, comparisons of writing results for

state and national assessments are essential. The sample of students used for analysis and reporting was

formed so that comparable inclusion rules were used.

There were four major steps in the analysis of the state assessment writing data, each of which is

described in a separate section:

• Conventional item and test analyses (Section 21.2)

• Item response theory (IRT) scaling (Section 21.3)

• Estimation of state and subgroup scale score distributions based on the “plausible

values” methodology (Section 21.4)

• Linking of the 1998 state assessment scales to the corresponding scales from the

1998 national assessment (Section 21.5)

For the context of the assessment instruments and administration procedures of the writing

assessments, see Section 19.2.

 

1 Jiahe Qian was the primary person responsible for the planning, specification, and coordination of the state writing analyses in

collaboration with Hua-Hua Chang. Computing activities for all writing scaling and data analyses were directed by Bruce A.

Kaplan and completed by Youn-Hee Lim and Ting Lu. Others contributing to the analysis of writing data were David S. Freund,

Jo-Lin Liang, and Katharine E. Pashley.
382

21.2 STATE ITEM ANALYSES

21.2.1 Conventional Item and Test Analyses

This section contains a detailed description of the item analysis performed on the state writing

data. As was discussed in Chapter 20, only the 25-minute writing blocks were included in the writing

scale. Because there is only one item per block, all twenty 25-minute blocks (items) were treated together

as one large block in the item analysis. The main statistics analyzed are mean item scores and frequencies

of responses at each score point. Table 21-1 contains summary statistics for overall samples and by the

order of the block within booklet, based on the data from all 40 jurisdictions. The senate weights were

used in item analysis and scaling procedure (see Sections 15.5 and 17.5). Use of the senate weights does

nothing to alter the value of statistics calculated separately within each jurisdiction. Items W006402 and

W007602 were presented to accommodated students in the writing assessment. To make the statistics

comparable with those of other items, the accommodated students were not included in the item analysis

calculation.

For statistics obtained from samples that combine students from different jurisdictions, use of the

senate weights results in a roughly equal contribution of each jurisdiction’s data to the final value of the

estimate. As discussed in Mazzeo (1991), equal contribution of each jurisdiction’s data to the results of

the IRT scaling was viewed as a desirable outcome and the same rescaled weights were only adjusted

slightly in carrying out that scaling. Hence, the item analysis statistics shown in Table 21-1 is

approximately consistent with the weighting used in scaling.

Table 21-1 shows the number of students assigned each item, the average item scores and the

percentage of students in each category of an item. For the constructed-response items in the writing

assessment, the score means were calculated as item score mean. As is evident from Table 21-1, the

difficulty of the items did not vary greatly.

This table also indicates that there was little variability in average item scores by block position

within the assessment booklet. The differences in item statistics were small for items appearing in blocks

in the first position and in the second position. However, differences were consistent in their direction.

The average item scores were higher when each block was presented in the first position.

In an attempt to maintain rigorous standardized administration procedures across the

jurisdictions, a Westat-trained quality control monitor would observe randomly selected sessions within

each jurisdiction. If a jurisdiction had never participated in a state assessment, a randomly selected 50

percent of the sessions within jurisdictions were monitored; otherwise, a 25 percent of sampled sessions

would be monitored within jurisdictions. Because all jurisdictions in the 1998 state writing assessment

had participated in previous state assessments, 25 percent of sessions were monitored in each

jurisdiction. Observations from the monitored sessions provided information about the quality of

administration procedures and the frequency of departures from standardized procedures in the

monitored sessions.

The 1998 state assessment in writing included students sampled from nonpublic schools. The

nonpublic-school population that was sampled included students from Catholic schools, private religious

schools, and private nonreligious schools (all referred to by the term “nonpublic school”). Table 21-2

contains the item descriptive statistics for total, public-school sessions, and nonpublic-school sessions,

respectively. Of the 40 jurisdictions that reported in the state assessment in writing, 39 had public-school

samples, while 18 of the 40 jurisdictions had nonpublic-school samples that met reporting requirements.
 Table 21-1

Descriptive Statistics Writing Prompts, Writing 25-Minute State Samples, Grade 8

Percentage of Students in Each Category

Item ID Description n Off-task 0 1 2 3 4 5

Total Item

Mean

1st P

Item 

W006002 Cartoon Story 9,190 3.70 0.81 12.14 30.79 33.76 17.47 5.04 3.70 W006102 President for a Day 9,272 1.67 0.66 12.30 30.43 38.37 14.37 3.87 3.65 

W006202 Plums 9,274 1.82 0.88 14.26 38.39 31.34 12.21 2.92 3.48 

W006302 Tower 9,300 1.64 6.20 6.18 23.63 38.62 21.50 3.87 3.75 

W006402† Principal for a Day 9,337 1.87 3.04 8.09 20.87 40.45 19.70 7.84 3.89 

W006502 Pioneer Journal 9,316 2.37 0.91 5.46 21.99 45.99 22.35 3.29 3.93 W006602 Space Visitor 9,376 2.43 0.96 10.52 20.51 49.25 14.48 4.29 3.79 

W006802 Performance Review 9,261 2.18 1.07 8.05 32.14 43.56 12.12 3.05 3.67 W006902 New Park 9,392 1.22 1.05 8.61 29.01 48.80 10.31 2.23 3.65 

W007002 Dream Weekend 9,428 1.43 1.01 6.76 28.58 48.04 12.72 2.88 3.73 W007102 Backpack 9,262 1.61 1.89 5.47 24.04 54.24 12.38 1.98 3.76 

W007202 Designing a TV Show 9,260 1.78 2.65 12.56 44.97 31.62 6.61 1.60 3.32 

W007302 Save a Book 9,286 2.38 3.10 9.55 32.05 47.23 6.20 1.87 3.49 

W007402 Life’s Lessons 9,291 2.14 3.14 7.90 26.78 41.83 15.98 4.36 3.73 

W007602† Lengthening School Year 9,430 1.47 2.67 11.04 36.07 34.86 12.15 3.22 3.52 

W007702 School Schedule 9,344 1.79 2.73 11.55 38.20 40.08 6.76 0.68 3.39 

W007802 Fast Food 9,335 1.57 4.99 7.63 27.52 40.82 15.16 3.87 3.65 

W007902 Class Trip 9,370 1.21 2.02 10.32 38.28 41.53 6.21 1.64 3.44 

W008002 Driving Age 9,315 1.87 1.40 10.72 34.24 41.94 10.91 0.78 3.53 

W008102 Teens in Malls 9,326 1.51 4.23 11.10 29.33 40.35 12.07 2.91 3.54 

* The means were calculated by coding responses from 1 to 6, according to standard IA procedures.† This item was presented to the accommodated students in the writing assessment. To make the comparisons of statistics comparable with those of 

accommodated students were not included in the item analysis calculation.

Key:n = Unweighted sample size0 = Unsatisfactory1 = Insufficient2 = Uneven3 = Sufficient

4 = Skilled5 = Excellent
384

Table 21-2

Descriptive Statistics for Each Item of the Writing State Assessment

Using Senate Weights (Scaled from 0 to 5) , Grade 8

Public and Private Public 

n Mean* n Mean* n 

Item ID Overall

1st

Position

2nd

Position Overall

1st

Position

2nd

Position Overall Mon. Unmon. Overall Mon. Unmon. Overall Mon. 

W006002 9,190 4,556 4,634 3.70 3.80 3.57 8,399 2,094 6,305 3.66 3.68 3.66 791 242 W006102 9,272 4,615 4,657 3.65 3.70 3.58 8,445 2,093 6,352 3.60 3.65 3.58 827 252 W006202 9,274 4,635 4,639 3.48 3.60 3.40 8,453 2,136 6,317 3.46 3.45 3.47 821 248 W006302 9,300 4,654 4,646 3.75 3.81 3.68 8,474 2,074 6,400 3.71 3.65 3.73 826 252 W006402† 9,337 4,603 4,734 3.89 4.03 3.83 8,539 2,168 6,371 3.91 3.92 3.90 798 255 W006502 9,316 4,694 4,622 3.93 4.00 3.86 8,466 2,108 6,358 3.91 3.91 3.91 850 260 W006602 9,376 4,694 4,682 3.79 3.87 3.73 8,567 2,116 6,451 3.78 3.81 3.77 809 253 W006802 9,261 4,643 4,618 3.67 3.74 3.62 8,458 2,031 6,427 3.66 3.65 3.66 803 233 W006902 9,392 4,663 4,729 3.65 3.79 3.56 8,590 2,102 6,488 3.65 3.67 3.64 802 235 W007002 9,428 4,722 4,706 3.73 3.85 3.66 8,601 2,146 6,455 3.72 3.78 3.70 827 251 W007102 9,262 4,611 4,651 3.76 3.86 3.67 8,485 2,090 6,395 3.75 3.71 3.76 777 240 W007202 9,260 4,624 4,636 3.32 3.38 3.26 8,443 2,041 6,402 3.29 3.29 3.28 817 244 W007302 9,286 4,606 4,680 3.49 3.60 3.41 8,474 2,021 6,453 3.47 3.46 3.48 812 242 W007402 9,291 4,638 4,653 3.73 3.84 3.65 8,465 2,066 6,399 3.72 3.72 3.72 826 244 W007602† 9,430 4,715 4,715 3.52 3.62 3.52 8,573 2,111 6,462 3.54 3.50 3.55 857 248 W007702 9,344 4,670 4,674 3.39 3.47 3.32 8,491 2,030 6,461 3.36 3.37 3.36 853 246 W007802 9,335 4,650 4,685 3.65 3.78 3.58 8,513 2,044 6,469 3.66 3.61 3.67 822 239 W007902 9,370 4,699 4,671 3.44 3.61 3.38 8,531 2,025 6,506 3.47 3.46 3.47 839 246 W008002 9,315 4,639 4,676 3.53 3.59 3.49 8,477 2,102 6,375 3.51 3.54 3.50 838 244 W008102 9,326 4,662 4,664 3.54 3.66 3.47 8,464 2,044 6,420 3.53 3.54 3.53 862 247 

 Mon. = Monitored Unmon. = Unmonitored

* The means were calculated by coding responses from 1 to 6, according to standard IA procedures.† This item was presented to the accommodated students in the writing assessment. To make the comparisons of statistics comparable with those of other i

were not included in the item analysis calculation.
385

Consistent differences were evident between the public- and nonpublic-school students. The

difference in average item score between public- and nonpublic-school students (i.e., public item mean

minus nonpublic item mean) range from -0.54 to -0.26 with an average of -0.40, indicating that publicschool students were generally lower in average item scores.

Within each school type session, Table 21-2 also provides the item descriptive statistics for the

monitored or unmonitored sessions. When results were aggregated over all participating jurisdictions,

there was little difference between the performance of students who attended monitored or unmonitored

sessions. When public-school results were aggregated over all participating jurisdictions, there was little

difference between the performance of students who attended monitored or unmonitored sessions. For

nonpublic-school data, the difference was also very small. The average item score was 3.62 for both

monitored public-school sessions and unmonitored public-school sessions. The average item score was

4.01 for monitored nonpublic-school sessions and 4.03 for unmonitored nonpublic-school sessions.

Table 21-3 summarizes the differences between monitored and unmonitored average item scores

for the jurisdictions. These are mean differences within a jurisdiction averaged over all items in all the

booklets. The information in the table combines public- and nonpublic-school data. The mean difference

and median difference were close to zero. There are 15 jurisdictions with negative differences (i.e.,

students from unmonitored sessions scored higher than students from monitored sessions). None were

larger in absolute magnitude than 0.083. The results indicate that across jurisdictions, the differences

between monitored and unmonitored sessions were relatively small.

21.3 STATE IRT SCALING

21.3.1 Samples Used in State IRT Scaling

As in other state assessments, a single set of item parameters for each item was estimated and

used for all jurisdictions (Mazzeo, 1991). Item parameter estimation was carried out using a 25 percent

systematic random sample of the public-school students participating in the 1998 state assessment and

included equal numbers of students from each participating jurisdiction, half from monitored sessions

and half from unmonitored sessions whenever possible. All students in the scaling sample were publicschool students. The sample consisted of 89,164 students, with 590 students being sampled from each of

the 39 participating jurisdictions (excluding DoDEA/DDESS2

 and DoDEA/DoDDS3 schools). Of the 590

records sampled from each jurisdiction, 295 were drawn from the monitored sessions and 295 were

drawn from the unmonitored sessions. There were not enough monitored students in the District of

Columbia and Virgin Islands to sample these two jurisdictions. All the monitored students were taken in

these two jurisdictions. The rescaled weights for the 25 percent sample of students used in item

calibration were adjusted slightly to ensure that (1) each jurisdiction’s data contributed equally to the

estimation process, and (2) data from monitored and unmonitored sessions contributed equally. All

calibrations were carried out using the rescaled sampling weights described in Section 11.3 in an effort to

ensure that each jurisdiction’s data contributed equally to the determination of the item parameter

estimates.

 

2 DoDEA/DDESS is the Department of Defense Education Activity Department of Defense Domestic Dependent Elementary and

Secondary Schools.

3 DoDEA/DoDDS is the Department of Defense Education Activity Department of Defense Dependents Schools.
386

Table 21-3

Effect of Monitoring Sessions by Jurisdiction:

Average Jurisdiction Item Scores for Monitored and Unmonitored Sessions, Grade 8

Jurisdiction

Monitored

Mean

Unmonitored

Mean Monitored – Unmonitored

Alabama 0.488 0.495 -0.007

Arizona 0.498 0.502 -0.004

Arkansas 0.488 0.475 0.013

California 0.508 0.494 0.014

Colorado 0.539 0.536 0.002

Connecticut 0.610 0.593 0.016

Delaware 0.556 0.487 0.069

Florida 0.497 0.491 0.006

Georgia 0.515 0.517 -0.002

Hawaii 0.483 0.452 0.031

Kentucky 0.525 0.512 0.014

Louisiana 0.472 0.486 -0.014

Maine 0.547 0.558 -0.012

Maryland 0.548 0.528 0.021

Massachusetts 0.562 0.563 -0.001

Minnesota 0.520 0.519 0.002

Mississippi 0.469 0.450 0.019

Missouri 0.527 0.512 0.015

Montana 0.511 0.538 -0.027

Nevada 0.497 0.477 0.021

New Mexico 0.502 0.496 0.006

New York 0.509 0.519 -0.010

North Carolina 0.552 0.541 0.011

Oklahoma 0.535 0.536 -0.001

Oregon 0.512 0.527 -0.015

Rhode Island 0.552 0.527 0.025

South Carolina 0.492 0.480 0.013

Tennessee 0.509 0.519 -0.010

Texas 0.533 0.550 -0.017

Utah 0.508 0.488 0.020

Virginia 0.560 0.537 0.024

Washington 0.512 0.526 -0.014

West Virginia 0.516 0.511 0.004

Wisconsin 0.564 0.536 0.028

Wyoming 0.510 0.509 0.000

District of Columbia 0.430 0.412 0.018

DoDEA/DDESS 0.598 0.564 0.034

DoDEA/DoDDS 0.550 0.558 -0.008

Virgin Islands 0.355 0.438 -0.083 

Mean 0.007

Median 0.006

Minimum -0.027

1st Quartile -0.006

3rd Quartile 0.019

Maximum 0.069
387

Only public-school data were used in the scaling models for the state assessments. Based on the

analysis of item response function plots for the public/nonpublic comparisons, the public/nonpublic data

have similar item response functions for the state writing sample. The plots of empirical and model-based

estimates of the item response function were used to study the appropriateness. Each plot contained three

estimates of each item category characteristic curve: two sets of empirical estimates that represented

public- and nonpublic-school samples, respectively, were compared with a third set that assumed the

partial-credit model, which was estimated from public-school data only. The plots for all the items

showed reasonable closeness between two empirical curves and the theoretical curve.

21.3.2 Item Parameter Estimation

For the 1998 state assessment, a writing IRT-based scale was developed using the generalized

partial-credit model described in Chapter 12. The item parameter estimates were obtained using the

NAEP BILOG/PARSCALE program, which combines Mislevy and Bock’s (1982) BILOG and Muraki

and Bock’s (1991) PARSCALE computer programs. The program uses marginal maximum likelihood

estimation procedures to estimate the parameters (Muraki, 1992).

All the items in writing assessments were extended constructed-response items. Each of these

items was also scaled using the generalized partial-credit model. Six scoring levels were defined:

0 = Unsatisfactory

1 = Insufficient Response

2 = Uneven Response

3 = Sufficient Response

4 = Skilled Response

5 = Excellent Response

As was done in previous assessments of writing, “missing” responses (i.e., students did not reach

the task, or provided an off-task response) were treated as if the item had not been presented to the

student. (See Section 12.3.1 for more information on this topic.)

Empirical Bayes modal estimates of all item parameters were obtained from the

BILOG/PARSCALE program. Item parameter estimation proceeded as follows. The subject ability

distribution was assumed fixed (normal [0,1]) and a stable solution was obtained. Starting values for the

item parameters were provided by item analysis routines. After each estimation cycle, the subject ability

distribution was restandardized to have a mean of 0 and standard deviation of 1. Correspondingly,

parameter estimates for that cycle were also linearly standardized. Two items, W006402 and W007602,

were presented to the accommodated students in the state assessment. The data of accommodated

students were calibrated as a separate population in the scaling procedure. Their weights were

appropriately reduced to the proportion of the students in the student group who took the items in the

test.

During and subsequent to item parameter estimation, evaluations of the fit of the IRT models

were carried out for each of the items in the item pool. These evaluations were conducted to determine

the final composition of the item pool making up the scales by identifying misfitting items that should not

be included. Evaluations of model fit were based primarily on graphical analyses.

As with most procedures that involve evaluating plots of data versus model predictions, a certain

degree of subjectivity is involved in determining the degree of fit necessary to justify use of the model.

There are a number of reasons why evaluation of model fit relied primarily on analyses of plots rather

than seemingly more objective procedures based on goodness-of-fit indices such as the “pseudo chi-
388

squares” produced in BILOG (Mislevy & Bock, 1982). First, the exact sampling distributions of these

indices when the model fits are not well understood, even for fairly long tests. Mislevy and Stocking

(1989) point out that the usefulness of these indices appears particularly limited in situations like NAEP,

where examinees have been administered relatively short tests. Studies by Stone, Ankenmann, Lane, and

Liu (1993), and by Stone, Mislevy, and Mazzeo (1994) using simulated data suggest that the correct

reference chi-square distributions for these indices have considerably fewer degrees of freedom than the

value indicated by the BILOG/PARSCALE program and require additional adjustments of scale.

However, it is not yet clear how to estimate the correct number of degrees of freedom and necessary

scale factor adjustment factors. Consequently, pseudo chi-square goodness-of-fit indices are used only as

rough guides in interpreting the severity of model departures.

Second, as discussed in Chapter 12, it is almost certainly the case that, for most items, item

response models hold only to a certain degree of approximation. Given the large sample sizes used in the

state assessment, there will be sets of items for which one is almost certain to reject the hypothesis that

the model fits the data even though departures are minimal in nature or involve kinds of misfit unlikely to

impact on important model-based inferences. In practice, one is almost always forced to temper decisions

based on hypothesis testing with judgments about the severity of model misfit and the potential impact of

such misfit on final results.

For all of the items of the state writing assessment, the fit of the model was extremely good.

Figure 21-1 and Figure 21-2 provide typical examples of what the plots look like for this class of items.

The item W006502 in Figure 21-1, an extended constructed-response item, has a good fit. This plot

shows two estimates of each item category characteristic curve, one set that does not assume the

generalized partial-credit model (shown as diamonds) and one that does (the solid curves). The estimates

for all parameters for the item in question are also indicated on the plot. As shown by the figure, the

estimates agree quite well, although some diamonds on the empirical curve lie above the theoretical

curve in the lowest category. They contain just a few students. The sizes of the diamonds are

proportional to the number of students categorized as having thetas at or close to the indicated value.

Although few student responses were categorized in the highest category, there were adequate data to

estimate the model-based estimates for those categories (the solid curves). Such results were typical for

the extended constructed-response items.

The plot of item W007602 in Figure 21-2 shows three estimates of each item category

characteristic curve, one that assumes the partial-credit model (the solid curves) that was fit on the

accommodated and nonaccommodated cases together, and two sets that do not assume the generalized

partial-credit model (shown as diamonds for nonaccommodated cases and circles for accommodated

cases). The figure also shows a very good fit, except for some accommodated cases lying above

theoretical curve in the third category.

As discussed above, all of the items retained for the final scaling display good model fit. No item

needed to be recoded for the state writing assessment. The IRT parameters for the items included in the

state assessment are listed in Appendix E.

21.4 GENERATION OF PLAUSIBLE VALUES

The scale score distributions in each jurisdiction (and for some demographic subgroups within

each jurisdiction) were estimated by using the univariate plausible values methodology and the

corresponding BGROUP computer program. As described in Chapter 12, the BGROUP program

estimates scale score distributions using information from student item responses, measures of student

background variables, and the item parameter estimates obtained from the BILOG/PARSCALE program.
389

Results from Mazzeo’s research (1991) suggested that separate conditioning models needed to be

estimated for each jurisdiction because the parameters estimated by the conditioning model differed

across jurisdictions. If a jurisdiction had a nonpublic-school sample, students from that sample were

included in this part of the analysis, and a conditioning variable differentiating between public- and

nonpublic-school students was included. This resulted in the estimation of 41 distinct conditioning

models for the eighth-grade 1998 state writing assessment.

Figure 21-1

Polytomous Item (W006502) Exhibiting Good Model Fit*

* Diamonds represent 1998 grade 8 writing assessment data. They indicate estimated conditional

probabilities obtained without assuming a specific model form; the curve indicates the estimated item

category response function (ICRF) using a generalized partial credit model.

Reporting each jurisdiction’s results required analyses describing the relationships between scale

scores and a large number of background variables. The background variables included in each

jurisdiction’s model were principal component scores derived from the within-jurisdiction correlation

matrix of selected main-effects and two-way interactions associated with a wide range of student,

teacher, school, and community variables. The background variables included student demographic
390

characteristics (e.g., the race/ethnicity of the student, highest level of education attained by parents, status

of test accommodation), students’ perceptions about writing, student behavior both in and out of school

(e.g., amount of TV watched daily, amount of writing homework done each day), the type of writing

class being taken, and a variety of other aspects of the students’ background and preparation, and the

educational, social, and financial environment of the schools they attended. Information also was

collected from students’ teachers about the types of educational practice, such as the amount of

classroom emphasis on various topics included in the assessment provided by the students’ teachers, the

background and preparation of their teachers.

Figure 21-2

Polytomous Item (W007602) Exhibiting Good Model Fit*

* Diamonds represent 1998 grade 8 writing assessment data. They indicate estimated

conditional probabilities obtained without assuming a specific model form; the curve

indicates the estimated item category response function (ICRF) using a generalized

partial credit model.

As described in the Chapter 12, to avoid biases in reporting results and to minimize biases in

secondary analyses, it is desirable to incorporate measures of a large number of independent variables in
391

the conditioning model. When expressed in terms of contrast-coded main effects and interactions, the

number of variables to be included totaled 1,129. Appendix F provides a listing of the full set of contrasts

defined. These contrasts were the common starting point in the development of the conditioning models

for each of the participating jurisdictions.

Because of the large number of these contrasts and the fact that, within each jurisdiction, some

contrasts had zero variance, some involved relatively small numbers of individuals, and some were

highly correlated with other contrasts or sets of contrasts, an effort was made to reduce the

dimensionality of the predictor variables in each jurisdiction’s BGROUP models. As was done for the

1990, 1992, and 1996 state assessments in mathematics and the 1992, 1994, and 1998 state assessment in

reading, the original background variable contrasts were standardized and transformed into a set of

linearly independent variables by extracting separate sets of principal components (one set for each of the

40 jurisdictions) from the within-jurisdiction correlation matrices of the original contrast variables. The

principal components, rather than the original variables, were used as the independent variables in the

conditioning model. As was done for the previous assessments, the number of principal components

included for each jurisdiction was the number required to account for approximately 90 percent of the

variance in the original contrast variables. Research based on data from the 1990 state assessment in

mathematics suggests that results obtained using such a subset of the components will differ only slightly

from those obtained using the full set (Mazzeo et al., 1992).

Table 21-4 lists the number of principal components included in and the proportion of scale score

variance accounted for by the conditioning model for each participating jurisdictions.

It is important to note that the proportion of variance accounted for by the conditioning model

differs across jurisdictions. Such variability is not unexpected for at least two reasons. First, there is no

reason to expect the strength of the relationship between scale score and demographics to be identical

across all jurisdictions. In fact, one of the reasons for fitting separate conditioning models is that the

strength and nature of this relationship may differ across jurisdictions. Second, the homogeneity of the

demographic profile also differs across jurisdictions. As with any correlation analysis, the restriction of

the range in the predictor variables will attenuate relationship.

Table 21-4

Proportion of Scale Score Variance Accounted by Conditioning Model

for the Writing State Assessment, Grade 8

Jurisdiction

Number of Principal

Components

Proportion of Scale

Score Variance*

Alabama 242 0.670

Arizona 264 0.704

Arkansas 249 0.731

California 270 0.752

Colorado 259 0.698

Connecticut 276 0.712

Delaware 198 0.775

Florida 284 0.647 

* (Total Variance - Residual Variance)/Total Variance, where Total Variance

consists of both sampling and measurement error variance

(continued)
392

Table 21-4 (continued)

Proportion of Scale Score Variance Accounted by Conditioning Model

for the Writing State Assessment, Grade 8

Jurisdiction

Number of Principal

Components

Proportion of Scale

Score Variance*

Georgia 293 0.732

Hawaii 213 0.665

Kentucky 240 0.699

Louisiana 274 0.696

Maine 228 0.657

Maryland 257 0.719

Massachusetts 256 0.714

Minnesota 219 0.705

Mississippi 241 0.663

Missouri 255 0.705

Montana 194 0.647

Nevada 229 0.685

New Mexico 260 0.709

New York 240 0.714

North Carolina 287 0.690

Oklahoma 232 0.680

Oregon 246 0.667

Rhode Island 225 0.718

South Carolina 290 0.766

Tennessee 234 0.711

Texas 263 0.664

Utah 267 0.621

Virginia 291 0.733

Washington 267 0.705

West Virginia 249 0.731

Wisconsin 214 0.672

Wyoming 200 0.641

District of Columbia 163 0.730

DoDEA/DDESS 142 0.834

DoDEA/DoDDS 173 0.667

Virgin Islands 138 0.841

* (Total Variance - Residual Variance)/Total Variance, where Total Variance

consists of both sampling and measurement error variance

As discussed in Chapter 12, NAEP scales are viewed as summaries of consistencies and

regularities that are present in item-level data. Such summaries should agree with other reasonable

summaries of the item-level data. In order to evaluate the reasonableness of the scaling and estimation

results, a variety of analyses were conducted to compare state-level and subgroup-level performance in

terms of the scaled scores and in terms of the average proportion correct for the set of items. High

agreement was found in all of these analyses. One set of such analyses is presented in Figure 21-3.
393

Figure 21-3

Plot of Mean Scale Score Versus Mean Item Score by Jurisdiction, Grade 8

The figure contains scatterplots of the state scaled score mean versus the state item score means,

for the writing scale. In calculating the statistics for both metrics, the accommodated students are

included. As is evident from the figures, there is an extremely strong relationship between the estimates

of state-level performance in the scale-score and item-score metrics.

21.5 FINAL SCORE SCALES

21.5.1 Linking State and National Scales

A major purpose of the state assessment program was to allow each participating jurisdiction to

compare its 1998 results with the nation as a whole and with the region of the country in which that

jurisdiction is located.

Although the students in the 1998 state writing assessment were administered the same test

booklets as the eighth-graders in the national assessment, separate state and national scalings were

carried out (for reasons explained in Mazzeo, 1991, and Yamamoto & Mazzeo, 1992). For meaningful

comparisons to be made between each of the state assessment jurisdictions and the relevant national

samples, results from these two assessments had to be expressed in terms of a similar system of scale

units. The purpose of this section is to describe the procedures used to align the 1998 state assessment

scales with their 1998 national counterparts. The procedures that were used represent an extension of the

common population equating procedures employed to link the previous national and state scales

(Mazzeo, 1991; Yamamoto & Mazzeo, 1992).

Using the house sampling weights provided by Westat, the combined sample of students from all

participating jurisdictions was used to estimate the distribution of scale scores for the population of

students enrolled in public schools that participated in the state assessment.4 The total sample size was

89,164. A subsample of the eighth-grade national sample, consisting of grade-eligible public-school

students from any of the 40 jurisdictions that participated in the 1998 state assessment, was used to

obtain estimates of the distribution of scale scores for the same target population. This subsample of

 

4 Students from Virgin Islands, DoDEA/DDESS, and DoDEA/DoDDS schools were excluded from the state aggregate sample for

purposes of linking.

Writing (Including Accommodated)

110

125

140

155

170

0.4 0.45 0.5 0.55 0.6

Mean Item Score

Mean Scale Score
394

national data is referred to as the national linking (NL)5

 sample, and appropriate NL weights were

obtained from Westat. Again, appropriate weights provided by Westat were used. Thus, for each scale,

two sets of scale score distributions were obtained and used in the linking process. One set, based on the

sample of combined data from the state assessment (referred to as the state aggregate, or SA), and using

item parameter estimates and conditioning results from that assessment, was in the metric of the 1998

state assessment. The other, based on the NL sample from the 1998 national assessment and obtained

using item parameters and conditioning results from the national assessment, was in the reporting metric

of the 1998 national assessment. The state assessment and national scales were made comparable by

constraining the mean and standard deviation of the two sets of estimates to be equal.

More specifically, the following steps were followed to linearly link the scales of the two

assessments:

1) For each scale, estimates of the scale score distribution for the SA sample was

obtained using the full set of plausible values generated by the BGROUP program.

The weights used were the final sampling weights provided by Westat (see Section

11.7). For each scale, the arithmetic mean of the five sets of plausible values was

taken as the overall estimated mean and the square root of arithmetic average of the

variances of the five sets of plausible values was taken as the overall estimated

standard deviation.

2) For each scale, the estimated scale score distribution of the NL sample was obtained,

again using the full set of plausible values generated by the BGROUP program. The

weights used were specially provided by Westat to allow for the estimation of scale

score distributions for the same target population of students estimated by the

jurisdiction data. The means and standard deviations of the distributions (in the 1998

national reporting metric) for each scale were obtained for this sample in the same

manner as described in Step 1.

3) For each scale, a set of linear transformation coefficients was obtained to link the

state scale to the corresponding national scale. The linking was of the form

 = A • + B

where

 = a scale score level in terms of the system of units of the

provisional BILOG/PARSCALE scale of the state assessment

scaling

 = a scale score level in terms of the system of units comparable to

those used for reporting the 1998 national writing results

A = [Standard DeviationNL]/[Standard DeviationSA]

B = MeanNL – A[MeanSA]

where the subscripts refer to the NL sample and to the SA sample.

 

5 Note that in previous state assessments, the national linking sample was called the state aggregate comparison, or SAC, sample.

Many people thought this was easy to confuse with state data, so the term “national linking” is used in this report.
395

The final conversion parameters for transforming plausible values from the provisional

BILOG/PARSCALE scales to the final state assessment reporting scales are given in Table 21-5. All

state assessment results are reported in terms of the  metric.

Table 21-5

Coefficients of Linear Transformations for the 1998 State Writing Assessment

Grade Writing Scale A B

8 State Writing 33.70 147.13

As is evident from the discussion above, a linear method was used to link the scales from the

state and national assessments. While these linear methods ensure equality of means and standard

deviations for the SA (after transformation) and the NL samples, they do not guarantee the shapes of the

estimated scale score distributions for the two samples to be the same. As these two samples are both

from a common target population, estimates of the scale score distribution of that target population based

on each of the samples should be quite similar in shape in order to justify strong claims of comparability

for the state and national scales. Substantial differences in the shapes of the two estimated distributions

would result in differing estimates of the percentages of students above achievement levels or of

percentile locations, depending on whether state or national scales were useda clearly unacceptable

result given claims about the comparability of the scales. In the face of such results, nonlinear linking

methods would be required.

Analyses were carried out to verify the degree to which the linear linking process described

above produced comparable scales for state and national results. Comparisons were made between two

estimated scale score distributions, one based on the SA sample and one based on the NL sample. The

comparisons were carried out using slightly modified versions of what Wainer (1974) refers to as

suspended rootograms. The final reporting scales for the state and national assessments were each

divided into 10-point intervals. Two sets of estimates of the percentage of students in each interval were

obtained, one based on the SA sample and one based on the NL sample. Following Tukey (1977), the

square roots of these estimated percentages were compared.6

 The comparisons are shown in Figure 21-4.

The height of each of the unshaded bar corresponds to the square root of the percentage of students from

the state assessment aggregate sample in each 10-point interval on the final reporting scale.

 

6 The square root transformation allows for more effective comparisons for counts (or equivalently, percentages) when the expected

number of counts in each interval is likely to vary greatly over the range of intervals, as is the case for the NAEP scales where the

expected counts of individuals in intervals near the extremes of the scale (e.g., below 50 and above 250) are dramatically smaller than

the counts obtained near the middle of the scale.
396

Figure 21-4

Rootogram Comparing Scale Score Distributions

for the State Assessment Aggregate Sample

and the National Linking Sample for the Composite Scale, Grade 8

The shaded bars show the differences in root percents between the NL and SA estimates. Positive

differences indicate intervals in which the estimated percentages from the SA sample are lower than

those obtained from the NL. Conversely, negative differences indicate intervals in which the estimated

percentages from the SA sample are higher. Differences in root percents are quite small, suggesting that

the shapes of the two estimated distributions are quite similar (i.e., unimodal with slight negative

skewness). There is some evidence that the estimates produced using the SA data are slightly heavier in

the extreme lower tails, below 50. However, even these differences at the extremes are small in

magnitude (0.3 in the root percent metric) and have little impact on estimates of reported statistics such

as percentages of students below the achievement levels.

21.6 PARTITIONING OF THE ESTIMATION ERROR VARIANCE

For each grade in state writing assessments, the error variance of the final transformed scale score

mean was partitioned as described in Chapter 10. The partition of error variance consists of two parts: the

proportion of error variance due to sampling students (sampling variance) and the proportion of error

variance due to the fact that scale score, θ , is a latent variable that is estimated rather than observed.

Table 21-6 contains estimates of the total error variance, the proportion of error variance due to sampling

students, and the proportion of error variance due to the latent nature of θ . Instead of using 100 plausible

values as in the national assessment, the calculations for the state samples are based on 5 plausible

values.

-1 0 1 2 3 4

0 50 100 150 200 250 300

Root Percent

State

Diff 
397

Table 21-6

Estimation Error Variance and Related Coefficients

for the Writing State Assessment, Grade 8

Proportion of Variance due to ...

State

Total

Estimation

Error

Variance Student Sampling Latency of θ

Alabama 1.958 0.95 0.05

Arizona 2.331 0.86 0.14

Arkansas 1.470 0.89 0.11

California 3.162 0.93 0.07

Colorado 1.719 0.91 0.09

Connecticut 1.843 0.83 0.17

Delaware 2.077 0.32 0.68

Florida 1.534 0.83 0.17

Georgia 1.822 0.83 0.17

Hawaii 1.019 0.37 0.63

Kentucky 2.320 0.92 0.08

Louisiana 1.902 0.93 0.07

Maine 2.110 0.47 0.53

Maryland 2.270 0.89 0.11

Massachusetts 2.814 0.94 0.06

Minnesota 3.492 0.81 0.19

Mississippi 1.689 0.71 0.29

Missouri 2.087 0.87 0.13

Montana 2.107 0.64 0.36

Nevada 0.750 0.48 0.52

New Mexico 0.663 0.80 0.20

New York 2.209 0.94 0.06

North Carolina 2.111 0.77 0.23

Oklahoma 1.603 0.90 0.10

Oregon 2.317 0.87 0.13

Rhode Island 0.431 0.84 0.16

South Carolina 1.196 0.82 0.18

Tennessee 3.121 0.94 0.06

Texas 2.246 0.88 0.12

Utah 1.522 0.63 0.37

Virginia 1.424 0.76 0.24

Washington 2.371 0.80 0.20

West Virginia 2.692 0.43 0.57

Wisconsin 1.746 0.96 0.04

Wyoming 2.043 0.28 0.72

District of Columbia 1.413 0.52 0.48

DoDEA/DDESS 6.695 0.40 0.60

DoDEA/DoDDS 1.476 0.47 0.53

Virgin Islands 14.194 0.14 0.86
398

21.7 WRITING TEACHER QUESTIONNAIRES

Teachers of the eighth-grade students were surveyed about their educational background and

teaching practices. The students were matched first with their writing teacher, and then the specific

classroom period. Variables derived from the questionnaire were used in the conditioning models. An

additional conditioning variable was included that indicated whether the student had been matched with a

teacher record. This contrast controlled estimates of subgroup means for differences that exist between

matched and nonmatched students. Of the 97,589 eighth-grade students in the sample, 84,605 (86.7%,

unweighted) were matched with teachers who answered both parts of the teacher questionnaire, and

6,920 (7.1%, unweighted) of the students had teachers who answered only the teacher background

section of the questionnaire.
399

Chapter 22

ASSESSMENT FRAMEWORKS AND INSTRUMENTS

FOR THE 1998 CIVICS ASSESSMENT1

Andrew R. Weiss and Terry L. Schoeps

Educational Testing Service

22.1 INTRODUCTION

In 1998, NAEP conducted a national main civics assessment and national special trend civics

assessment at grades 4, 8, and 12.2 Chapters 22, 23, and 24 cover only the main assessment; a

forthcoming report will detail the procedures and analyses of the special trend assessment.

The framework that was used for the 1998 NAEP civics assessment detailed the structure of the

assessment to be given at grades 4, 8, and 12 at the national level. The framework for the civics

assessment is available on the National Assessment Governing Board (NAGB) web site at

http://www.nagb.org.

Sections 22.2 through 22.5 include a detailed description of the development of the framework,

objectives, and items for the 1998 NAEP civics assessment. Sections 22.6 and 22.7 describe the final

cognitive instruments. Section 22.8 describes the student background questionnaires and the civics

teacher questionnaire. Additional information on the structure and content of assessment booklets can be

found in Section 22.9. Section 22.10 mentions the special trend study in civics. Various committees

worked on developing the framework, objectives, and items for the civics assessment. The list of

committee members and consultants who participated in the 1998 development process is provided in

Appendix K.

Samples of assessment questions and student responses are published in the NAEP 1998 Civics

Report Card for the Nation (Lutkus, Weiss, Campbell, Mazzeo, & Lazer, 1999).

22.2 DEVELOPING THE CIVICS ASSESSMENT FRAMEWORK

NAGB is responsible for setting policy for NAEP; this policymaking role includes developing

assessment frameworks and test specifications. Appointed by the Secretary of Education from lists of

nominees proposed by the board itself in various statutory categories, the 24-member board is composed

of state, local, and federal officials, as well as educators and members of the public.

NAGB began the development process for the 1998 civics objectives by establishing the NAEP

Civics Consensus Project in February 1995 with the award of the framework contract to the Council of

Chief State School Officers (CCSSO). The project’s committees gained input through public hearings,

student forums, and written reviews of successive drafts of the framework.

 

1 Andrew R. Weiss manages the item-development process for NAEP civics assessments. Terry L. Schoeps coordinates the

production of NAEP technical reports.

2 Civics was not part of the NAEP state asssesments in 1998.
400

For more detail on the development and specifications of the civics framework, refer to the

Civics Framework for the 1998 National Assessment of Educational Progress (CCSSO, 1996).

Additional information on the NAEP Civics Framework can be found in three technical

publications available through NAGB—Civics Assessment and Exercise Specifications,

Recommendations for Background Questions, and Reporting Recommendations.

22.3 CIVICS FRAMEWORK AND ASSESSMENT DESIGN PRINCIPLES

The framework authors stated that given the extreme importance of competent citizenship and

effective civic education for the well-being of our constitutional democracy, it is imperative that we have

adequate information about what students know and are able to do with regard to civics and government.

The aim of the 1998 NAEP Civics assessment was to indicate generally how much and how well students

are learning essential knowledge and skills about democratic citizenship and government.

22.4 FRAMEWORK FOR THE 1998 CIVICS ASSESSMENT

The framework comprised three interrelated components: knowledge, intellectual skills, and civic

dispositions. Of these, the knowledge component served as the core of the framework. The framework

designers established five content areas of knowledge on which to base civics test questions:

• What are civic life, politics, and government?

• What are the foundations of the U.S. political system?

• How does the government established by the Constitution embody the purposes,

values, and principles of U.S. democracy?

• What is the relationship of the United States to other nations and to

world affairs?

• What are the roles of citizens in U.S. democracy?

The second component, intellectual skills, includes:

• identifying and describing,

• explaining and analyzing, and

• evaluating, taking, and defending a position.

The distribution of questions by intellectual skill across grade levels recommended in the

assessment framework is provided in Table 22-1. Table 22-2 shows the actual distribution of these

questions in the assessment.
401

Table 22-1

Percentage Distribution of Questions by Intellectual Skill

as Recommended in the NAEP Civics Framework

Intellectual Skill

Identifying

and

Describing

Explaining

and

Analyzing

Evaluating,

Taking, and

Defending a

Position

Grade 4 40% 30% 30%

Grade 8 35% 35% 30%

Grade 12 25% 40% 35%

Table 22-2

Actual Percentage Distribution of Questions by Intellectual Skill

Intellectual Skill

Identifying

and

Describing

Explaining

and

Analyzing

Evaluating,

Taking, and

Defending a

Position

Grade 4 33% 37% 30%

Grade 8 29% 38% 33%

Grade 12 18% 33% 38%

Civic dispositions refers to those aspects of a person’s character that drive him or her to

contribute to the preservation and improvement of United States constitutional democracy.

All three components are summarized in the civics framework (CCSSO, 1996) as shown in

Figure 22-1.

22.5 DEVELOPING THE CIVICS COGNITIVE ITEMS

Civics questions were developed by NAEP test developers and outside consultants to meet the

requirements of the civics framework. In addition to matching the content and intellectual skills

components, NAEP staff had to balance the question pool by question format. The question format

included multiple-choice, short constructed-response, and extended constructed-response questions.

Short constructed-response questions required answers ranging from a few words to a few sentences and

were intended to be answered in up to two minutes. Extended constructed-response questions generally

required longer written answers or more time for thinking and were intended to be answered in up to five

minutes. The decision to use a specific question format was based on a consideration of how best to

measure particular civics knowledge and skills.
402

Figure 22-1

Description of the NAEP 1998 Civics Framework Components

Knowledge

The knowledge component is embodied in the form of five significant and enduring questions: (1)

What are civic life, politics, and government? (2) What are the foundations of the American

political system? (3) How does the government established by the Constitution embody the

purposes, values, and principles of American democracy? (4) What is the relationship of the

United States to other nations and to world affairs? (5) What are the roles of citizens in American

democracy?

Intellectual and Participatory Skills

The intellectual and participatory skills component involves the use of knowledge to think and act

effectively in a constitutional democracy. Intellectual skills enable students to learn and apply civic

knowledge in the many and varied roles of citizens. These skills help citizens identify, describe,

explain, and analyze information and arguments as well as evaluate, take, and defend positions on

public policies. Participatory skills enable citizens to monitor and influence public and civic life by

working with others, clearly articulating ideas and interests, building coalitions, seeking consensus,

negotiating compromise, and managing conflict.

Civic Dispositions

Civics dispositions refer to the inclination or "habits of the heart," as de Tocqueville called them,

that pervade all aspects of citizenship. In a constitutional democracy, these dispositions pertain to

the rights and responsibilities of individuals in society and to the advancement of ponsibilities of

individuals in society and to the ideals of the polity. They include the dispositions to become an

independent member of society; respect individual worth and responsibilities of a citizen; abide by

the "rules of the game," such as accepting the legitimate decisions of the majority while protecting

the rights of the minority; participate in civic affairs in an informed, thoughtful, and effective

manner; and promote the healthy functioning of American constitutional democracy.

Table 22-3 contains the percent of assessment time for each question format as specified in the

framework and as estimated for the questions selected for the assessment. Grades 8 and 12 estimated

percents are closer to the target percent.
403

Table 22-3

NAEP 1998 Civics Assessment

Percentage of Student Assessment Time by Question Format

Actual Percentage of Time

Question Type

Specified in

Framework* Grade 4 Grade 8 Grade 12

Multiple Choice 60% 53% 61% 61%

Short Constructed-Response 30% 29% 27% 30%

Extended Constructed-Response 10% 18% 12% 9%

* These percentages were specified to be the same for all three grades.

Finally, the assessment framework directed test developers to ensure that 15 percent of the

questions measured civic dispositions and participatory skills, and that a significant portion of questions

were based on textual and visual stimulus material.

22.6 DEVELOPING THE CIVICS OPERATIONAL FORMS

In preparation for the 1998 operational assessment, questions were field-tested in 1997. The

purpose of the field test was to administer a large pool of questions so that those with the best content

and statistical properties could be selected for the 1998 operational assessment. The civics field test was

conducted in January and February of 1997 and involved national samples of fourth-, eighth-, and

twelfth-grade students. A total of 555 questions were developed for the field test. Two hundred questions

were administered at grade 4, 224 at grade 8, and 244 at grade 12. The questions were organized in a

series of 25-minute blocks, each containing multiple-choice, short constructed-response questions, and

extended constructed-response questions. Each student received two blocks. Thirty blocks were

administered as follows:

• Eight blocks at grade 4 only,

• Four blocks at grade 4 and grade 8,

• Six blocks at grade 8 only,

• Three blocks at grade 8 and grade 12, and

• Nine blocks at grade 12 only.

Field test results were used by ETS test developers to assemble the 1998 operational instruments.

Approximately 500 responses were obtained for each question in the field test. Multiple-choice questions

were machine scored and constructed-response questions were read and scored by staff at the National

Computer Systems scoring center under the direction of NAEP/ETS staff. The raw field test data were

subjected to statistical analyses by NAEP/ETS data analysts. The resulting question analyses yielded

mean percentage correct, polyserial correlations, difficulty levels, and other information for each

question. NAEP test developers reviewed the analyses to help determine:

• which items best measured civics knowledge and skills,

• the need for revisions of items that lacked clarity or had ineffective item formats,

and

• the appropriate number of items to include in each operational assessment test

book.
404

The items chosen for the operational assessment were revised as needed and assembled into new

blocks. With the approval of the Civics Instrument Development Committee, cross-grade blocks were

eliminated, because it was believed that few questions were successful measures of student knowledge at

more than one grade. The blocks were reviewed by the committee in May 1997 for content and balance.

Once approved by the committee, all items were subjected to content, measurement, fairness, and

editorial reviews by appropriate ETS staff. The draft materials, including background questionnaires,

were submitted to the Office of Management and Budget (OMB) in July 1997 for clearance. Changes

requested by OMB were made in August 1997, and upon receiving approval, the assessment was sent to

print.

 Six blocks were assembled for grade 4 and eight blocks were assembled for each of grades 8 and

12. Each student participating in NAEP received two blocks of items. Grade 4 blocks included 15 items

each, whereas the blocks at grade 8 and grade 12 included 19 items each.

22.7 DISTRIBUTION OF CIVICS ASSESSMENT ITEMS

Of the total of 393 items, there are 315 multiple-choice items, 61 short constructed-response

items, and 17 extended constructed-response items that make up the 1998 civics assessment. A few of

these items are used at more than one grade level. As a result, the sum of the items that appear at each

grade level is greater than the total number of unique items.

Figure 22-2

Distribution of Items for the 1998 Civics Assessment

22.8 BACKGROUND QUESTIONS FOR THE 1998 CIVICS ASSESSMENT

To gather contextual information, NAEP assessments include background questions designed to

provide insight into the factors that may influence civics proficiency.

NAEP includes both general background questionnaires given to participants in all subjects and

subject-specific questionnaires for both students and their teachers. The development of the general

background questionnaires is discussed below. It is worth noting that members of the Civics Instrument

Development Committee were consulted on the appropriateness of the issues addressed in all

questionnaires that may relate to civics instruction and achievement. Like the civics questions, all

background questions were submitted for extensive review and field testing. Recognizing the validity

problems inherent in self-reported data, particular attention was given to developing questions that were

meaningful and unambiguous and that would encourage accurate reporting.

The 1998 assessment included two five-minute sets of general and civics background questions

designed to gather contextual information about students and their instructional experiences in civics.

Students in the fourth grade were given additional time for these sections (up to fifteen minutes per

section), because the items in the general questionnaire were read aloud for them. A one-minute

Grade 4

 69 Multiple-Choice

 15 Short Constructed-Response

 6 Extended Constructed-Response

Grade 8

123 Multiple-Choice

22 Short Constructed-Response

6 Extended Constructed-Response

Grade 12

123 Multiple-Choice

24 Short Constructed-Response

5 Extended Constructed-Response
405

questionnaire was also given to students at the end of each booklet to determine students’ motivation in

completing the assessment and their familiarity with assessment tasks.

22.8.1 Student Civics Questionnaires

Three sets of multiple-choice background questions were included as separate sections in each

student booklet:

General Background: The general background questions collected demographic information

about race/ethnicity, language spoken at home, mother’s and father’s level of education, reading

materials in the home, homework, school attendance, which parents live at home, and which

parents work outside the home.

Civics Background: Students were asked to report their instructional experiences related

to civics including the amount of civics instruction they received and the topics they

studied. In addition, they were asked about the instructional practices of their civics

teachers including, for example, how often they used textbooks, discussed current events,

and took part in classroom activities that simulated civic participation.

Motivation: Students were asked five questions about their attitudes and perceptions

about reading and self-evaluation of their performance on the NAEP assessment.

Table 22-4 shows the number of questions per background section and notes the placement of

each within student booklets.

Table 22-4

NAEP 1998 Background Sections of Student Civics Booklets

Number of Questions

Placement in Student Booklet

(of 5 Sections)

Grade 4

General Background 21 Section 3

Civics Background 22 Section 4

Motivation 5 Section 5

Grade 8

General Background 22 Section 3

Civics Background 24 Section 4

Motivation 5 Section 5

Grade 12

General Background 24 Section 3

Civics Background 29 Section 4

Motivation 5 Section 5

22.8.2 Civics Teacher Questionnaire

To supplement the information on instruction reported by students, the civics teachers of the

fourth and eighth graders participating in the NAEP civics assessment were asked to complete a

questionnaire about their backgrounds, education, experience, and instructional practices. To make the
406

link between student data and teacher information as complete as possible, teachers were asked to

provide information for each class containing an assessed student.

The Teacher Questionnaire, Part I: Background, Education, and Resources ( 49 questions at

grade 4 and 47 at grade 8) included questions pertaining to:

• years of teaching experience;

• certification, degrees, major and minor fields of study;

• coursework in education;

• coursework in specific subject areas;

• amount of in-service training;

• extent of control over instructional issues; and

• availability of resources for their classroom.

The Teacher Questionnaire, Part IIA: Civics Preparation ( 7 questions at grade four and

7 at grade eight) included questions on the teacher’s preparedness in various areas related to

civics education, for example:

• preparedness in social studies instruction;

• preparedness in use of community resources in instruction;

• preparedness in using national standards for civics; and

• preparedness in using software for social studies.

The Teacher Questionnaire, Part IIB: Civics Classroom Information ( 33 questions at

grade four and 32 at grade eight) included questions pertaining to:

• ability level of students in the class;

• whether students were assigned to the class by ability level;

• time on task;

• homework assignments;

• frequency of instructional activities used in class;

• methods of assessing student progress in civics;

• instructional emphasis given to the civics abilities covered in the assessment; and

• use of particular resources.
407

22.9 STUDENT BOOKLETS FOR THE 1998 CIVICS ASSESSMENT

Each student assessed in civics received a booklet containing two blocks of test questions, a fiveminute section of general background questions, a five-minute section of civics background questions, a

one-minute section of questions about his or her motivation and familiarity with the assessment

materials, and content questions. The test questions were assembled into sections or blocks, each

containing a range of questions covering the five knowledge categories.

The assembly of civics blocks into booklets and their subsequent assignment to sampled students

was determined by a balanced incomplete block (BIB) design with spiraled administration. The civics

blocks were assigned to booklets in such a way that every block was paired with every other block at

least once. The BIB design balanced the order of presentation of the blocks of items so that every block

appears as the first question block and as the second question block an equal number of times. This

design allows for some reduction of the impact of context and fatigue effects to be measured and

reported. The BIB design in Table 22-5 would call for 15 booklets to allow each of the six blocks to be

paired with every other block. Three additional booklets (316-318) were added to ensure that each block

appeared equally often in the first and second position. These booklets are the reverse of booklets

313-315.

Once assembled, the assessment booklets were then spiraled and packaged. Spiraling involves

interweaving the booklets in a systematic sequence so that each booklet appears an appropriate number

of times in the sample. The packages were designed so that each booklet would appear equally often in

each position in a package.

The final step in the BIB-spiraling procedure was the assigning of the booklets to the assessed

students. The students within an assessment session were assigned booklets in the order in which the

booklets were bundled. Thus, most students in an assessment session received different booklets. Tables

22-5 through 22-7 detail the configuration of booklets administered in the 1998 civics assessment.
408

Table 22-5

NAEP 1998 Civics Grade 4 Booklet Configuration

Booklet

Number

Question

Block 1

Question

Block 2

Common Core

Background

Civics

Background Motivation

301 C3 C4 CW PB PA

302 C4 C5 CW PB PA

303 C5 C6 CW PB PA

304 C6 C7 CW PB PA

305 C7 C8 CW PB PA

306 C8 C3 CW PB PA

307* C3 C5 CW PB PA

308 C4 C6 CW PB PA

309 C5 C7 CW PB PA

310 C6 C8 CW PB PA

311 C7 C3 CW PB PA

312 C8 C4 CW PB PA

313 C3 C6 CW PB PA

314 C4 C7 CW PB PA

315 C5 C8 CW PB PA

316 C6 C3 CW PB PA

317 C7 C4 CW PB PA

318 C8 C5 CW PB PA

* A large-type version of this booklet was administered as an accommodation to students who had a

visual disability.
409

Table 22-6

NAEP 1998 Civics Grade 8 Booklet Configuration

Booklet

Number

Question

Block 1

Question

Block 2

Common Core

Background

Civics

Background Motivation

301 C3 C4 CW PB PA

302 C4 C5 CW PB PA

303 C5 C6 CW PB PA

304 C6 C7 CW PB PA

305 C7 C8 CW PB PA

306 C8 C9 CW PB PA

307 C9 C10 CW PB PA

308 C10 C3 CW PB PA

309 C3 C5 CW PB PA

310* C4 C6 CW PB PA

311 C5 C7 CW PB PA

312 C6 C8 CW PB PA

313 C7 C9 CW PB PA

314 C8 C10 CW PB PA

315 C9 C3 CW PB PA

316 C10 C4 CW PB PA

317 C3 C6 CW PB PA

318 C4 C7 CW PB PA

319 C5 C8 CW PB PA

320 C6 C9 CW PB PA

321 C7 C10 CW PB PA

322 C8 C3 CW PB PA

323 C9 C4 CW PB PA

324 C10 C5 CW PB PA

325 C3 C7 CW PB PA

326 C4 C8 CW PB PA

327 C5 C9 CW PB PA

328 C6 C10 CW PB PA

329 C7 C3 CW PB PA

330 C8 C4 CW PB PA

331 C9 C5 CW PB PA

332 C10 C6 CW PB PA

* A large-type version of this booklet was administered as an accommodation to students who had a

visual disability.
410

Table 22-7

NAEP 1998 Civics Grade 12 Booklet Configuration

Booklet

Number

Question

Block 1

Question

Block 2

Common Core

Background

Civics

Background Motivation

301* C3 C4 CW PB PA

302 C4 C5 CW PB PA

303 C5 C6 CW PB PA

304 C6 C7 CW PB PA

305 C7 C8 CW PB PA

306 C8 C9 CW PB PA

307 C9 C10 CW PB PA

308 C10 C3 CW PB PA

309 C3 C5 CW PB PA

310 C4 C6 CW PB PA

311 C5 C7 CW PB PA

312 C6 C8 CW PB PA

313 C7 C9 CW PB PA

314 C8 C10 CW PB PA

315 C9 C3 CW PB PA

316 C10 C4 CW PB PA

317 C3 C6 CW PB PA

318 C4 C7 CW PB PA

319 C5 C8 CW PB PA

320 C6 C9 CW PB PA

321 C7 C10 CW PB PA

322 C8 C3 CW PB PA

323 C9 C4 CW PB PA

324 C10 C5 CW PB PA

325 C3 C7 CW PB PA

326 C4 C8 CW PB PA

327 C5 C9 CW PB PA

328 C6 C10 CW PB PA

329 C7 C3 CW PB PA

330 C8 C4 CW PB PA

331 C9 C5 CW PB PA

332 C10 C6 CW PB PA

* A large-type version of this booklet was administered as an accommodation to students who had a

visual disability.
411

22.10 CIVICS SPECIAL TREND STUDY IN 1998

In 1998, NAEP conducted a special study designed to compare trends in civics proficiency

between 1988 and 1998. Students participating in this special trend study were given booklets from the

1988 NAEP civics assessment. Because the questions in the trend study were based on the 1988

framework, the results cannot be linked to 1998 national assessment results. At the fourth grade level,

2,087 student participated. For grades 8 and 12 the number of students participating totaled 2,053 and

2,181, respectively. Differences in mean item scores for the 1988 booklet were calculated. Results from

this special trend study appear in a separate report (Weiss et al., 2000).

413

Chapter 23

INTRODUCTION TO THE DATA ANALYSIS

FOR THE CIVICS ASSESSMENT1

Spencer S. Swinton and Edward Kulick

Educational Testing Service

23.1 INTRODUCTION

This chapter gives an introduction to the analyses performed on the responses to the cognitive

and background items in the 1998 assessment of civics. These analyses led to the results presented in the

NAEP 1998 Civics Report Card for the Nation (Lutkus et al., 1999). This chapter describes the student

samples, items, assessment booklets, administrative procedures, student weights, and the process used in

scoring constructed-response items, as well as the methods and results of differential item functioning

(DIF) analyses. The major analysis components are discussed in Chapter 24.

The objectives of the civics analyses were to prepare scale values and estimate subgroup scale

score distributions for samples of students who were administered civics items from the national main

assessment.

23.2 DESCRIPTION OF STUDENT SAMPLES, ITEMS, ASSESSMENT BOOKLETS,

AND ADMINISTRATIVE PROCEDURES

The student samples that were administered civics items in the 1998 assessment are shown in

Table 23-1. The data from the national main focused balanced in completed block (BIB) assessment (see

Section 1.5) of civics (4 [Civics-Main], 8 [Civics-Main], and 12 [Civics-Main]) were used for national

main analyses comparing the levels of civics achievement for various subgroups of the 1998 target

populations. Chapters 1 and 3 contain descriptions of the target populations and the sample design used

for the assessment. The target populations were grade 4, grade 8, and grade 12 students in the United

States. (See Appendix A for tables describing the students assessed and the reporting sample for each

component of the civics assessment).

The items in the assessment were based on the framework described in Civics Framework for the

National Assessment of Educational Progress (NAGB, 1996a). Five areas are described in the civics

framework, and were used in developing the assessment questions. For purposes of scaling, all items

were fit to a single scale.

In the national main samples, each student was administered a booklet containing two separately

timed 25-minute blocks of cognitive civics items. In addition, each student was administered a block of

background questions, a block of civics-related background questions, and a block of questions

concerning the student’s motivation and his or her perception of the difficulty of the cognitive items;

these blocks were common to all civics booklets for a particular grade level. Eight 25-minute blocks of

 

1 Spencer S. Swinton was the primary person responsible for the planning, specification, and coordination of the civics analyses.

Computing activities for all civics scaling and data analyses were directed by Edward Kulick and completed by Venus Leung.

Others contributing to the analysis of civics data were David S. Freund, Bruce A. Kaplan, and Katharine E. Pashley.
414

civics items were administered at grade 4, and 10 at each of grades 8 and 12. As described in Chapter 22,

the 25-minute blocks were combined into booklets according to a BIB design. See Chapter 22 for more

information about the blocks and booklets.

At each grade, two civics blocks were repeated from the 1988 assessment of citizenship and

social studies to provide data for a special trend study. These items were not scaled with the national

main civics assessment items, but were reported using a mean percent-correct metric. The results are

reported in The Next Generation of Citizens: NAEP Trends in Civics, 1988 to 1998 (Weiss, Lutkus,

Grigg, & Niemi, 2000).

The mean percent-correct metric involves the percent of people who answered the item correctly.

Since all students in the civics trend special study took all items, it was possible to report results for

single items and subsets of items by demographic groups. In contrast, the main civics items were scaled

using item response theory (IRT). IRT scaling provides parameters that describe the overall difficulty and

discrimination of the item. The scale score metric defined by IRT makes comparisons possible across

assessments, even if different students took different items.

Table 23-1

NAEP 1998 National Main Civics Assessment Student Samples

Sample Booklet ID Number*

Cohort

Assessed Time of Testing†

Reporting

Sample Size

4 [Civics–Main] C301-C318 Grade 4 1/5/98 – 3/27/98 5,948

8 [Civics–Main] C301-C332 Grade 8 1/5/98 – 3/27/98 8,212

12 [Civics–Main] C301-C332 Grade 12 1/5/98 – 3/27/98 7,763

* Common labeling of booklet numbers across grade levels does not denote common items

(e.g., Booklet C301 at grade 8 does not contain the same items as Booklet C301 at grade 12).

† Final makeup sessions were held March 30–April 3, 1998.

The total number of scaled items in the main civics assessments was 89, 149, and 151,

respectively, for grades 4, 8, and 12. Note that some items overlap across grade. Table 23-2 shows the

numbers of items within civics purpose subscales for each grade—both for the original item pool, and

after the necessary adjustments were made during scaling.

The composition of each block of items by item type is given in Tables 23-3, 23-5, and 23-7.

Common labeling of these blocks across grade levels does not necessarily denote common items (e.g.,

Block C3 at grade 4 does not contain the same items as Block C3 at grade 8). The numbers of items

scaled in 1998 for each grade are presented in Tables 23-4, 23-6, and 23-8.
415

Table 23-2

Number of Items in the National Main Civics Assessment by Content Area

Content Areas

Grade 1 2 3 4 5 Total

4 Prescaling

Postscaling

19

19

17

16

16

16

8

8

30

30

90

89

8 Prescaling

Postscaling

19

18

35

35

44

43

22

22

31

31

151

149

12 Prescaling

Postscaling

14

14

29

29

43

43

30

29

37

37

152

151

CONTENT-AREA LEGEND 1 What are civic life, politics, and government?

2 What are the foundations of the U.S. political system?

3 How does the government established by the Constitution embody

the purposes, values, and principles of U.S. democracy?

4 What is the relationship of the United States to other nations and to

world affairs?

5 What are the roles of citizens in U.S. democracy?

Table 23-3

1998 NAEP Civics Block Composition

As Defined Before Scaling, Grade 4

MultipleConstructed-Response Items ScoredPolytomously Total

Block Choice Items 2-category* 3-category 4-category Items

Total 69 0 15 6 90

C3 11 0 3 1 15

C4 11 0 4 0 15

C5 12 0 2 1 15

C6 12 0 3 0 15

C7 11 0 1 3 15

C8 12 0 2 1 15

* For a small number of constructed-response items, adjacent categories were combined.

Table 23-4

1998 NAEP Civics Block Composition

After Scaling, Grade 4

MultipleConstructed-Response Items ScoredPolytomously Total

Block Choice Items 2-category* 3-category 4-category Items

Total 68 1 15 5 89

C3 11 0 3 1 15

C4 11 1 3 0 15

C5 12 0 2 1 15

C6 12 0 3 0 15

C7 11 0 2 2 15

C8 11 0 2 1 14

* For a small number of constructed-response items, adjacent categories were combined.
416

Table 23-5

1998 NAEP Civics Block Composition

As Defined Before Scaling, Grade 8

MultipleConstructed-Response Items ScoredPolytomously Total

Block Choice Items 2-category* 3-category 4-category Items

Total 123 0 22 6 151

C3 15 0 4 0 19

C4 16 0 1 2 19

C5 15 0 4 0 19

C6 15 0 4 0 19

C7 15 0 3 1 19

C8 16 0 2 1 19

C9 16 0 2 1 19

C10 15 0 2 1 18

* For a small number of constructed-response items, adjacent categories were combined.

Table 23-6

1998 NAEP Civics Block Composition

After Scaling, Grade 8

MultipleConstructed-Response Items ScoredPolytomously Total

Block Choice Items 2-category* 3-category 4-category Items

Total 121 1 21 6 149

C3 15 0 4 0 19

C4 16 0 1 2 19

C5 14 0 4 0 18

C6 15 1 3 0 19

C7 15 0 3 1 19

C8 15 0 2 1 18

C9 16 0 2 1 19

C10 15 0 2 1 18

* For a small number of constructed-response items, adjacent categories were combined.

Table 23-7

1998 NAEP Civics Block Composition

As Defined Before Scaling, Grade 12

MultipleConstructed-Response Items ScoredPolytomously Total

Block Choice Items 2-category* 3-category 4-category Items

Total 123 0 23 6 152

C3 15 0 3 1 19

C4 16 0 3 0 19

C5 15 0 3 1 19

C6 16 0 3 0 19

C7 15 0 2 2 19

C8 15 0 4 0 19

C9 16 0 2 1 19

C10 15 0 3 1 19

* For a small number of constructed-response items, adjacent categories were combined.
417

Table 23-8

1998 NAEP Civics Block Composition

After Scaling, Grade 12

MultipleConstructed-Response Items ScoredPolytomously Total

Block Choice Items 2-category* 3-category 4-category Items

Total 122 1 22 6 151

C3 15 0 3 1 19

C4 16 0 3 0 19

C5 15 1 2 1 19

C6 15 0 3 0 18

C7 15 0 2 2 19

C8 15 0 4 0 19

C9 16 0 2 1 19

C10 15 0 3 1 19

* For a small number of constructed-response items, adjacent categories were combined.

23.3 SCORING CONSTRUCTED-RESPONSE ITEMS

In addition to multiple-choice items, each block contained a number of constructed-response

items, accounting for 47 percent of testing time in grade 4 and 39 percent of testing time in grades 8 and

12. Constructed-response items were scored by specially trained readers. (Chapter 7 describes scoring

procedures and ranges of interrater reliability for constructed-response items.) Some of the constructedresponse items required only a few sentences or a paragraph response. These short constructed-response

items were scored dichotomously as correct or incorrect. Other constructed-response items required

somewhat more elaborated responses, and were scored polytomously on a 3-point (0–2) scale:

0 = Unsatisfactory (and omit)

1 = Partial

2 = Complete

In addition, most blocks contained at least one constructed-response item that required a more indepth, elaborated response.These items were scored polytomously on a 4-point (0–3) scale:

0 = Unsatisfactory (and omit)

1 = Partial

2 = Essential

3 = Extensive, which demonstrates more in-depth understanding

Adjacent categories of a small number of constructed-response items were combined (collapsed).

These changes were made so that the scaling model used for these items fit the data more closely, and are

described more fully in Chapter 12.

23.4 DIF ANALYSIS

A differential item functioning (DIF) analysis of items was done to identify potentially biased

items that were differentially difficult for members of various subgroups with comparable overall scores.
418

Sample sizes were large enough to compare male and female students, White and Black students, and

White and Hispanic students. Table A-9 of Appendix A specifies the sample size for each of these

groups. The purpose of the analysis was to identify items that should be examined more closely by a

committee of trained test developers and subject-matter specialists for possible bias and consequent

exclusion from the assessment. The presence of DIF in an item means that the item is differentially

harder for one group of students than another, while controlling for the ability level of the students. DIF

analyses were conducted separately by grade for national samples.

For dichotomous items, the Mantel-Haenszel procedure as adapted by Holland and Thayer

(1988) was used as a test of DIF (this is described in Chapter 9). The Mantel procedure (Mantel, 1963)

was used for detection of DIF in polytomous items and also as described by Zwick, Donoghue, and

Grima (1993). This procedure assumes ordered categories.

For dichotomous items, the DIF index generated by the Mantel-Haenszel procedure is used to

place items into one of three categories: “A,” “B,” or “C.” “A” items exhibit little or no DIF, while “C”

items exhibit a strong indication of DIF and should be examined more closely. Positive values of the

index indicate items that are differentially easier for the focal group (female, Black, or Hispanic students)

than for the reference groups (male or White students). Similarly, negative values indicate items that are

differentially harder for the focal group than the reference group. An item that was classified as a “C”

item in any analysis was considered to be a “C” item.

For polytomous items (regular constructed-response items and extended constructed-response

items), the Mantel statistic provides a statistical test of the hypothesis of no DIF. A categorization similar

to that described for dichotomous items was developed to classify items (this is discussed in detail in

Donoghue, 2000). Polytomous items were placed into one of three categories: “AA”, “BB”, or “CC”

similar to dichotomous items. “AA” items exhibit no DIF, while CC items exhibit a strong indication of

DIF and should be examined more closely. The classification criterion for polytomous items is presented

in Donoghue (2000). As with dichotomous items, positive values of the index indicate items that are

differentially easier for the focal group (female, Black, or Hispanic students) than for the reference

group (male or White students). Similarly, negative values indicate items that are differentially harder for

the focal group than for the reference group. An item that was classified as a CC item in any analysis

was considered to be a CC item.

Table 23-9 summarizes the results of DIF analyses for dichotomously scored items. One C item

was identified in grade 4, 2 in grade 8, and 3 in grade 12. The committee decided that only the C item in

grade 8 showed evidence of bias. The item tested for understanding that the rights of United States

citizens date back to the Constitution and Bill of Rights, but used a World War II poster as a stimulus. It

was judged that the concept being tested did not require a military theme, making it unnecessarily more

difficult for females. Note that if the concept in the framework being assessed had required a military

context, the same performance differential would not necessarily have resulted in the dropping of the

item.
419

Table 23-9

DIF Category by Grade for Dichotomous Civics Items

DIF Analysis

Grade Category* Male/Female White/Black White/Hispanic

 4 CBAA+

B+

C+

 0

3

31

35

0

0

0

5

30

30

3

1

0

0

29

37

3

0

 8 CBAA+

B+

C+

 1

 5

70

46

1

0

0

4

51

58

10

0

0

2

52

65

3

1

12 CBAA+

B+

C+

0

14

49

55

4

1

0

6

45

65

5

2

0

4

46

68

5

0

* Positive values of the index indicate items that are differentially easier for the focal group (female,

Black, or Hispanic students) than for the reference groups (male or White students). “A+” or “A-”

means no indication of DIF, “B+” means a weak indication of DIF in favor of the focal group, “B-”

means a weak indication of DIF in favor of the reference group and “C+” or “C-” means a strong

indication of DIF.
420

Table 23-10

DIF Category by Grade for Polytomous Civics Items

DIF Analysis

Grade Category* Male/Female White/Black White/Hispanic

4 CCBBAAAA+

BB+

CC+

0

0

 7

14

0

0

0

2

 9

 9

1

0

0

0

10

11

0

0

 8 CCBBAAAA+

BB+

CC+

0

1

 5

16

5

1

0

1

13

13

1

0

0

2

11

15

0

0

12 CCBBAAAA+

BB+

CC+

0

0

 6

21

2

0

3

2

10

13

0

1

0

1

12

14

2

0

* Positive values of the index indicate items that are differentially easier for the focal group (female,

Black, or Hispanic students) than for the reference groups (male or White students). “AA+” or “AA-”

means no indication of DIF, “BB+” means a weak indication of DIF in favor of the focal group, “BB-”

means a weak indication of DIF in favor of the reference group, and “CC+” or “CC-” means a strong

indication of DIF.

In addition to the Mantel-Haenszel DIF procedure, a second bias test was performed using a

SIBTEST analysis (Shealy & Stout, 1993). This analysis identified essentially the same items as were

flagged by the other DIF procedure.

23.5 THE WEIGHT FILES

To include special-needs students in its assessment, NAEP test developers established

accommodations or adaptations of test forms for students with disabilities (SD) and those characterized

as having limited English proficiency (LEP). Inclusion criteria for these students were developed by the

Department of Education in consultation with a number of other federal government offices. Its goal was

to achieve optimal inclusion of students with disabilities and increase the salience of subject-related

instructional matters in inclusion decisions.

For the 1998 civics assessments, the sampling contractor Westat produced the final student and

school weights and the corresponding replicate weights. Information for the creation of the weight files

was supplied by National Computer Systems (NCS) under the direction of Educational Testing Service

(ETS).
421

Chapter 24

DATA ANALYSIS FOR THE CIVICS ASSESSMENT1

Spencer S. Swinton, Edward Kulick, and Venus Leung

Educational Testing Service

24.1 INTRODUCTION

This chapter describes the analyses performed on the responses to the cognitive and background

items in the 1998 assessment of civics. The focus of this chapter is on the methods and procedures used

to estimate scale score distributions for subgroups of students. This includes a wide array of topics, such

as the scoring of constructed-response items, classical item statistics, item response theory (IRT) analysis

of civics scales, and estimation of subgroup means by the imputation of plausible values. The statistical

bases of the IRT and plausible values methodology described in this chapter are given in Chapter 12.

These analyses serve as a basis for the results presented in NAEP 1998 Civics Report Card for the Nation

(Lutkus et al., 1999).

The student samples that were administered civics items in the 1998 national assessment were

shown in Table 23-1. (See Chapters 1 and 3 for descriptions of the target populations and the sample

design used for the assessment.). These samples were defined only by grade (4, 8, or 12) and not by age

of the student. Data from the samples denoted (Civics–Main) comprised the spiraled partially balanced

incomplete block design (spiral BIB design, described in Chapter 22) and the present chapter contains

information about the scaling of data from these samples. The analyses for the special trend study of

1988–1998 civics will be published in a separate report through the National Center for Education

Statistics (NCES).

24.2 ITEM ANALYSIS

This section contains a detailed description of the item analysis performed using sample data.

The analysis examines items within blocks. In preparation for this step, constructed-response items were

polytomously scored, and derived background variables were calculated. Item statistics such as mean

percent correct, average score, item to total score correlations, and percent responding in each item

category were calculated.

Tables 24-1, 24-2, and 24-3 show the number of scaled items, number of constructed-response

items, unweighted sample size, weighted mean item score, weighted alpha reliability, weighted mean

item to total score correlation, and the weighted proportion of students attempting the last item in the

block for each block administered at each grade level for the national main assessment for grades 4, 8,

and 12, respectively. These values were calculated within block only for those items used in the scaling

process. For these item analyses, accommodated students were excluded, because they were not evenly

distributed across items; all of the accommodated students in a grade received the same two blocks.

Because of the concentration in these blocks of accommodated students, who are generally lowerscoring, inclusion of the accommodated students in the data for these blocks would have made these

 

1

 Spencer S. Swinton was the primary person responsible for the planning, specification, and coordination of the civics analyses.

Computing activities for all civics scaling and data analyses were directed by Edward Kulick and completed by Venus Leung.

Others contributing to the analysis of civics data were David S. Freund, Bruce A. Kaplan, and Katharine E. Pashley.
422

items appear more difficult than they would have in other blocks. Student weights were used, except for

the sample sizes. The results for the blocks administered to each grade level indicated that despite nearly

identical numbers of items, the blocks differ in average difficulty (i.e., weighted average item score

[Block C4=.48 – Block C7=.55]), reliability (i.e., weighted alpha reliability [Block C8=.68 – Block

C3=.74]), and proportion reaching the last item (Block C3=.84 – Block C6=.93]). Note that these tables

are descriptive, since no significance tests of differences were done.

As described in Chapter 9, in NAEP analyses (both conventional and IRT-based) a distinction is

made between missing responses at the end of each block (not-reached) and missing responses prior to

the last completed response (omitted). Not-reached items are those occurring after the last item the

student completed in a block. Items that were not reached are treated as if they had not been presented to

the examinee, while omitted items are regarded as incorrect.

The r-polyserial is a generalization of the r-biserial statistic traditionally employed in item

analysis. Like the alpha reliability, the r-biserial and r-polyserial statistic provides information about the

reliability of the block of items. Smaller values are less desirable than large values. The proportion of

students attempting the last item of a block (or, equivalently, one minus the proportion not reaching the

last item) is often used as an index of the degree of speededness of the block of items.

Tables 24-1 to 24-3 also contain information about the effect of the position of blocks within

booklets on the average item score for items within each block presented to the national main samples for

each grade. Because the special trend study 1988–1998 blocks appeared in only one position, they are not

included in these tables. The averages for the national main samples show that the order of blocks within

booklets has a small, but consistent, effect on mean item score in the national main civics assessment.

Table 24-1

Descriptive Statistics for Item Blocks by Position Within Test Booklet and Overall

Occurrences for the National Main Civics Sample, Grade 4, As Defined After Scaling

Statistic Position C3 C4 C5 C6 C7 C8

Number of Scaled Items

Number Constructed-Response Items

15

4

15

4

15

3

15

3

15

4

14

3

Unweighted Sample Size First

Second

Both

942

985

1,927

921

904

1,825

984

947

1,931

965

938

1,903

975

969

1,944

946

971

1,917

Weighted Average Item Score First

Second

Both

.53

.52

.52

.50

.47

.48

.51

.48

.49

.55

.53

.54

.56

.54

.55

.51

.50

.50

Weighted Alpha Reliability First

Second

Both

.73

.75

.74

.73

.72

.72

.72

.71

.71

.69

.70

.70

.69

.71

.70

.68

.68

.68

Weighted Average R-Polyserial* First

Second

Both

.52

.56

.54

.54

.55

.55

.54

.55

.54

.54

.55

.55

.50

.54

.52

.48

.51

.50

Weighted Proportion of Students

Attempting Last Item

First

Second

Both

.77

.91

.84

.86

.91

.88

.83

.90

.86

.93

.94

.93

.87

.92

.90

.82

.93

.88
Table 24-2

Descriptive Statistics for Item Blocks by Position Within Test Booklet and Overall

Occurrences for the National Main Civics Sample, Grade 8, As Defined After Scaling

Statistic Position C3 C4 C5 C6 C7 C8 CNumber of Scaled ItemsNumber Constructed-Response Items

1915

1916

1814

1915

1915

1815

Unweighted Sample Size First

SecondBoth

1,0001,0032,003

980

1,0121,992

981992

1,973

1,0021,0092,011

993974

1,967

1,021

975

1,996

9

1,01,9

Weighted Average Item Score First

SecondBoth

.50.47.48

.44.43.44

.47.46.47

.56.54.55

.49.47.48

.56.55.55

...

Weighted Alpha Reliability First

SecondBoth

.77.76.76

.78.77.77

.75.76.75

.77.77.77

.71.73.72

.72.73.73

...

Weighted Average R-Polyserial First

SecondBoth

.53.53.53

.57.55.56

.53.54.53

.55.54.55

.48.50.49

.51.52.52

...

Weighted Proportion of StudentsAttempting Last Item

FirstSecondBoth

.88.93.90

.94.94.94

.90.92.91

.95.96.95

.82.90.86

.93.96.94

...
424

Table 24-3

Descriptive Statistics for Item Blocks by Position Within Test Booklet and Overall

 Occurrences for the National Main Civics Sample, Grade 12, As Defined After Scaling

Statistic Position C3 C4 C5 C6 C7 C8 Number of Scaled ItemsNumber Constructed-Response Items

1915

1916

1915

1815

1915

1915

Unweighted Sample Size First

SecondBoth

988931

1,919

970976

1,946

929924

1,853

940996

1,936

922947

1,869

957928

1,885

Weighted Average Item Score First

SecondBoth

.54.51.53

.56.53.54

.53.51.52

.57.55.56

.50.49.50

.51.48.50

Weighted Alpha Reliability First

SecondBoth

.83.85.84

.75.77.76

.79.81.80

.75.76.76

.77.79.78

.72.75.74

Weighted Average R-Polyserial First

SecondBoth

.61.63.62

.54.54.54

.54.57.56

.54.54.54

.54.55.55

.48.51.50

Weighted Proportion of StudentsAttempting Last Item

FirstSecondBoth

.87.91.89

.95.95.95

.76.85.80

.94.92.93

.88.90.89

.86.91.89
425

In grades 4 and 8, and in most grade 12 blocks, the proportion of students attempting the last item

is higher for blocks in the second position. This suggests that students learn to pace themselves better as

they go through the assessment. Since slower students are more likely to be somewhat lower-scoring, if

more of them run out of time in the first block and do not attempt the final items, they will not contribute

to those item statistics, which will be based on a group of relatively more able individuals. This will

make the average item appear somewhat easier in the first position than in the second.

24.2.1 Constructed-Response Items

As indicated previously in Tables 23-3, 23-5, and 23-7, about 20 percent of the civics items were

constructed-response. Constructed-response items were scored in 3 or 4 categories. The categories of

responses for the items and the number of responses that were rescored for each item are indicated in

Appendix C. The percent agreement for the raters and the intraclass correlation, a rater reliability

estimate appropriate for items with several categories, are also given in the appendix. The sample sizes

listed in the tables correspond to the samples used in calculating the rater reliability.

In general, the rater reliability of the scoring for dichotomized responses was reasonably high.

Reliabilities ranged over items from 0.69 to 0.96 for grade 4, mean 0.82; from 0.50 to 0.94 for grade 8,

mean 0.80; and from 0.61 to 0.90 for grade 12, mean 0.78. The item in grade 8 with unusually low scorer

reliability, P040903, was a 3-category item requiring the student to explain characteristics of a good

representative.

Chapter 7 discusses the definition of the item ratings and describes the process by which teams

of raters scored the constructed-response items. This discussion includes the rating definitions for short

and extended constructed-response items as well as the range of interrater reliabilities that occurred.

Constructed-response items were scored on a scale from 1 to 4 or 1 to 3 to reflect degrees of knowledge.

In scaling, this scale is shifted to 0 to 3 or 0 to 2, respectively. Rating information on constructedresponse items can be found in Appendix C, which lists the sample sizes, percent agreement, and

Cohen’s Kappa reliability index. No items were excluded because of low rater reliabilities.

24.3 ITEM RESPONSE THEORY (IRT) SCALING

For each grade, a separate univariate IRT scale was constructed. The BILOG/PARSCALE

computer program was used to estimate the item parameters for the national main assessment. For

dichotomous multiple-choice and dichotomized constructed-response items, a three-parameter IRT model

was used. Three- and four-category items were polytomously scored and were analyzed with a

generalized partial- credit model (Muraki, 1992).

Recall from Section 24.2 that for calibration, item responses that were missing prior to the last

completed item in a block were considered omitted and scored as wrong. Also, items that were not

reached were treated as if they were not presented to the examinees (and therefore, not counted as

wrong). Omitted multiple-choice items were treated as fractionally [ 1 / (number of alternatives) ]

correct. Responses to constructed-response items that were classified by scorers as “off-task” (not

responsive to the question) were treated as omitted and assigned to the lowest category (0 = omitted). For

score-point descriptions, see Section 15.3; for details on scaling procedures, see Section 12.3.1.

The item parameter estimation was done separately within grade, with accommodated student

responses included as a separate population. Empirical Bayes modal estimates of all item parameters

were obtained from the BILOG/PARSCALE program. Prior distributions were imposed on item
426

parameters with the following starting values: thresholds, normal [0,2]; slopes, log-normal [0,.5]; and

asymptotes, two-parameter beta with parameter values determined as functions of the number of response

options for an item and a weight factor of 50. The locations (but not the dispersions) were updated at

each program estimation cycle in accordance with provisional estimates of the item parameters.

Item parameter estimation proceeded in two phases. First, the subject ability distribution was

assumed fixed (normal [0,1]) and a stable solution was obtained. Starting values for the item parameters

were provided by item analysis routines. The parameter estimates from this initial solution were then

used as starting values for a subsequent set of runs in which the subject ability distribution was freed

(modeled as a multinomial distribution) and estimated concurrently with item parameter estimates. After

each estimation cycle, the subject ability distribution was standardized to have a mean of zero and

standard deviation of one. Correspondingly, parameter estimates for that cycle were also linearly

standardized.

In the final BILOG/PARSCALE run, the prior distributions of the population abilities were free

to be estimated and the overall distribution was set to range from –6 to +4. The calibration was based on

student weights that were rescaled so that the their sum equaled the unweighted sample size of the 1998

sample. The weights of accommodated students were further rescaled so that for a given item from the

accommodation blocks, the proportion of responses from accommodated students was made similar to

their proportion in the weighted sample. As a result, the sum of population weights for accommodated

students is smaller than the sum of population weights for nonaccommodated students.

Items that received special treatment in the scaling procedure are listed in Table 24-4, along with

the reason for special treatment. Items were either dropped or collapsed. If items had empirical item

response functions that were severely nonmonotonic, they were dropped. If polytomous items had sparse

or nonmonotonic responses in one or more categories, the items were collapsed so that some adjacent

response categories were combined into a single category. Only eight of the total items were given

special treatment.

Table 24-4

1998 Civics Items Receiving Special Treatment

Grade NAEP ID Block Treatment

4 P040102 C4 Collapsed: (0,1,2) becomes (0,0,1)

P040402 C7 Collapsed: (0,1,2,3) becomes (0,0,1,2)

P040506 C8 Dropped due to lack of fit

8 P040905 C5 Dropped due to DIF

P041003 C6 Collapsed: (0,1,2) becomes (0,0,1)

P041204 C8 Dropped due to lack of fit

12 P041705 C5 Collapsed: (0,1,2) becomes (0,1,1)

P041810 C6 Dropped due to lack of fit

24.3.1 Evaluating the Fit of the IRT Model

During the course of estimating an IRT model, individual items were evaluated to determine how

well the item response model fit the data. This was done by visual inspection of plots comparing

empirically based and theoretical item response functions. Specifically, for dichotomous items these plots

consisted of empirically based estimates of the expected proportion correct for each level of civics

performance compared to the proportion correct for each level of civics scale score as predicted by the

theoretical item response function. For polytomous extended constructed-response items, similar plots
427

were produced for each item category response function. See Chapter 12 for a fuller explanation of these

plots.

In making decisions about excluding items from the final scales, a balance was sought between

being too stringent, hence deleting too many items and possibly damaging the content representativeness

of the pool of scaled items, and being too lenient, hence including items with model fit so poor as to

weaken the types of model-based inferences made from NAEP results. Items showing extreme misfit

were not included in the final scales; however, a certain degree of misfit was tolerated for a number of

items included in the final scales.

For most items, the model fit reasonably well in the scale score region containing most of the

observations. In a few cases, poor fit with the data led to special treatment or deletion of the item. Figures

24-1, 24-3, and 24-5 give item response plots of dichotomous items. In the plots, the x-axis depicts scale

score (theta), and the y-axis the probability of a correct response. The solid line is the logistic model

prediction, and the symbols (diamonds) are the empirically based proportions. The size of the symbols

are proportional to the estimated number of students at a particular scale score level. The item parameter

values are also included in the plot.

Item response plots for polytomously scored items are given in Figures 24-2, 24-4, 24-6, and

24-7. These are similar to the plots for dichotomous items except that there are several solid lines, one for

each item category, with each line indicating the probability of responding in the respective item

category. As before, the diamonds indicate the empirical response function, with the size of the symbols

proportional to the estimated number of students at a scale score level.

In the plots, good fit of the model to the data is indicated when the model-based functions (solid

lines) coincide with the empirical functions (diamonds). When the empirical plot is far away from the

model-based line, there is poor fit of the model to the data.

Four examples of fit are illustrated. First there is good model fit, which is shown by Figure 24-1

for a dichotomous item and Figure 24-2 for a polytomous item. In both cases empirical and theoretical

lines nearly coincide.

Second are examples of items that displayed moderate lack of fit to the theoretical function.

Figure 24-3 shows a dichotomous item and Figure 24-4 a polytomous item with moderate model misfit.

Third (Figure 24-5) is an example of a dichotomous item exhibiting unacceptably poor model fit.

This item was dropped from the assessment. This item asked the student to identify a function of a

nongovernmental organization.

The fourth example is of a poorly fitting polytomous item that was modified by collapsing

categories. Figure 24-6 shows a 4-category item that evidences poor fit mostly in the lower categories. As

a result, the lower two categories were collapsed, resulting in a 3-category item, as illustrated in Figure

24-7. This plot still exhibits some degree of misfit, but was judged to fit satisfactorily to be included in

the scale. This item asked the student to write on the contrast between a rule and a law.
428

24.3.2 Derived Background Variables

Derived variables are variables that use information from more than one background question.

They were used for two purposes: as conditioning variables and as reporting variables used to define

subgroups. Some of these variables are common to all the subject areas; others are specific to the 1998

civics assessment. Derived variables used for conditioning and reporting are described in Appendix G.

Figure 24-1

Dichotomous Item (P040719) Exhibiting Good Model Fit*

* Diamonds represent 1998 grade 12 civics assessment data. They indicate estimated

conditional probabilities obtained without assuming a specific model form; the curve

indicates the estimated item response function (IRF) assuming a logistic form.
429

Figure 24-2

Polytomous Item (P042008) Exhibiting Good Model Fit*

* Diamonds represent 1998 grade 12 civics assessment data. They indicate estimated

conditional probabilities obtained without assuming a specific model form; the curve

indicates the estimated item category response function (ICRF) using a generalized partial

credit model.
430

Figure 24-3

Dichotomous Item (P041209) Exhibiting Moderate Model Misfit*

* Diamonds represent 1998 grade 12 civics assessment data. They indicate estimated

conditional probabilities obtained without assuming a specific model form; the curve

indicates the estimated item response function (IRF) assuming a logistic form.
431

Figure 24-4

Polytomous Item (P041902) Exhibiting Moderate Model Misfit

* Diamonds represent 1998 grade 12 civics assessment data. They indicate estimated

conditional probabilities obtained without assuming a specific model form; the curve

indicates the estimated item category response function (ICRF) using a generalized partial

credit model.
432

Figure 24-5

Dichotomous Item (P040506) Exhibiting Poor Model Fit*

(Deleted from the Assessment)

* Diamonds represent 1998 grade 4 civics assessment data. They indicate estimated

conditional probabilities obtained without assuming a specific model form; the curve

indicates the estimated item response function (IRF) assuming a logistic form.
433

Figure 24-6

Polytomous Item (P040402) Exhibiting Poor Model Fit in the Lower Two Categories*

* Diamonds represent 1998 grade 4 civics assessment data. They indicate estimated

conditional probabilities obtained without assuming a specific model form; the curve

indicates the estimated item category response function (ICRF) using a generalized partial

credit model.
434

Figure 24-7

Same Polytomous Item (P040402) with the Lower Two Categories Collapsed,

Now Exhibiting Improved Model Fit*

* Diamonds represent 1998 grade 4 civics assessment data. They indicate estimated

conditional probabilities obtained without assuming a specific model form; the curve

indicates the estimated item category response function (ICRF) using a generalized partial

credit model.
435

24.4 GENERATION OF PLAUSIBLE VALUES

For the grade sample, univariate plausible values for a single overall civics score scale were

generated using the univariate conditioning program BGROUP. As with the scaling, student weights

were used at this stage of the analysis. To avoid bias in reporting results and to minimize biases in

secondary analyses, it was desirable to incorporate a large number of independent variables in the

conditioning model. When expressed in terms of contrast-coded main effects and interactions, the

number of variables to be included totaled 869 for grade 4, 866 for grade 8, and 699 for grade 12. The

much larger numbers for grade 4 and grade 8 reflect the number of contrasts from the teacher

questionnaires.

Some of these contrasts involved relatively small numbers of individuals and some were highly

correlated with other contrasts or sets of contrasts. Given the large number of contrasts, an effort was

made to reduce the dimensionality of the predictor variables. The original background variable contrasts

were standardized and transformed into a set of linearly independent variables by extracting separate sets

of principal components at each grade level. The principal components, rather than the original variables,

were used as the independent variables in the conditioning model. The number of principal components

was the number required to account for at least 90 percent of the variance in the original contrast

variables. Research based on data from the 1990 trial state assessment in mathematics suggests that

results obtained using such a subset of components will differ only slightly from those obtained using the

full set (Mazzeo, Johnson, Bowker, & Fong, 1992). The principal component procedure reduced the

number of variables to 318 in grade 4, 320 in grade 8, and 263 in grade 12.

Research based on data from the 1990 trial state assessment suggests that results obtained using

the 90 percent subset of components will differ only slightly from those obtained using the full set

(Mazzeo, Johnson, Bowker, & Fong, 1992). Table 24-5 contains a list of the number of principal

components included in conditioning, as well as the proportion of scale score variance accounted for by

the conditioning model (as described in Chapter 12) for each grade.

The codings of the original civics-specific conditioning variables, before principal components

were calculated, are presented in Appendix F. The BGROUP program estimates distributions of scale

scores by combining information from item responses of individuals and information from linear

regression of scale score on conditioning variables. For each individual, five plausible values are

randomly drawn from their estimated scale score distribution.

Table 24-5

Proportion of Scale Score Variance Accounted for by the Conditioning Model

for the National Main Civics Assessment

Grade

Number of

Conditioning Contrasts

Number of

Principal Components

Proportion of Scale

Score Variance

Accounted for

 4 869 319 .64

 8 866 320 .58

12 699 262 .55

The conditioning model reduces redundancy by extracting principal components from a large

number of conditioning variables and basing conditioning on the components that account for 90 percent

of the variance of the components (see Sections 17.4 and 20.4).
436

The proportion of variance of each original conditioning variable accounted for by the principal

components included in the conditioning model is listed in Appendix C. The estimated conditioning

effects for the principal components of the samples defined by the three grade groups are also given in

Appendix C. The values of the conditioning effects are expressed in the metrics of the original

calibration scale. Definitions of derived conditioning variables are given in Appendix G.

24.5 TRANSFORMATION OF THE CIVICS CALIBRATION SCALE

FOR REPORTING

Since the 1998 civics assessment was developed and scaled using within-grade procedures, and

since there was no prior civics assessment with a comparable framework to which it was being linked, a

new reporting metric was adopted. The results are reported on 0–300 scales with identical means at each

grade. As is shown in Table 24-6, the mean of the civics scale was set at 150 for each grade, and the

standard deviation at 35.

Table 24-6

Means and Standard Deviations for the Civics Scale

All Five Plausible Values

Grade Mean S. D.

4 150.0 35.0

8 150.0 35.0

12 150.0 35.0

If the achievement distribution were normal, we would expect this range to cover about 99.998 percent of

the distribution. Note that any transformed scale scores below 0 were censored to values of 0. A total of

three scores in grade 4, six scores in grade 8, and five scores in grade 12 were censored to values of 0.

Had any transformed scale scores been greater than 300, they would have been censored to values of 300;

however, no such cases were encountered.

Constraining the mean and standard deviation of the scales in this way also constrained, to some

degree, the percentile distributions for the total group. However, within-grade comparisons of percentiles

across subgroups continue to provide valuable comparative information, although cross-grade

comparisons, with each grade set to the same mean and standard deviation, do not have meaning.

For each grade, the target mean and standard transformation resulted from applying the linear

transformation:

θtarget = A • θcalibrated + B,

where A and B are linear transformation constants. The values of A and B for each grade are given in

Table 24-7. These numbers are documented for researchers who wish to reproduce these analyses, and

equally, for archival purposes for those who carried out these analyses.
437

Table 24-7

Transformation Constants for the National Main Civics Assessment

Grade A B

4 39.98 149.36

8 38.49 149.68

12 37.87 149.46

24.6 PARTITIONING OF THE ESTIMATION ERROR VARIANCE

Within each grade, the error variance of the reporting scale mean was partitioned according to

the procedure described in Chapter 12. The variance is partitioned into two parts: the proportion of error

variance due to sampling students (sampling variance) and the proportion of error variance due to the fact

that scale score, θ, is a latent variable that is estimated rather than observed. Table 24-8 contains

estimates of the total error variance, the proportion of error variance due to sampling students, and the

proportion of error variance due to the latent nature of θ (for stability, the estimates of the betweenimputation variance, B, in Equation 12.12 are based on 100 imputations for each student). Table 24-8

shows that the preponderance of error variance is attributable to student sampling. More detailed

information by gender and race/ethnicity is presented in Appendix H.

Table 24-8

Estimation Error Variance and Related Coefficients

for the National Main Civics Assessment

Proportion of Variance Due to...

Grade

Total Estimation

Error Variance

Student

Sampling

Latency

of θ

4 .54 .90 .10

8 .32 .91 .09

12 .62 .95 .05

* Since θ is unobserved, or "latent," a proportion of the estimation error is due to the

fact that θ is known imperfectly.

24.7 CIVICS TEACHER QUESTIONNAIRE

Teachers of fourth- and eighth-grade students assessed in civics were surveyed. Along with a

variable that indicated whether a student record had been matched with a teacher record, variables

derived from the questionnaire were used in the conditioning models for the grade 4 and the grade 8

samples. These variables were included, so that means for subgroups defined by these variables could be

compared with no bias. Of the 5,948 fourth-grade students in the main sample, 5,110 (86%) were

matched with both parts of the teacher questionnaire and 277 (5%) were matched with only the first part

of the questionnaire. Of the 8,212 eighth-grade students in the main sample, 6,053 (74%) were matched

with both parts of the teacher questionnaire and 649 (8%) were matched with only the first part of the

questionnaire. Thus, 91 percent of the fourth-graders and 82 percent of the eighth-graders were matched

with at least the background information about their civics teachers.

439

Appendix A

STATISTICAL SUMMARY OF THE 1998 NAEP SAMPLES1

Bruce A. Kaplan and Youn-Hee Lim

Educational Testing Service

A.1 INTRODUCTION

The analysis of the 1998 NAEP data has resulted in the production of thousands of tables

presenting estimates of the scale score of students, and various subgroups of students, in American

schools. This appendix provides a statistical summary of the 1998 NAEP national samples. The appendix

assumes a general familiarity with the structure of NAEP as summarized in the Introduction and in the

overviews presented in Chapters 1 and 9. Similar results for the individual state samples appear on the

NCES website (http://nces.ed.gov/nationsreportcard).

Two of the many types of NAEP results are presented here:

1. the results of the instrument development process, including the sizes of

the item pools and numbers of booklets; and

2. the results of the sampling process, including the numbers of students in

each sample by selected subgroups.

A.2 MEASUREMENT INSTRUMENTS

For the 1998 assessment, 79 different assessment booklets and questionnaires were printed for

grade 4, 99 for grade 8, and 98 for grade 12. These instruments are shown by age level and type in Table

A-1.

The item pool contributing to all booklets is described in Table A-2. In general, there are two

types of items, cognitive and noncognitive. The cognitive items are developed to measure proficiency in

subject areas (reading, writing, and civics). Cognitive items may be constructed-response or multiplechoice. The noncognitive items are usually questions about the student’s or teacher’s backgrounds and

perceptions but may also probe other areas, such as school policies or teaching methods. Because many

items were used at more than grade class, the total number of items in an item pool is not the sum of the

item pools used for the three grade classes.

The SD/LEP Student Questionnaires, Teacher Questionnaires, and School Characteristics and

Policies Questionnaires contained only noncognitive questions. The number of items in the noncognitive

pools is the same as the number of items on the questionnaires. More information about the instruments

that were developed is provided in Chapters 2, 14, 18, and 22.

 

1 Bruce A. Kaplan was responsible for the text, specifying the tables, and coordinating table production. Youn-Hee Lim, Ting Lu,

and Michael Narcowich produced most of the tables in this chapter. The advice of David S. Freund and Nancy L. Allen was

invaluable in the production of this chapter.
440

A.3 SAMPLE CHARACTERISTICS

In this section, the characteristics of the final reporting NAEP samples are described. The

process by which the samples were selected is discussed in Chapters 3 (national) and 4 (state).

In the 1998 main assessment, NAEP contacted 2,866 schools (2,698 original and 168

replacements), of which 2,102 contributed data to the assessment. The disposition of these schools is

shown in Table A-3. Some of the schools were unwilling to cooperate; others were believed to be eligible

from the sampling frame, but were not. The cooperation rate is calculated as the sum of cooperating

schools and the schools that were found to have no eligible students divided by the same sum plus the

schools that refused or were from districts that refused to cooperate.

Table A-4 shows the number of schools in several categories: region of the country (Northeast,

Southeast, Central, West), school type (public, nonpublic, Catholic, Bureau of Indian Affairs,

Department of Defense Education Activity), type of location, number of teachers, and number of

students.

The numbers of respondents to the teacher questionnaires are summarized in Table A-5. The first

column in this table includes the number of teachers who responded, by grade and subject area. The

second column is the number of students who were not linked to teachers. The third column is the

number of students linked to teachers, but not specific classes of these teachers (for eighth grade) or

teachers who did not answer classroom information (for fourth grade). The last column is the number of

students linked to their teachers and their specific classes.

Table A-6 lists the total number of students assessed, accommodated, and excluded. This is done

by grade crossed with subject area. Note that the number of accommodated students is included in the

assessed students. No accommodations were offered in the reading reporting sample, the writing 50-

minute sample, and the civics special trend sample. Also for reading, the numbers are for the reporting

sample only.

Tables A-7 through A-9 display the distribution of the students assessed in the national and state

NAEP assessment in several basic categories: gender, racial/ethnic grouping, region of the country,

parental education, type of location, school type, modal age, and students with disabilities (SD), and

students with Limited English Proficiency (LEP) status. These data are presented for assessed students in

the reading samples in Table A-7, the writing samples in Table A-8, and the civics samples in Table A-9.

Tables A-10, A-11, and A-12 provide equivalent information, respectively, for excluded students.

Table A-13 for writing and Table A-14 for civics contain similar information for the accommodated

students. The reading reporting sample, due to the necessity of linking to trend, did not contain students

who were offered accommodations.

A.4 POPULATION ESTIMATES

The 1998 NAEP samples were designed for estimating the size and attributes of a number of

different populations of students. The estimation procedures use sampling weights, developed by Westat,

Inc., that are associated with the members of the sample (see Chapters 3 and 4). In this appendix, all

estimates of population parameters are calculated using these sampling weights. Note these estimates are

for the reporting samples (see Chapters 3 and 4 for an explanation of the reporting and modular samples).

The sum of the initial weights for a given sample is an estimate of the number of students who

are in the population represented by the sample. In other words, the sum of the initial weights is taken as

the estimated population size. In analyses, however, this sum of weights was rescaled to sum to the

sample size. For example, in Table A-15, the estimated number of fourth graders in the nation is
441

3,588,382, as estimated from the main reading sample, as opposed to the 7,672 students in the sample

given in Table A-7. The sum of the weights of the students in the state assessment estimate the total

number of grade eligible students in the participating jurisdictions.

The sum of the weights of the excluded students estimates the number of ineligible students at

the respective grade levels.

An estimate of the total number of students in a grade sample can be made by summing the initial

weights of grade-eligible students plus the initial weights of grade-eligible students from the appropriate

excluded student sample.

Tables A-15 to A-17 show the sizes of the estimated populations of assessable students and the

weighted percentages for the NAEP reporting categories of gender, race/ethnicity, region of the country,

parents’ education level, type of location, school type, modal age and SD/LEP status. The estimated

subpopulation percentages for the national and state samples are shown in Tables A-15 through A-17.

Tables A-18 to A-20 show the estimated total population of excluded students and the weighted

percentages by demographic subgroups (data about parents’ education level is not collected for excluded

students and therefore not reported; data about reasons for exclusion are included instead).

Tables showing selected scale score results for assessed students, as an aid to readers who are

interested in the estimates of scale scores that led to the interpretive results provided in the NAEP subject

area reports, can be accessed from Summary Data Tables posted on the National Center for Education

Statistics website at http://nces.ed.gov/nationsreportcard.

Table A-1

Measurement Instruments Used in 1998 NAEP

Grade 4 Grade 8 Grade 12

Total 78 98 97

Student Booklets

Reading 16 19 20

Writing 25-Minute 40 40 40

Writing 50-Minute — 3 3

Civics Main 18 32 32

Questionnaires

SD/LEP 1 1 1

Teacher 2 2 —

School 1 1 1

Note: “—” indicates that this category was not applicable.
442

Table A-2

Number of Items Administered, by Sample and Age Class

Grade 4 Grade 8 Grade 12

Common Background

Reading 21 17 18

Writing 25-Minute 21 22 24

Writing 50-Minute 21 22 24

Civics Main 21 22 24

Reading

Background 22 24 25

Cognitive 82 110 119

Motivation 5 5 5

Writing 25-Minute

Background 17 28 28

Cognitive 20 20 20

Motivation 5 5 5

Writing 50-Minute

Background — 28 28

Cognitive — 3 3

Motivation — 5 5

Civics

Background 22 24 29

Cognitive 90 151 152

Motivation 5 5 5

Questionnaires

SD/LEP 46 46 46

Teacher Reading/Writing 145 145 —

Teacher Civics 89 88 —

School 52 52 52

 Note: “—” indicates that this category was not applicable.
443

Table A-3

School Participation in NAEP 1998 Main Samples (All Subsamples)

Grade 4 Grade 8 Grade 12

 Total Original Sample 889 957 852

Cooperating 671 703 560

No Eligibles Enrolled 7 7 4

School Refused 104 118 135

Cooperation Rate 81% 81% 75%

Cooperating Replacements for Refusals 62 58 48

 Totals

Cooperating Schools 733 761 608

Completing Questionnaires 700 721 569

Missing 33 40 39
444

Table A-4

School Characteristics in NAEP 1998 Main Samples

Grade 4 Grade 8 Grade 12

Region

Northeast 161 170 123

Southeast 174 175 167

Central 173 187 121

West 225 229 197

School Type

Public 473 427 446

Private 93 114 82

Catholic 28 33 19

BIA 138 186 59

DODEA 102

Size and Type of Community

 Rural 157 166 113

 Disadvantaged Urban 148 141 108

 Advantaged Urban 192 209 153

 Big City 49 54 45

 Fringe 9 10 8

 Medium City 80 76 77

 Small Place 98 105 104

Number of Enrolled Students

10-250 194 192 101

251-500 245 194 105

501-1000 208 209 106

1,001-2,000 28 91 158

2,000+ 1 7 78
445

Table A-5

Numbers of Responses to Teacher Questionnaires and Students

Matched with Teacher Data

Number of

Teachers

Responding No Match

Partial

Match

Complete

Match

Reading

Grade 4 National 1,252 597 334 6,741

Grade 4 State 14,707 7,099 13 105,026

Grade 8 National 1,266 1,181 935 8,935

Grade 8 State 10,209 5,736 6,575 82,118

Writing

Grade 4 25-Minute 1,799 1,395 830 17,591

Grade 8 25-Minute 1,565 4,279 1,574 14,733

Grade 8 50-Minute 1,286 1,277 467 4,266

Grade 8 State 10,695 6,064 6,920 84,605

Civics

Grade 4 Main 1,606 561 277 5,110

Grade 8 Main 1,275 1,510 649 6,053
446

Table A-6

Number of Students Assessed, Accommodated, and Excluded

by Reporting Sample and Grade

Grade 4 Grade 8 Grade 12

ASSESSED STUDENTS 145,574 237,877 45,751

Reading 119,810 105,480 12,675

National 7,672 11,051 12,675

State 112,138 94,429 —

Writing 19,816 124,185 25,313

25-Minute 19,816 20,586 19,505

50-Minute — 6,010 5,808

State — 97,589 —

Civics 5,948 8,212 7,763

ACCOMMODATED STUDENTS 953 3,670 432

Reading — ——

National — — —

State — — —

Writing 746 3,449 326

25-Minute 746 678 326

50-Minute — — —

State — 2,771 —

Civics 207 221 106

EXCLUDED STUDENTS 7,605 10,461 1,582

Reading 5,748 4,074 448

National 545 623 448

State 5,203 3,451 —

Writing 1,450 6,046 887

25-Minute 1,450 877 658

50-Minute — 270 229

State — 4,899 —

Civics 407 341 247

 Note: “—” indicates that this category was not applicable.
447

Table A-7

Number of Students in the Reading Reporting Samples

by Subgroup Classification, National Grades 4, 8, and 12 & State Grades 4 and 8

National State

Grade 4 Grade 8 Grade 12 Grade 4 Grade 8

Total 7,672 11,051 12,675 112,138 94,429

Gender

Male 3,749 5,512 6,086 55,393 46,636

Female 3,923 5,539 6,589 56,745 47,793

Race/Ethnicity

White 4,277 6,457 7,585 71,446 62,082

Black 1,300 1,745 2,052 19,124 15,222

Hispanic 1,624 2,141 2,234 13,733 10,379

Asian American 283 564 689 4,634 4,600

American Indian 173 119 94 3,007 1,940

Unclassified 15 25 21 194 206

Region

Northeast 1,547 2,006 2,533 22,981 19,092

Southeast 2,212 3,046 3,570 31,713 29,483

Central 1,455 2,273 2,325 17,925 11,216

West 2,458 3,726 4,247 32,368 31,019

Unclassified (Territories) — — — 7,151 3,619

Parent’s Education

Less Than High School 248 831 1,118 3,166 6,316

High School 966 2,230 2,359 15,139 20,344

Greater Than High School 1,282 1,889 3,150 19,906 18,506

Graduated College 4,228 4,996 5,626 60,907 41,158

Unknown 948 1,105 422 13,020 8,105

Type of Location

Central City 3,119 4,455 4,891 36,251 28,841

Urban Fringe/Large Town 2,812 4,068 4,743 34,426 30,633

Rural/Small Town 1,741 2,528 3,041 38,581 32,715

Unclassified (Territories) — — — 2,880 2,240

School Type

Public 6,300 9,091 10,664 98,873 86,201

Nonpublic 1,372 1,960 2,011 7,676 5,264

Private 493 746 588 2,917 1,833

Catholic 879 1,214 1,423 4,759 3,431

BIA 0 0 0 124 104

DODEA 0 0 0 5,465 2,860

Note: “—” indicates that this category was not applicable.

(continued)
448

Table A-7 (continued)

Number of Students in the Reading Reporting Samples

by Subgroup Classification, National Grades 4, 8, and 12 & State Grades 4 and 8

National State

Grade 4 Grade 8 Grade 12 Grade 4 Grade 8

Modal Age

< Modal Age 44 71 164 611 501

= Modal Age 4,979 6,729 8,204 72,369 56,697

> Modal Age 2,649 4,251 4,307 39,158 37,231

SD/LEP

SD Only 236 427 286 3,075 2,784

LEP Only 197 291 259 1,333 794

SD & LEP 7 24 18 75 57

Non SD/LEP 7,232 10,309 12,112 107,655 90,794
449

Table A-8

Number of Students in the Writing 25-Minute and 50-Minute Samples

by Subgroup Classification, Grades 4, 8, and 12 & State Grade 8

Main 25-minute Main 50-minute

State

25-minute

Grade 4 Grade 8 Grade 12 Grade 8 Grade 12 Grade 8

Total 19,816 20,586 19,505 6,009 5,804 97,589

Gender

Male 9,971 10,298 9,302 2,999 2,770 48,834

Female 9,845 10,288 10,203 3,010 3,034 48,755

Race/Ethnicity

White 10,612 11,774 11,628 3,531 3,476 62,490

Black 3,242 3,271 3,139 910 925 15,583

Hispanic 4,537 4,261 3,383 1,184 1,003 12,148

Asian American 760 930 1,088 275 316 4,723

American Indian 603 298 199 93 65 2,423

Unclassified 62 52 68 16 19 222

Region

Northeast 4169 4,042 4,068 1,152 1,183 20,342

Southeast 5,541 5,643 5,479 1,653 1,644 30,946

Central 3,534 3,936 3,618 1,159 1,085 9,723

West 6,572 6,965 6,340 2,045 1,892 32,939

Unclassified (Territories) — — — — — 3,639

Parent’s Education

Less Than High School 595 1,056 1,412 316 393 4,090

High School 2,241 2,855 2,492 878 767 14,814

Greater Than High School 3,207 5,665 5,283 1,621 1629 27,702

Graduated College 11,363 10,261 9,886 2,953 2,890 47,653

Unknown 2,410 749 432 241 125 3,330

Type of Location

Central City 8,024 8,305 7,640 2,449 2,276 30,070

Urban Fringe/Large Town 7,117 7,940 7,237 2,293 2,185 31,571

Rural/Small Town 4,675 4,341 4,628 1,267 1,343 33,652

Unclassified (Territories) — — — — — 2,296

School Type

Public 16,330 17,005 16,221 4,941 4,821 89164

Nonpublic 3,464 3,581 3,267 1,068 977 5,411

Private 1,118 1,388 963 405 285 1,849

Catholic 2,346 2,193 2,304 663 692 3,562

BIA 22 0 17 0 6 73

DODEA 0 0 0 0 0 2,941

Note: “—” indicates that this category was not applicable.

(continued)
450

Table A-8 (continued)

Number of Students in the Writing 25-Minute and 50-Minute Samples

by Subgroup Classification, Grades 4, 8, and 12 & State Grade 8

Main 25-minute Main 50-minute

State

25-minute

Grade 4 Grade 8 Grade 12 Grade 8 Grade 12 Grade 8

Modal Age

< Modal Age 87 146 231 46 76 523

= Modal Age 12,814 12,311 12,523 3,587 3,723 58,004

> Modal Age 6,915 8,129 6,751 2,376 2,005 39,062

SD/LEP

SD Only 1,342 1,407 785 242 154 6,859

LEP Only 785 591 508 160 127 1,678

SD & LEP 44 60 24 15 3 157

Non SD/LEP 17,645 18,528 18,188 5,592 5,520 88,895
451

Table A-9

Number of Students in the Civics Main Samples

by Subgroup Classification, Grades 4, 8, and 12

Grade 4 Grade 8 Grade 12

Total 5,948 8,212 7,763

Gender

Male 3,017 4,078 3,654

Female 2,931 4,134 4,109

Race/Ethnicity

White 3,200 4,732 4,597

Black 937 1,280 1,240

Hispanic 1,415 1,720 1,398

Asian American 217 348 433

American Indian 164 116 74

Unclassified 15 16 21

Region

Northeast 1,241 1,616 1,641

Southeast 1,656 2,258 2,196

Central 1,078 1,568 1,404

West 1,973 2,770 2,522

Parent’s Education

Less Than High School 173 448 560

High School 683 1,184 1,048

Greater Than High School 977 2,249 2,106

Graduated College 3,449 4,013 3,883

Unknown 642 265 74

Type of Location

Central City 2,416 3,311 3,069

Urban Fringe/Large Town 2,121 3,157 2,854

Rural/Small Town 1,411 1,744 1,840

School Type

Public 4,893 6,795 6,437

Nonpublic 1,048 1,417 1,319

Private 340 553 383

Catholic 708 864 936

BIA 7 0 7

DODEA 0 0 0

Modal Age

< Modal Age 10 50 103

= Modal Age 3,827 4,963 4,927

> Modal Age 2,111 3,199 2,733

SD/LEP

SD Only 385 542 292

LEP Only 262 199 211

SD & LEP 10 17 9

Non SD/LEP 5,291 7,454 7,251
452

Table A-10

Number of Excluded Students in the Reading Reporting Samples

by Subgroup Classification, Grades 4, 8, and 12 & State Grades 4 and 8

 National State

 Grade 4 Grade 8 Grade 12 Grade 4 Grade 8

 Total 545 623 448 5,203 3,451

 Gender

 Male 311 390 282 3,326 2,294

 Female 234 233 166 1,877 1,157

 Race/Ethnicity

 White 154 243 225 2,603 1,716

 Black 52 174 78 1,051 785

 Hispanic 317 175 117 1,181 681

 Asian American 22 21 26 233 161

 American Indian 0 9 1 85 85

 Unclassified 0 1 1 50 23

 Region

 Northeast 59 112 128 1,130 782

 Southeast 99 183 112 1,481 1,144

 Central 81 122 55 681 369

 West 306 206 153 1,697 1,071

Unclassified (Territories) — — — 214 85

 Type of Location

 Central City 253 298 207 2,013 1,243

 Urban Fringe/Large Town 207 204 160 1,512 1,067

 Rural/Small Town 85 121 81 1,600 1,088

Unclassified (Territories) — — — 78 53

 School Type

 Public 540 622 440 5,019 3,363

 Nonpublic 5 1 8 31 20

 Private 1 0 6 6 8

 Catholic 4 1 2 25 12

 BIA 0 0 0 2 3

 DODEA 0 0 0 151 65

Note: “—” indicates that this category was not applicable.

(continued)
453

Table A-10 (continued)

Number of Excluded Students in the Reading Reporting Samples

by Subgroup Classification, Grades 4, 8, and 12 & State Grades 4 and 8

 National State

 Grade 4 Grade 8 Grade 12 Grade 4 Grade 8

 Modal Age

 < Modal Age 0 1 10 29 26

 = Modal Age 326 216 125 2,433 1,184

 > Modal Age 219 406 313 2,741 2,241

 SD/LEP

 SD Only 222 489 333 3,979 2,787

 LEP Only 298 99 83 1,016 535

 SD & LEP 25 35 32 208 129
454

Table A-11

Number of Excluded Students in the Writing Samples

by Subgroup Classification, Grades 4, 8, and 12 & State Grade 8

Main 25 min. Main 50 min.

 State

 25 min.

 Grade 4 Grade 8 Grade 12 Grade 8 Grade 12 Grade 8

Total 1,450 877 658 266 201 4,899

 Gender

 Male 874 549 404 144 127 3,190

 Female 576 328 254 122 74 1,709

 Race/Ethnicity

 White 336 334 286 84 84 2,217

 Black 266 218 169 65 52 1,091

 Hispanic 797 286 171 101 55 1,164

 Asian American 42 35 29 14 8 270

 American Indian 5 3 2 2 2 108

 Unclassified 4 1 1 0 0 49

 Region

 Northeast 153 129 95 44 44 1,066

 Southeast 275 221 204 62 45 1,596

 Central 158 183 88 51 35 344

 West 864 344 271 109 77 1,781

Unclassified (Territories) — — — — — 112

 Type of Location

 Central City 788 443 291 145 82 1,873

 Urban Fringe/Large Town 459 250 232 72 70 1,433

 Rural/Small Town 203 184 135 49 49 1,549

Unclassified (Territories) — — — — — 44

 School Type

 Public 1,436 865 656 265 201 4,819

 Nonpublic 14 12 2 1 0 24

 Private 6 3 1 0 0 10

 Catholic 8 9 1 1 0 14

 BIA 0 0 0 0 0 1

 DODEA 0 0 0 0 0 55

 Modal Age

 < Modal Age 9 8 10 3 3 54

 = Modal Age 786 311 168 110 46 1,615

 > Modal Age 655 558 480 153 152 3,230

 SD/LEP

 SD Only 697 604 536 182 162 3,697

 LEP Only 644 205 92 70 34 988

 SD & LEP 109 68 30 14 5 214

Note: “—” indicates that this category was not applicable.
455

Table A-12

Number of Excluded Students in the Civics Main Samples

by Subgroup Classification, Grades 4, 8, and 12

 Grade 4 Grade 8 Grade 12

 Total 407 341 247

 Gender

 Male 257 207 153

 Female 150 134 94

 Race/Ethnicity

 White 98 130 109

 Black 66 74 61

 Hispanic 223 118 56

 Asian American 18 16 18

 American Indian 1 2 2

 Unclassified 1 1 1

 Region

 Northeast 38 51 36

 Southeast 76 82 63

 Central 34 63 42

 West 259 145 106

 Type of Location

 Central City 205 159 106

 Urban Fringe/Large Town 150 107 101

 Rural/Small Town 52 75 40

 School Type

 Public 405 337 247

 Nonpublic 2 4 0

 Private 0 1 0

 Catholic 2 3 0

 BIA 0 0 0

 DODEA 0 0 0

 Modal Age

 < Modal Age 3 2 0

 = Modal Age 233 121 61

 > Modal Age 171 218 186

 SD/LEP

 SD Only 186 225 201

 LEP Only 194 89 35

 SD & LEP 27 27 11
456

Table A-13

Number of Accommodated Students in the Writing Samples

by Subgroup Classification, Grades 4, 8, and 12 & State Grade 8*

Main 25 min. State 25 min.

Grade 4 Grade 8 Grade 12 Grade 8

Total 746 678 326 2,652

Gender

Male 475 444 196 1,809

Female 271 234 130 843

Race/Ethnicity

White 352 337 189 1,578

Black 150 130 46 370

Hispanic 197 175 72 509

Asian American 19 20 17 94

American Indian 24 16 2 98

Unclassified 4 0 0 3

Region

Northeast 182 179 135 751

Southeast 248 223 91 806

Central 123 103 46 283

West 193 173 54 752

Unclassified (Territories) — — — 60

Parent’s Education

Less Than High School 41 50 37 200

High School 86 109 45 453

Greater Than High School 102 200 75 756

Graduated College 382 273 151 1,025

Unknown 135 46 18 218

Type of Location

Central City 280 218 101 686

Urban Fringe/Large Town 304 293 127 929

Rural/Small Town 162 167 98 991

Unclassified (Territories) — — — 46

School Type

Public 700 641 306 2,534

Nonpublic 46 37 20 52

Private 18 9 10 29

Catholic 28 28 10 23

BIA 0 0 0 6

DODEA 0 0 0 60

* Accommodations were not offered in the 50-minute study.

Note: “—” indicates that this category was not applicable.
457

Table A-13 (continued)

Number of Accommodated Students in the Writing Samples

by Subgroup Classification, Grades 4, 8, and 12 & State Grade 8*

Main 25 min. State 25 min.

Grade 4 Grade 8 Grade 12 Grade 8

Modal Age

< Modal Age 2 7 5 5

= Modal Age 344 253 108 991

> Modal Age 400 418 213 1,656

Type of Accommodation

Large-Print Book 3 5 5 16

Extended Time 181 211 120 1,040

Read Aloud 42 24 6 313

Bilingual Dictionary 5 14 8 53

Small Groups 449 379 152 944

One on One 32 29 14 121

Scribe/Computer 27 10 9 112

Other 7 6 12 53

Accommodation Book

Yes 737 672 319 2,427

No 9 6 7 225

SD/LEP

SD Only 626 588 268 2,408

LEP Only 104 74 53 177

SD & LEP 16 16 5 67

* Accommodations were not offered in the 50-minute study.
458

Table A-14

Number of Accommodated Students in the Civics Main Samples

 by Subgroup Classification, Grades 4, 8, and 12

Grade 4 Grade 8 Grade 12

Total 207 221 106

Gender

Male 132 142 66

Female 75 79 40

Race\Ethnicity

White 93 121 52

Black 34 37 21

Hispanic 65 57 26

Asian American 5 3 2

American Indian 10 3 5

Unclassified 0 0 0

Region

Northeast 47 71 44

Southeast 63 58 29

Central 32 53 18

West 65 39 15

Parent’s Education

Less Than High School 13 12 11

High School 16 26 16

Greater Than High School 36 74 27

Graduated College 105 88 48

Unknown 37 21 4

Type of Location

Central City 84 77 32

Urban Fringe/Large Town 82 93 43

Rural/Small Town 41 51 31

School Type

Public 197 209 97

Nonpublic 10 12 9

Private 4 2 4

Catholic 6 10 5

BIA 0 0 0

DODEA 0 0 0

(continued)
459

Table A-14 (continued)

Number of Accommodated Students in the Civics Main Samples

 by Subgroup Classification, Grades 4, 8, and 12

Grade 4 Grade 8 Grade 12

Modal Age

< Modal Age 0 1 1

= Modal Age 116 87 30

> Modal Age 91 133 75

Type of Accommodation

Large-Print Book 1 1 1

Extended Time 51 70 40

Read Aloud 6 9 1

Bilingual Dictionary 1 1 2

Small Groups 125 128 54

One on One 15 8 6

Scribe/Computer 3 2 0

Other 5 2 2

Accommodation Book

Yes 202 218 105

No 5 3 1

SD/LEP

SD Only 175 197 87

LEP Only 28 16 14

SD & LEP 4 8 5
460

Table A-15

Weighted Percentages of Students in the Reading Reporting Samples

by Subgroup Classification, National Grades 4, 8, and 12 & State Grades 4 and 8

National State

Grade 4 Grade 8 Grade 12 Grade 4 Grade 8

Total 3,588,382 3,464,591 3,061,170 2,833,845 2,599,198

Gender

Male 50 51 48 50 50

Female 50 50 52 50 50

Race/Ethnicity

White 67 67 69 61 61

Black 16 14 14 16 15

Hispanic 13 14 12 17 17

Asian American 3 4 4 4 5

American Indian 2 1 1 2 2

Unclassified 0 0 0 0 0

Region

Northeast 22 22 22 17 17

Southeast 26 24 23 28 29

Central 24 25 26 20 15

West 28 29 29 35 39

Unclassified (Territories) — — — 0 0

Parent’s Education

Less Than High School 3 7 7 3 8

High School 13 22 19 13 21

Greater Than High School 17 18 25 17 19

Graduated College 55 44 46 55 43

Unknown 12 9 3 12 10

Type of Location

Central City 35 34 31 35 35

Urban Fringe/Large Town 36 40 39 38 39

Rural/Small Town 29 27 30 27 26

Unclassified (Territories) — — — 0 0

School Type

Public 89 89 89 92 93

Nonpublic 11 11 11 7 7

Private 4 4 4 3 2

Catholic 7 7 8 5 4

BIA 0 0 0 0 0

DODEA 0 0 0 0 0

Note: “—” indicates that this category was not applicable.

(continued)
461

Table A-15 (continued)

Weighted Percentages of Students in the Reading Reporting Samples

by Subgroup Classification, National Grades 4, 8, and 12 & State Grades 4 and 8

National State

Grade 4 Grade 8 Grade 12 Grade 4 Grade 8

Modal Age

< Modal Age 1 1 1 0 0

= Modal Age 61 57 63 66 60

> Modal Age 38 43 36 34 39

SD/LEP

SD Only 5 5 3 5 6

LEP Only 2 2 2 4 3

SD & LEP 0 0 0 0 0

Non SD/LEP 92 93 96 91 91
462

Table A-16

Weighted Percentages of Students in the Writing 25-Minute and 50-Minute Samples

by Subgroup Classification, Grades 4, 8, and 12 & State Grade 8

Main 25-Minute Main 50-Minute

State

25-Minute

Grade 4 Grade 8 Grade 12 Grade 8 Grade 12 Grade 8

Total 3,730,723 3,526,984 3,103,590 3,429,355 3,085,458 2,602,998

Gender

Male 51 51 48 51 48 50

Female 49 49 52 49 52 50

Race/Ethnicity

White 67 67 69 68 69 60

Black 15 14 14 14 14 15

Hispanic 13 14 12 13 12 18

Asian American 2 3 4 3 4 5

American Indian 2 1 1 1 1 2

Unclassified 0 0 0 0 0 0

Region

Northeast 23 21 22 21 22 17

Southeast 25 25 23 24 24 30

Central 24 25 26 25 25 14

West 28 29 29 29 29 39

Unclassified (Territories) — — — — — 0

Parent’s Education

Less Than High School 3 5 6 5 6 5

High School 12 15 13 16 14 15

Greater Than High School 16 27 27 28 27 28

Graduated College 57 50 52 48 51 48

Unknown 12 3 2 4 2 4

Type of Location

Central City 35 33 32 33 31 35

Urban Fringe/Large Town 36 40 39 39 40 40

Rural/Small Town 30 27 30 27 29 26

Unclassified (Territories) — — — — — 0

School Type

Public 89 89 89 89 89 93

Nonpublic 12 11 12 11 12 7

Private 4 5 3 5 3 3

Catholic 8 7 8 7 8 4

BIA 0 0 0 0 0 0

DODEA 0 0 0 0 0 0

Note: “—” indicates that this category was not applicable.

(continued)
463

Table A-16 (continued)

Weighted Percentages of Students in the Writing 25-Minute and 50-Minute Samples

by Subgroup Classification, Grades 4, 8, and 12 & State Grade 8

Main 25-Minute Main 50-Minute

State

25-Minute

Grade 4 Grade 8 Grade 12 Grade 8 Grade 12 Grade 8

Modal Age

< Modal Age 1 1 1 1 1 1

= Modal Age 61 56 63 55 63 60

> Modal Age 39 44 36 44 36 39

SD/LEP

SD Only 8 7 4 5 3 7

LEP Only 2 2 2 2 2 3

SD & LEP 0 0 0 0 0 0

Non SD/LEP 90 91 94 93 95 89
464

Table A-17

Weighted Percentages of Students in the Civics Main Samples

by Subgroup Classification, Grades 4, 8, and 12

Grade 4 Grade 8 Grade 12

Total 3,745,108 3,533,641 3,137,172

Gender

Male 52 51 48

Female 48 49 52

Race/Ethnicity

White 67 67 69

Black 15 15 14

Hispanic 14 14 12

Asian American 2 3 4

American Indian 2 1 1

Unclassified 0 0 0

Region

Northeast 23 22 23

Southeast 25 25 23

Central 24 25 25

West 27 29 30

Parent’s Education

Less Than High School 3 5 6

High School 12 16 14

Greater Than High School 17 27 27

Graduated College 58 49 52

Unknown 10 3 1

Type of Location

Central City 35 33 32

Urban Fringe/Large Town 36 40 39

Rural/Small Town 30 28 30

School Type

Public 88 89 88

Nonpublic 12 11 12

Private 4 5 3

Catholic 8 6 8

BIA 0 0 0

DODEA 0 0 0

(continued)
465

Table A-17 (continued)

Weighted Percentages of Students in the Civics Main Samples

by Subgroup Classification, Grades 4, 8, and 12

Grade 4 Grade 8 Grade 12

Modal Age

< Modal Age 0 1 1

= Modal Age 60 55 62

>Modal Age 39 44 37

SD/LEP

SD Only 0 0 0

LEP Only 0 0 0

SD & LEP 0 0 0

Non SD/LEP 0 0 0
466

Table A-18

Weighted Percentages of Excluded Students in the Reading Reporting Samples

by Subgroup Classification, National Grades 4, 8, and 12 & State Grades 4 and 8

 National State

 Grade 4 Grade 8 Grade12 Grade 4 Grade 8

Total 356,547 209,148 98,674 286,313 177,631

 Gender

 Male 62 65 66 61 64

 Female 38 35 34 39 36

 Race/Ethnicity

 White 51 52 65 40 43

 Black 12 25 16 15 19

 Hispanic 32 20 15 37 30

 Asian American 5 2 3 6 5

 American Indian 0 1 0 1 2

 Unclassified 0 0 0 0 1

 Region

 Northeast 15 20 28 15 20

 Southeast 20 23 29 23 27

 Central 29 29 18 14 12

 West 36 28 25 47 41

 Unclassified (Territories) — — — 0 0

 Type of Location

 Central City 42 39 29 45 38

 Urban Fringe/Large Town 30 37 41 35 38

 Rural/Small Town 28 25 30 20 24

 Unclassified (Territories) — — — 0 0

 School Type

 Public 99 100 98 99 99

 Nonpublic 1 0 2 1 1

 Private 0 0 1 0 0

 Catholic 1 0 1 1 1

 BIA 0 0 0 0 0

 DODEA 0 0 0 0 0

 Modal Age

 < Modal Age 0 0 1 1 1

 = Modal Age 46 28 26 55 39

 > Modal Age 54 72 73 45 60

 SD/LEP

 SD Only 62 86 86 61 70

 LEP Only 34 11 9 35 23

 SD & LEP 4 4 5 5 7

Note: “—” indicates that this category was not applicable.
467

Table A-19

Weighted Percentages of Excluded Students in the Writing Samples

by Subgroup Classification, Grades 4, 8, and 12 & State Grade 8

 Main 25 min. Main 50 min. State 25 min.

 Grade 4 Grade 8 Grade 12 Grade 8 Grade 12 Grade 8

Total 214,210 146,762 78,648 140,951 86,351 125,288

Gender

Male 62 63 61 57 66 63

 Female 38 37 39 43 34 37

 Race/Ethnicity

 White 39 51 54 44 57 38

 Black 22 24 25 25 22 19

 Hispanic 36 22 18 27 17 36

 Asian American 2 3 3 4 3 5

 American Indian 0 0 0 1 1 1

Unclassified 0 0 0 0 0 1

 Region

 Northeast 15 17 15 20 20 17

 Southeast 23 24 31 25 22 28

 Central 18 28 19 23 30 9

 West 45 31 35 32 28 46

 Unclassified (Territories) — — — — — 0

 Type of Location

Central City 48 43 35 46 32 41

 Urban Fringe/Large Town 32 30 38 31 35 36

 Rural/Small Town 20 26 27 23 34 24

 Unclassified (Territories) — — — — — 0

 School Type

 Public 99 99 100 100 100 99

 Nonpublic 1 1 0 0 0 1

 Private 0 0 0 0 0 1

 Catholic 0 1 0 0 0 0

 BIA 0 0 0 0 0 0

 DODEA 0 0 0 0 0 0

 Modal Age

 < Modal Age 1 1 2 1 1 1

 = Modal Age 44 29 25 33 23 36

 > Modal Age 55 71 74 66 76 63

 SD/LEP

 SD Only 64 78 87 77 84 65

 LEP Only 31 17 10 19 13 29

 SD & LEP 6 6 3 4 3 6

Note: “—” indicates that this category was not applicable.
468

Table A-20

Weighted Percentages of Excluded Students in the Civics Main Samples

by Subgroup Classification, Grades 4, 8, and 12

Grade 4 Grade 8 Grade 12

Total 199,822 140,098 76,333

Gender

Male 65 62 61

Female 35 38 39

Race/Ethnicity

White 38 52 55

Black 23 22 24

Hispanic 36 22 14

Asian American 4 4 5

American Indian 0 0 1

Unclassified 0 0 1

Region

Northeast 12 17 15

Southeast 23 25 23

Central 15 25 26

West 51 33 37

Type of Location

Central City 45 40 36

Urban Fringe/Large Town 38 36 44

Rural/Small Town 17 24 21

School Type

Public 100 99 100

Nonpublic 0 1 0

Private 0 0 0

Catholic 0 1 0

BIA 0 0 0

DODEA 0 0 0

Modal Age

< Modal Age 1 1 0

= Modal Age 48 31 23

>Modal Age 51 69 77

SD/LEP

SD Only 0 0 0

LEP Only 0 0 0

SD & LEP 0 0 0
469

 Table A-21

Weighted Percentages of Accommodated Students in the Writing Samples

by Subgroup Classification, Grades 4, 8, and 12 & State Grade 8*

Main 25 min. State 25 min.

Grade 4 Grade 8 Grade 12 Grade 8

Total 150,096 101,366 43,165 76,570

Gender

Male 65 66 61 68

Female 35 34 39 32

Race/Ethnicity

White 63 60 69 56

Black 19 18 11 14

Hispanic 15 17 16 25

Asian American 1 2 3 3

American Indian 2 2 1 3

Unclassified 0 0 0 0

Region

Northeast 29 26 42 31

Southeast 31 31 22 28

Central 21 22 23 13

West 20 21 14 28

Unclassified (Territories) — — — 0

Parent’s Education

Less Than High School 5 7 9 8

High School 13 17 14 16

Greater Than High School 14 30 23 29

Graduated College 50 39 48 40

Unknown 19 7 6 8

Type of Location

Central City 30 27 23 27

Urban Fringe/Large Town 42 42 42 40

Rural/Small Town 28 31 36 32

Unclassified (Territories) — — — 0

School Type

Public 97 97 97 98

Nonpublic 3 3 3 2

Private 1 1 2 1

Catholic 2 2 2 1

BIA 0 0 0 0

DODEA 0 0 0 0

*Accommodations were not offered in the 50-minute study.

Note: “—” indicates that this category was not applicable.

(continued)
470

Table A-21 (continued)

Weighted Percentages of Accommodated Students in the Writing Samples

by Subgroup Classification, Grades 4, 8, and 12 & State Grade 8*

Main 25 min. State 25 min.

Grade 4 Grade 8 Grade 12 Grade 8

Modal Age

< Modal Age 0 1 1 0

= Modal Age 40 31 29 37

> Modal Age 60 68 70 63

Type of Accommodation

Large-Print Book 0 1 1 1

Extended Time 20 30 33 42

Read Aloud 7 3 3 12

Bilingual Dictionary 0 2 2 2

Small Groups 61 58 50 34

One on One 6 4 5 4

Scribe/Computer 5 2 3 4

Other 1 1 4 2

Accommodation Book

Yes 99 99 97 90

No 1 1 3 10

SD/LEP

SD Only 90 91 88 90

LEP Only 8 8 11 6

SD & LEP 2 1 1 4

* Accommodations were not offered in the 50-minute study.
471

Table A-22

Weighted Percentages of Accommodated Students in the Civics Main Samples

by Subgroup Classification, Grades 4, 8, and 12

Grade 4 Grade 8 Grade 12

Total 136,538 116,685 41,679

Gender

Male 67 67 65

Female 33 33 35

Race/Ethnicity

White 59 67 62

Black 18 15 17

Hispanic 18 17 15

Asian American 2 1 1

American Indian 4 1 4

Unclassified 0 0 0

Region

Northeast 28 34 46

Southeast 28 23 20

Central 19 33 25

West 24 11 10

Parent’s Education

Less Than High School 5 6 6

High School 9 14 13

Greater Than High School 18 32 25

Graduated College 51 38 51

Unknown 18 10 5

Type of Location

Central City 35 28 23

Urban Fringe/Large Town 38 41 49

Rural/Small Town 27 32 29

School Type

Public 97 97 95

Nonpublic 3 3 5

Private 1 1 2

Catholic 3 2 3

BIA 0 0 0

DODEA 0 0 0

(continued)
472

Table A-22 (continued)

Weighted Percentages of Accommodated Students in the Civics Main Samples

by Subgroup Classification, Grades 4, 8, and 12

Grade 4 Grade 8 Grade 12

Modal Age

< Modal Age 0 0 1

= Modal Age 46 31 29

> Modal Age 54 69 71

Type of Accommodation

Large-Print Book 1 0 1

Extended Time 22 31 35

Read Aloud 3 4 1

Bilingual Dictionary 0 0 1

Small Groups 63 58 55

One on One 8 4 5

Scribe/Computer 2 2 0

Other 2 1 2

Accommodation Book

Yes 98 99 99

No 2 1 1

SD/LEP

SD Only 91 94 88

LEP Only 8 4 9

SD & LEP 1 2 3
473

Appendix B

SUMMARY INFORMATION FOR THE NAEP 1998 STATE SAMPLES

AND FOR WEIGHTING THE NAEP 1998 STATE SAMPLES

Keith F. Rust and Leslie Wallace

Westat

This appendix supplements the text of Chapters 4 and 11 (State Sampling and Weighting

Procedures and Variance Estimation). It contains summary information for the 1998 NAEP state samples

and includes the following tables:

Table B-1 Weighted Mean Values Derived from Sampled Public Schools - Grade 4, Reading

Table B-2 Weighted Mean Values Derived from Sampled Public Schools - Grade 8, Reading

Table B-3 Weighted Mean Values Derived from Sampled Public Schools - Grade 8, Writing

Table B-4 Weighted Mean Values Derived from Sampled Nonpublic Schools - Grade 4, Reading

Table B-5 Weighted Mean Values Derived from Sampled Nonpublic Schools - Grade 8, Reading

Table B-6 Weighted Mean Values Derived from Sampled Nonpublic Schools - Grade 8, Writing

Table B-7 Weighted Student Percentages Derived From Sampled Public Schools - Grade 4,

Reading

Table B-8 Weighted Student Percentages Derived From Sampled Public Schools - Grade 8,

Reading

Table B-9 Weighted Student Percentages Derived from Sampled Public Schools - Grade 8,

Writing

Table B-10 Weighted Student Percentages Derived From All Schools Sampled - Grade 4, Reading

Table B-11 Weighted Student Percentages Derived From All Schools Sampled - Grade 8, Reading

Table B-12 Weighted Student Percentages Derived From All Schools Sampled - Grade 8, Writing

Table B-13 Final Collapsed Levels Used for Raking Dimensions for All Jurisdictions - Grade 4

Reading

Table B-14 Final Collapsed Levels Used for Raking Dimensions for All Jurisdictions - Grade 8

Reading

Table B-15 Distribution of Selected Public Schools by Sampling Strata, Fourth Grade

Table B-16 Distribution of Selected Public Schools by Sampling Strata, Eighth Grade

Table B-17 Distribution of Selected Nonpublic Schools by Sampling Strata, Fourth Grade
474

Table B-18 Distribution of Selected Nonpublic Schools by Sampling Strata, Eighth Grade

Table B-19 Weighted School Participation Rates and Sample Counts - Grade 4, Reading for Public

Schools

Table B-20 Weighted School Participation Rates and Sample Counts - Grade 4, Reading for

Nonpublic Schools

Table B-21 Weighted School Participation Rates and Sample Counts - Grade 8, Reading and

Writing for Public Schools

Table B-22 Weighted School Participation Rates and Sample Counts - Grade 8, Reading and

Writing for Nonpublic Schools

Table B-23 Weighted School Participation Rates, Exclusion Rates, and Sample Counts for the

Reporting Samples - Grade 4, Reading for Public Schools

Table B-24 Weighted School Participation Rates, Exclusion Rates, and Sample Counts for the

Reporting Samples - Grade 4, Reading for Nonpublic Schools

Table B-25 Weighted School Participation Rates, Exclusion Rates, and Sample Counts for the

Reporting Samples - Grade 8, Reading for Public Schools

Table B-26 Weighted School Participation Rates, Exclusion Rates, and Sample Counts for the

Reporting Samples - Grade 8, Reading for Nonpublic Schools

Table B-27 Weighted School Participation Rates, Exclusion Rates, and Sample Counts for the

Reporting Samples - Grade 8, Writing for Public Schools

Table B-28 Weighted School Participation Rates, Exclusion Rates, and Sample Counts for the

Reporting Samples - Grade 8, Writing for Nonpublic Schools

Table B-29 Results of the Logistic Regression Analysis of School Nonresponse - Grade 4 Reading

Table B-30 Results of the Logistic Regression Analysis of School Nonresponse - Grade 8 Reading

Table B-31 Results of the Logistic Regression Analysis of School Nonresponse - Grade 8 Writing
Table B-1

Weighted Mean Values Derived from Sampled Public Schools - Grade 4, Reading

Weighted Mean Value

Derived from Full Sample

Weighted Mean Value Der

Responding Sample with Sub

School Nonresponse Adju

Jurisdiction

Weighted

Participation

Rate After

Substitution

(%)

Percent

Black

Percent

Hispanic

Median

Income

Type of

Location

Percent

Black

Percent

Hispanic

Media

Incom

Alabama 90.81 35.11 0.37 $23,727 4.43 34.19 0.33 $24,26Arizona 97.86 3.89 29.31 $30,835 2.50 3.97 29.59 $30,89Arkansas 97.39 22.46 1.04 $22,164 4.88 23.31 1.03 $22,18California 79.92 7.79 38.31 $35,521 2.64 7.05 37.10 $36,05Colorado 95.43 6.19 14.44 $33,220 3.45 6.54 14.01 $33,07Connecticut 98.22 12.54 11.61 $47,008 3.87 12.75 11.73 $46,81Delaware 100.00 28.96 3.74 $28,464 3.54 28.96 3.74 $28,46District of Columbia 100.00 86.04 6.88 $28,020 1.00 86.04 6.88 $28,02DoDEA/ DDESS 100.00 — — $23,976 3.61 — — $23,97DoDEA/ DoDDS 100.00 — — — — — — — Florida 99.04 26.76 13.68 $28,805 3.07 26.61 13.64 $28,77Georgia 99.05 38.77 1.65 $30,325 4.22 39.58 1.63 $30,16Hawaii 100.00 3.31 4.69 $35,848 3.48 3.31 4.69 $35,84Illinois 84.13 20.92 11.54 $34,772 3.18 26.33 12.89 $33,98Iowa 83.94 3.71 1.78 $27,640 5.01 4.29 1.96 $27,78Kansas 70.42 8.89 5.80 $30,715 4.48 10.19 6.14 $29,96Kentucky 92.39 9.95 0.28 $24,466 4.75 9.41 0.28 $24,36Louisiana 100.00 43.59 0.86 $23,560 3.97 44.94 0.86 $23,56Maine 95.99 0.78 0.42 $28,760 5.89 0.82 0.42 $28,82Maryland 88.42 33.69 3.19 $40,410 3.34 33.67 3.36 $40,58Massachusetts 88.15 7.94 8.72 $41,555 3.75 7.75 8.70 $40,59Michigan 89.62 17.00 3.15 $32,952 3.78 18.78 3.22 $32,17Minnesota 85.82 5.81 1.58 $33,160 4.17 5.91 1.69 $32,92

(continued)
476

Table B-1 (continued)

Weighted Mean Values Derived from Sampled Public Schools - Grade 4, Reading

Weighted Mean Value

Derived from Full Sample

Weighted Mean Value Der

Responding Sample with Sub

School Nonresponse Adju

Jurisdiction

Weighted

Participation

Rate After

Substitution

(%)

Percent

Black

Percent

Hispanic

Median

Income

Type of

Location

Percent

Black

Percent

Hispanic

Media

Incom

Mississippi 94.12 47.48 0.25 $21,459 5.39 47.68 0.26 $21,44Missouri 99.03 14.93 1.03 $28,886 4.10 14.96 1.05 $28,87Montana 78.48 0.59 1.68 $24,569 5.34 0.62 1.52 $24,67Nevada 100.00 10.32 16.02 $32,280 2.76 10.32 16.02 $32,28New Hampshire 70.48 0.86 0.85 $40,014 4.98 0.83 0.84 $39,92New Mexico 99.06 2.66 47.20 $24,434 3.99 2.67 46.93 $24,48New York 83.92 17.90 16.39 $34,708 2.96 19.77 16.23 $34,07North Carolina 99.05 29.41 1.91 $28,065 4.36 29.35 1.96 $28,17Oklahoma 100.00 9.94 4.83 $25,948 4.06 9.94 4.83 $25,94Oregon 94.23 2.17 7.27 $30,173 3.78 2.37 7.08 $30,60Rhode Island 100.00 7.20 10.21 $31,644 3.75 7.20 10.21 $31,64South Carolina 97.02 41.37 0.62 $27,099 4.59 41.65 0.61 $26,91Tennessee 97.15 24.30 0.58 $25,857 3.78 24.40 0.61 $26,07Texas 97.08 16.14 34.49 $28,298 2.90 15.92 34.93 $28,32Utah 100.00 0.62 5.68 $32,177 3.94 0.62 5.68 $32,17Virgin Islands 100.00 82.83 14.89 — 7.00 82.83 14.89 — Virginia 100.00 25.72 3.25 $38,201 3.83 25.72 3.25 $38,20Washington 89.25 4.79 7.77 $34,636 3.53 4.76 7.85 $34,52West Virginia 100.00 3.81 0.21 $22,356 5.55 3.81 0.21 $22,35Wisconsin 82.04 10.03 3.65 $32,285 3.96 10.37 3.93 $32,05Wyoming 100.00 1.04 6.40 $30,865 5.15 1.04 6.40 $30,86
Table B-2

Weighted Mean Values Derived from Sampled Public Schools - Grade 8, Reading

Weighted Mean Value

Derived From Full Sample

Weighted Mean Value De

Responding Sample with 

 and School Nonresponse 

Jurisdiction

Weighted

Participation

Rate After

Substitution

(%)

Percent

Black

Percent

Hispanic

Median

Income

Type of

Location

Percent

Black

Percent

Hispanic

Med

Inco

Alabama 90.94 36.04 0.46 $23,757 4.51 36.11 0.46 $24,Arizona 97.48 4.17 27.95 $30,706 2.61 4.17 27.85 $30,Arkansas 96.79 23.18 0.99 $22,166 5.00 23.98 0.98 $22,California 83.74 8.64 36.69 $36,334 2.58 9.12 38.89 $36,Colorado 96.57 4.80 18.18 $32,528 3.38 5.17 18.76 $32,Connecticut 99.07 12.71 10.13 $45,855 4.06 12.71 10.13 $45,Delaware 100.00 29.33 3.49 $35,472 3.89 29.33 3.49 $35,District of Columbia 100.00 87.33 6.90 $30,015 1.00 87.33 6.90 $30,DoDEA/DDESS 100.00 — — $23,801 3.13 — — $23,DoDEA/DoDDS 100.00 — — — — — — — Florida 100.00 27.56 12.02 $28,843 2.88 27.56 12.02 $28,Georgia 100.00 37.20 1.72 $30,407 4.20 37.58 1.75 $30,Hawaii 100.00 2.32 4.96 $35,496 3.69 2.32 4.96 $35,Illinois 81.12 20.86 11.93 $34,509 3.15 24.16 13.22 $33,Kansas 70.60 7.97 5.15 $31,206 4.48 8.87 5.34 $30,Kentucky 87.32 10.26 0.28 $23,952 4.70 10.11 0.24 $23,Louisiana 100.00 41.69 1.03 $23,383 4.00 40.74 1.05 $23,Maine 97.33 0.68 0.37 $28,822 5.83 0.67 0.39 $28,Maryland 85.45 34.50 2.89 $41,452 3.33 34.61 3.11 $42,Massachusetts 89.20 7.86 9.61 $41,967 3.72 7.84 9.89 $42,Minnesota 73.73 4.69 1.79 $33,552 4.15 5.56 1.94 $34,

(continued)
478

Table B-2 (continued)

Weighted Mean Values Derived from Sampled Public Schools - Grade 8, Reading

Weighted Mean Value

Derived From Full Sample

Weighted Mean Value De

Responding Sample with 

 and School Nonresponse 

Jurisdiction

Weighted

Participation

Rate After

Substitution

(%)

Percent

Black

Percent

Hispanic

Median

Income

Type of

Location

Percent

Black

Percent

Hispanic

Med

Inco

Mississippi 92.16 49.41 0.11 $21,266 5.38 49.81 0.12 $21,Missouri 96.51 16.07 0.66 $28,409 4.15 16.41 0.64 $28,Montana 77.81 0.32 1.51 $24,647 5.46 0.38 1.53 $24,Nevada 99.08 9.26 15.33 $32,757 2.72 9.31 15.36 $32,New Mexico 96.37 2.11 45.88 $24,403 4.18 2.13 45.85 $24,New York 77.27 19.20 16.29 $35,042 3.06 20.29 19.15 $34,North Carolina 99.94 30.84 1.39 $28,520 4.32 30.83 1.38 $28,Oklahoma 100.00 8.95 3.56 $25,690 4.31 8.95 3.56 $25,Oregon 87.53 2.41 5.56 $30,064 3.81 2.62 5.58 $30,Rhode Island 100.00 6.14 8.07 $32,573 3.76 6.14 8.07 $32,South Carolina 94.51 41.24 0.51 $27,018 4.51 41.45 0.51 $27,Tennessee 89.03 22.51 0.58 $26,085 3.76 21.91 0.51 $26,Texas 95.78 13.09 33.47 $28,382 3.09 12.68 32.84 $28,Utah 100.00 0.49 4.63 $32,171 3.94 0.49 4.63 $32,Virgin Islands 100.00 84.29 15.45 — 7.00 84.29 15.45 — Virginia 100.00 26.61 2.35 $38,728 3.82 26.61 2.35 $38,Washington 86.13 4.58 6.26 $34,473 3.61 4.52 6.40 $34,West Virginia 100.00 3.28 0.13 $22,394 5.48 3.28 0.13 $22,Wisconsin 73.18 7.99 3.15 $32,278 4.13 9.56 3.67 $31,Wyoming 94.91 0.84 6.33 $31,294 5.15 0.87 6.32 $31,
Table B-3

Weighted Mean Values Derived from Sampled Public Schools - Grade 8, Writing

Weighted Mean Value Derived

from Full Sample

Weighted Mean Value De

Responding Sample with Su

School Nonresponse Ad

Jurisdiction

Weighted

Participation

Rate After

Substitution

(%)

Percent

Black

Percent

Hispanic

Median

Income

Type of

Location

Percent

Black

Percent

Hispanic

Med

Inco

Alabama 90.31 36.11 0.46 $23,703 4.50 35.96 0.46 $24,Arizona 97.84 4.23 27.28 $30,917 2.55 4.22 27.20 $30,Arkansas 96.79 23.73 0.98 $22,211 5.00 24.54 0.97 $22,California 83.15 8.35 36.80 $36,356 2.59 8.77 39.24 $36,Colorado 96.57 4.92 18.20 $32,609 3.35 5.16 18.49 $32,Connecticut 99.07 12.71 10.13 $45,855 4.06 12.71 10.13 $45,Delaware 100.00 29.31 3.49 $35,484 3.89 29.31 3.49 $35,District of Columbia 100.00 87.33 6.90 $29,977 1.00 87.33 6.90 $29,DoDEA/DDESS 100.00 — — $24,229 3.26 — — $24,DoDEA/DoDDS 100.00 — — — — — — — Florida 100.00 27.38 11.95 $28,800 2.91 27.38 11.95 $28,Georgia 100.00 37.20 1.72 $30,407 4.20 37.58 1.75 $30,Hawaii 100.00 2.34 4.84 $35,546 3.66 2.34 4.84 $35,Illinois 80.28 20.91 11.39 $34,569 3.21 24.30 12.59 $33,Kentucky 87.14 10.41 0.29 $24,020 4.67 10.25 0.25 $23,Louisiana 100.00 42.61 1.02 $23,443 3.97 42.08 1.03 $23,Maine 97.34 0.63 0.39 $28,769 5.85 0.63 0.41 $28,Maryland 86.42 34.50 2.89 $41,452 3.33 34.63 3.11 $41,Massachusetts 89.28 7.81 9.54 $41,838 3.75 7.79 9.82 $41,Minnesota 73.51 4.72 1.80 $33,491 4.16 5.60 1.95 $34,Mississippi 92.16 49.25 0.11 $21,275 5.38 49.66 0.12 $21,

(continued)
480

Table B-3 (continued)

Weighted Mean Values Derived from Sampled Public Schools - Grade 8, Writing

Weighted Mean Value Derived

from Full Sample

Weighted Mean Value De

Responding Sample with Su

School Nonresponse Ad

Jurisdiction

Weighted

Participation

Rate After

Substitution

(%)

Percent

Black

Percent

Hispanic

Median

Income

Type of

Location

Percent

Black

Percent

Hispanic

Med

Inco

Missouri 97.08 16.37 0.67 $28,644 4.10 16.72 0.65 $28,Montana 77.60 0.34 1.45 $24,700 5.45 0.36 1.53 $24,Nevada 99.08 9.26 15.57 $32,788 2.72 9.32 15.61 $32,New Mexico 96.40 2.09 45.88 $24,324 4.21 2.12 45.99 $24,New York 77.27 19.20 16.29 $35,042 3.06 20.29 19.15 $34,North Carolina 100.00 30.84 1.38 $28,472 4.31 30.84 1.38 $28,Oklahoma 100.00 8.99 3.56 $25,777 4.31 8.99 3.56 $25,Oregon 87.53 2.42 5.84 $30,089 3.83 2.64 5.91 $30,Rhode Island 100.00 6.14 8.10 $32,571 3.77 6.14 8.10 $32,South Carolina 94.48 41.42 0.51 $27,090 4.50 41.63 0.51 $27,Tennessee 89.03 22.48 0.58 $26,073 3.76 21.88 0.52 $26,Texas 96.41 12.83 33.97 $28,487 3.07 12.55 33.34 $28,Utah 100.00 0.49 4.63 $32,148 3.94 0.49 4.63 $32,Virgin Islands 100.00 84.29 15.45 — 7.00 84.29 15.45 — Virginia 100.00 26.61 2.35 $38,728 3.82 26.61 2.35 $38,Washington 86.59 4.63 6.33 $34,606 3.58 4.57 6.49 $34,West Virginia 100.00 3.36 0.12 $22,408 5.49 3.36 0.12 $22,Wisconsin 72.91 8.02 3.16 $32,321 4.11 9.63 3.67 $31,Wyoming 100.00 0.87 6.23 $31,336 5.13 0.87 6.23 $31,
481

Table B-4

Weighted Mean Values Derived from Sampled Nonpublic Schools – Grade 4, Reading

Weighted Mean Value

Derived from Full Sample

Weighted Mean Values Derived from

Responding Sample, with Substitutes

and School Nonresponse Adjustment

Jurisdiction

Weighted

Participation

Rate after

Substitution

(%)

Percent

Catholic

Percent

Urban

Percent

Catholic

Percent

Urban

Colorado 85.98 38 94 49 100

Connecticut 81.59 76 94 77 92

Florida 78.21 35 95 45 94

Georgia 80.08 12 80 15 84

Hawaii 85.46 39 84 42 86

Iowa 91.89 75 48 82 48

Louisiana 80.86 57 86 57 79

Massachusetts 84.12 72 100 79 100

Michigan 73.38 49 89 49 85

Minnesota 81.21 65 75 66 69

Missouri 79.81 70 85 80 91

Montana 87.76 56 5 64 0

Nebraska 98.90 75 64 80 62

Nevada 88.57 29 98 37 98

New Mexico 91.48 23 57 32 68

North Carolina 90.44 10 81 12 76

Rhode Island 95.94 80 94 89 93

South Carolina 95.80 17 70 20 69

Utah 75.49 29 96 39 95

Virgin Islands 96.29 14 0 14 0

Washington 76.74 36 97 33 96

West Virginia 85.63 74 58 95 75

Wyoming 96.10 51 49 68 43
482

Table B-5

Weighted Mean Values Derived from Sampled Nonpublic Schools – Grade 8, Reading

Weighted Mean Value Derived

from Full Sample

Weighted Mean Values Derived from

Responding Sample, with Substitutes and

School Nonresponse Adjustment

Jurisdiction

Weighted

Participation

Rate After

Substitution

(%)

Percent

Catholic

Percent

Urban

Percent

Catholic

Percent

Urban

Arkansas 85.71 23 80 26 93

California 79.46 53 97 66 96

Colorado 100.00 44 100 37 100

Connecticut 84.02 75 93 81 100

Florida 73.72 34 100 46 100

Georgia 100.00 21 72 21 72

Louisiana 77.87 70 92 70 93

Maryland 82.35 57 98 69 100

Missouri 89.62 85 83 91 81

Montana 81.78 64 24 67 30

Nebraska 89.01 70 62 79 62

Nevada 88.29 39 100 49 100

New Mexico 83.14 35 60 37 50

New York 72.51 52 93 71 95

North Carolina 83.60 13 73 16 81

Rhode Island 85.33 80 97 86 96

Virgin Islands 100.00 13 0 13 0

Washington 100.00 61 100 49 100

West Virginia 87.38 58 71 67 81

Wyoming 95.33 49 56 51 58
483

Table B-6

Weighted Mean Values Derived from Sampled Nonpublic Schools – Grade 8, Writing

Weighted Mean Value Derived

from Full Sample

Weighted Mean Values Derived from

Responding Sample, with Substitutes and

School Nonresponse Adjustment

Jurisdiction

Weighted

Participation

Rate After

Substitution

(%)

Percent

Catholic

Percent

Urban

Percent

Catholic

Percent

Urban

Arkansas 82.74 0.34 0.70 0.47 0.87

California 84.12 0.44 0.96 0.53 0.95

Florida 84.97 0.38 0.94 0.50 0.92

Georgia 87.91 0.14 0.84 0.16 0.81

Louisiana 90.30 0.74 0.89 0.74 0.88

Maryland 78.57 0.51 0.91 0.55 0.95

Massachusetts 70.49 0.69 1.00 0.84 1.00

Montana 100.00 0.47 0.27 0.52 0.20

Nebraska 91.58 0.71 0.60 0.71 0.62

Nevada 95.06 0.67 0.77 0.70 0.81

New Mexico 80.00 0.32 0.63 0.36 0.50

New York 80.16 0.57 0.99 0.57 1.00

North Carolina 78.35 0.00 0.88 0.00 0.92

Rhode Island 82.09 0.81 0.92 0.86 0.95

Virgin Islands 82.01 0.24 0.00 0.30 0.00

Washington 92.25 0.33 0.89 0.30 0.88

West Virginia 84.99 0.58 0.49 0.68 0.58

Wyoming 76.55 0.39 0.37 0.52 0.17
484

Table B-7

Weighted Student Percentages Derived from Sampled Public Schools – Grade 4, Reading

Weighted Estimates Derived from Full Sample

Weighted Estimates Deriv

Student Nonre

Jurisdiction

Weighted

Student

Participation

(%)

Percent

Male

Percent

White

Percent

Black

Percent

Hispanic

Percent

SD

Percent

LEP

Mean

Age

(Months)

Percent

Male

Percent

White

Percent

Black

Per

Hisp

Alabama 96.00 51.11 60.44 31.02 5.34 5.18 0.13 121.40 51.19 61.19 30.17 5Arizona 93.84 49.22 54.35 4.66 33.81 5.57 8.49 120.29 49.43 54.09 4.63 33Arkansas 95.06 50.53 69.94 21.53 5.91 5.86 0.52 120.81 50.43 70.48 20.47 6California 93.19 48.12 41.90 9.01 33.58 3.01 15.48 117.42 47.54 42.48 8.56 33Colorado 94.08 49.90 69.24 5.71 20.17 6.82 2.40 120.17 49.50 68.77 5.82 20Connecticut 94.04 47.93 72.74 10.53 12.37 6.24 1.55 117.78 47.95 72.64 10.57 12Delaware 93.88 50.79 60.14 26.26 9.42 10.22 2.10 117.82 51.01 59.97 26.03 9District of Columbia 93.06 48.64 6.77 75.98 13.98 3.12 4.20 118.56 48.05 6.98 74.81 14DoDEA/DDESS 95.53 49.10 46.83 28.39 18.58 3.65 0.39 119.27 49.35 46.73 28.60 18DoDEA/DoDDS 94.03 50.26 47.31 17.48 15.31 3.17 0.94 118.78 50.31 47.22 17.88 15Florida 93.87 49.85 52.68 24.28 19.68 8.71 3.17 120.47 49.91 52.08 24.38 20Georgia 95.51 49.57 50.85 37.85 7.50 4.73 0.40 120.95 49.73 51.16 37.37 7Hawaii 94.50 50.65 19.72 5.12 19.76 6.75 4.32 116.50 50.43 19.95 5.26 20Illinois 94.84 50.45 56.87 22.24 14.76 5.77 2.50 120.35 50.56 61.97 18.61 13Iowa 96.10 50.71 86.36 3.97 6.19 8.67 0.75 120.83 50.55 86.49 3.80 6Kansas 93.36 53.13 74.33 9.80 10.42 6.47 1.26 121.41 52.86 73.93 9.83 10Kentucky 95.97 50.40 86.27 8.37 3.22 4.69 0.12 120.47 50.04 85.60 8.83 3Louisiana 95.19 49.82 47.58 41.90 7.29 5.48 0.54 121.01 49.34 50.10 39.58 7Maine 92.99 51.46 91.15 1.58 4.02 7.24 0.70 120.11 51.42 90.57 1.57 4Maryland 95.05 50.32 52.70 32.49 8.41 4.88 1.26 116.73 49.68 52.61 32.16 8Massachusetts 94.90 48.44 78.27 6.32 9.95 11.14 2.46 119.27 48.39 77.77 6.49 10

Michigan 93.38 50.03 70.63 16.96 8.65 2.82 0.89 119.38 49.37 72.49 13.99 8Minnesota 93.91 50.87 82.27 5.59 6.28 9.24 3.00 120.74 50.59 81.93 5.73 6Mississippi 94.96 49.44 50.16 42.71 5.59 2.89 0.09 122.63 49.28 50.00 42.55 5

(continued)
Table B-7 (continued)

Weighted Student Percentages Derived from Sampled Public Schools – Grade 4, Reading

Weighted Estimates Derived from Full Sample

Weighted Estimates Deriv

Student Nonre

Jurisdiction

Weighted

Student

Participation

(%)

Percent

Male

Percent

White

Percent

Black

Percent

Hispanic

Percent

SD

Percent

LEP

Mean

Age

(Months)

Percent

Male

Percent

White

Percent

Black

Per

Hisp

Missouri 95.31 51.12 75.35 15.15 6.60 7.42 0.47 122.12 51.46 75.51 14.74 6Montana 95.43 50.38 83.04 1.19 6.85 6.70 0.27 121.67 50.51 82.99 1.15 6Nevada 94.40 50.50 60.20 9.12 22.93 4.53 4.57 119.13 50.31 59.53 8.77 23New Hampshire 92.91 51.16 89.35 1.60 5.52 10.00 0.47 120.04 50.81 89.02 1.61 5

New Mexico 94.45 49.47 39.39 3.38 45.86 6.81 13.07 120.40 49.48 39.09 3.44 46New York 95.09 48.91 57.15 17.28 19.57 4.76 2.47 117.54 48.71 59.78 15.91 18North Carolina 94.00 50.18 62.61 26.74 6.89 6.12 1.28 119.64 49.71 62.01 26.86 7Oklahoma 94.92 49.96 69.79 7.58 9.65 4.65 2.00 123.03 49.85 70.22 7.62 9Oregon 94.54 49.60 74.75 3.13 13.04 9.41 4.85 119.85 49.15 74.58 2.97 13Rhode Island 94.23 52.30 74.93 6.68 13.92 9.80 4.27 117.93 52.88 74.59 6.77 14South Carolina 95.45 48.80 54.38 37.06 5.84 7.04 0.51 119.06 48.59 53.58 37.47 6Tennessee 94.43 50.00 69.68 23.79 4.30 8.73 0.47 120.80 50.11 69.89 23.37 4Texas 95.32 51.42 48.64 14.57 32.94 8.97 6.71 120.76 50.14 47.13 15.10 33Utah 95.26 52.00 78.63 1.79 13.79 6.39 2.83 119.94 52.07 78.26 1.88 14Virgin Islands 95.62 47.55 3.16 75.45 19.33 0.64 1.55 118.73 47.37 3.28 75.10 19

Virginia 94.79 49.98 62.85 23.67 7.92 6.62 1.82 119.61 49.96 62.51 23.57 8Washington 94.42 50.92 73.82 4.58 10.28 7.82 3.07 120.40 51.03 73.45 4.44 10West Virginia 94.03 48.50 88.10 3.63 5.30 3.36 0.11 120.73 48.25 87.63 3.63 5Wisconsin 94.95 50.98 79.07 9.17 8.05 6.33 1.29 120.71 50.71 79.00 8.85 8Wyoming 95.19 51.34 80.91 1.20 11.70 9.97 0.51 121.06 51.41 80.51 1.23 11
486

Table B-8

Weighted Student Percentages Derived from Sampled Public Schools – Grade 8, Reading

Weighted Estimates Derived from Full Sample

Weighted Estimates Deriv

Student Nonres

Jurisdiction

Weighted

Student

Participation

(%)

Percent

Male

Percent

White

Percent

Black

Percent

Hispanic

Percent

SD

Percent

LEP

Mean

Age

(Months)

Percent

Male

Percent

White

Percent

Black

Per

Hisp

Alabama 92.73 50.07 59.48 33.81 4.39 5.93 0.15 170.16 49.94 62.04 31.49 4Arizona 90.61 50.70 57.72 4.23 29.99 5.42 7.06 169.18 50.59 57.16 4.19 30Arkansas 92.23 51.50 72.67 21.49 3.29 5.43 0.44 169.73 51.45 73.06 20.68 3California 90.86 49.53 35.90 9.18 41.56 5.58 13.95 166.64 50.22 37.43 8.33 41Colorado 91.07 52.19 68.12 4.63 22.40 7.01 3.03 169.04 52.19 68.28 4.63 22Connecticut 91.38 52.27 74.04 11.64 9.87 8.28 0.59 166.93 52.01 74.05 11.42 9Delaware 90.73 50.48 62.72 27.20 6.75 9.04 1.31 167.38 50.41 62.77 26.48 7District of Columbia 85.62 46.28 4.31 84.37 8.76 5.95 1.53 168.33 47.36 4.44 82.74 9

DoDEA/DDESS 95.02 53.32 41.09 27.04 26.13 5.59 0.76 167.75 52.52 41.24 26.59 26DoDEA/DoDDS 93.59 51.45 46.94 19.01 14.20 4.93 0.80 167.07 51.24 46.68 19.10 14Florida 89.41 49.42 52.68 25.03 18.34 9.86 2.78 169.79 49.37 51.88 24.36 19Georgia 90.33 50.95 57.26 34.31 4.64 6.28 1.09 170.12 51.13 57.31 34.01 4Hawaii 90.80 50.30 16.20 2.53 15.13 7.33 2.81 165.58 50.47 15.89 2.70 16Illinois 92.99 46.83 58.19 21.62 15.95 6.23 2.01 168.61 47.51 64.51 17.40 14

Kansas 91.56 50.49 80.50 8.05 7.84 6.49 1.13 169.91 50.25 79.98 8.15 8Kentucky 93.17 51.80 87.54 9.14 1.75 5.50 0.30 169.92 51.83 87.33 9.07 1Louisiana 91.38 49.80 55.98 37.63 4.50 6.91 — 171.41 50.04 55.09 38.15 4Maine 91.98 50.55 93.47 1.07 1.90 8.16 0.25 169.97 50.17 93.12 1.10 2Maryland 88.89 50.61 56.52 32.38 6.17 6.51 0.55 165.59 50.51 57.38 30.62 6Massachusetts 90.50 50.75 76.45 6.93 11.12 11.18 0.96 168.28 50.87 75.92 6.88 11Minnesota 92.92 51.21 83.93 4.17 4.49 8.40 2.74 169.44 51.62 84.16 3.82 4Mississippi 92.18 48.87 49.94 44.20 4.14 4.31 0.18 172.43 48.49 49.88 44.02 4Missouri 92.30 51.56 80.38 14.82 2.39 7.79 0.28 170.45 51.74 81.16 13.91 2Montana 91.53 47.98 86.93 0.69 3.85 7.74 0.34 169.94 48.31 87.04 0.76 4Nevada 90.78 51.98 62.49 8.05 21.90 5.91 3.39 167.87 51.84 61.77 8.14 22

(continued)
Table B-8 (continued)

Weighted Student Percentages Derived from Sampled Public Schools – Grade 8, Reading

Weighted Estimates Derived from Full Sample

Weighted Estimates Deriv

Student Nonres

Jurisdiction

Weighted

Student

Participation

(%)

Percent

Male

Percent

White

Percent

Black

Percent

Hispanic

Percent

SD

Percent

LEP

Mean

Age

(Months)

Percent

Male

Percent

White

Percent

Black

Per

Hisp

New York 88.35 49.57 54.97 18.14 20.43 5.83 2.09 167.20 49.74 56.80 16.59 19North Carolina 92.34 48.67 63.34 27.76 3.83 6.60 0.56 168.64 48.48 63.15 27.75 3

Oklahoma 91.20 48.99 71.06 8.49 6.84 3.74 1.40 171.49 49.37 71.66 8.51 7Oregon 89.34 50.38 81.20 2.56 8.43 8.85 2.24 168.49 51.06 81.13 2.54 8Rhode Island 88.47 50.69 79.34 5.86 10.29 9.85 1.93 167.42 50.18 78.88 5.92 10

South Carolina 92.55 48.55 56.29 37.53 4.08 6.20 0.23 168.89 48.37 55.99 37.45 4

Tennessee 90.45 48.90 73.42 21.33 3.32 8.98 0.28 170.02 48.86 73.49 20.79 3Texas 92.76 50.31 47.80 11.96 35.59 8.67 5.53 169.99 49.83 47.53 12.20 35Utah 89.68 50.96 85.95 0.55 7.60 5.97 1.64 167.97 51.06 85.70 0.52 7Virgin Islands 87.84 48.63 0.96 80.19 17.35 — — 170.89 47.66 1.11 78.88 18Virginia 91.20 49.93 64.99 24.65 5.71 6.54 0.63 168.06 49.89 64.81 24.35 5Washington 90.95 51.80 75.05 3.52 10.30 7.41 1.77 168.77 51.22 74.92 3.52 10West Virginia 91.07 50.03 92.46 3.39 1.87 6.57 0.06 169.56 49.69 92.03 3.45 2

Wisconsin 92.44 50.50 81.27 9.42 4.96 7.78 0.59 169.56 50.51 81.71 8.88 5Wyoming 91.15 51.88 84.34 1.20 9.12 8.08 0.28 169.78 52.30 84.22 1.25 9
488

Table B-9

Weighted Student Percentages Derived from Sampled Public Schools – Grade 8, Writing

Weighted Estimates Derived from Full Sample

Weighted Estimates Derive

Student Nonres

Jurisdiction

Weighted

Student

Participation

(%)

Percent

Male

Percent

White

Percent

Black

Percent

Hispanic

Percent

SD

Percent

LEP

Mean

Age

(Months)

Percent

Male

Percent

White

Percent

Black

Per

His

Alabama 92.42 49.82 62.44 30.83 4.17 6.28 0.29 170.13 49.25 64.05 28.82 Arizona 89.23 51.19 55.02 3.83 31.01 5.83 7.50 169.14 50.85 54.75 3.87 3Arkansas 92.28 50.05 71.01 21.85 4.56 6.71 0.44 169.79 49.94 71.11 21.38 California 91.78 47.92 34.75 7.95 44.30 5.34 13.71 166.34 47.71 36.88 7.15 4Colorado 90.78 50.80 69.54 4.49 20.75 6.59 2.88 168.97 50.79 69.06 4.48 2Connecticut 90.21 50.64 73.78 11.79 11.48 8.51 0.32 166.98 50.28 73.88 11.35 1Delaware 90.95 50.81 63.15 25.19 7.72 10.71 0.86 167.24 50.70 63.47 24.04 District of Columbia 84.59 48.49 3.96 84.05 8.83 5.79 2.00 168.40 48.27 4.10 82.82 

DoDEA/DDESS 95.45 50.37 41.56 26.73 25.99 6.31 0.88 167.82 50.61 41.24 26.08 2DoDEA/DoDDS 92.77 49.32 47.11 17.64 15.90 4.55 1.11 167.17 49.42 46.47 18.03 1Florida 88.60 49.67 52.08 25.70 18.36 9.36 2.55 169.78 49.01 50.61 26.04 1Georgia 90.08 51.63 57.50 34.02 5.02 5.95 0.95 169.94 51.69 57.81 33.50 Hawaii 91.62 52.58 14.52 3.16 18.67 7.67 3.80 165.71 52.59 14.93 3.21 1Illinois 92.42 51.29 59.54 21.37 15.76 6.87 2.19 168.83 52.00 64.34 17.52 1Kentucky 92.82 49.39 85.50 9.58 2.84 6.89 0.38 169.41 49.50 85.43 9.49 Louisiana 90.65 47.39 55.85 37.62 4.10 8.00 0.19 170.91 47.32 55.20 37.59 Maine 91.04 49.49 92.03 1.59 2.16 8.36 0.45 169.45 49.40 91.53 1.59 Maryland 88.53 50.69 55.25 33.37 5.94 10.64 0.60 165.91 50.02 55.89 31.94 Massachusetts 91.89 50.82 77.89 5.41 11.29 12.25 0.68 168.09 51.00 77.61 5.53 1Minnesota 90.23 51.61 82.44 4.76 5.20 8.69 3.20 169.52 51.14 82.25 4.83 Mississippi 92.20 49.57 48.72 44.43 4.61 4.90 0.07 171.98 49.31 48.55 44.11 Missouri 91.84 51.01 79.92 13.51 4.05 10.41 0.38 170.73 50.58 80.34 12.80 Montana 92.50 49.82 86.17 1.09 5.68 8.39 0.29 170.39 50.36 85.90 1.25 Nevada 89.22 50.94 59.46 8.40 23.53 7.33 4.11 167.88 50.49 58.68 8.24 2New York 87.35 51.20 53.49 18.29 21.48 8.04 1.96 167.03 50.64 55.21 16.63 2

(continued)
Table B-9 (continued)

Weighted Student Percentages Derived from Sampled Public Schools – Grade 8, Writing

Weighted Estimates Derived from Full Sample

Weighted Estimates Derive

Student Nonres

Jurisdiction

Weighted

Student

Participation

(%)

Percent

Male

Percent

White

Percent

Black

Percent

Hispanic

Percent

SD

Percent

LEP

Mean

Age

(Months)

Percent

Male

Percent

White

Percent

Black

Per

His

North Carolina 92.50 51.36 62.14 27.11 4.45 9.52 0.93 169.01 50.91 61.81 26.97 Oklahoma 92.16 52.05 73.50 7.23 7.46 4.17 0.81 171.38 51.90 73.04 7.07 Oregon 89.36 51.54 79.82 2.36 9.87 10.50 1.76 168.38 51.21 79.82 2.44 Rhode Island 88.92 50.97 76.08 7.17 11.86 10.93 2.43 167.39 50.81 75.46 7.01 1

South Carolina 91.43 51.35 54.73 36.74 5.06 7.19 0.08 169.11 51.22 54.46 36.46 

Tennessee 90.97 48.24 73.97 20.59 3.29 8.94 0.54 170.13 48.21 73.29 20.81 Texas 92.77 49.05 47.70 11.92 36.25 9.52 4.60 170.04 49.14 47.65 12.15 3Utah 89.86 49.51 84.23 1.17 8.93 5.45 1.23 167.86 48.80 83.60 1.19 Virgin Islands 86.97 46.60 0.85 76.63 20.54 0.14 — 170.83 44.43 0.99 76.46 2Virginia 90.91 51.80 64.62 24.24 6.09 8.95 0.89 168.21 51.94 64.57 23.88 Washington 89.17 49.21 74.38 4.31 10.96 7.65 2.37 168.83 48.92 73.60 3.99 1West Virginia 90.97 52.03 90.33 3.96 2.83 9.23 0.03 169.65 51.55 89.83 4.08 Wisconsin 92.14 50.85 80.31 8.89 6.15 6.75 0.76 169.38 50.93 80.50 7.99 Wyoming 92.13 52.46 83.09 1.35 10.34 6.86 0.11 169.83 52.18 82.87 1.45 1
490

Table B-10

Weighted Student Percentages Derived from All Schools Sampled – Grade 4, Reading

Weighted

Student

Participation

(%) Weighted Estimates Derived from Full Sample

Weighted Estimates Deri

Student Nonr

Jurisdiction Public

Non

Public

Percent

Male

Percent

White

Percent

Black

Percent

Hispanic

Percent

SD

Percent

LEP

Mean

Age

(Months)

Percent

Male

Percent

White

Percent

Black

P

Hi

Arkansas 95.06 96.98 50.75 71.06 20.49 5.77 5.69 0.49 120.79 50.57 71.89 19.17 Colorado 94.08 95.32 50.09 70.23 5.39 19.46 6.61 2.23 120.14 49.53 70.02 5.42 

Connecticut 94.04 95.31 47.49 72.91 9.99 12.93 5.96 1.40 117.71 47.54 72.79 10.07 

Florida 93.87 93.85 49.92 55.32 21.75 19.47 8.11 2.80 120.48 49.93 54.84 21.85 

Georgia 95.51 97.18 49.73 51.20 37.30 7.63 4.45 0.41 120.85 49.92 51.36 36.93 

Hawaii 94.50 96.79 50.94 19.68 4.64 18.83 5.93 3.83 116.65 50.88 19.95 4.72 Illinois 94.84 96.20 50.69 61.00 18.54 15.27 4.63 2.00 120.13 50.66 64.20 16.35 

Iowa 96.10 98.11 51.16 87.45 3.61 5.85 7.74 0.68 120.89 51.01 87.42 3.51 Louisiana 95.19 94.91 49.46 53.50 36.24 6.88 5.24 0.44 120.79 49.04 55.41 34.38 

Maine 92.99 94.01 51.37 91.12 1.55 4.13 7.16 0.76 120.09 51.25 90.53 1.53 Maryland 95.05 98.02 50.10 54.85 30.97 8.21 4.43 1.09 116.80 49.59 54.71 30.70 

Massachusetts 94.90 93.50 48.45 79.02 6.05 9.77 9.96 2.22 119.33 48.45 78.65 6.12 Michigan 93.38 94.69 49.92 73.20 15.16 8.25 2.48 0.76 119.33 49.26 74.43 12.82 

Minnesota 93.91 95.07 51.26 83.57 4.97 5.92 8.40 2.64 120.73 50.93 83.24 5.10 

Mississippi 94.96 97.94 49.12 52.69 40.19 5.36 2.65 0.08 122.48 49.01 52.81 39.75 

Missouri 95.31 95.89 51.31 76.24 14.78 6.20 6.60 0.41 122.01 51.31 76.44 14.37 

Montana 95.43 93.62 50.55 82.44 1.18 7.15 6.66 0.26 121.68 50.68 82.36 1.14 

Nevada 94.40 95.32 50.69 60.44 8.87 23.00 4.35 4.39 119.15 50.39 60.11 8.45 2New Hampshire 92.91 51.16 89.35 1.60 5.52 10.00 0.47 120.04 50.81 89.02 1.61 

New Mexico 94.45 95.25 49.91 38.32 3.23 44.99 6.88 12.25 120.38 49.92 36.96 3.20 4

New York 95.09 95.54 49.50 54.91 18.84 20.08 3.93 2.04 117.47 49.28 57.36 17.50 

North Carolina 94.00 95.40 50.23 63.68 25.95 6.68 6.23 1.24 119.63 49.84 63.27 25.90 

(continued)
Table B-10 (continued)

Weighted Student Percentages Derived from All Schools Sampled – Grade 4, Reading

Weighted

Student

Participation

(%) Weighted Estimates Derived from Full Sample

Weighted Estimates Der

Student Non

Jurisdiction Public

Non

Public

Percent

Male

Percent

White

Percent

Black

Percent

Hispanic

Percent

SD

Percent

LEP

Mean

Age

(Months)

Percent

Male

Percent

White

Percent

Black

P

H

Rhode Island 94.23 94.57 51.77 76.39 6.17 13.21 8.96 3.83 117.80 52.48 76.13 6.20 South Carolina 95.45 94.42 48.74 56.89 34.59 5.66 6.56 0.47 119.10 48.43 56.53 34.53 

Virgin Islands 95.62 95.95 46.62 7.13 71.17 18.72 0.50 1.19 118.12 46.58 7.27 70.86 

Washington 94.42 94.90 50.38 74.20 4.44 10.13 7.34 2.86 120.38 50.45 73.84 4.31 

West Virginia 94.03 100.00 48.15 88.43 3.51 5.08 3.25 0.10 120.69 47.84 87.99 3.49 

Wisconsin 94.95 96.43 51.30 80.41 7.73 8.10 5.33 1.07 120.60 51.02 79.58 7.40 

Wyoming 95.19 90.68 51.43 80.89 1.16 11.89 9.67 0.50 121.02 51.64 80.54 1.19 
492

Table B-11

Weighted Student Percentages Derived from All Schools Sampled – Grade 8, Reading

Weighted

Student

Participation

(%) Weighted Estimates Derived from Full Sample

Weighted Estimates Deriv

Student Nonre

Jurisdiction Public

NonPublic

Percent

Male

Percent

White

Percent

Black

Percent

Hispanic

Percent

SD

Percent

LEP

Mean

Age

(Months)

Percent

Male

Percent

White

Percent

Black

P

H

Arizona 90.61 89.53 50.20 54.64 4.28 29.03 5.35 6.63 169.09 50.33 53.77 4.28 Arkansas 92.23 96.96 51.33 73.46 20.77 3.23 5.18 0.42 169.68 51.34 73.83 19.99 

California 90.86 96.6 50.75 37.94 8.30 39.55 4.93 12.21 166.52 51.27 39.10 7.66 

Colorado 91.07 96.94 51.82 68.75 4.62 21.91 6.73 2.91 169.02 51.86 69.03 4.60 

Connecticut 91.38 94.65 52.25 74.96 10.84 9.77 8.40 0.54 166.86 52.03 74.96 10.62 

Florida 89.41 92.73 49.33 54.30 23.25 17.89 9.04 2.58 169.66 49.48 53.60 22.67 

Georgia 90.33 95.24 50.84 58.81 32.73 4.71 5.83 1.01 169.95 51.07 58.72 32.55 

Illinois 92.99 97.51 46.96 60.66 19.89 14.80 5.42 1.66 168.38 47.54 65.66 16.61 

Louisiana 91.38 94.9 50.73 62.47 31.36 4.19 6.27 — 170.81 50.80 61.33 32.14 Maine 91.98 95.7 50.65 93.32 1.07 2.09 7.90 0.24 169.90 50.22 92.98 1.10 Maryland 88.89 95.63 50.98 59.98 29.56 5.91 5.81 0.48 165.56 50.90 60.68 28.02 

Massachusetts 90.50 95.86 49.23 77.60 6.77 10.39 10.17 0.97 168.11 49.43 77.18 6.68 

Missouri 92.30 96.15 52.30 80.93 14.29 2.30 7.30 0.25 170.36 52.40 81.61 13.47 

Montana 91.53 98.34 47.93 87.08 0.73 3.72 7.33 0.32 169.94 48.25 87.17 0.79 

Nevada 90.78 96.33 52.05 62.66 7.91 21.79 5.74 3.25 167.87 51.84 62.21 7.96 New Mexico 89.71 95.48 48.98 37.49 1.98 49.26 9.79 6.91 169.21 48.69 37.58 2.05 

New York 88.35 94.92 50.20 56.87 17.72 19.32 5.37 1.81 166.91 50.29 58.16 16.49 

North Carolina 92.34 95.56 48.67 64.39 26.70 3.94 6.26 0.53 168.60 48.56 63.95 26.89 Rhode Island 88.47 94.3 50.27 79.43 6.33 9.86 8.76 1.69 167.32 49.76 78.95 6.42 Virgin Islands 87.84 96.19 48.39 4.84 76.64 16.31 — — 169.81 47.68 5.19 75.48 Washington 90.95 94.04 51.69 75.68 3.41 9.87 7.20 1.67 168.75 50.95 75.72 3.36 

West Virginia 91.07 93.08 49.67 92.39 3.29 1.85 6.46 0.05 169.51 49.37 92.00 3.35 

Wyoming 91.15 98.57 52.02 84.32 1.28 9.06 7.95 0.27 169.80 52.45 84.19 1.33 
Table B-12

Weighted Student Percentages Derived from All Schools Sampled – Grade 8, Writing

Weighted

Student

Participation

(%) Weighted Estimates Derived from Full Sample

Weighted Estimates Derived 

Nonrespo

Jurisdiction Public

Non

Public

Percent

Male

Percent

White

Percent

Black

Percent

Hispanic

Percent

SD

Percent

LEP

Mean

Age

(Months)

Percent

Male

Percent

White

Percent

Black

H

Arizona 89.23 96.03 51.35 54.77 3.98 31.23 5.73 7.11 169.19 51.15 54.73 4.01 Arkansas 92.28 95.89 49.76 72.36 20.67 4.37 6.36 0.42 169.74 49.57 72.48 20.13 

California 91.78 97.52 47.26 36.37 7.65 42.59 4.91 12.48 166.30 47.16 38.11 6.97 Colorado 90.78 93.39 50.35 69.39 4.49 20.76 7.04 2.80 168.97 50.34 69.06 4.42 

Connecticut 90.21 92.76 50.59 74.31 11.62 11.14 8.10 0.29 166.90 50.18 74.33 11.29 

Florida 88.60 94.21 49.66 54.15 24.30 17.49 9.01 2.36 169.74 49.06 52.90 24.49 

Georgia 90.08 94.18 51.38 59.51 32.00 4.99 5.57 0.89 169.82 51.43 59.79 31.54 

Illinois 92.42 96.26 51.26 64.60 17.91 14.13 5.75 1.78 168.69 51.88 67.68 15.31 

Louisiana 90.65 96.78 48.35 60.15 33.36 4.07 7.27 0.15 170.32 48.34 59.23 33.68 

Maine 91.04 96.25 49.72 91.74 1.57 2.42 7.93 0.43 169.41 49.68 91.24 1.57 Maryland 88.53 96.39 50.75 57.18 31.36 5.60 9.37 0.55 165.91 50.41 57.98 29.84 

Massachusetts 91.89 92.21 49.44 78.44 5.36 10.65 11.32 0.67 168.01 49.51 78.08 5.52 

Missouri 91.84 96.2 50.90 81.41 12.07 4.03 9.34 0.37 170.62 50.97 81.84 11.45 

Montana 92.50 95.68 49.92 84.82 1.06 6.02 7.95 0.27 170.33 50.36 84.64 1.20 

Nevada 89.22 92.47 50.87 59.80 8.20 23.45 7.11 3.95 167.86 50.36 58.96 8.06 New Mexico 88.95 95.79 52.01 35.37 2.44 49.33 11.27 8.21 169.03 51.59 35.84 2.60 

New York 87.35 95.56 51.07 53.80 19.71 20.13 6.98 2.62 166.62 50.60 55.12 18.24 

North Carolina 92.50 95.16 51.21 63.52 25.83 4.50 8.97 0.86 168.99 50.82 63.15 25.73 Rhode Island 88.92 95.87 51.02 76.70 7.16 11.60 9.62 2.10 167.19 50.99 76.08 7.02 Virgin Islands 86.97 98.09 45.90 4.88 71.76 19.96 0.11 — 169.46 44.26 5.10 71.58 Washington 89.17 94.43 49.23 74.78 4.43 10.64 7.49 2.25 168.85 48.99 74.00 4.14 

West Virginia 90.97 97.69 51.66 90.32 3.89 2.88 8.90 0.03 169.57 51.18 89.87 4.01 

Wyoming 92.13 94.81 52.80 82.12 1.34 10.37 6.94 0.11 169.87 52.54 81.98 1.43 
494

Table B-13

Final Collapsed Levels Used for Raking Dimensions for All Jurisdictions

Grade 4, Reading

Jurisdiction Gender Age Race SD LEP

Alabama X X W / * — —

Arizona X X H / * X X

Arkansas X X W / * — —

California X X W / H / * X X

Colorado X X W / H / * X X

Connecticut X X W / H / * X X

Delaware X X W / B / * — —

District of Columbia X X B / * X X

DoDEA/DDESS X X W / * — —

DoDEA/DoDDS X X W / * — —

Florida X X W / H / * X X

Georgia X X W / * — —

Hawaii X X H / A / * X X

Illinois X X W / H / * X X

Iowa X X — — —

Kansas X X W / * — —

Kentucky X X — — —

Louisiana X X B / * — —

Maine X X — — —

Maryland X X W / * — —

Massachusetts X X W / H / * X X

Michigan X X — — -

Minnesota X X W / * X X

Mississippi — X B / * — —

Missouri X X W / * — —

Montana X X — — —

Nevada X X W / H / * X X

New Hampshire X X — — —

New Mexico X X W / H / * X X

LEGEND

X = Variable was not collapsed for raking

W = White

B = Black

A = Asian or Pacific Islander

N = American Indian or Alaskan Native

— = Variable was not used as a raking dimension (i.e., all levels were

combined)

* = All other levels of the dimension were combined into one level

(e.g., in fourth grade for Florida, there are three levels of race:

White, Hispanic, and all others combined)

(continued)
495

Table B-13 (continued)

Final Collapsed Levels Used for Raking Dimensions for All Jurisdictions

Grade 4, Reading

Jurisdiction Gender Age Race SD LEP

New York X X W / * X X

North Carolina X X W / B / * — —

Oklahoma X X W / N / * — —

Oregon X X W / * X X

Rhode Island X X W / * X X

South Carolina X X W / * — —

Tennessee X X W / * — —

Texas X X W / H / * X X

Utah X X W / * X X

Virgin Islands — — — — —

Virginia X X W / B / * — —

Washington X X W / H / * X X

West Virginia X X - - -

Wisconsin X X W / B / * X X

Wyoming X X W / * - -

LEGEND

X = Variable was not collapsed for raking

W = White

B = Black

A = Asian or Pacific Islander

N = American Indian or Alaskan Native

— = Variable was not used as a raking dimension (i.e., all levels were

combined)

* = All other levels of the dimension were combined into one level

(e.g., in fourth grade for Florida, there are three levels of race:

White, Hispanic, and all others combined)
496

Table B-14

Final Collapsed Levels Used for Raking Dimensions for All Jurisdictions

Grade 8, Reading

Jurisdiction Gender Age Race SD LEP

Alabama X X W / * — —

Arizona X X W / H / * X X

Arkansas X X W / * — —

California X X H / * X X

Colorado X X H / * X X

Connecticut X X W / * — —

Delaware X X W / * — —

District of Columbia X X — — —

DoDEA/DDESS — — — — —

DoDEA/DoDDS X — — — —

Florida X X W / H / * X X

Georgia X X W / * — —

Hawaii X X W / A / * X X

Illinois X X W / * — X

Kansas X X — — —

Kentucky — X — — —

Louisiana X X W / * — —

Maine X X — — —

Maryland X — W / * — —

Massachusetts X X W / * — —

Minnesota X X W / * X X

Mississippi — X B / * — —

Missouri X X W / * — —

Montana X X — — —

Nevada X X H / * X X

New Mexico X X H / * X X

New York X X W / * X X

North Carolina X X W / * — —

Oklahoma X X W / N / * — —

LEGEND

X = Variable was not collapsed for raking

W = White

B = Black

A = Asian or Pacific Islander

N = American Indian or Alaskan Native

— = Variable was not used as a raking dimension (i.e., all levels were

combined)

* = All other levels of the dimension were combined into one level

(e.g., in eighth grade for Oklahoma, there are three levels of race:

White, American Indian, and all others combined)

(continued)
497

Table B-14 (continued)

Final Collapsed Levels Used for Raking Dimensions for All Jurisdictions

Grade 8, Reading

Jurisdiction Gender Age Race SD LEP

Oregon X X W / * X X

Rhode Island X X W / * X X

South Carolina X X / * — —

Tennessee X X W / * — —

Texas X X W / H / * X X

Utah X — W / * — X

Virgin Islands — — — — —

Virginia X X W / * — —

Washington X X W / * X X

West Virginia X X — — —

Wisconsin X X — — —

Wyoming X X W / * — —

LEGEND

X = Variable was not collapsed for raking

W = White

B = Black

A = Asian or Pacific Islander

N = American Indian or Alaskan Native

— = Variable was not used as a raking dimension (i.e., all levels were

combined)

* = All other levels of the dimension were combined into one level

(e.g., in eighth grade for Oklahoma, there are three levels of race:

White, American Indian, and all others combined)
498

Table B-15

Distribution of Selected Public Schools by Sampling Strata, Fourth Grade

Small or Large

District

Small or Large

School Urbanization

Percent

of

Minority

Originally

Selected

Schools

Alabama

Small Small Large/Small Town Low 1

Small Large Large/Mid-Size Central City Low 11

Small Large Large/Mid- size Central City Medium 10

Small Large Large/Mid- size Central City High 11

Small Large Urban Fringe of Large/Mid-Size Central City Low 8

Small Large Urban Fringe of Large/Mid-Size Central City Medium 8

Small Large Urban Fringe of Large/Mid-Size Central City High 9

Small Large Large/Small Town Low 8

Small Large Large/Small Town Medium 7

Small Large Large/Small Town High 7

Small Large Rural Low 8

Small Large Rural Medium 9

Small Large Rural High 9

Arizona

Small Small Large Central City High 1

Small Small Large/Small Town/Rural Medium 1

Small Large Large Central City Low 16

Small Large Large Central City Medium 16

Small Large Large Central City High 16

Small Large Mid-Size Central City Low 4

Small Large Mid-Size Central City Medium 4

Small Large Mid-Size Central City High 4

Small Large Urban Fringe of Large Central City Low 9

Small Large Urban Fringe of Large Central City Medium 8

Small Large Urban Fringe of Large Central City High 8

Small Large Large/Small Town/Rural Low 6

Small Large Large/Small Town/Rural Medium 6

Small Large Large/Small Town/Rural High 6

Arkansas

Small Small Rural Low 1

Small Small Rural High 1

Small Large Mid-Size Central City Low 10

Small Large Mid-Size Central City Medium 10

Small Large Mid-Size Central City High 10

Small Large Urban Fringe of Large Central City None 11

Small Large Large/Small Town Low 12

Small Large Large/Small Town Medium 11

Small Large Large/Small Town High 12

Small Large Rural Low 10

Small Large Rural Medium 9

Small Large Rural High 10
499

Table B-15 (continued)

Distribution of Selected Public Schools by Sampling Strata, Fourth Grade

Small or Large

District

Small or Large

School Urbanization

Percent

of

Minority

Originally

Selected

Schools

California

Small Small Large Central City Low 1

Small Small Large/Small Town/Rural Low 1

Small Large Large Central City Low 9

Small Large Large Central City Medium 9

Small Large Large Central City High 9

Small Large Mid-Size Central City Low 5

Small Large Mid-Size Central City Medium 6

Small Large Mid-Size Central City High 5

Small Large Urban Fringe of Large/Mid-Size Central City Low 18

Small Large Urban Fringe of Large/Mid-Size Central City Medium 18

Small Large Urban Fringe of Large/Mid-Size Central City High 18

Small Large Large/Small Town/Rural Low 2

Small Large Large/Small Town/Rural Medium 2

Small Large Large/Small Town/Rural High 3

Colorado

Small Small Rural Low 2

Small Small Rural High 1

Small Large Large Central City Low 7

Small Large Large Central City Medium 7

Small Large Large Central City High 7

Small Large Mid-Size Central City Low 5

Small Large Mid-Size Central City Medium 5

Small Large Mid-Size Central City High 4

Small Large Urban Fringe of Large/Mid-Size Central City Low 15

Small Large Urban Fringe of Large/Mid-Size Central City Medium 15

Small Large Urban Fringe of Large/Mid-Size Central City High 15

Small Large Large/Small Town Low 4

Small Large Large/Small Town Medium 4

Small Large Large/Small Town High 4

Small Large Rural Low 4

Small Large Rural Medium 4

Small Large Rural High 4
Table B-15 (continued)

Distribution of Selected Public Schools by Sampling Strata, Fourth Grade

500

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Connecticut

Small Large Mid-Size Central City Low Black/Low Hispanic Small Large Mid-Size Central City Low Black/High Hispanic Small Large Mid-Size Central City High Black/Low Hispanic Small Large Mid-Size Central City High Black/High Hispanic Small Large Urban Fringe of Large Central City None Small Large Urban Fringe of Mid-Size Central City Low Small Large Urban Fringe of Mid-Size Central City Medium Small Large Urban Fringe of Mid-Size Central City High Small Large Large/Small Town None Small Large Rural None 

Florida

Small Large Large Central City Low Black/Low Hispanic Small Large Large Central City Low Black/High Hispanic Small Large Large Central City High Black/Low Hispanic Small Large Large Central City High Black/High Hispanic Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large Central City Low Small Large Urban Fringe of Large Central City Medium Small Large Urban Fringe of Large Central City High Small Large Urban Fringe of Mid-Size Central City Low Small Large Urban Fringe of Mid-Size Central City Medium Small Large Urban Fringe of Mid-Size Central City High Small Large Large/Small Town/Rural Low Small Large Large/Small Town/Rural Medium Small Large Large/Small Town/Rural High 
Table B-15 (continued)

Distribution of Selected Public Schools by Sampling Strata, Fourth Grade

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Georgia

Small Large Large/Mid-Size Central City Low Small Large Large/Mid-Size Central City Medium Small Large Large/Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Urban Fringe of Large/Mid-Size Central City High Small Large Large/Small Town Low Small Large Large/Small Town Medium Small Large Large/Small Town High Small Large Rural Low Small Large Rural Medium Small Large Rural High 

Hawaii

Large Small Rural None Large Large Large Central City None Large Large Urban Fringe of Large/Mid-Size Central City None Large Large Large/Small Town None Large Large Rural None 
Table B-15 (continued)

Distribution of Selected Public Schools by Sampling Strata, Fourth Grade

502

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Illinois

Small Small Large Central City Low Black/Low Hispanic Small Small Large Central City Low Black/High Hispanic Small Small Large Central City High Black/Low Hispanic Small Small Large Central City High Black/High Hispanic Small Large Large/Small Town/Rural None Small Large Large Central City Low Black/Low Hispanic Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Urban Fringe of Large/Mid-Size Central City High Small Large Large/Small Town/Rural None 

Iowa

Small Small Rural None Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City None Small Large Large/Small Town None Small Large Rural None 
Table B-15 (continued)

Distribution of Selected Public Schools by Sampling Strata, Fourth Grade

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Kansas

Small Small Large/Small Town Low Small Small Rural None Small Large Large Central City Low Small Large Large Central City Medium Small Large Large Central City High Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large Central City None Small Large Large/Small Town Low Small Large Large/Small Town Medium Small Large Large/Small Town High Small Large Rural None 

Kentucky

Small Small Rural None Small Large Large Central City Low Small Large Large Central City Medium Small Large Large Central City High Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City None Small Large Large/Small Town None Small Large Rural None 
Table B-15 (continued)

Distribution of Selected Public Schools by Sampling Strata, Fourth Grade

504

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Louisiana

Small Small Large/Small Town Low Small Large Large Central City Low Small Large Large Central City Medium Small Large Large Central City High Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large Central City Low Small Large Urban Fringe of Large Central City Medium Small Large Urban Fringe of Large Central City High Small Large Urban Fringe of Mid-Size Central City Low Small Large Urban Fringe of Mid-Size Central City Medium Small Large Urban Fringe of Mid-Size Central City High Small Large Large/Small Town Low Small Large Large/Small Town Medium Small Large Large/Small Town High Small Large Rural Low Small Large Rural Medium Small Large Rural High 

Maine

Small Small Mid-Size Central City None Small Small Small Town None Small Small Rural None Small Large Mid-Size Central City None Small Large Urban Fringe of Large/Mid-Size Central City None Small Large Small Town None Small Large Rural None 
Table B-15 (continued)

Distribution of Selected Public Schools by Sampling Strata, Fourth Grade

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Maryland

Small Small Small Town/Rural Low Small Large Large/Mid-Size Central City Low Small Large Large/Mid-Size Central City Medium Small Large Large/Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Urban Fringe of Large/Mid-Size Central City High Small Large Small Town/Rural Low Small Large Small Town/Rural Medium Small Large Small Town/Rural High 

Massachusetts

Small Large Large/Mid-Size Central City Low Black/Low Hispanic Small Large Large/Mid-Size Central City Low Black/High Hispanic Small Large Large/Mid-Size Central City High Black/Low Hispanic Small Large Large/Mid-Size Central City High Black/High Hispanic Small Large Urban Fringe of Large/Mid-Size Central City None Small Large Large/Small Town None Small Large Rural None 
Table B-15 (continued)

Distribution of Selected Public Schools by Sampling Strata, Fourth Grade

506

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Michigan

Small Small Large/Small Town/Rural None Small Large Large Central City Low Small Large Large Central City Medium Small Large Large Central City High Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City None Small Large Large/Small Town/Rural None 

Minnesota

Small Small Rural None Small Large Large/Mid-Size Central City Low Small Large Large/Mid-Size Central City Medium Small Large Large/Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City None Small Large Large/Small Town None Small Large Rural None 

Mississippi

Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Urban Fringe of Large/Mid-Size Central City High Small Large Large/Small Town Low Small Large Large/Small Town Medium Small Large Large/Small Town High Small Large Rural Low Small Large Rural Medium Small Large Rural High 
Table B-15 (continued)

Distribution of Selected Public Schools by Sampling Strata, Fourth Grade

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Missouri

Small Small Rural None Small Large Large/Mid-Size Central City Low Small Large Large/Mid-Size Central City Medium Small Large Large/Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Urban Fringe of Large/Mid-Size Central City High Small Large Large/Small Town None Small Large Rural None 

Montana

Small Small Mid-Size Central City/Urban Fringe None Small Small Small Town None Small Small Rural None Small Large Mid-Size Central City/Urban Fringe None Small Large Large Town None Small Large Small Town None Small Large Rural None 
Table B-15 (continued)

Distribution of Selected Public Schools by Sampling Strata, Fourth Grade

508

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Nevada

Large Small Large/Small Town/Rural Low Large Large Large Central City Low Large Large Large Central City Medium Large Large Large Central City High Large Large Mid-Size Central City Low Large Large Mid-Size Central City Medium Large Large Mid-Size Central City High Large Large Urban Fringe of Large/Mid-Size Central City Low Large Large Urban Fringe of Large/Mid-Size Central City Medium Large Large Urban Fringe of Large/Mid-Size Central City High Large Large Large/Small Town/Rural High Small Small Large/Small Town/Rural Low Small Small Large/Small Town/Rural High Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Urban Fringe of Large/Mid-Size Central City High Small Large Large/Small Town/Rural Low Small Large Large/Small Town/Rural Medium Small Large Large/Small Town/Rural High 

New Hampshire

Small Small Rural None Small Large Mid-Size Central City None Small Large Urban Fringe of Large/Mid-Size Central City None Small Large Large/Small Town None Small Large Rural None 
Table B-15 (continued)

Distribution of Selected Public Schools by Sampling Strata, Fourth Grade

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

New Mexico

Large Large Large Central City Low Large Large Large Central City Medium Large Large Large Central City High Large Large Rural Low Small Small Rural Low Small Small Rural Medium Small Small Rural High Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Urban Fringe of Large/Mid-Size Central City High Small Large Large Town Low Small Large Large Town Medium Small Large Large Town High Small Large Small Town Low Small Large Small Town Medium Small Large Small Town High Small Large Rural Low Small Large Rural Medium Small Large Rural High 
Table B-15 (continued)

Distribution of Selected Public Schools by Sampling Strata, Fourth Grade

510

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

New York

Large Large Large/Mid-Size Central City Low Black/Low Hispanic Large Large Large/Mid-Size Central City Low Black/High Hispanic Large Large Large/Mid-Size Central City High Black/Low Hispanic Large Large Large/Mid-Size Central City High Black/High Hispanic Large Large Large/Small Town/Rural None Small Large Large/Mid-Size Central City Low Black/Low Hispanic Small Large Large/Mid-Size Central City Low Black/High Hispanic Small Large Large/Mid-Size Central City High Black/Low Hispanic Small Large Large/Mid-Size Central City High Black/High Hispanic Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Urban Fringe of Large/Mid-Size Central City High Small Large Large/Small Town/Rural None 

North Carolina

Small Small Rural High Small Large Large/Mid-Size Central City Low Small Large Large/Mid-Size Central City Medium Small Large Large/Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Urban Fringe of Large/Mid-Size Central City High Small Large Large/Small Town Low Small Large Large/Small Town Medium Small Large Large/Small Town High Small Large Rural Low Small Large Rural Medium Small Large Rural High 
Table B-15 (continued)

Distribution of Selected Public Schools by Sampling Strata, Fourth Grade

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Oklahoma

Small Small Large/Small Town None Small Small Rural None Small Large Large/Mid-Size Central City Low Small Large Large/Mid-Size Central City Medium Small Large Large/Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City None Small Large Large/Small Town None Small Large Rural None 

Oregon

Small Small Mid-Size Central City Low Small Small Urban Fringe of Large/Mid-Size Central City Low Small Small Large/Small Town Low Small Small Rural Low Small Small Rural High Small Large Large Central City Low Small Large Large Central City Medium Small Large Large Central City High Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Urban Fringe of Large/Mid-Size Central City High Small Large Large/Small Town Low Small Large Large/Small Town Medium Small Large Large/Small Town High Small Large Rural Low Small Large Rural Medium Small Large Rural High 
Table B-15 (continued)

Distribution of Selected Public Schools by Sampling Strata, Fourth Grade

512

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Rhode Island

Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large Central City None Small Large Urban Fringe of Mid-Size Central City None Small Large Large/Small Town None Small Large Rural None 

South Carolina

Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Urban Fringe of Large/Mid-Size Central City High Small Large Small Town Low Small Large Small Town Medium Small Large Small Town High Small Large Rural Low Small Large Rural Medium Small Large Rural High 
Table B-15 (continued)

Distribution of Selected Public Schools by Sampling Strata, Fourth Grade

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Tennessee

Small Small Large/Small Town Low Small Small Rural None Small Large Large Central City Low Small Large Large Central City Medium Small Large Large Central City High Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City None Small Large Large/Small Town Low Small Large Large/Small Town Medium Small Large Large/Small Town High Small Large Rural None 

Texas

Small Small Rural Low Small Small Rural High Small Large Large Central City Low Hispanic/Low Black Small Large Large Central City Low Hispanic/High Black Small Large Large Central City High Hispanic/Low Black Small Large Large Central City High Hispanic/High Black Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Urban Fringe of Large/Mid-Size Central City High Small Large Large/Small Town Low Small Large Large/Small Town Medium Small Large Large/Small Town High Small Large Rural Low Small Large Rural Medium Small Large Rural High 
Table B-15 (continued)

Distribution of Selected Public Schools by Sampling Strata, Fourth Grade

514

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Utah

Small Small Rural None Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Mid-Size Central City None Small Large Large/Small Town None Small Large Rural None 

Vermont

Small Small Small Town None Small Small Rural None Small Large Mid-Size Central City/Urban Fringe None Small Large Small Town None Small Large Rural None 

Virginia

Small Small Rural Low Small Large Large Central City Low Small Large Large Central City Medium Small Large Large Central City High Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Urban Fringe of Large/Mid-Size Central City High Small Large Large/Small Town/Rural Low Small Large Large/Small Town/Rural Medium Small Large Large/Small Town/Rural High 
Table B-15 (continued)

Distribution of Selected Public Schools by Sampling Strata, Fourth Grade

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Or

Washington

Small Small Large/Mid-Size Central City High Small Small Rural Low Small Large Large/Mid-Size Central City Low Small Large Large/Mid-Size Central City Medium Small Large Large/Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Urban Fringe of Large/Mid-Size Central City High Small Large Large/Small Town Low Small Large Large/Small Town Medium Small Large Large/Small Town High Small Large Rural Low Small Large Rural Medium Small Large Rural High 

West Virginia

Small Small Mid-Size Central City None Small Small Large/Small Town None Small Small Rural None Small Large Mid-Size Central City None Small Large Urban Fringe of Large/Mid-Size Central City None Small Large Large/Small Town None Small Large Rural None 
Table B-15 (continued)

Distribution of Selected Public Schools by Sampling Strata, Fourth Grade

516

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Or

Wisconsin

Small Small Large/Small Town None Small Small Rural None Small Large Large Central City Low Small Large Large Central City Medium Small Large Large Central City High Small Large Mid-Size Central City None Small Large Urban Fringe of Large/Mid-Size Central City None Small Large Large/Small Town None Small Large Rural None 

Wyoming

Small Small Mid-Size Central City Low Small Small Large/Small Town None Small Small Rural None Small Large Mid-Size Central City/Urban Fringe Low Small Large Mid-Size Central City/Urban Fringe Medium Small Large Mid-Size Central City/Urban Fringe High Small Large Large/Small Town None Small Large Rural None 
Table B-16

Distribution of Selected Public Schools by Sampling Strata, Eighth Grade

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Alabama

Small Small Large/Mid-Size Central City Low Small Small Rural Low Small Large Large/Mid-Size Central City Low Small Large Large/Mid-Size Central City Medium Small Large Large/Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Urban Fringe of Large/Mid-Size Central City High Small Large Large/Small Town Low Small Large Large/Small Town Medium Small Large Large/Small Town High Small Large Rural Low Small Large Rural Medium Small Large Rural High 

Arizona

Small Small Mid-Size Central City Low Small Small Large/Small Town/Rural Medium Small Small Large/Small Town/Rural High Small Large Large Central City Low Small Large Large Central City Medium Small Large Large Central City High Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large Central City Low Small Large Urban Fringe of Large Central City Medium Small Large Urban Fringe of Large Central City High Small Large Large/Small Town/Rural Low Small Large Large/Small Town/Rural Medium Small Large Large/Small Town/Rural High 
Table B-16 (continued)

Distribution of Selected Public Schools by Sampling Strata, Eighth Grade

518

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Arkansas

Small Small Large/Small Town Low Small Small Rural Low Small Small Rural High Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large Central City None Small Large Large/Small Town Low Small Large Large/Small Town Medium Small Large Large/Small Town High Small Large Rural Low Small Large Rural Medium Small Large Rural High 

California

Small Small Large/Small Town/Rural Low Small Small Large/Small Town/Rural Medium Small Large Large Central City Low Small Large Large Central City Medium Small Large Large Central City High Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Urban Fringe of Large/Mid-Size Central City High Small Large Large/Small Town/Rural Low Small Large Large/Small Town/Rural Medium Small Large Large/Small Town/Rural High 
Table B-16 (continued)

Distribution of Selected Public Schools by Sampling Strata, Eighth Grade

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Colorado

Small Small Large Central City High Small Small Rural Low Small Small Rural Medium Small Small Rural High Small Large Large Central City Low Small Large Large Central City Medium Small Large Large Central City High Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Urban Fringe of Large/Mid-Size Central City High Small Large Large/Small Town Low Small Large Large/Small Town Medium Small Large Large/Small Town High Small Large Rural Low Small Large Rural Medium Small Large Rural High 
Table B-16 (continued)

Distribution of Selected Public Schools by Sampling Strata, Eighth Grade

520

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Connecticut

Small Large Mid-Size Central City Low Black/Low Hispanic Small Large Mid-Size Central City Low Black/High Hispanic Small Large Mid-Size Central City High Black/Low Hispanic Small Large Mid-Size Central City High Black/High Hispanic Small Large Urban Fringe of Large Central City Low Small Large Urban Fringe of Large Central City Medium Small Large Urban Fringe of Large Central City High Small Large Urban Fringe of Mid-Size Central City Low Small Large Urban Fringe of Mid-Size Central City Medium Small Large Urban Fringe of Mid-Size Central City High Small Large Large/Small Town None Small Large Rural None 
Table B-16 (continued)

Distribution of Selected Public Schools by Sampling Strata, Eighth Grade

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Florida

Small Small Large/Small Town/Rural Low Small Large Large Central City Low Black/Low Hispanic Small Large Large Central City Low Black/High Hispanic Small Large Large Central City High Black/Low Hispanic Small Large Large Central City High Black/High Hispanic Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large Central City Low Small Large Urban Fringe of Large Central City Medium Small Large Urban Fringe of Large Central City High Small Large Urban Fringe of Mid-Size Central City Low Small Large Urban Fringe of Mid-Size Central City Medium Small Large Urban Fringe of Mid-Size Central City High Small Large Large/Small Town/Rural Low Small Large Large/Small Town/Rural Medium Small Large Large/Small Town/Rural High 
Table B-16 (continued)

Distribution of Selected Public Schools by Sampling Strata, Eighth Grade

522

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Georgia

Small Large Large/Mid-Size Central City Low Small Large Large/Mid-Size Central City Medium Small Large Large/Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Urban Fringe of Large/Mid-Size Central City High Small Large Large/Small Town Low Small Large Large/Small Town Medium Small Large Large/Small Town High Small Large Rural Low Small Large Rural Medium Small Large Rural High 

Illinois

Large Small Large Central City High Black/Low Hispanic Large Large Large Central City Low Black/Low Hispanic Large Large Large Central City Low Black/High Hispanic Large Large Large Central City High Black/Low Hispanic Large Large Large Central City High Black/High Hispanic Small Small Rural None Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large Central City Low Small Large Urban Fringe of Large Central City Medium Small Large Urban Fringe of Large Central City High Small Large Large/Small Town None Small Large Rural None 
Table B-16 (continued)

Distribution of Selected Public Schools by Sampling Strata, Eighth Grade

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Kansas

Small Small Urban Fringe of Large/Mid-Size Central City None Small Small Rural None Small Large Large/Mid-Size Central City Low Small Large Large/Mid-Size Central City Medium Small Large Large/Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City None Small Large Large/Small Town Low Small Large Large/Small Town Medium Small Large Large/Small Town High Small Large Rural None 

Kentucky

Small Small Urban Fringe of Large/Mid-Size Central City None Small Small Rural None Small Large Large Central City Low Small Large Large Central City Medium Small Large Large Central City High Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City None Small Large Large/Small Town None Small Large Rural None 
Table B-16 (continued)

Distribution of Selected Public Schools by Sampling Strata, Eighth Grade

524

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Louisiana

Small Small Large/Small Town Low Small Small Rural High Small Large Large/Mid-Size Central City Low Small Large Large/Mid-Size Central City Medium Small Large Large/Mid-Size Central City High Small Large Urban Fringe of Large Central City Low Small Large Urban Fringe of Large Central City Medium Small Large Urban Fringe of Large Central City High Small Large Urban Fringe of Mid-Size Central City Low Small Large Urban Fringe of Mid-Size Central City Medium Small Large Urban Fringe of Mid-Size Central City High Small Large Large/Small Town Low Small Large Large/Small Town Medium Small Large Large/Small Town High Small Large Rural Low Small Large Rural Medium Small Large Rural High 

Maine

Small Small Rural None Small Large Mid-Size Central City None Small Large Urban Fringe of Large/Mid-Size Central City None Small Large Small Town None Small Large Rural None 
Table B-16 (continued)

Distribution of Selected Public Schools by Sampling Strata, Eighth Grade

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Maryland

Small Small Urban Fringe of Large Central City Medium Small Large Large/Mid-Size Central City Low Small Large Large/Mid-Size Central City Medium Small Large Large/Mid-Size Central City High Small Large Urban Fringe of Large Central City Low Small Large Urban Fringe of Large Central City Medium Small Large Urban Fringe of Large Central City High Small Large Small Town/Rural Low Small Large Small Town/Rural Medium Small Large Small Town/Rural High 

Massachusetts

Small Large Large/Mid-Size Central City Low Small Large Large/Mid-Size Central City Medium Small Large Large/Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City None Small Large Large/Small Town None Small Large Rural None 

Minnesota

Small Small Rural None Small Large Large/Mid-Size Central City Low Small Large Large/Mid-Size Central City Medium Small Large Large/Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City None Small Large Large/Small Town None Small Large Rural None 
Table B-16 (continued)

Distribution of Selected Public Schools by Sampling Strata, Eighth Grade

526

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Mississippi

Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Urban Fringe of Large/Mid-Size Central City High Small Large Large/Small Town Low Small Large Large/Small Town Medium Small Large Large/Small Town High Small Large Rural Low Small Large Rural Medium Small Large Rural High 

Missouri

Small Small Large/Small Town None Small Small Rural None Small Large Large/Mid-Size Central City Low Small Large Large/Mid-Size Central City Medium Small Large Large/Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Urban Fringe of Large/Mid-Size Central City High Small Large Large/Small Town None Small Large Rural None 
Table B-16 (continued)

Distribution of Selected Public Schools by Sampling Strata, Eighth Grade

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Montana

Small Small Mid-Size Central City/Urban Fringe None Small Small Small Town None Small Small Rural None Small Large Mid-Size Central City/Urban Fringe None Small Large Large Town None Small Large Small Town None Small Large Rural None 

North Carolina

Small Large Large/Mid-Size Central City Low Small Large Large/Mid-Size Central City Medium Small Large Large/Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Urban Fringe of Large/Mid-Size Central City High Small Large Large/Small Town Low Small Large Large/Small Town Medium Small Large Large/Small Town High Small Large Rural Low Small Large Rural Medium Small Large Rural High 
Table B-16 (continued)

Distribution of Selected Public Schools by Sampling Strata, Eighth Grade

528

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Nevada

Large Large Large Central City Low Large Large Large Central City Medium Large Large Large Central City High Large Large Mid-Size Central City Medium Large Large Mid-Size Central City High Large Large Urban Fringe of Large/Mid-Size Central City Low Large Large Urban Fringe of Large/Mid-Size Central City Medium Large Large Urban Fringe of Large/Mid-Size Central City High Large Large Large/Small Town/Rural Medium Large Large Large/Small Town/Rural High Small Small Large/Small Town/Rural Low Small Small Large/Small Town/Rural Medium Small Small Large/Small Town/Rural High Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Large/Small Town/Rural Low Small Large Large/Small Town/Rural Medium Small Large Large/Small Town/Rural High 
Table B-16 (continued)

Distribution of Selected Public Schools by Sampling Strata, Eighth Grade

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

New Mexico

Large Large Large Central City Low Large Large Large Central City Medium Large Large Large Central City High Large Large Rural Medium Small Small Rural Low Small Small Rural Medium Small Small Rural High Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Urban Fringe of Large/Mid-Size Central City High Small Large Large Town Low Small Large Large Town Medium Small Large Large Town High Small Large Small Town Low Small Large Small Town Medium Small Large Small Town High Small Large Rural Low Small Large Rural Medium Small Large Rural High 
Table B-16 (continued)

Distribution of Selected Public Schools by Sampling Strata, Eighth Grade

530

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

New York

Large Large Large/Mid-Size Central City Low Black/Low Hispanic Large Large Large/Mid-Size Central City Low Black/High Hispanic Large Large Large/Mid-Size Central City High Black/Low Hispanic Large Large Large/Mid-Size Central City High Black/High Hispanic Small Large Large/Mid-Size Central City Low Black/Low Hispanic Small Large Large/Mid-Size Central City High Black/Low Hispanic Small Large Large/Mid-Size Central City High Black/High Hispanic Small Large Urban Fringe of Large Central City Low Small Large Urban Fringe of Large Central City Medium Small Large Urban Fringe of Large Central City High Small Large Urban Fringe of Mid-Size Central City None Small Large Large/Small Town/Rural None 

Oklahoma

Small Small Urban Fringe of Large/Mid-Size Central City None Small Small Large/Small Town None Small Small Rural None Small Large Large Central City Low Small Large Large Central City Medium Small Large Large Central City High Small Large Urban Fringe of Large/Mid-Size Central City None Small Large Large/Small Town None Small Large Rural None 
Table B-16 (continued)

Distribution of Selected Public Schools by Sampling Strata, Eighth Grade

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Oregon

Small Small Urban Fringe of Large/Mid-Size Central City None Small Small Large/Small Town High Small Small Rural None Small Large Large Central City Low Small Large Large Central City Medium Small Large Large Central City High Small Large Mid-Size Central City None Small Large Urban Fringe of Large/Mid-Size Central City None Small Large Large/Small Town Low Small Large Large/Small Town Medium Small Large Large/Small Town High Small Large Rural None 

South Carolina

Small Small Rural Low Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Urban Fringe of Large/Mid-Size Central City High Small Large Small Town Low Small Large Small Town Medium Small Large Small Town High Small Large Rural Low Small Large Rural Medium Small Large Rural High 
Table B-16 (continued)

Distribution of Selected Public Schools by Sampling Strata, Eighth Grade

532

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Tennessee

Small Small Large/Small Town Low Small Small Rural None Small Large Large Central City Low Small Large Large Central City Medium Small Large Large Central City High Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City None Small Large Large/Small Town Low Small Large Large/Small Town Medium Small Large Large/Small Town High Small Large Rural None 
Table B-16 (continued)

Distribution of Selected Public Schools by Sampling Strata, Eighth Grade

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Texas

Small Small Rural Low Small Small Rural Medium Small Small Rural High Small Large Large Central City Low Small Large Large Central City Medium Small Large Large Central City High Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Urban Fringe of Large/Mid-Size Central City High Small Large Large/Small Town Low Small Large Large/Small Town Medium Small Large Large/Small Town High Small Large Rural Low Small Large Rural Medium Small Large Rural High 

Utah

Small Small Large/Small Town/Rural None Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Mid-Size Central City None Small Large Large/Small Town/Rural None 
Table B-16 (continued)

Distribution of Selected Public Schools by Sampling Strata, Eighth Grade

534

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Virginia

Small Large Large Central City Low Small Large Large Central City Medium Small Large Large Central City High Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City Low Small Large Urban Fringe of Large/Mid-Size Central City Medium Small Large Urban Fringe of Large/Mid-Size Central City High Small Large Large/Small Town/Rural Low Small Large Large/Small Town/Rural Medium Small Large Large/Small Town/Rural High 

Washington

Small Small Large/Small Town Low Small Small Rural Low Small Large Large/Mid-Size Central City Low Small Large Large/Mid-Size Central City Medium Small Large Large/Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City None Small Large Large/Small Town Low Small Large Large/Small Town Medium Small Large Large/Small Town High Small Large Rural Low Small Large Rural Medium Small Large Rural High 
Table B-16 (continued)

Distribution of Selected Public Schools by Sampling Strata, Eighth Grade

Small or Large

District

Small or Large

School Urbanization

Percent of

Minority

Wisconsin

Small Small Large/Small Town None Small Small Rural None Small Large Large/Mid-Size Central City Low Small Large Large/Mid-Size Central City Medium Small Large Large/Mid-Size Central City High Small Large Urban Fringe of Large/Mid-Size Central City None Small Large Large/Small Town None Small Large Rural None 

West Virginia

Small Small Large/Small Town None Small Small Rural None Small Large Mid-Size Central City None Small Large Urban Fringe of Large/Mid-Size Central City None Small Large Large/Small Town None Small Large Rural None 

Wyoming

Small Small Large/Small Town None Small Small Rural None Small Large Mid-Size Central City Low Small Large Mid-Size Central City Medium Small Large Mid-Size Central City High Small Large Large/Small Town None Small Large Rural None 
Table B-17

Distribution of Selected Nonpublic Schools by Sampling Strata, Fourth Grade

536

Small or Large

School

Metro

Status

School

Type

Originally Selected

Schools

Arkansas

Small Not In Metro Area Other Nonpublic 3

Small In Metro Area Other Nonpublic 2

Large Not In Metro Area Other Nonpublic 2

Large In Metro Area Catholic 2

Large In Metro Area Other Nonpublic 4

Colorado

Small Not In Metro Area Other Nonpublic 2

Small In Metro Area Catholic 1

Small In Metro Area Other Nonpublic 4

Large In Metro Area Catholic 4

Large In Metro Area Other Nonpublic 5

Connecticut

Small In Metro Area Catholic 1

Small In Metro Area Other Nonpublic 4

Large Not In Metro Area Catholic 1

Large In Metro Area Catholic 9

Large In Metro Area Other Nonpublic 3

Florida

Small Not In Metro Area Other Nonpublic 1

Small In Metro Area Other Nonpublic 5

Large In Metro Area Catholic 5

Large In Metro Area Other Nonpublic 9

Georgia

Small Not In Metro Area Other Nonpublic 2

Small In Metro Area Other Nonpublic 2

Large Not In Metro Area Other Nonpublic 1

Large In Metro Area Catholic 1

Large In Metro Area Other Nonpublic 5

Hawaii

Small Not In Metro Area Catholic 1

Small Not In Metro Area Other Nonpublic 3

Small In Metro Area Other Nonpublic 3

Large Not In Metro Area Catholic 1

Large Not In Metro Area Other Nonpublic 1

Large In Metro Area Catholic 6

Large In Metro Area Other Nonpublic 8
Table B-17 (continued)

Distribution of Selected Nonpublic Schools by Sampling Strata, Fourth Grade

 537

Small or Large

School

Metro

Status

School

Type

Originally Selected

Schools

Iowa

Small Not In Metro Area Catholic 2

Small Not In Metro Area Other Nonpublic 3

Small In Metro Area Other Nonpublic 1

Large Not In Metro Area Catholic 4

Large Not In Metro Area Other Nonpublic 1

Large In Metro Area Catholic 6

Large In Metro Area Other Nonpublic 1

Illinois

Small Not In Metro Area Other Nonpublic 1

Small In Metro Area Catholic 1

Small In Metro Area Other Nonpublic 5

Large Not In Metro Area Catholic 1

Large Not In Metro Area Other Nonpublic 1

Large In Metro Area Catholic 12

Large In Metro Area Other Nonpublic 4

Louisiana

Small Not In Metro Area Other Nonpublic 1

Small In Metro Area Other Nonpublic 3

Large Not In Metro Area Catholic 1

Large Not In Metro Area Other Nonpublic 1

Large In Metro Area Catholic 11

Large In Metro Area Other Nonpublic 7

Massachusetts

Small Not In Metro Area Other Nonpublic 1

Small In Metro Area Catholic 1

Small In Metro Area Other Nonpublic 4

Large In Metro Area Catholic 10

Large In Metro Area Other Nonpublic 3

Maryland

Small Not In Metro Area Other Nonpublic 1

Small In Metro Area Other Nonpublic 5

Large Not In Metro Area Other Nonpublic 1

Large In Metro Area Catholic 8

Large In Metro Area Other Nonpublic 9

Maine

Small Not In Metro Area Catholic 1

Small Not In Metro Area Other Nonpublic 6

Small In Metro Area Catholic 1

Small In Metro Area Other Nonpublic 2

Large Not In Metro Area Catholic 1

Large Not In Metro Area Other Nonpublic 2

Large In Metro Area Catholic 1

Large In Metro Area Other Nonpublic 2
Table B-17 (continued)

Distribution of Selected Nonpublic Schools by Sampling Strata, Fourth Grade

538

Small or Large

School

Metro

Status

School

Type

Originally Selected

Schools

Michigan

Small Not In Metro Area Other Nonpublic 2

Small In Metro Area Catholic 1

Small In Metro Area Other Nonpublic 5

Large Not In Metro Area Catholic 1

Large In Metro Area Catholic 6

Large In Metro Area Other Nonpublic 6

Minnesota

Small Not In Metro Area Catholic 1

Small Not In Metro Area Other Nonpublic 4

Small In Metro Area Catholic 2

Small In Metro Area Other Nonpublic 3

Large Not In Metro Area Catholic 2

Large In Metro Area Catholic 7

Large In Metro Area Other Nonpublic 3

Missouri

Small Not In Metro Area Catholic 2

Small Not In Metro Area Other Nonpublic 1

Small In Metro Area Catholic 1

Small In Metro Area Other Nonpublic 4

Large Not In Metro Area Catholic 1

Large Not In Metro Area Other Nonpublic 1

Large In Metro Area Catholic 10

Large In Metro Area Other Nonpublic 4

Mississippi

Small Not In Metro Area Other Nonpublic 2

Small In Metro Area Catholic 1

Small In Metro Area Other Nonpublic 1

Large Not In Metro Area Other Nonpublic 6

Large In Metro Area Catholic 1

Large In Metro Area Other Nonpublic 3

Montana

Small Not In Metro Area Catholic 1

Small Not In Metro Area Other Nonpublic 5

Small In Metro Area Other Nonpublic 1

Large Not In Metro Area Catholic 3

Large Not In Metro Area Other Nonpublic 2

Large In Metro Area Other Nonpublic 1

North Carolina

Small Not In Metro Area Other Nonpublic 2

Small In Metro Area Other Nonpublic 3

Large Not In Metro Area Other Nonpublic 1

Large In Metro Area Catholic 1

Large In Metro Area Other Nonpublic 5
Table B-17 (continued)

Distribution of Selected Nonpublic Schools by Sampling Strata, Fourth Grade

 539

Small or Large

School

Metro

Status

School

Type

Originally Selected

Schools

Nebraska

Small Not In Metro Area Catholic 4

Small Not In Metro Area Other Nonpublic 4

Small In Metro Area Catholic 1

Small In Metro Area Other Nonpublic 2

Large Not In Metro Area Catholic 2

Large Not In Metro Area Other Nonpublic 1

Large In Metro Area Catholic 9

Large In Metro Area Other Nonpublic 3

New Mexico

Small Not In Metro Area Catholic 1

Small Not In Metro Area Other Nonpublic 3

Small In Metro Area Catholic 1

Small In Metro Area Other Nonpublic 3

Large In Metro Area Catholic 2

Large In Metro Area Other Nonpublic 3

Nevada

Small Not In Metro Area Other Nonpublic 1

Small In Metro Area Other Nonpublic 3

Large In Metro Area Catholic 2

Large In Metro Area Other Nonpublic 4

New York

Small Not In Metro Area Other Nonpublic 1

Small In Metro Area Catholic 1

Small In Metro Area Other Nonpublic 3

Large Not In Metro Area Catholic 1

Large In Metro Area Catholic 12

Large In Metro Area Other Nonpublic 9

Rhode Island

Small Not In Metro Area Other Nonpublic 1

Small In Metro Area Catholic 3

Small In Metro Area Other Nonpublic 3

Large Not In Metro Area Catholic 1

Large In Metro Area Catholic 11

Large In Metro Area Other Nonpublic 4

South Carolina

Small Not In Metro Area Other Nonpublic 2

Small In Metro Area Catholic 1

Small In Metro Area Other Nonpublic 3

Large Not In Metro Area Other Nonpublic 2

Large In Metro Area Catholic 1

Large In Metro Area Other Nonpublic 6

Utah

Small Not In Metro Area Other Nonpublic 1

Small In Metro Area Other Nonpublic 3

Large In Metro Area Catholic 2

Large In Metro Area Other Nonpublic 4
Table B-17 (continued)

Distribution of Selected Nonpublic Schools by Sampling Strata, Fourth Grade

540

Small or Large

School

Metro

Status

School

Type

Originally Selected

Schools

Washington

Small Not In Metro Area Other Nonpublic 1

Small In Metro Area Catholic 1

Small In Metro Area Other Nonpublic 3

Large Not In Metro Area Catholic 1

Large In Metro Area Catholic 3

Large In Metro Area Other Nonpublic 5

Wisconsin

Small Not In Metro Area Catholic 2

Small Not In Metro Area Other Nonpublic 5

Small In Metro Area Catholic 3

Small In Metro Area Other Nonpublic 6

Large Not In Metro Area Catholic 3

Large Not In Metro Area Other Nonpublic 2

Large In Metro Area Catholic 9

Large In Metro Area Other Nonpublic 5

West Virginia

Small Not In Metro Area Other Nonpublic 5

Small In Metro Area Catholic 1

Small In Metro Area Other Nonpublic 2

Large Not In Metro Area Catholic 1

Large In Metro Area Catholic 3

Large In Metro Area Other Nonpublic 1

Wyoming

Small Not In Metro Area Catholic 1

Small Not In Metro Area Other Nonpublic 4

Small In Metro Area Other Nonpublic 2

Large Not In Metro Area Catholic 1

Large Not In Metro Area Other Nonpublic 1

Large In Metro Area Catholic 1

Large In Metro Area Other Nonpublic 1
Table B-18

Distribution of Selected Nonpublic Schools by Sampling Strata, Eighth Grade

541

Small or Large

School

Metro

Status

School

Type

Originally Selected

Schools

Arkansas

Small Not In Metro Area Other Nonpublic 4

Small In Metro Area Other Nonpublic 4

Large Not In Metro Area Other Nonpublic 3

Large In Metro Area Catholic 3

Large In Metro Area Other Nonpublic 4

Arizona

Small Not In Metro Area Other Nonpublic 1

Small In Metro Area Other Nonpublic 6

Large Not In Metro Area Catholic 1

Large In Metro Area Catholic 3

Large In Metro Area Other Nonpublic 7

California

Small In Metro Area Other Nonpublic 9

Large Not In Metro Area Catholic 1

Large In Metro Area Catholic 9

Large In Metro Area Other Nonpublic 14

Colorado

Small Not In Metro Area Other Nonpublic 3

Small In Metro Area Catholic 1

Small In Metro Area Other Nonpublic 7

Large Not In Metro Area Other Nonpublic 1

Large In Metro Area Catholic 4

Large In Metro Area Other Nonpublic 7

Connecticut

Small Not In Metro Area Other Nonpublic 1

Small In Metro Area Catholic 2

Small In Metro Area Other Nonpublic 8

Large Not In Metro Area Catholic 1

Large Not In Metro Area Other Nonpublic 1

Large In Metro Area Catholic 18

Large In Metro Area Other Nonpublic 7

Florida

Small Not In Metro Area Other Nonpublic 1

Small In Metro Area Other Nonpublic 9

Large In Metro Area Catholic 6

Large In Metro Area Other Nonpublic 13
Table B-18 (continued)

Distribution of Selected Nonpublic Schools by Sampling Strata, Eighth Grade

542

Small or Large

School

Metro

Status

School

Type

Originally Selected

Schools

Georgia

Small Not In Metro Area Other Nonpublic 3

Small In Metro Area Other Nonpublic 5

Large Not In Metro Area Other Nonpublic 3

Large In Metro Area Catholic 2

Large In Metro Area Other Nonpublic 7

Illinois

Small Not In Metro Area Catholic 1

Small Not In Metro Area Other Nonpublic 2

Small In Metro Area Catholic 2

Small In Metro Area Other Nonpublic 8

Large Not In Metro Area Catholic 2

Large In Metro Area Catholic 19

Large In Metro Area Other Nonpublic 9

Louisiana

Small Not In Metro Area Other Nonpublic 1

Small In Metro Area Catholic 1

Small In Metro Area Other Nonpublic 5

Large Not In Metro Area Catholic 2

Large Not In Metro Area Other Nonpublic 2

Large In Metro Area Catholic 20

Large In Metro Area Other Nonpublic 9

Massachusetts

Small In Metro Area Catholic 2

Small In Metro Area Other Nonpublic 11

Large In Metro Area Catholic 18

Large In Metro Area Other Nonpublic 8

Maryland

Small Not In Metro Area Other Nonpublic 2

Small In Metro Area Catholic 1

Small In Metro Area Other Nonpublic 8

Large Not In Metro Area Catholic 1

Large Not In Metro Area Other Nonpublic 1

Large In Metro Area Catholic 14

Large In Metro Area Other Nonpublic 12

Maine

Small Not In Metro Area Catholic 1

Small Not In Metro Area Other Nonpublic 10

Small In Metro Area Catholic 1

Small In Metro Area Other Nonpublic 4

Large Not In Metro Area Catholic 1

Large Not In Metro Area Other Nonpublic 5

Large In Metro Area Catholic 3

Large In Metro Area Other Nonpublic 4
Table B-18 (continued)

Distribution of Selected Nonpublic Schools by Sampling Strata, Eighth Grade

543

Small or Large

School

Metro

Status

School

Type

Originally Selected

Schools

Missouri

Small Not In Metro Area Catholic 2

Small Not In Metro Area Other Nonpublic 3

Small In Metro Area Catholic 2

Small In Metro Area Other Nonpublic 6

Large Not In Metro Area Catholic 2

Large Not In Metro Area Other Nonpublic 2

Large In Metro Area Catholic 16

Large In Metro Area Other Nonpublic 6

Montana

Small Not In Metro Area Catholic 1

Small Not In Metro Area Other Nonpublic 11

Small In Metro Area Other Nonpublic 3

Large Not In Metro Area Catholic 5

Large Not In Metro Area Other Nonpublic 3

Large In Metro Area Catholic 2

Large In Metro Area Other Nonpublic 1

North Carolina

Small Not In Metro Area Other Nonpublic 4

Small In Metro Area Other Nonpublic 6

Large Not In Metro Area Other Nonpublic 2

Large In Metro Area Catholic 1

Large In Metro Area Other Nonpublic 7

Nebraska

Small Not In Metro Area Catholic 3

Small Not In Metro Area Other Nonpublic 7

Small In Metro Area Catholic 1

Small In Metro Area Other Nonpublic 4

Large Not In Metro Area Catholic 4

Large Not In Metro Area Other Nonpublic 2

Large In Metro Area Catholic 12

Large In Metro Area Other Nonpublic 3

New Mexico

Small Not In Metro Area Catholic 1

Small Not In Metro Area Other Nonpublic 3

Small In Metro Area Catholic 1

Small In Metro Area Other Nonpublic 6

Large Not In Metro Area Catholic 1

Large Not In Metro Area Other Nonpublic 3

Large In Metro Area Catholic 4

Large In Metro Area Other Nonpublic 4
Table B-18 (continued)

Distribution of Selected Nonpublic Schools by Sampling Strata, Eighth Grade

544

Small or Large

School

Metro

Status

School

Type

Originally Selected

Schools

Nevada

Small Not In Metro Area Other Nonpublic 1

Small In Metro Area Other Nonpublic 3

Large Not In Metro Area Catholic 1

Large In Metro Area Catholic 4

Large In Metro Area Other Nonpublic 5

New York

Small Not In Metro Area Other Nonpublic 2

Small In Metro Area Catholic 1

Small In Metro Area Other Nonpublic 7

Large Not In Metro Area Catholic 1

Large In Metro Area Catholic 19

Large In Metro Area Other Nonpublic 15

Rhode Island

Small Not In Metro Area Other Nonpublic 1

Small In Metro Area Catholic 5

Small In Metro Area Other Nonpublic 7

Large Not In Metro Area Catholic 1

Large Not In Metro Area Other Nonpublic 1

Large In Metro Area Catholic 23

Large In Metro Area Other Nonpublic 8

Washington

Small Not In Metro Area Other Nonpublic 2

Small In Metro Area Other Nonpublic 6

Large Not In Metro Area Other Nonpublic 1

Large In Metro Area Catholic 6

Large In Metro Area Other Nonpublic 6

West Virginia

Small Not In Metro Area Catholic 1

Small Not In Metro Area Other Nonpublic 8

Small In Metro Area Catholic 2

Small In Metro Area Other Nonpublic 2

Large Not In Metro Area Catholic 1

Large Not In Metro Area Other Nonpublic 1

Large In Metro Area Catholic 3

Large In Metro Area Other Nonpublic 2

Wyoming

Small Not In Metro Area Other Nonpublic 8

Small In Metro Area Other Nonpublic 3

Large Not In Metro Area Catholic 1

Large Not In Metro Area Other Nonpublic 1

Large In Metro Area Catholic 2

Large In Metro Area Other Nonpublic 1
545

Table B-19

Weighted School Participation Rates and Sample Counts

Grade 4 Reading for Public Schools

Jurisdiction

Weighted

Participation

Rate After

Substitution

(%)

Number of

Schools in

Original Sample

Total Number of

Schools That

Participated

(Including

Substitutes)

Alabama 91 108 98

Arizona 98 111 108

Arkansas 97 107 102

California 80 107 84

Colorado 95 110 104

Connecticut 98 109 107

Delaware 100 65 65

District of Columbia 100 114 104

DoDEA/DoDDS 100 41 39

DoDEA/DDESS 100 104 103

Florida 99 105 103

Georgia 99 105 104

Hawaii 100 108 105

Illinois 84 107 89

Iowa 84 109 92

Kansas 70 112 79

Kentucky 92 108 99

Louisiana 100 110 109

Maine 96 119 106

Maryland 88 105 92

Massachusetts 88 111 95

Michigan 90 107 95

Minnesota 86 107 92

Mississippi 94 105 96

Missouri 99 110 105

Montana 78 115 83

Nevada 100 113 113

New Hampshire 70 109 74

New Mexico 99 110 109

New York 84 106 89

North Carolina 99 106 103

Oklahoma 100 110 109

Oregon 94 109 102

(continued)
546

Table B-19 (continued)

Weighted School Participation Rates and Sample Counts

Grade 4 Reading for Public Schools

Jurisdiction

Weighted

Participation

Rate After

Substitution

(%)

Number of

Schools in

Original Sample

Total Number of

Schools That

Participated

(Including

Substitutes)

Rhode Island 100 107 106

South Carolina 97 105 98

Tennessee 97 108 103

Texas 97 108 102

Utah 100 107 106

Virgin Islands 100 24 24

Virginia 100 106 106

Washington 89 107 93

West Virginia 100 111 110

Wisconsin 82 108 88

Wyoming 100 121 117
547

Table B-20

Weighted School Participation Rates and Sample Counts

Grade 4 Reading for Nonpublic Schools

Jurisdiction

Weighted

Participation

Rate After

Substitution

(%)

Number of

Schools in

Original Sample

Total Number of

Schools That

Participated

(Including

Substitutes)

Arkansas 76 13 7

Colorado 86 16 11

Connecticut 82 18 12

Florida 78 20 12

Georgia 80 13 9

Hawaii 85 23 18

Illinois 70 25 14

Iowa 92 18 16

Louisiana 81 24 17

Maine 80 16 10

Maryland 66 24 14

Massachusetts 84 19 13

Michigan 73 21 13

Minnesota 81 22 17

Mississippi 74 14 10

Missouri 80 24 17

Montana 88 14 5

Nebraska 99 26 22

Nevada 89 10 9

New Mexico 91 20 16

New York 67 27 17

North Carolina 90 13 10

Rhode Island 96 23 18

South Carolina 96 15 11

Utah 75 10 8

Virgin Islands 96 27 24

Washington 77 15 10

West Virginia 86 13 7

Wisconsin 75 36 24

Wyoming 96 11 7
548

Table B-21

Weighted School Participation Rates and Sample Counts

Grade 8 Reading and Writing for Public Schools

Reading Writing

Jurisdiction

Weighted

Percentage

School

Participation

After

Substitution

Number of

Schools in

Original

Sample

Total Number

of Schools

That

Participated

(Including

Substitutes)

Weighted

Percentage

School

Participation

After

Substitution

Number of

Schools in

Original

Sample

Total Number

of Schools

That

Participated

(Including

Substitutes)

Alabama 91 113 102 90 113 101

Arizona 97 110 105 98 111 104

Arkansas 97 113 105 97 114 105

California 84 108 90 83 107 88

Colorado 97 110 106 97 110 106

Connecticut 99 107 104 99 107 104

Delaware 100 32 31 100 31 30

District of Columbia 100 37 30 100 38 31

DoDEA/DoDDS 100 11 11 100 12 12

DoDEA/DDESS 100 59 57 100 57 55

Florida 100 104 103 100 105 104

Georgia 100 106 104 100 106 104

Hawaii 100 55 51 100 53 49

Illinois 81 111 89 80 110 88

Kansas 71 116 81 — — —

Kentucky 87 108 91 87 109 89

Louisiana 100 112 110 100 113 112

Maine 97 104 97 97 104 98

Maryland 85 107 88 86 108 89

Massachusetts 89 105 91 89 106 92

Minnesota 74 109 81 74 108 80

Mississippi 92 103 92 92 104 92

Missouri 97 117 109 97 115 108

Montana 78 92 59 78 93 62

Nevada 99 58 55 99 58 55

New Mexico 96 93 88 96 94 89

New York 77 109 81 77 108 81

North Carolina 100 107 104 100 107 104

Oklahoma 100 105 103 100 104 101

Oregon 88 109 96 88 109 96

Rhode Island 100 51 50 100 51 50

(continued)
549

Table B-21 (continued)

Weighted School Participation Rates and Sample Counts

Grade 8 Reading and Writing for Public Schools

Reading Writing

Jurisdiction

Weighted

Percentage

School

Participation

After

Substitution

Number of

Schools in

Original

Sample

Total Number

of Schools

That

Participated

(Including

Substitutes)

Weighted

Percentage

School

Participation

After

Substitution

Number of

Schools in

Original

Sample

Total Number

of Schools

That

Participated

(Including

Substitutes)

South Carolina 95 105 99 94 105 99

Tennessee 89 109 95 89 109 95

Texas 96 109 100 96 108 100

Utah 100 96 94 100 96 94

Virgin Islands 100 6 6 100 6 6

Virginia 100 104 103 100 104 103

Washington 86 108 93 87 107 92

West Virginia 100 107 106 100 107 106

Wisconsin 73 111 81 73 111 80

Wyoming 95 69 67 100 70 65
550

Table B-22

Weighted School Participation Rates and Sample Counts

Grade 8 Reading and Writing for Nonpublic Schools

Reading Writing

Jurisdiction

Weighted

Percentage

School

Participation

After

Substitution

Number of

Schools in

Original

Sample

Total Number

of Schools

That

Participated

(Including

Substitutes)

Weighted

Percentage

School

Participation

After

Substitution

Number of

Schools in

Original

Sample

Total Number

of Schools

That

Participated

(Including

Substitutes)

Arizona 78 13 9 76 12 7

Arkansas 86 12 8 83 11 9

California 79 20 13 84 20 9

Colorado 100 15 10 78 14 8

Connecticut 84 27 19 70 23 13

Florida 74 19 11 85 19 11

Georgia 100 13 11 88 13 9

Illinois 61 28 14 59 28 14

Louisiana 78 31 22 90 33 27

Maine 78 16 8 58 17 5

Maryland 82 23 14 79 25 16

Massachusetts 65 22 10 70 25 15

Missouri 90 23 16 69 26 16

Montana 82 15 9 100 15 13

Nebraska 89 25 20 92 25 21

Nevada 88 10 8 95 9 7

New Mexico 83 18 13 80 16 11

New York 73 31 18 80 29 19

North Carolina 84 13 9 78 13 8

Rhode Island 85 30 21 82 30 20

Virgin Islands 100 15 14 82 14 10

Washington 100 15 11 92 13 8

West Virginia 87 10 7 85 11 7

Wyoming 95 9 6 77 9 6
551

Table B-23

Weighted Student Participation Rates, Exclusion Rates, and Sample Counts for the Reporting Samples*

Grade 4 Reading for Public Schools

Jurisdiction

Weighted

Percentage

Student

Participation

After Makeups

Number of

Students

Sampled

Number of

Nonaccommodated

Students†

Assessed

Total

Number of

Students

Assessed

Weighted

Percentage of

Students

Identified as

SD or LEP

Weighted

Percentage

of Students

Excluded

Alabama 96 2,819 2,506 2,506 15 9

Arizona 94 2,901 2,432 2,432 22 10

Arkansas 95 2,956 2,580 2,580 11 5

California 93 2,112 1,722 1,722 30 15

Colorado 94 2,899 2,528 2,528 15 7

Connecticut 94 2,940 2,484 2,484 17 12

Delaware 94 2,684 2,309 2,309 18 8

District of Columbia 93 2,815 2,353 2,353 15 10

DoDEA/DDESS 96 3,122 2,647 2,647 10 5

DoDEA/DDESS 94 3,175 2,609 2,609 8 5

Florida 94 2,953 2,463 2,463 17 9

Georgia 96 3,051 2,647 2,647 10 7

Hawaii 95 2,943 2,600 2,600 15 5

Illinois 95 2,459 2,161 2,161 13 9

Iowa 96 2,456 2,232 2,232 15 8

Kansas 93 2,116 1,845 1,845 13 6

Kentucky 96 2,787 2,442 2,442 13 10

Louisiana 95 3,029 2,587 2,587 16 13

Maine 93 2,687 2,355 2,355 14 8

Maryland 95 2,600 2,241 2,241 13 10

Massachusetts 95 2,604 2,306 2,306 20 9

Michigan 93 2,723 2,365 2,365 10 7

Minnesota 94 2,535 2,271 2,271 15 4

Mississippi 95 2,842 2,552 2,552 7 4

Missouri 95 2,858 2,482 2,482 13 7

Montana 95 2,024 1,847 1,847 9 4

Nevada 94 3,159 2,597 2,597 20 12

New Hampshire 93 2,056 1,805 1,805 15 5

New Mexico 94 2,726 2,284 2,284 28 11

New York 95 2,474 2,221 2,221 13 8

North Carolina 94 2,960 2,514 2,514 16 11

Oklahoma 95 3,035 2,576 2,576 16 10

* The reporting samples for reading include all non SD/LEP students plus SD/LEP students from sample type 2.

† No accommodated students were assessed.

(continued)
552

Table B-23 (continued)

Weighted Student Participation Rates, Exclusion Rates, and Sample Counts for fhe Reporting Samples*

Grade 4 Reading for Public Schools

Jurisdiction

Weighted

Percentage

Student

Participation

After Makeups

Number of

Students

Sampled

Number of

Nonaccommodated

Students† 

Assessed

Total

Number of

Students

Assessed

Weighted

Percentage of

Students

Identified as

SD or LEP

Weighted

Percentage

of Students

Excluded

Oregon 95 2,783 2,396 2,396 23 8

Rhode Island 94 2,919 2,533 2,533 21 8

South Carolina 95 2,799 2,411 2,411 18 12

Tennessee 94 2,972 2,627 2,627 14 5

Texas 95 2,694 2,241 2,241 28 14

Utah 95 3,034 2,678 2,678 15 6

Virgin Islands 96 1,645 1,469 1,469 10 8

Virginia 95 2,999 2,602 2,602 16 9

Washington 94 2,635 2,378 2,378 17 6

West Virginia 94 2,927 2,518 2,518 13 10

Wisconsin 95 2,343 2,071 2,071 14 9

Wyoming 95 2,948 2,642 2,642 14 4

* The reporting samples for reading include all non SD/LEP students plus SD/LEP students from sample type 2.

† No accommodated students were assessed.
553

Table B-24

Weighted Student Participation Rates, Exclusion Rates, and Sample Counts for the Reporting Samples*

Grade 4 Reading for Nonpublic Schools

Jurisdiction

Weighted

Percentage

Student

Participation

After Makeups

Number of

Students

Sampled

Number of

Nonaccommodated

Students† 

Assessed

Total

Number of

Students

Assessed

Weighted

Percentage of

Students

Identified as

SD or LEP

Weighted

Percentage

of Students

Excluded

Arkansas 97 168 163 163 0 0

Colorado 95 233 221 221 2 0

Connecticut 95 277 261 261 7 1

Florida 94 291 271 271 5 1

Georgia 97 298 266 266 2 1

Hawaii 97 395 379 379 1 0

Illinois 96 368 353 353 0 0

Iowa 98 336 329 329 3 0

Louisiana 95 439 413 413 5 2

Maine 94 135 127 127 3 0

Maryland 98 306 297 297 4 1

Massachusetts 94 308 282 282 3 3

Michigan 95 280 264 264 1 1

Minnesota 95 356 335 335 3 1

Mississippi 98 230 224 224 2 0

Missouri 96 333 317 317 2 1

Montana 94 108 99 99 7 2

Nebraska 96 498 476 476 2 1

Nevada 95 159 150 150 1 1

New Mexico 95 246 221 221 13 6

New York 96 404 377 377 2 2

North Carolina 95 246 227 227 5 0

Rhode Island 95 405 379 379 3 0

South Carolina 94 245 227 227 2 1

Utah 94 114 107 107 0 0

Virgin Islands 96 444 426 426 0 0

Washington 95 186 175 175 2 0

West Virginia 100 125 124 124 0 0

Wisconsin 96 443 424 424 0 0

Wyoming 91 105 95 95 0 0

* The reporting samples for reading include all non SD/LEP students plus SD/LEP students from sample type 2.

† No accommodated students were assessed.
554

Table B-25

Weighted Student Participation Rates, Exclusion Rates, and Sample Counts for the Reporting Samples*

Grade 8 Reading for Public Schools

Jurisdiction

Weighted

Percentage

Student

Participation

After Makeups

Number of

Students

Sampled

Number of

Nonaccommodated

Students†

Assessed

Total

Number of

Students

Assessed

Weighted

Percentage of

Students

Identified as

SD or LEP

Weighted

Percentage

of Students

Excluded

Alabama 93 2,820 2,428 2,428 14 7

Arizona 91 2,788 2,325 2,325 15 6

Arkansas 92 2,904 2,412 2,412 12 7

California 91 2,331 1,944 1,944 23 8

Colorado 91 2,971 2,542 2,542 15 5

Connecticut 91 2,928 2,489 2,489 15 8

Delaware 91 2,396 1,987 1,987 17 8

District of Columbia 86 1,968 1,528 1,528 15 9

DoDEA/DoDDS 95 732 610 610 13 7

DoDEA/DDESS 94 2,578 2,138 2,138 7 3

Florida 89 2,928 2,392 2,392 16 5

Georgia 90 3,007 2,499 2,499 12 6

Hawaii 91 2,877 2,461 2,461 14 6

Illinois 93 2,316 2,051 2,051 12 6

Kansas 92 2,164 1,857 1,857 11 5

Kentucky 93 2,649 2,282 2,282 11 5

Louisiana 91 3,001 2,479 2,479 13 9

Maine 92 2,712 2,363 2,363 13 6

Maryland 89 2,539 2,087 2,087 12 7

Massachusetts 91 2,495 2,141 2,141 16 6

Minnesota 93 2,218 1,926 1,926 12 4

Mississippi 92 2,676 2,274 2,274 11 7

Missouri 92 2,935 2,526 2,526 13 6

Montana 92 2,142 1,877 1,877 10 3

Nevada 91 3,020 2,449 2,449 16 8

New Mexico 90 2,700 2,183 2,183 20 7

New York 88 2,244 1,842 1,842 14 9

North Carolina 92 2,954 2,487 2,487 14 9

Oklahoma 91 2,682 2,182 2,182 14 9

Oregon 89 2,624 2,169 2,169 14 4

Rhode Island 88 2,838 2,393 2,393 17 5

* The reporting samples for reading include all non SD/LEP students plus SD/LEP students from sample type 2.

† No accommodated students were assessed. 

(continued)
555

Table B-25 (continued)

Weighted Student Participation Rates, Exclusion Rates, and Sample Counts for the Reporting Samples*

Grade 8 Reading for Public Schools

Jurisdiction

Weighted

Percentage

Student

Participation

After Makeups

Number of

Students

Sampled

Number of

Nonaccommodated

Students†

Assessed

Total

Number of

Students

Assessed

Weighted

Percentage of

Students

Identified as

SD or LEP

Weighted

Percentage

of Students

Excluded

South Carolina 93 2,838 2,429 2,429 12 6

Tennessee 90 2,558 2,159 2,159 14 4

Texas 93 2,730 2,318 2,318 18 7

Utah 90 3,004 2,510 2,510 12 5

Virgin Islands 88 767 643 643 5 5

Virginia 91 2,958 2,493 2,493 13 7

Washington 91 2,573 2,205 2,205 12 4

West Virginia 91 2,916 2,442 2,442 12 8

Wisconsin 92 2,209 1,918 1,918 14 8

Wyoming 91 2,891 2,509 2,509 11 2

* The reporting samples for reading include all non SD/LEP students plus SD/LEP students from sample type 2.

† No accommodated students were assessed.
556

Table B-26

Weighted Student Participation Rates, Exclusion Rates, and Sample Counts for the Reporting Samples*

Grade 8 Reading for Nonpublic Schools

Jurisdiction

Weighted

Percentage

Student

Participation

After Makeups

Number of

Students

Sampled

Number of

Nonaccommodated

Students† 

Assessed

Total

Number of

Students

Assessed

Weighted

Percentage of

Students

Identified as

SD or LEP

Weighted

Percentage

of Students

Excluded

Arizona 90 204 174 174 17 2

Arkansas 97 140 132 132 4 4

California 97 305 295 295 1 0

Colorado 97 159 154 154 0 0

Connecticut 95 367 343 343 1 0

Florida 93 204 189 189 1 0

Georgia 95 194 185 185 0 0

Illinois 98 298 288 288 3 1

Louisiana 95 480 453 453 4 0

Maine 96 82 78 78 0 0

Maryland 96 344 326 326 1 0

Massachusetts 96 191 183 183 0 0

Missouri 96 300 288 288 0 0

Montana 98 151 147 147 0 0

Nebraska 95 384 362 362 2 2

Nevada 96 138 129 129 4 2

New Mexico 95 184 166 166 26 2

New York 95 368 345 345 4 2

North Carolina 96 259 238 238 8 5

Rhode Island 94 423 401 401 2 1

Virgin Islands 96 238 228 228 0 0

Washington 94 247 229 229 7 3

West Virginia 93 105 96 96 0 0

Wyoming 99 52 51 51 0 0

* The reporting samples for reading include all non SD/LEP students plus SD/LEP students from sample type 2.

† No accommodated students were assessed.
Table B-27

Weighted Student Participation Rates, Exclusion Rates, and Sample Counts for the Reporting Sample

Grade 8 Writing for Public Schools

Jurisdiction

Weighted

Percentage Student

Participation After

Makeups

Number of

Students

Sampled

Number of

Nonaccommodated

StudentsAssessed

Number of

Accommodated

Students

Assessed

Total

Number of

StudentsAssessed

Weighte

Percentag

Students Ide

as SD or L

Alabama 92 2,938 2,427 22 2,449 12 Arizona 89 3,111 2,437 62 2,499 17 Arkansas 92 3,041 2,428 34 2,462 13 California 92 2,618 2,122 35 2,157 23 Colorado 91 3,197 2,619 78 2,697 13 Connecticut 90 3,186 2,514 78 2,592 15 Delaware 91 2,522 2,048 71 2,119 14 District of Columbia 85 2,115 1,571 21 1,592 13 DoDEA/DoDDS 95 765 628 22 650 10 DoDEA/DDESS 93 2,650 2,144 38 2,182 7 Florida 89 3,222 2,518 56 2,574 16 Georgia 90 3,208 2,550 55 2,605 11 Hawaii 92 3,092 2,584 63 2,647 15 Illinois 92 2,457 2,096 49 2,145 12 Kentucky 93 2,713 2,235 106 2,341 10 Louisiana 91 3,222 2,530 123 2,653 13 Maine 91 2,970 2,431 77 2,508 14 Maryland 89 2,726 2,119 144 2,263 13 Massachusetts 92 2,806 2,273 126 2,399 17 Minnesota 90 2,344 1,923 57 1,980 14 Mississippi 92 2,839 2,378 23 2,401 9 Missouri 92 3,080 2,510 111 2,621 13 Montana 93 2,326 1,981 43 2,024 11 * The reporting samples for writing included both accommodated and nonaccommodated students.

(continued)
558

Table B-27 (continued)

Weighted Student Participation Rates, Exclusion Rates, and Sample Counts for the Reporting Sample

Grade 8 Writing for Public Schools

Jurisdiction

Weighted

Percentage Student

Participation After

Makeups

Number of

Students

Sampled

Number of

Nonaccommodated

StudentsAssessed

Number of

Accommodated

Students

Assessed

Total

Number of

StudentsAssessed

Weighte

Percentag

Students Ide

as SD or L

Nevada 89 3,258 2,482 71 2,553 16 New Mexico 89 3,109 2,339 87 2,426 23 New York 87 2,443 1,865 116 1,981 15 North Carolina 93 3,147 2,505 164 2,669 14 Oklahoma 92 2,868 2,233 25 2,258 13 Oregon 89 2,851 2,257 66 2,323 15 Rhode Island 89 3,071 2,441 75 2,516 17 South Carolina 91 2,993 2,425 44 2,469 12 Tennessee 91 2,739 2,253 22 2,275 13 Texas 93 3,068 2,467 63 2,530 19 Utah 90 3,152 2,564 24 2,588 10 Virgin Islands 87 777 614 0 614 8 Virginia 91 3,156 2,523 82 2,605 14 Washington 89 2,753 2,223 63 2,286 13 West Virginia 91 3,168 2,525 86 2,611 14 Wisconsin 92 2,332 1,952 54 2,006 11 Wyoming 92 3,142 2,668 58 2,726 9 * The reporting samples for writing included both accommodated and nonaccommodated students.
Table B-28

Weighted Student Participation Rates, Exclusion Rates, and Sample Counts for the Reporting Sample

Grade 8 Writing for Nonpublic Schools

Jurisdiction

Weighted

Percentage Student

Participation After

Makeups

Number of

Students

Sampled

Number of

Nonaccommodated

StudentsAssessed

Number of

Accommodated

Students

Assessed

Total

Number of

StudentsAssessed

Weighte

Percentag

Students Ide

as SD or L

Arizona 96 149 129 1 130 14 Arkansas 96 146 140 0 140 1 California 98 232 224 0 224 0 Colorado 93 147 137 0 137 14 Connecticut 93 261 235 5 240 5 Florida 94 235 210 3 213 5 Georgia 94 156 144 0 144 1 Illinois 96 328 313 1 314 1 Louisiana 97 603 570 10 580 4 Maine 96 100 95 0 95 0 Maryland 96 367 347 3 350 1 Massachusetts 92 288 255 8 263 5 Missouri 96 314 300 3 303 2 Montana 96 217 203 3 206 3 Nebraska 97 370 346 8 354 2 Nevada 92 122 108 0 108 2 New Mexico 96 223 198 6 204 16 New York 96 403 378 2 380 9 North Carolina 95 271 247 1 248 2 Rhode Island 96 453 434 0 434 1 Virgin Islands 98 198 193 0 193 0 Washington 94 168 153 2 155 4 West Virginia 98 122 117 0 117 2 Wyoming 95 64 59 2 61 9 * The reporting samples for writing included both accommodated and nonaccommodated students.
560

Table B-29

Results of Logistic Regression Analysis of School Nonresponse – Grade 4 Reading

Model with All Variables 

Jurisdiction

School

Participation

Rate (%)

Percent of

Population

Covered

by Model

Degrees

of

Freedom Significance Significant Variables

F

California 83.15 85.83 8 p=0.646 none Illinois 80.28 100.00 9 p=0.003 percent black, p=0.006 Kentucky 87.14 73.23 9 p=0.677 none Massachusetts 89.28 77.42 10 p=0.218 none Maryland 86.42 81.62 8 p=0.494 none Minnesota 73.51 100.00 7 p=0.018 estimated grade enrollment, p=0.001 Montana 77.60 82.51 6 p=0.045 None New York 77.27 100.00 8 p=0.099 None Oregon 87.53 86.66 11 p=0.079 estimated grade enrollment, p=0.038 Tennessee 89.03 60.07 8 p=0.354 None Washington 86.59 95.16 11 p=0.506 None Wisconsin 72.91 100.00 8 p=0.246 percent of black students, p=0.007 
Table B-30

Results of Logistic Regression Analysis of School Nonresponse – Grade 8 Reading

Model with All Variables 

Jurisdiction

School

Participation

Rate (%)

Percent of

Population

Covered

by Model

Degrees

of

Freedom Significance Significant Variables

F

California 83.74 79.87 7 p=0.400 none Illinois 81.12 100.00 9 p=0.001 none Kansas 70.60 100.00 9 p=0.748 none Kentucky 87.32 72.63 9 p=0.701 none Massachusetts 89.20 77.59 10 p=0.818 none Maryland 85.45 81.62 8 p=0.413 none Minnesota 73.73 100.00 7 p=0.009 estimated grade enrollment, p=0.001 Montana 77.81 79.74 6 p=0.008 nonresponse cell 5, p=0.028 New York 77.27 100.00 8 p=0.198 none Oregon 87.53 86.66 11 p=0.000 none Tennessee 89.03 60.09 8 p=0.203 none Washington 86.13 95.22 11 p=0.701 none Wisconsin 73.18 100.00 8 p=0.331 percent black, p=0.013 
562

 Table B-31

Results of Logistic Regression Analysis of School Nonresponse – Grade 8 Writing

Model with All Variables 

Jurisdiction

School

Participation

Rate (%)

Percent of

Population

Covered

by Model

Degrees

of

Freedom Significance Significant Variables

D

F

California 83.15 85.83 8 p=0.646 none Illinois 80.28 100.00 9 p=0.003 percent black, p=0.006 Kentucky 87.14 73.23 9 p=0.677 None Massachusetts 89.28 77.42 10 p=0.218 None Maryland 86.42 81.62 8 p=0.494 None Minnesota 73.51 100.00 7 p=0.018 estimated grade enrollment, p=0.001 Montana 77.60 82.51 6 p=0.045 None New York 77.27 100.00 8 p=0.099 None Oregon 87.53 86.66 11 p=0.079 estimated grade enrollment, p=0.038 Tennessee 89.03 60.07 8 p=0.354 None Washington 86.59 95.16 11 p=0.506 None Wisconsin 72.91 100.00 8 p=0.246 percent of black students, p=0.007 
563

Appendix C

CONSTRUCTED-RESPONSE ITEM SCORE STATISTICS

This appendix contains information about the constructed-response items included in the scaling

of data from the 1998 assessments of reading, writing, and civics. For each subject area and grade, the

information in the tables includes the NAEP item numbers for each of the constructed-response items

included in scaling, and the block that contains the item. The tables also indicate the codes from the

NAEP database that denote the range of responses and the correct responses where appropriate. A

portion of the responses to the constructed-response items were scored twice for the purpose of

examining rater reliability. For each item, the number of papers with responses that were scored a second

time is listed, along with the percent agreement between raters and an index of reliability based on those

responses. Cohen’s Kappa (Cohen, 1968) is the reliability estimate used for dichotomized items. For

items that are not dichotomized (i.e., polytomous items), the intraclass correlation coefficient is used as

the index of reliability. See Chapter 9 for more information about score reliability for constructedresponse items.

565

Table C-1

Score Range, Percent Agreement, and Cohen’s Kappa*

for the Dichotomously Scored Constructed-Response Reading Items

Used in 1998 National Main Assessment Scaling, Grade 4 †

Item Block

Range of

Response

Codes

Correct

Response

Codes

Sample

Size

Percent

Agreement

Cohen’s

Kappa

R012102 R4 1-2 2 1,923 98 0.970

R012104 R4 1-2 2 1,900 96 0.910

R012106 R4 1-2 2 1,862 93 0.859

R012108 R4 1-2 2 1,761 97 0.920

R012109 R4 1-2 2 1,752 97 0.922

R012112 R4 1-2 2 1,299 94 0.870

R012201 R6 1-2 2 1,925 96 0.923

R012206 R6 1-2 2 1,697 98 0.956

R012208 R6 1-2 2 1,547 93 0.852

R012210 R6 1-2 2 1,452 95 0.820

R012503 R10 1-2 2 1,921 96 0.922

R012504 R10 1-2 2 1,897 98 0.969

R012506 R10 1-2 2 1,865 97 0.949

R012508 R10 1-2 2 1,794 98 0.956

R012511 R10 1-2 2 1,637 97 0.941

R012601 R5 1-2 2 1,897 90 0.759

R012604 R5 1-2 2 1,855 93 0.834

R012611 R5 1-2 2 1,475 89 0.779

R012702 R7 1-2 2 1,908 97 0.913

R012703 R7 1-2 2 1,878 94 0.877

R012705 R7 1-2 2 1,798 94 0.860

R012706 R7 1-2 2 1,765 87 0.705

R012710 R7 1-2 2 1,227 92 0.839

R015802 R9 1-2 2 1,909 92 0.786

R017001 R3 1-2 2 2,035 96 0.902

R017004 R3 1-2 2 1,988 97 0.927

R017006 R3 1-2 2 1,938 96 0.908 , * Cohen’s Kappa is a measure of reliability that is appropriate for items that are dichotomized. These items are

dichotomized into right and wrong.

†

 Rescored responses from the national and state assessment samples contributed to these statistics.
566

Table C-2

Score Range, Percent Agreement, and Cohen’s Kappa*

for the Dichotomously Scored Constructed-Response Reading Items

Used in 1998 National Main Assessment Scaling, Grade 8 †

Item Block

Range of

Response

Codes

Correct

Response

Codes

Sample

Size

Percent

Agreement

Cohen’s

Kappa

R012601 R5 1-2 2 1,323 85 0.713

R012604 R5 1-2 2 1,326 91 0.824

R012611 R5 1-2 2 1,258 87 0.699

R012702 R7 1-2 2 1,233 97 0.872

R012703 R7 1-2 2 1,222 91 0.819

R012705 R7 1-2 2 1,231 89 0.785

R012706 R7 1-2 2 1,229 85 0.703

R012710 R7 1-2 2 1,180 92 0.829

R012713 R7 1-2 2 1,065 99 0.979

R013001 R11 1-2 2 1,237 94 0.816

R013003 R11 1-2 2 1,239 100 0.996

R013005 R11 1-2 2 1,215 96 0.891

R013007 R11 1-2 2 1,183 99 0.957

R013008 R11 1-2 2 1,152 91 0.832

R013009 R11 1-2 2 1,131 96 0.874

R013010 R11 1-2 2 1,115 97 0.920

R013011 R11 1-2 2 1,098 86 0.747

R013203 R6 1-2 2 1,236 99 0.932

R013205 R6 1-2 2 1,232 96 0.836

R013207 R6 1-2 2 1,226 90 0.772

R013209 R6 1-2 2 1,210 98 0.944

R013211 R6 1-2 2 1,088 84 0.665

R013402 R10 1-2 2 1,233 98 0.964

R013405 R10 1-2 2 1,229 96 0.922

R013407 R10 1-2 2 1,184 95 0.900

R013409 R10 1-2 2 1,151 96 0.924

R013411 R10 1-2 2 1,049 94 0.868

R013412 R10 1-2 2 998 94 0.741

R015901 R4 1-2 2 1,219 94 0.894

R017101 R3 1-2 2 1,238 94 0.891

R017108 R3 1-2 2 1,136 98 0.972

R017210 R8 1-2 2 1,075 91 0.705 , * Cohen’s Kappa is a measure of reliability that is appropriate for items that are dichotomized. These items are

dichotomized into right and wrong.

†

 Rescored responses from the national and state assessment samples contributed to these statistics.
567

Table C-3

Score Range, Percent Agreement, and Cohen’s Kappa*

for the Dichotomously Scored Constructed-Response Reading Items

Used in 1998 National Main Assessment Scaling, Grade 12

Item Block

Range of

Response

Codes

Correct

Response

Codes

Sample

Size

Percent

Agreement

Cohen’s

Kappa

R013203 R6 1-2 2 536 100 0.967

R013205 R6 1-2 2 529 98 0.873

R013207 R6 1-2 2 527 89 0.666

R013209 R6 1-2 2 520 97 0.939

R013211 R6 1-2 2 496 85 0.709

R013402 R10 1-2 2 508 98 0.962

R013405 R10 1-2 2 503 94 0.835

R013407 R10 1-2 2 485 94 0.839

R013409 R10 1-2 2 475 94 0.844

R013411 R10 1-2 2 426 94 0.798

R013412 R10 1-2 2 416 89 0.596

R013501 R4 1-2 2 489 92 0.864

R013503 R4 1-2 2 485 97 0.949

R013505 R4 1-2 2 467 89 0.749

R013508 R4 1-2 2 358 90 0.797

R013509 R4 1-2 2 333 92 0.831

R013701 R7 1-2 2 494 84 0.672

R013702 R7 1-2 2 496 83 0.677

R013704 R7 1-2 2 493 90 0.785

R013708 R7 1-2 2 474 86 0.734

R013710 R7 1-2 2 460 95 0.890

R013712 R7 1-2 2 405 84 0.689

R013902 R11 1-2 2 508 97 0.933

R013903 R11 1-2 2 508 98 0.965

R013904 R11 1-2 2 505 98 0.932

R013906 R11 1-2 2 505 93 0.882

R013908 R11 1-2 2 503 89 0.806

R013910 R11 1-2 2 497 97 0.945

R013913 R11 1-2 2 488 94 0.889

R015503 R13 1-2 2 506 95 0.714

R015505 R13 1-2 2 502 89 0.804

R015509 R13 1-2 2 504 90 0.838

R015512 R13 1-2 2 498 94 0.863

R017101 R3 1-2 2 496 96 0.908

R017108 R3 1-2 2 464 96 0.949

* Cohen’s Kappa is a measure of reliability that is appropriate for items that are dichotomized. These items are

dichotomized into right and wrong.
568

Table C-4

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Reading Items

Used in 1998 National Main Assessment Scaling, Grade 4 *

Item Block

Range of

Response

Codes†

Sample

Size

Percent

Agreement

Intraclass

Correlation

R012111 R4 1-4 1,555 91 0.946

R012204 R6 1-4 1,894 81 0.906

R012512 R10 1-4 1,591 90 0.957

R012607 R5 1-4 1,770 85 0.867

R012708 R7 1-4 1,637 87 0.908

R015702 R8 1-3 2,036 87 0.841

R015703 R8 1-3 2,017 89 0.862

R015704 R8 1-3 1,978 84 0.870

R015705 R8 1-3 1,963 90 0.942

R015707 R8 1-4 1,834 89 0.904

R015709 R8 1-3 1,558 88 0.881

R015803 R9 1-3 1,891 88 0.841

R015804 R9 1-4 1,845 83 0.873

R015806 R9 1-3 1,706 87 0.884

R015807 R9 1-3 1,548 87 0.880

R015809 R9 1-3 1,389 89 0.858

R017003 R3 1-3 2,019 90 0.917

R017007 R3 1-4 1,868 78 0.899

R017009 R3 1-3 1,613 87 0.821

*

 Rescored responses from the national and state assessment samples contributed to these statistics.

†

 Response codes used here are from the scoring process. They do not reflect the credit students received for the

responses. See the subject-area analysis chapters (Reading-Chapter 15, Writing-Chapter 19; Civics-Chapter 23)

for the scoring categories used in all other analyses.
569

Table C-5

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Reading Items

Used in 1998 National Main Assessment Scaling, Grade 8 *

Item Block

Range of

Response

Codes

Sample

Size

Percent

Agreement

Intraclass

Correlation

R012607 R5 1-4 1,315 84 0.882

R012708 R7 1-4 1,210 86 0.913

R013004 R11 1-4 1,234 88 0.950

R013201 R6 1-4 1,244 80 0.867

R013212 R6 1-4 1,106 87 0.914

R013403 R10 1-4 1,233 98 0.987

R013406 R10 1-4 1,217 91 0.963

R015902 R4 1-3 1,222 94 0.949

R015904 R4 1-3 1,203 92 0.881

R015905 R4 1-3 1,157 91 0.918

R015906 R4 1-4 1,155 82 0.781

R015907 R4 1-3 1,063 84 0.791

R015908 R4 1-3 899 88 0.888

R016101 R9 1-3 1,317 94 0.953

R016104 R9 1-3 1,315 91 0.894

R016107 R9 1-3 1,151 94 0.950

R016108 R9 1-3 1,250 85 0.837

R016109 R9 1-3 1,248 93 0.911

R016201 R13 1-3 1,251 98 0.824

R016202 R13 1-3 1,247 94 0.886

R016204 R13 1-4 1,243 91 0.831

R016205 R13 1-3 1,244 92 0.906

R016207 R13 1-3 1,248 96 0.977

R016210 R13 1-4 1,112 82 0.839

R016211 R13 1-3 1,220 93 0.890

R016212 R13 1-3 1,189 92 0.939

R016213 R13 1-3 1,201 89 0.818

R017102 R3 1-3 1,240 90 0.929

R017104 R3 1-3 1,228 96 0.964

R017105 R3 1-4 1,219 84 0.909

R017107 R3 1-3 1,171 88 0.925

R017110 R3 1-3 1,034 94 0.940

R017204 R8 1-3 1,230 84 0.822

R017205 R8 1-4 1,227 64 0.761

R017208 R8 1-3 1,167 92 0.928

*

 Rescored responses from the national and state assessment samples contributed to these statistics.
570

Table C-6

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Reading Items

Used in 1998 National Main Assessment Scaling, Grade 12

Item Block

Range of

Response

Codes

Sample

Size

Percent

Agreement

Intraclass

Correlation

R013201 R6 1-4 540 81 0.910

R013212 R6 1-4 497 86 0.914

R013403 R10 1-4 505 97 0.987

R013406 R10 1-4 498 88 0.960

R013506 R4 1-4 444 85 0.913

R013706 R7 1-4 488 80 0.852

R013915 R11 1-4 414 97 0.992

R015507 R13 1-4 505 83 0.927

R015514 R13 1-4 497 90 0.957

R016101 R9 1-3 549 93 0.950

R016104 R9 1-3 543 87 0.798

R016107 R9 1-3 486 94 0.933

R016108 R9 1-3 513 82 0.811

R016109 R9 1-3 518 94 0.930

R016301 R5 1-3 495 85 0.862

R016302 R5 1-3 491 83 0.828

R016303 R5 1-3 480 85 0.884

R016305 R5 1-3 459 77 0.774

R016306 R5 1-3 419 84 0.856

R016307 R5 1-3 381 81 0.864

R016308 R5 1-4 373 85 0.920

R016401 R8 1-3 497 85 0.850

R016402 R8 1-3 494 66 0.669

R016403 R8 1-3 497 82 0.840

R016404 R8 1-3 493 88 0.886

R016405 R8 1-3 490 94 0.934

R016407 R8 1-3 430 83 0.866

R016408 R8 1-4 450 86 0.916

R016501 R14 1-3 509 90 0.859

R016502 R14 1-3 491 92 0.881

R016601 R14 1-3 499 79 0.715

R016602 R14 1-3 491 89 0.904

R016603 R14 1-3 502 82 0.762

R016604 R14 1-3 493 83 0.785

R016605 R14 1-3 479 80 0.668

R016701 R14 1-4 488 81 0.821

(continued)
571

Table C-6 (continued)

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Reading Items

Used in 1998 National Main Assessment Scaling, Grade 12

Item Block

Range of

Response

Codes

Sample

Size

Percent

Agreement

Intraclass

Correlation

R017102 R3 1-3 498 87 0.930

R017104 R3 1-3 499 93 0.954

R017105 R3 1-4 487 79 0.899

R017107 R3 1-3 473 89 0.939

R017110 R3 1-3 421 91 0.924
572

Table C-7

Score Range, Percent Agreement, and Cohen’s Kappa*

for the Dichotomously Scored Constructed-Response Reading Items

from 1994 That Were Rescored in 1998, Grade 4

Item Block

Range of

Response

Codes

Correct

Response

Code

Sample

Size

Percent

Agreement

Cohen’s

Kappa

R012102 R4 1 - 2 2 1,004 95.518 0.918

R012104 R4 1 - 2 2 1,004 92.331 0.871

R012106 R4 1 - 2 2 1,004 87.351 0.792

R012108 R4 1 - 2 2 1,004 95.717 0.919

R012109 R4 1 - 2 2 1,004 94.323 0.898

R012112 R4 1 - 2 2 1,004 93.526 0.900

R012201 R6 1 - 2 2 995 93.266 0.871

R012206 R6 1 - 2 2 995 96.482 0.944

R012208 R6 1 - 2 2 995 92.965 0.890

R012210 R6 1 - 2 2 995 93.367 0.892

R012503 R10 1 - 2 2 887 88.726 0.797

R012504 R10 1 - 2 2 887 95.265 0.924

R012506 R10 1 - 2 2 887 92.559 0.883

R012508 R10 1 - 2 2 887 95.378 0.925

R012511 R10 1 - 2 2 887 93.574 0.896

R012601 R5 1 - 2 2 848 87.736 0.753

R012604 R5 1 - 2 2 848 94.222 0.889

R012611 R5 1 - 2 2 848 92.335 0.884

R012702 R7 1 - 2 2 1,151 93.571 0.860

R012703 R7 1 - 2 2 1,151 90.791 0.841

R012705 R7 1 - 2 2 1,151 93.831 0.886

R012706 R7 1 - 2 2 1,151 88.358 0.786

R012710 R7 1 - 2 2 1,151 95.743 0.930

R015802 R9 1 - 2 2 958 85.908 0.722

* Cohen’s Kappa is a measure of reliability that is appropriate for items that are dichotomized. These items are

dichotomized into right and wrong.
573

Table C-8

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Reading Items

from 1994 That Were Rescored in 1998, Grade 4

Item Block

Range of

Response

Codes

Sample

Size

Percent

Agreement

Intraclass

Correlation

R012111 R4 1 - 4 1,004 90.438 0.968

R012204 R6 1 - 4 995 78.291 0.914

R012512 R10 1 - 4 887 82.976 0.946

R012607 R5 1 - 4 848 86.792 0.880

R012708 R7 1 - 4 1,151 85.491 0.911

R015702 R8 1 - 3 908 83.921 0.858

R015703 R8 1 - 3 908 86.013 0.886

R015704 R8 1 - 3 908 82.159 0.894

R015705 R8 1 - 3 908 88.436 0.949

R015707 R8 1 - 4 908 86.344 0.913

R015709 R8 1 - 3 908 91.520 0.942

R015803 R9 1 - 3 958 84.760 0.855

R015804 R9 1 - 4 958 80.167 0.892

R015806 R9 1 - 3 958 81.315 0.888

R015807 R9 1 - 3 958 82.463 0.923

R015809 R9 1 - 3 958 87.265 0.936
574

Table C-9

Score Range, Percent Agreement, and Cohen’s Kappa*

for the Dichotomously Scored Constructed-Response Reading Items

from 1994 That Were Rescored in 1998, Grade 8

Item Block

Range of

Response

Codes

Correct

Response

Code

Sample

Size

Percent

Agreement

Cohen’s

Kappa

R012601 R5 1 - 2 2 1,090 86.330 0.742

R012604 R5 1 - 2 2 1,090 91.284 0.844

R012611 R5 1 - 2 2 1,090 89.083 0.786

R012702 R7 1 - 2 2 887 95.716 0.824

R012703 R7 1 - 2 2 887 83.766 0.706

R012705 R7 1 - 2 2 887 88.050 0.791

R012706 R7 1 - 2 2 887 83.315 0.678

R012710 R7 1 - 2 2 887 91.657 0.857

R012713 R7 1 - 2 2 887 98.760 0.979

R013001 R11 1 - 2 2 820 92.927 0.825

R013003 R11 1 - 2 2 820 99.146 0.983

R013005 R11 1 - 2 2 820 89.512 0.766

R013007 R11 1 - 2 2 820 97.927 0.953

R013008 R11 1 - 2 2 820 91.341 0.861

R013009 R11 1 - 2 2 820 92.561 0.838

R013010 R11 1 - 2 2 820 92.927 0.863

R013011 R11 1 - 2 2 820 84.512 0.760

R013203 R6 1 - 2 2 1,004 92.729 0.758

R013205 R6 1 - 2 2 1,004 94.920 0.883

R013207 R6 1 - 2 2 1,004 87.948 0.780

R013209 R6 1 - 2 2 1,004 96.215 0.910

R013211 R6 1 - 2 2 1,004 87.450 0.791

R013402 R10 1 - 2 2 824 97.937 0.962

R013405 R10 1 - 2 2 824 90.413 0.827

R013407 R10 1 - 2 2 824 97.816 0.961

R013409 R10 1 - 2 2 824 90.777 0.849

R013411 R10 1 - 2 2 824 93.447 0.890

R013412 R10 1 - 2 2 824 88.107 0.790

R015901 R4 1 - 2 2 973 90.236 0.834

* Cohen’s Kappa is a measure of reliability that is appropriate for items that are dichotomized. These items are

dichotomized into right and wrong.
575

Table C-10

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Reading Items

from 1994 That Were Rescored in 1998, Grade 8

Item Block

Range of

Response

Codes

Sample

Size

Percent

Agreement

Intraclass

Correlation

R012607 R5 1 - 4 1,090 77.523 0.848

R012708 R7 1 - 4 887 77.339 0.866

R013004 R11 1 - 4 820 63.659 0.856

R013201 R6 1 - 4 1,004 83.865 0.906

R013212 R6 1 - 4 1,004 89.044 0.923

R013403 R10 1 - 4 824 95.995 0.978

R013406 R10 1 - 4 824 85.194 0.946

R015902 R4 1 - 3 973 87.770 0.914

R015904 R4 1 - 3 973 87.359 0.821

R015905 R4 1 - 3 973 86.639 0.899

R015906 R4 1 - 4 973 73.895 0.807

R015907 R4 1 - 3 973 84.275 0.896

R015908 R4 1 - 3 973 86.228 0.913

R016101 R9 1 - 3 794 86.272 0.902

R016104 R9 1 - 3 794 83.879 0.798

R016107 R9 1 - 3 794 93.451 0.973

R016108 R9 1 - 3 794 80.101 0.863

R016109 R9 1 - 3 794 88.791 0.897

R016201 R13 1 - 3 794 94.081 0.804

R016202 R13 1 - 3 794 88.917 0.832

R016204 R13 1 - 4 794 87.531 0.851

R016205 R13 1 - 3 794 87.154 0.886

R016207 R13 1 - 3 794 88.665 0.930

R016210 R13 1 - 4 794 73.552 0.890

R016211 R13 1 - 3 794 85.642 0.851

R016212 R13 1 - 3 794 89.924 0.944

R016213 R13 1 - 3 794 87.406 0.914
576

Table C-11

Score Range, Percent Agreement, and Cohen’s Kappa*

for the Dichotomously Scored Constructed-Response Reading Items

from 1994 That Were Rescored in 1998, Grade 12

Item Block

Range of

Response

Codes

Correct

Response

Code

Sample

Size

Percent

Agreement

Cohen’s

Kappa

R013203 R6 1 - 2 2 987 96.150 0.775

R013205 R6 1 - 2 2 987 98.176 0.910

R013207 R6 1 - 2 2 987 87.335 0.691

R013209 R6 1 - 2 2 987 96.150 0.926

R013211 R6 1 - 2 2 987 83.992 0.736

R013402 R10 1 - 2 2 716 97.626 0.952

R013405 R10 1 - 2 2 716 91.480 0.801

R013407 R10 1 - 2 2 716 94.832 0.887

R013409 R10 1 - 2 2 716 90.922 0.801

R013411 R10 1 - 2 2 716 94.972 0.894

R013412 R10 1 - 2 2 716 86.173 0.740

R013501 R4 1 - 2 2 1,074 90.782 0.838

R013503 R4 1 - 2 2 1,074 93.948 0.905

R013505 R4 1 - 2 2 1,074 88.082 0.777

R013508 R4 1 - 2 2 1,074 90.223 0.846

R013509 R4 1 - 2 2 1,074 91.993 0.877

R013701 R7 1 - 2 2 894 76.510 0.535

R013702 R7 1 - 2 2 894 79.866 0.616

R013704 R7 1 - 2 2 894 88.479 0.777

R013708 R7 1 - 2 2 894 84.452 0.729

R013710 R7 1 - 2 2 894 89.597 0.797

R013712 R7 1 - 2 2 894 86.242 0.789

R013902 R11 1 - 2 2 731 90.971 0.800

R013903 R11 1 - 2 2 731 93.844 0.890

R013904 R11 1 - 2 2 731 95.486 0.866

R013906 R11 1 - 2 2 731 90.424 0.831

R013908 R11 1 - 2 2 731 85.636 0.745

R013910 R11 1 - 2 2 731 94.938 0.919

R013913 R11 1 - 2 2 731 92.886 0.883

R015503 R13 1 - 2 2 789 94.297 0.745

R015505 R13 1 - 2 2 789 87.706 0.787

R015509 R13 1 - 2 2 789 86.946 0.788

R015512 R13 1 - 2 2 789 91.255 0.841

* Cohen’s Kappa is a measure of reliability that is appropriate for items that are dichotomized. These items are

dichotomized into right and wrong.
577

Table C-12

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Reading Items

from 1994 That Were Rescored in 1998, Grade 12

Item Block

Range of

Response

Codes

Sample

Size

Percent

Agreement

Intraclass

Correlation

R013201 R6 1 - 4 987 78.014 0.892

R013212 R6 1 - 4 987 85.816 0.935

R013403 R10 1 - 4 716 94.972 0.974

R013406 R10 1 - 4 716 85.894 0.949

R013506 R4 1 - 4 1,074 83.426 0.906

R013706 R7 1 - 4 894 76.063 0.826

R015507 R13 1 - 4 789 83.650 0.927

R015514 R13 1 - 4 789 83.523 0.926

R016101 R9 1 - 3 717 88.703 0.907

R016104 R9 1 - 3 717 82.287 0.699

R016107 R9 1 - 3 717 92.608 0.962

R016108 R9 1 - 3 717 77.964 0.834

R016109 R9 1 - 3 717 89.958 0.920

R016301 R5 1 - 3 1,073 69.059 0.762

R016302 R5 1 - 3 1,073 84.716 0.873

R016303 R5 1 - 3 1,073 87.512 0.915

R016305 R5 1 - 3 1,073 80.336 0.899

R016306 R5 1 - 3 1,073 80.522 0.888

R016307 R5 1 - 3 1,073 85.834 0.913

R016308 R5 1 - 4 1,073 82.665 0.897

R016401 R8 1 - 3 992 84.375 0.861

R016402 R8 1 - 3 992 62.500 0.744

R016403 R8 1 - 3 992 82.157 0.876

R016404 R8 1 - 3 992 88.710 0.908

R016405 R8 1 - 3 992 88.306 0.908

R016407 R8 1 - 3 992 81.754 0.911

R016408 R8 1 - 4 992 85.181 0.909

R016501 R14 1 - 3 746 84.584 0.812

R016502 R14 1 - 3 746 87.399 0.871

R016601 R14 1 - 4 746 76.810 0.781

R016602 R14 1 - 3 746 80.965 0.861

R016603 R14 1 - 3 746 81.769 0.779

R016604 R14 1 - 3 746 78.552 0.803

R016605 R14 1 - 3 746 79.893 0.698

R016701 R14 1 - 4 746 82.440 0.879
578

Table C-13

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Writing Items

Used in 1998 National Main Assessment Scaling, Grade 4

Item Block

Range of

Response

Codes

Sample

Size

Percent

Agreement

Intraclass

Correlation

W004002 W3 1-6 433 76 0.943

W004102 W4 1-6 507 68 0.883

W004202 W5 1-6 540 72 0.903

W004302 W6 1-6 440 78 0.930

W004402 W7 1-6 446 78 0.917

W004502 W8 1-6 432 82 0.942

W004602 W9 1-6 449 78 0.918

W004702 W10 1-6 448 80 0.953

W004802 W11 1-6 467 76 0.926

W004902 W12 1-6 494 78 0.925

W005002 W13 1-6 454 80 0.905

W005102 W14 1-6 457 79 0.886

W005202 W15 1-6 536 75 0.915

W005302 W16 1-6 548 78 0.893

W005402 W17 1-6 751 81 0.927

W005502 W18 1-6 444 76 0.922

W005602 W19 1-6 641 70 0.911

W005702 W20 1-6 440 79 0.928

W005802 W21 1-6 432 78 0.932

W005902 W22 1-6 444 75 0.911
579

Table C-14

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Writing Items

Used in 1998 National Main Assessment Scaling, Grade 8*

Item Block

Range of

Response

Codes

Sample

Size

Percent

Agreement

Intraclass

Correlation

W006002 W3 1-6 1,127 69 0.866

W006102 W4 1-6 1,120 66 0.809

W006202 W5 1-6 1,135 76 0.896

W006302 W6 1-6 1,129 64 0.828

W006402 W7 1-6 1,438 72 0.892

W006502 W8 1-6 1,132 83 0.921

W006602 W9 1-6 1,141 81 0.909

W006702 W10 1-6 478 62 0.797

W006802 W11 1-6 1,116 71 0.850

W006902 W12 1-6 1,137 78 0.893

W007002 W13 1-6 1,130 72 0.822

W007102 W14 1-6 1,120 68 0.793

W007202 W15 1-6 1,120 79 0.888

W007302 W16 1-6 1,129 69 0.851

W007402 W17 1-6 1,130 75 0.893

W007502 W18 1-6 483 73 0.863

W007602 W19 1-6 1,452 74 0.887

W007702 W20 1-6 1,129 71 0.847

W007802 W21 1-6 1,129 66 0.842

W007902 W22 1-6 1,127 68 0.824

W008002 W23 1-6 1,134 76 0.875

W008102 W24 1-6 1,129 64 0.834

W008202 W25 1-6 563 67 0.881

*

 Rescored responses from the national and state assessment samples contributed to these statistics.
580

Table C-15

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Writing Items

Used in 1998 National Main Assessment Scaling, Grade 12

Item Block

Range of

Response

Codes

Sample

Size

Percent

Agreement

Intraclass

Correlation

W008302 W3 1-6 435 87 0.924

W008402 W4 1-6 449 74 0.857

W008502 W5 1-6 526 86 0.933

W008602 W6 1-6 466 73 0.815

W008702 W7 1-6 463 79 0.884

W008802 W8 1-6 454 65 0.832

W008902 W9 1-6 447 84 0.906

W009002 W10 1-6 496 73 0.878

W009102 W11 1-6 535 76 0.888

W009202 W12 1-6 436 81 0.925

W009302 W13 1-6 444 63 0.833

W009402 W14 1-6 430 79 0.917

W009502 W15 1-6 433 85 0.936

W009602 W16 1-6 519 76 0.882

W009702 W17 1-6 455 75 0.886

W009802 W18 1-6 507 76 0.870

W009902 W19 1-6 437 68 0.843

W010002 W20 1-6 515 63 0.789

W010102 W21 1-6 607 70 0.861

W010202 W22 1-6 449 58 0.790

W010302 W23 1-6 513 66 0.864

W010402 W24 1-6 439 79 0.914

W010502 W25 1-6 446 69 0.873
581

Table C-16

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Civics Items

Used in 1998 National Main Assessment Scaling, Grade 4

Item Block

Range of

Response

Codes

Sample

Size

Percent

Agreement

Intraclass

Correlation

P030004 C3 1-3 533 82 0.868

P030005 C3 1-3 522 92 0.913

P030007 C3 1-3 513 85 0.875

P030010 C3 1-4 489 91 0.942

P040102 C4 1-3 457 90 0.898

P040105 C4 1-3 452 86 0.868

P040109 C4 1-3 441 95 0.943

P040111 C4 1-3 414 93 0.946

P040203 C5 1-4 534 89 0.900

P040206 C5 1-3 529 95 0.946

P040209 C5 1-3 490 81 0.826

P040304 C6 1-3 474 94 0.959

P040310 C6 1-3 468 94 0.941

P040311 C6 1-3 456 98 0.974

P040402 C7 1-4 486 90 0.868

P040404 C7 1-4 484 82 0.867

P040409 C7 1-3 478 84 0.802

P040412 C7 1-4 447 81 0.905

P040502 C8 1-4 479 86 0.934

P040507 C8 1-3 470 92 0.839

P040510 C8 1-3 435 83 0.834
582

Table C-17

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Civics Items

Used in 1998 National Main Assessment Scaling, Grade 8

Item Block

Range of

Response

Codes

Sample

Size

Percent

Agreement

Intraclass

Correlation

P040602 C10 1-3 500 89 0.938

P040608 C10 1-4 502 84 0.922

P040613 C10 1-3 490 85 0.906

P040703 C3 1-3 502 90 0.900

P040705 C3 1-3 500 92 0.898

P040708 C3 1-3 499 78 0.793

P040715 C3 1-3 479 86 0.911

P040803 C4 1-3 552 84 0.904

P040807 C4 1-4 551 95 0.964

P040813 C4 1-4 539 84 0.891

P040903 C5 1-3 491 68 0.707

P040906 C5 1-3 492 89 0.917

P040910 C5 1-3 487 85 0.834

P040913 C5 1-3 476 96 0.974

P041003 C6 1-3 551 86 0.898

P041007 C6 1-3 558 83 0.849

P041013 C6 1-3 553 88 0.846

P041014 C6 1-3 549 91 0.923

P041102 C7 1-3 494 90 0.942

P041106 C7 1-4 494 84 0.839

P041111 C7 1-3 474 92 0.933

P041116 C7 1-3 449 85 0.873

P041202 C8 1-3 500 89 0.918

P041205 C8 1-3 498 87 0.860

P041213 C8 1-4 494 83 0.905

P041307 C9 1-4 498 86 0.932

P041309 C9 1-3 496 92 0.952

P041315 C9 1-3 491 86 0.924
583

Table C-18

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Civics Items

Used in 1998 National Main Assessment Scaling, Grade 12

Item Block

Range of

Response

Codes

Sample

Size

Percent

Agreement

Intraclass

Correlation

P041404 C10 1-4 482 77 0.813

P041408 C10 1-3 476 81 0.888

P041412 C10 1-3 463 90 0.928

P041413 C10 1-3 463 83 0.871

P041503 C3 1-3 504 91 0.958

P041505 C3 1-3 510 75 0.840

P041509 C3 1-4 503 82 0.891

P041511 C3 1-3 500 87 0.923

P041606 C4 1-3 514 81 0.856

P041613 C4 1-3 506 93 0.958

P041614 C4 1-3 510 90 0.950

P041705 C5 1-3 459 73 0.806

P041706 C5 1-4 460 83 0.889

P041711 C5 1-3 449 88 0.913

P041713 C5 1-3 430 88 0.939

P041804 C6 1-3 487 72 0.815

P041806 C6 1-3 482 92 0.962

P041815 C6 1-3 473 87 0.903

P041902 C7 1-3 465 87 0.918

P041905 C7 1-4 466 79 0.829

P041907 C7 1-4 465 91 0.949

P041912 C7 1-3 459 80 0.792

P042002 C8 1-3 472 85 0.927

P042008 C8 1-3 472 82 0.904

P042009 C8 1-3 469 88 0.926

P042012 C8 1-3 464 93 0.971

P042102 C9 1-3 479 86 0.917

P042103 C9 1-4 479 86 0.887

585

Appendix D

DIFFERENTIAL ITEM FUNCTIONING (DIF) RESULTS

Differential item functioning (DIF) results for the reading and civics assessments are given in the tables

below. Results for the writing assessment DIF analysis are detailed in Chapter 19.

Table D-1

1998 Reading Items Identified as “C” or “CC” Items in at Least One Comparison*

Item Block Scale Category Grade Comparison

Group

Favored

R017203 R8 Reading to Gain Information C 8 Male/Female Male

*

 For each grade for which an item was administered, three comparisons were performed: Male/Female, White/Black, and

White/Hispanic.

Table D-2

1998 Civics Items Identified as “C” or “CC” Items in at Least One Comparison*

Item Block Scale Category Grade Comparison Group Favored

P040505 C8 Overall C 4 White/Black Black

P040801 C4 Overall C 8 White/Hispanic Hispanic

P040905 C5 Overall C 8 Male/Female Male

P040608 C10 Overall CC 8 Male/Female Female

P041816 C6 Overall C 12 White/Black Black

P042013 C8 Overall C 12 White/Black Black

P041705 C5 Overall CC 12 White/Black Black

P041804 C6 Overall CC 12 White/Black White

P042008 C8 Overall CC 12 White/Black White

P042012 C8 Overall CC 12 White/Black White

P041507 C3 Overall C 12 Male/Female Female

* For each grade for which an item was administered, three comparisons were performed: Male/Female, White/Black, and

White/Hispanic.

587

Appendix E

IRT PARAMETERS

This appendix contains tables of IRT (item response theory) parameters for NAEP items that

were scaled in each subject area for which IRT scales were created, as well as the block in which each

item appears for each age class (Block) and the position of each item within its block (Item). Note that

item parameters shown in this appendix are in the metrics used for the original calibration of the scales.

Table E-1

IRT Parameters for the 1998 Reading Items

Reading for Literary Experience Scale, Grade 4

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 (1R017001 R3 1A 0.623 (0.038) -0.872 (0.069) 0.000 (0.000)1R017002 R3 2 1.506 (0.110) -0.495 (0.056) 0.215 (0.030)1R017003 R3 3A 0.476 (0.026) 0.431 (0.040) 0.000 (0.000) 0.012 (0.073) -0.012 (0.078)1R017004 R3 4A 0.920 (0.059) 1.008 (0.050) 0.000 (0.000)1R017005 R3 5 0.607 (0.094) 0.712 (0.136) 0.251 (0.041)1R017006 R3 6A 1.052 (0.066) 1.009 (0.045) 0.000 (0.000)1R017007 R3 7A 0.641 (0.030) 0.407 (0.026) 0.000 (0.000) 0.359 (0.065) 0.138 (0.066) -0.497 1R017008 R3 8 1.288 (0.126) 0.554 (0.052) 0.190 (0.024)1R017009 R3 9A 0.496 (0.021) -0.278 (0.058) 0.000 (0.000) 1.842 (0.102) -1.842 (0.079)1R012101 R4 1 1.798 (0.105) -0.899 (0.044) 0.248 (0.025)1R012102 R4 2A 0.754 (0.031) 0.015 (0.029) 0.000 (0.000)1R012103 R4 3 1.342 (0.068) -0.456 (0.042) 0.175 (0.021)1R012104 R4 4A 0.763 (0.031) -0.284 (0.032) 0.000 (0.000)1R012105 R4 5 1.110 (0.073) 0.148 (0.049) 0.244 (0.021)1R012106 R4 6A 1.025 (0.039) 0.107 (0.023) 0.000 (0.000)1R012107 R4 7 1.228 (0.083) 0.259 (0.044) 0.247 (0.020)1R012108 R4 8A 0.647 (0.029) -1.008 (0.054) 0.000 (0.000)1R012109 R4 9A 0.520 (0.027) -1.425 (0.080) 0.000 (0.000)1R012110 R4 10 0.951 (0.068) -0.864 (0.103) 0.319 (0.039)1R012111 R4 11A 1.026 (0.037) 1.248 (0.024) 0.000 (0.000) 0.851 (0.025) -0.851 (0.056)1R012112 R4 12A 0.757 (0.038) -0.630 (0.048) 0.000 (0.000)1R012601 R5 1A 0.832 (0.040) 1.118 (0.042) 0.000 (0.000)1R012602 R5 2 1.472 (0.108) 1.204 (0.036) 0.167 (0.010)1R012603 R5 3 1.859 (0.110) 0.213 (0.030) 0.265 (0.017)1R012604 R5 4A 1.123 (0.050) 1.057 (0.031) 0.000 (0.000)1R012605 R5 5 1.133 (0.113) 0.916 (0.048) 0.297 (0.018)1R012606 R5 6 1.374 (0.092) 0.307 (0.041) 0.269 (0.019)1R012607 R5 7A 1.212 (0.041) 1.102 (0.016) 0.000 (0.000) 0.627 (0.023) -0.059 (0.031) -0.568 1R012608 R5 8 0.504 (0.044) -0.932 (0.199) 0.247 (0.051)1R012609 R5 9 1.415 (0.134) 0.891 (0.039) 0.271 (0.016)1R012610 R5 10 2.303 (0.177) 0.609 (0.030) 0.418 (0.015)1R012611 R5 11A 0.814 (0.037) 0.306 (0.030) 0.000 (0.000)1R015801 R9 1 0.966 (0.059) -1.318 (0.099) 0.244 (0.039)
Table E-1 (continued)

IRT Parameters for the 1998 Reading Items

Reading for Literary Experience Scale, Grade 4

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 (5901R015802 R9 2A 0.506 (0.035) -1.272 (0.099) 0.000 (0.000)1R015803 R9 3A 0.646 (0.024) -0.386 (0.040) 0.000 (0.000) 1.573 (0.074) -1.573 (0.052)1R015804 R9 4A 0.659 (0.017) 0.693 (0.024) 0.000 (0.000) 2.081 (0.046) -0.361 (0.038) -1.720 1R015805 R9 5 1.029 (0.082) 0.327 (0.059) 0.300 (0.023)1R015806 R9 6A 0.698 (0.021) 0.268 (0.026) 0.000 (0.000) 1.089 (0.039) -1.089 (0.040)1R015807 R9 7A 0.625 (0.027) -0.087 (0.042) 0.000 (0.000) 1.293 (0.071) -1.293 (0.059)1R015808 R9 8 0.721 (0.053) -1.193 (0.142) 0.247 (0.046)1R015809 R9 9A 0.623 (0.019) 0.106 (0.032) 0.000 (0.000) 1.381 (0.052) -1.381 (0.048)
Table E-2

IRT Parameters for the 1998 Reading Items

Reading to Gain Information Scale, Grade 4

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 (2R012201 R6 1A 0.269 (0.020) -0.904 (0.097) 0.000 (0.000)2R012202 R6 2 0.941 (0.073) 0.401 (0.061) 0.264 (0.023)2R012203 R6 3 0.793 (0.071) 0.642 (0.069) 0.247 (0.024)2R012204 R6 4A 0.509 (0.017) 0.133 (0.022) 0.000 (0.000) 1.139 (0.055) -0.350 (0.051) -0.789 2R012205 R6 5 1.032 (0.082) 0.507 (0.054) 0.248 (0.022)2R012206 R6 6A 1.172 (0.045) 0.645 (0.024) 0.000 (0.000)2R012207 R6 7 0.533 (0.042) -0.835 (0.159) 0.218 (0.045)2R012208 R6 8A 0.877 (0.036) -0.523 (0.034) 0.000 (0.000)2R012209 R6 9 1.203 (0.074) 0.257 (0.042) 0.165 (0.019)2R012210 R6 10A 0.761 (0.036) -1.242 (0.058) 0.000 (0.000)2R012701 R7 1 1.104 (0.066) -0.155 (0.057) 0.247 (0.026)2R012702 R7 2A 0.619 (0.028) -1.113 (0.056) 0.000 (0.000)2R012703 R7 8A 1.154 (0.042) 0.645 (0.023) 0.000 (0.000)2R012704 R7 4 1.464 (0.093) 0.774 (0.028) 0.138 (0.012)2R012705 R7 5A 1.536 (0.067) 1.192 (0.027) 0.000 (0.000)2R012706 R7 6A 0.597 (0.034) 1.341 (0.065) 0.000 (0.000)2R012707 R7 3 2.300 (0.146) 0.416 (0.025) 0.264 (0.014)2R012708 R7 10A 0.673 (0.024) 1.734 (0.028) 0.000 (0.000) 1.378 (0.037) 0.441 (0.049) -1.819 2R012709 R7 9 0.562 (0.055) -0.073 (0.150) 0.237 (0.044)2R012710 R7 11A 0.970 (0.048) 0.906 (0.035) 0.000 (0.000)2R015701 R8 1 0.883 (0.059) -1.015 (0.109) 0.310 (0.042)2R015702 R8 2A 0.718 (0.025) 0.161 (0.038) 0.000 (0.000) 1.517 (0.056) -1.517 (0.057)2R015703 R8 3A 0.716 (0.018) 0.077 (0.026) 0.000 (0.000) 1.417 (0.040) -1.417 (0.038)2R015704 R8 4A 0.621 (0.022) -0.145 (0.024) 0.000 (0.000) 0.402 (0.043) -0.402 (0.038)2R015705 R8 5A 0.823 (0.027) 0.275 (0.021) 0.000 (0.000) 0.740 (0.032) -0.740 (0.033)2R015706 R8 6 1.261 (0.113) 1.084 (0.039) 0.206 (0.014)2R015707 R8 7A 0.562 (0.018) 0.419 (0.030) 0.000 (0.000) 1.209 (0.045) -1.209 (0.051)2R015708 R8 8 0.597 (0.043) -0.206 (0.102) 0.156 (0.033)2R015709 R8 9A 0.524 (0.025) 1.137 (0.044) 0.000 (0.000) 0.366 (0.050) -0.366 (0.069)2R012501 R10 1 0.609 (0.222) 3.921 (1.005) 0.309 (0.013)2R012502 R10 2 0.938 (0.063) -1.691 (0.121) 0.294 (0.046)2R012503 R10 3A 1.086 (0.037) -0.060 (0.022) 0.000 (0.000)2R012504 R10 4A 0.795 (0.030) -0.238 (0.029) 0.000 (0.000)
Table E-2 (continued)

IRT Parameters for the 1998 Reading Items

Reading to Gain Information Scale, Grade 4

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 (5922R012505 R10 5 1.414 (0.080) -0.608 (0.051) 0.275 (0.027)2R012506 R10 6A 0.838 (0.032) -0.076 (0.027) 0.000 (0.000)2R012507 R10 7 1.185 (0.074) -0.590 (0.067) 0.312 (0.031)2R012508 R10 8A 1.031 (0.037) -0.310 (0.026) 0.000 (0.000)2R012509 R10 9 0.579 (0.049) -0.688 (0.167) 0.276 (0.048)2R012510 R10 10 0.970 (0.062) -0.502 (0.078) 0.270 (0.032)2R012511 R10 11A 1.002 (0.039) -0.530 (0.031) 0.000 (0.000)2R012512 R10 12A 0.413 (0.016) 0.512 (0.029) 0.000 (0.000) 0.892 (0.069) 0.242 (0.067) -1.133 
Table E-3

IRT Parameters for the 1998 Reading Items

Reading for Literary Experience Scale, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 (1R017101 R3 1A 1.169 (0.052) -0.299 (0.032) 0.000 (0.000)1R017102 R3 2A 0.566 (0.032) 1.150 (0.052) 0.000 (0.000) 0.182 (0.058) -0.182 (0.081)1R017103 R3 3 0.664 (0.080) 0.423 (0.116) 0.218 (0.038)1R017104 R3 4A 1.240 (0.055) 0.784 (0.021) 0.000 (0.000) 0.269 (0.030) -0.269 (0.037)1R017105 R3 5A 0.892 (0.035) 0.922 (0.023) 0.000 (0.000) 0.716 (0.042) 0.392 (0.044) -1.108 1R017106 R3 6 0.858 (0.182) 1.754 (0.161) 0.247 (0.022)1R017107 R3 7A 0.556 (0.030) 0.724 (0.041) 0.000 (0.000) 0.437 (0.059) -0.437 (0.072)1R017108 R3 8A 1.491 (0.092) 1.106 (0.036) 0.000 (0.000)1R017109 R3 9 0.759 (0.067) -0.801 (0.146) 0.243 (0.052)1R017110 R3 10A 1.221 (0.063) 0.063 (0.031) 0.000 (0.000)1R015901 R4 1A 0.517 (0.023) -0.176 (0.040) 0.000 (0.000)1R015902 R4 2A 0.650 (0.020) 0.241 (0.024) 0.000 (0.000) 0.977 (0.037) -0.977 (0.039)1R015903 R4 3 0.849 (0.064) 0.241 (0.069) 0.241 (0.025)1R015904 R4 4A 0.595 (0.020) 1.682 (0.036) 0.000 (0.000) 1.293 (0.036) -1.293 (0.084)1R015905 R4 5A 0.548 (0.020) 0.422 (0.027) 0.000 (0.000) 0.606 (0.043) -0.606 (0.047)1R015906 R4 6A 0.506 (0.015) 2.189 (0.032) 0.000 (0.000) 2.803 (0.045) 0.245 (0.058) -3.048 1R015907 R4 7A 0.488 (0.013) 0.378 (0.038) 0.000 (0.000) 1.836 (0.055) -1.836 (0.063)1R015908 R4 8A 0.673 (0.027) 0.986 (0.032) 0.000 (0.000) 0.881 (0.040) -0.881 (0.058)1R012601 R5 1A 0.751 (0.028) 0.061 (0.028) 0.000 (0.000)1R012602 R5 2 1.029 (0.063) 0.398 (0.042) 0.158 (0.018)1R012603 R5 3 1.147 (0.065) -0.928 (0.073) 0.258 (0.035)1R012604 R5 4A 0.818 (0.029) 0.053 (0.027) 0.000 (0.000)1R012605 R5 5 0.671 (0.049) -0.210 (0.108) 0.217 (0.036)1R012606 R5 6 1.490 (0.082) -0.703 (0.051) 0.259 (0.027)1R012607 R5 7A 0.635 (0.027) 0.502 (0.027) 0.000 (0.000) 1.141 (0.059) -0.066 (0.055) -1.075 1R012608 R5 8 0.616 (0.041) -1.548 (0.173) 0.257 (0.058)1R012609 R5 9 1.331 (0.076) -0.173 (0.047) 0.254 (0.022)1R012610 R5 10 1.365 (0.090) -0.438 (0.064) 0.396 (0.027)1R012611 R5 11A 0.635 (0.038) -0.768 (0.064) 0.000 (0.000)
Table E-4

IRT Parameters for the 1998 Reading Items

Reading to Gain Information Scale, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 (5942R013201 R6 1A 0.709 (0.022) 0.743 (0.019) 0.000 (0.000) 0.969 (0.035) -0.020 (0.037) -0.948 2R013202 R6 2 0.733 (0.056) -0.234 (0.101) 0.266 (0.034)2R013203 R6 3A 1.173 (0.050) -1.793 (0.048) 0.000 (0.000)2R013204 R6 4 1.036 (0.070) -0.176 (0.067) 0.316 (0.027)2R013205 R6 5A 1.078 (0.040) -1.210 (0.038) 0.000 (0.000)2R013206 R6 6 0.772 (0.051) 0.065 (0.067) 0.160 (0.025)2R013207 R6 7A 0.720 (0.028) -0.574 (0.037) 0.000 (0.000)2R013208 R6 8 1.531 (0.086) -0.116 (0.038) 0.239 (0.020)2R013209 R6 9A 0.788 (0.038) 1.223 (0.047) 0.000 (0.000)2R013210 R6 10 1.530 (0.146) 1.694 (0.073) 0.283 (0.009)2R013211 R6 11A 0.580 (0.030) 0.797 (0.045) 0.000 (0.000)2R013212 R6 12A 0.593 (0.025) 1.756 (0.044) 0.000 (0.000) 1.903 (0.054) -0.604 (0.090) -1.299 2R012701 R7 1 1.052 (0.062) -1.389 (0.092) 0.281 (0.041)2R012702 R7 2A 0.662 (0.030) -1.996 (0.078) 0.000 (0.000)2R012707 R7 3 1.984 (0.127) -0.676 (0.045) 0.373 (0.024)2R012704 R7 4 1.119 (0.063) -0.424 (0.056) 0.219 (0.025)2R012705 R7 5A 0.966 (0.034) 0.040 (0.023) 0.000 (0.000)2R012706 R7 6A 0.510 (0.026) 0.500 (0.043) 0.000 (0.000)2R012711 R7 7 1.309 (0.109) 0.019 (0.061) 0.252 (0.029)2R012703 R7 8A 0.942 (0.033) -0.205 (0.025) 0.000 (0.000)2R012709 R7 9 0.899 (0.071) -0.415 (0.103) 0.400 (0.034)2R012708 R7 10A 0.584 (0.016) 0.668 (0.023) 0.000 (0.000) 1.594 (0.047) 0.275 (0.040) -1.869 2R012710 R7 11A 0.833 (0.032) -0.580 (0.033) 0.000 (0.000)2R012712 R7 12 0.995 (0.087) 0.340 (0.067) 0.367 (0.024)2R012713 R7 13A 1.251 (0.047) -0.591 (0.028) 0.000 (0.000)2R017201 R8 1 0.808 (0.075) -0.740 (0.143) 0.301 (0.049)2R017202 R8 2A 0.583 (0.029) -0.389 (0.038) 0.000 (0.000) 0.555 (0.068) -0.555 (0.054)2R017203 R8 3 0.888 (0.077) -0.300 (0.099) 0.237 (0.038)2R017204 R8 4A 0.760 (0.030) 0.721 (0.033) 0.000 (0.000) 1.086 (0.043) -1.086 (0.060)2R017205 R8 5A 0.632 (0.023) 0.598 (0.032) 0.000 (0.000) 1.902 (0.064) -0.216 (0.052) -1.686 2R017206 R8 6 0.808 (0.103) 0.637 (0.094) 0.271 (0.033)2R017207 R8 7A 0.360 (0.025) 1.523 (0.088) 0.000 (0.000) 0.859 (0.084) -0.859 (0.130)2R017208 R8 8A 0.767 (0.028) 1.045 (0.036) 0.000 (0.000) 1.419 (0.043) -1.419 (0.082)
Table E-4 (continued)

IRT Parameters for the 1998 Reading Items

Reading to Gain Information Scale, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 (2R017209 R8 9 1.635 (0.155) 0.445 (0.048) 0.314 (0.023)2R017210 R8 10A 0.586 (0.055) 1.799 (0.137) 0.000 (0.000)2R016201 R13 1A 0.491 (0.025) -3.618 (0.144) 0.000 (0.000) -0.077 (0.207) 0.077 (0.085)2R016202 R13 2A 0.677 (0.016) -0.757 (0.035) 0.000 (0.000) 2.431 (0.074) -2.431 (0.039)2R016203 R13 3 0.461 (0.041) -0.765 (0.209) 0.270 (0.051)2R016204 R13 4A 0.556 (0.012) -0.418 (0.034) 0.000 (0.000) 2.900 (0.110) 0.813 (0.043) -3.713 2R016205 R13 5A 0.641 (0.016) 0.690 (0.028) 0.000 (0.000) 1.571 (0.037) -1.571 (0.053)2R016206 R13 6 0.986 (0.062) -0.879 (0.091) 0.318 (0.036)2R016207 R13 7A 0.570 (0.020) 0.276 (0.024) 0.000 (0.000) 0.184 (0.042) -0.184 (0.044)2R016208 R13 8 0.824 (0.053) -1.046 (0.110) 0.291 (0.041)2R016209 R13 9 1.119 (0.062) -0.750 (0.066) 0.246 (0.030)2R016210 R13 10A 0.606 (0.023) 0.881 (0.034) 0.000 (0.000) 1.952 (0.062) -0.058 (0.056) -1.894 2R016211 R13 11A 0.500 (0.016) -2.064 (0.039) 0.000 (0.000) 2.272 (0.128) -2.272 (0.040)2R016212 R13 12A 0.395 (0.017) -0.001 (0.033) 0.000 (0.000) 0.200 (0.062) -0.200 (0.060)2R016213 R13 13A 0.396 (0.018) -2.137 (0.074) 0.000 (0.000) 1.131 (0.118) -1.131 (0.055)
Table E-5

IRT Parameters for the 1998 Reading Items

Reading to Perform a Task Scale, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 (5963R016101 R9 1A 0.534 (0.018) -0.111 (0.025) 0.000 (0.000) 0.069 (0.049) -0.069 (0.045)3R016102 R9 2 1.015 (0.068) -0.624 (0.084) 0.348 (0.033)3R016103 R9 3 1.525 (0.102) 0.418 (0.036) 0.308 (0.016)3R016104 R9 4A 0.739 (0.032) -1.823 (0.065) 0.000 (0.000)3R016105 R9 5 1.437 (0.078) -0.572 (0.046) 0.253 (0.024)3R016106 R9 6 1.020 (0.082) 0.996 (0.044) 0.180 (0.015)3R016107 R9 7A 0.700 (0.019) -0.150 (0.020) 0.000 (0.000) -0.162 (0.041) 0.162 (0.037)3R016108 R9 8A 0.396 (0.012) 0.069 (0.030) 0.000 (0.000) -1.046 (0.071) 1.046 (0.070)3R016109 R9 9A 0.441 (0.012) 0.780 (0.038) 0.000 (0.000) 1.793 (0.052) -1.793 (0.071)3R013401 R10 1 1.096 (0.064) 0.248 (0.042) 0.175 (0.019)3R013402 R10 2A 0.829 (0.031) 0.114 (0.026) 0.000 (0.000)3R013403 R10 3A 0.455 (0.011) 0.503 (0.026) 0.000 (0.000) -2.178 (0.084) 2.178 (0.087)3R013404 R10 4 1.090 (0.075) -0.043 (0.063) 0.337 (0.025)3R013405 R10 5A 0.971 (0.034) -0.280 (0.025) 0.000 (0.000)3R013406 R10 6A 0.637 (0.028) 0.576 (0.037) 0.000 (0.000)3R013407 R10 7A 0.648 (0.027) -0.626 (0.040) 0.000 (0.000)3R013408 R10 8 0.605 (0.048) -0.004 (0.107) 0.203 (0.034)3R013409 R10 9A 0.714 (0.029) -0.355 (0.034) 0.000 (0.000)3R013410 R10 10 0.875 (0.066) -0.357 (0.096) 0.328 (0.035)3R013411 R10 11A 0.510 (0.026) -1.023 (0.064) 0.000 (0.000)3R013412 R10 12A 0.405 (0.027) -1.923 (0.128) 0.000 (0.000)3R013001 R11 1A 0.960 (0.036) -1.069 (0.037) 0.000 (0.000)3R013002 R11 2 1.564 (0.079) -0.351 (0.036) 0.198 (0.020)3R013003 R11 3A 0.975 (0.034) -0.498 (0.027) 0.000 (0.000)3R013004 R11 4A 0.452 (0.025) 0.598 (0.045) 0.000 (0.000) 0.235 (0.072) -0.235 (0.082)3R013005 R11 5A 0.825 (0.032) -1.033 (0.041) 0.000 (0.000)3R013006 R11 6 0.886 (0.054) -0.351 (0.073) 0.206 (0.030)3R013007 R11 7A 0.691 (0.030) -1.292 (0.055) 0.000 (0.000)3R013008 R11 8A 0.730 (0.030) -0.085 (0.031) 0.000 (0.000)3R013009 R11 9A 1.010 (0.041) -1.258 (0.043) 0.000 (0.000)3R013010 R11 10A 0.846 (0.035) -1.035 (0.043) 0.000 (0.000)3R013011 R11 11A 0.477 (0.025) -0.316 (0.049) 0.000 (0.000)3R013012 R11 12 1.134 (0.075) 0.021 (0.055) 0.249 (0.025)
Table E-6

IRT Parameters for the 1998 Reading Items

Reading for Literary Experience Scale, Grade 12

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 1R017101 R3 1A 1.210 (0.061) -0.945 (0.043) 0.000 (0.000)1R017102 R3 2A 0.610 (0.028) 0.309 (0.032) 0.000 (0.000) 0.012 (0.059) -0.012 (0.061)1R017103 R3 3 0.618 (0.061) -0.449 (0.151) 0.220 (0.045)1R017104 R3 4A 1.011 (0.042) 0.021 (0.022) 0.000 (0.000) 0.205 (0.040) -0.205 (0.036)1R017105 R3 5A 0.649 (0.025) 0.406 (0.027) 0.000 (0.000) 0.752 (0.065) 0.641 (0.057) -1.31R017106 R3 6 0.549 (0.079) 0.824 (0.139) 0.228 (0.038)1R017107 R3 7A 0.457 (0.025) 0.382 (0.046) 0.000 (0.000) 0.828 (0.073) -0.828 (0.080)1R017108 R3 8A 1.229 (0.063) 0.486 (0.029) 0.000 (0.000)1R017109 R3 9 0.594 (0.049) -1.237 (0.157) 0.177 (0.045)1R017110 R3 10A 0.910 (0.050) -0.473 (0.046) 0.000 (0.000)1R013501 R4 1A 0.985 (0.035) -0.294 (0.026) 0.000 (0.000)1R013502 R4 2 1.463 (0.084) -0.617 (0.050) 0.254 (0.026)1R013503 R4 3A 0.420 (0.024) 0.308 (0.050) 0.000 (0.000)1R013504 R4 4 0.646 (0.045) -0.353 (0.104) 0.177 (0.035)1R013505 R4 5A 0.624 (0.027) -1.020 (0.052) 0.000 (0.000)1R013506 R4 6A 0.413 (0.014) 1.632 (0.055) 0.000 (0.000) -2.273 (0.107) 2.273 (0.122)1R013507 R4 7 1.011 (0.067) 0.234 (0.050) 0.183 (0.021)1R013508 R4 8A 0.319 (0.027) 1.410 (0.126) 0.000 (0.000)1R013509 R4 9A 0.709 (0.037) 0.623 (0.041) 0.000 (0.000)1R016301 R5 1A 0.452 (0.022) 0.394 (0.048) 0.000 (0.000) 1.214 (0.074) -1.214 (0.082)1R016302 R5 2 0.395 (0.021) 0.108 (0.052) 0.000 (0.000) 1.153 (0.087) -1.153 (0.087)1R016303 R5 3A 0.619 (0.022) 0.691 (0.027) 0.000 (0.000) 0.787 (0.037) -0.787 (0.048)1R016304 R5 4 1.500 (0.098) 0.017 (0.044) 0.341 (0.021)1R016305 R5 5A 0.441 (0.024) -0.351 (0.050) 0.000 (0.000) 0.953 (0.089) -0.953 (0.074)1R016306 R5 6 0.540 (0.021) 0.484 (0.028) 0.000 (0.000) 0.211 (0.047) -0.211 (0.052)1R016307 R5 7A 0.634 (0.026) 0.816 (0.030) 0.000 (0.000) 0.515 (0.041) -0.515 (0.052)1R016308 R5 8A 0.469 (0.021) 1.586 (0.052) 0.000 (0.000) 0.312 (0.056) -0.324 (0.092) 0.0
Table E-7

IRT Parameters for the 1998 Reading Items

Reading to Gain Information Scale, Grade 12

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 5982R013201 R6 1A 0.567 (0.018) 0.020 (0.021) 0.000 (0.000) 1.096 (0.053) -0.106 (0.044) -0.92R013202 R6 2 0.654 (0.044) -1.243 (0.140) 0.227 (0.046)2R013203 R6 3A 1.095 (0.053) -2.293 (0.068) 0.000 (0.000)2R013204 R6 4 0.745 (0.049) -1.573 (0.140) 0.250 (0.049)2R013205 R6 5A 1.102 (0.050) -1.999 (0.059) 0.000 (0.000)2R013206 R6 6 0.752 (0.048) -0.625 (0.096) 0.206 (0.036)2R013207 R6 7A 0.733 (0.045) -1.455 (0.085) 0.000 (0.000)2R013208 R6 8 1.384 (0.087) -1.127 (0.069) 0.302 (0.034)2R013209 R6 9A 0.524 (0.027) 0.756 (0.048) 0.000 (0.000)2R013210 R6 10 0.823 (0.092) 1.227 (0.066) 0.224 (0.020)2R013211 R6 11A 0.292 (0.030) 0.266 (0.098) 0.000 (0.000)2R013212 R6 12A 0.444 (0.013) 1.602 (0.037) 0.000 (0.000) 2.196 (0.052) -0.694 (0.072) -1.52R013701 R7 1A 0.418 (0.024) 0.298 (0.049) 0.000 (0.000)2R013702 R7 2A 0.558 (0.026) 0.125 (0.037) 0.000 (0.000)2R013703 R7 3 0.780 (0.051) -2.151 (0.151) 0.239 (0.054)2R013704 R7 4A 0.577 (0.056) 2.371 (0.189) 0.000 (0.000)2R013705 R7 5 1.156 (0.070) -1.140 (0.078) 0.281 (0.036)2R013706 R7 6A 0.611 (0.027) -0.209 (0.036) 0.000 (0.000)2R013707 R7 7 0.678 (0.042) -0.149 (0.075) 0.135 (0.027)2R013708 R7 8 0.202 (0.021) 1.130 (0.145) 0.000 (0.000)2R013709 R7 9 0.486 (0.062) 0.833 (0.140) 0.248 (0.037)2R013710 R7 10A 0.724 (0.038) 1.333 (0.057) 0.000 (0.000)2R013711 R7 11 0.376 (0.054) 1.134 (0.186) 0.223 (0.041)2R013712 R7 12A 0.558 (0.029) -0.066 (0.042) 0.000 (0.000)2R016401 R8 1A 0.633 (0.030) -0.667 (0.040) 0.000 (0.000) 0.963 (0.074) -0.963 (0.049)2R016402 R8 2A 0.324 (0.017) -1.164 (0.075) 0.000 (0.000) 1.627 (0.138) -1.627 (0.092)2R016403 R8 3A 0.643 (0.020) -0.149 (0.025) 0.000 (0.000) 0.989 (0.043) -0.989 (0.037)2R016404 R8 4A 0.449 (0.011) 0.115 (0.041) 0.000 (0.000) 2.211 (0.062) -2.211 (0.063)2R016405 R8 5A 0.422 (0.015) -0.565 (0.045) 0.000 (0.000) -2.853 (0.161) 2.853 (0.154)2R016406 R8 6 0.467 (0.045) -0.889 (0.238) 0.288 (0.057)2R016407 R8 7A 0.547 (0.021) -0.074 (0.028) 0.000 (0.000) 0.670 (0.050) -0.670 (0.046)2R016408 R8 8A 0.437 (0.015) 1.190 (0.032) 0.000 (0.000) 0.334 (0.063) 1.048 (0.069) -1.32R015501 R13 1 0.808 (0.056) -0.765 (0.107) 0.306 (0.038)
Table E-7 (continued)

IRT Parameters for the 1998 Reading Items

Reading to Gain Information Scale, Grade 12

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 2R015502 R13 2 0.798 (0.059) 0.147 (0.074) 0.224 (0.028)2R015503 R13 3A 0.517 (0.030) -2.991 (0.149) 0.000 (0.000)2R015504 R13 4 0.650 (0.048) -0.174 (0.101) 0.206 (0.033)2R015505 R13 5A 0.562 (0.025) -0.099 (0.037) 0.000 (0.000)2R015506 R13 6 0.800 (0.061) 0.067 (0.081) 0.263 (0.029)2R015507 R13 7A 0.786 (0.021) 0.304 (0.018) 0.000 (0.000) 1.376 (0.039) 0.002 (0.031) -1.32R015508 R13 8 0.953 (0.052) -0.711 (0.068) 0.183 (0.029)2R015509 R13 9A 1.064 (0.038) 0.394 (0.022) 0.000 (0.000)2R015510 R13 10 1.770 (0.107) 0.108 (0.034) 0.308 (0.018)2R015511 R13 11 0.768 (0.051) -0.742 (0.103) 0.243 (0.037)2R015512 R13 12A 0.842 (0.032) -0.551 (0.032) 0.000 (0.000)2R015513 R13 13 0.895 (0.051) -0.898 (0.080) 0.197 (0.033)2R015514 R13 14A 0.501 (0.022) 0.358 (0.034) 0.000 (0.000) 1.353 (0.081) 0.141 (0.069) -1.42R015515 R13 15 0.809 (0.060) 0.922 (0.049) 0.112 (0.016)2R015516 R13 16 0.674 (0.060) -0.068 (0.121) 0.315 (0.037)2R016501 R14 1A 1.001 (0.022) 0.325 (0.023) 0.000 (0.000) 1.564 (0.031) -1.564 (0.037)2R016502 R14 2A 1.009 (0.025) 0.842 (0.021) 0.000 (0.000) 1.242 (0.024) -1.242 (0.044)2R016601 R14 3A 0.578 (0.016) 0.269 (0.029) 0.000 (0.000) 1.343 (0.043) -1.343 (0.047)2R016602 R14 4A 0.379 (0.023) 0.631 (0.055) 0.000 (0.000) 0.790 (0.083) -0.790 (0.097)2R016604 R14 6A 0.567 (0.016) -0.608 (0.031) 0.000 (0.000) 1.513 (0.058) -1.513 (0.040)2R016605 R14 7A 0.230 (0.009) 1.718 (0.085) 0.000 (0.000) -2.207 (0.123) 2.207 (0.146)2R016701 R14 8A 0.773 (0.017) 0.572 (0.023) 0.000 (0.000) 2.301 (0.044) -0.763 (0.035) -1.5
Table E-8

IRT Parameters for the 1998 Reading ItemsReading to Perform a Task Scale, Grade 12

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.)6003R016101 R9 1A 0.548 (0.018) -0.583 (0.030) 0.000 (0.000) 0.073 (0.053) -0.073 (0.043)3R016102 R9 2 1.008 (0.063) -1.628 (0.103) 0.260 (0.043)3R016103 R9 3 1.135 (0.073) -0.417 (0.069) 0.355 (0.029)3R016104 R9 4A 0.767 (0.035) -2.040 (0.076) 0.000 (0.000)3R016105 R9 5 1.027 (0.066) -1.651 (0.106) 0.277 (0.044)3R016106 R9 6 0.899 (0.054) -0.324 (0.069) 0.208 (0.029)3R016107 R9 7A 0.619 (0.018) -0.949 (0.032) 0.000 (0.000) -0.310 (0.058) 0.310 (0.044)3R016108 R9 8A 0.369 (0.013) -0.608 (0.040) 0.000 (0.000) -0.628 (0.078) 0.628 (0.067)3R016109 R9 9A 0.406 (0.011) 0.214 (0.041) 0.000 (0.000) 1.953 (0.062) -1.953 (0.065)3R013401 R10 1 0.906 (0.055) -0.152 (0.064) 0.202 (0.026)3R013402 R10 2A 0.717 (0.029) -0.495 (0.035) 0.000 (0.000)3R013403 R10 3A 0.464 (0.011) -0.146 (0.026) 0.000 (0.000) -2.073 (0.083) 2.073 (0.081)3R013404 R10 4 0.775 (0.056) -0.909 (0.127) 0.343 (0.042)3R013405 R10 5A 0.953 (0.036) -0.960 (0.036) 0.000 (0.000)3R013406 R10 6A 0.694 (0.029) 0.361 (0.031) 0.000 (0.000)3R013407 R10 7A 0.636 (0.028) -1.120 (0.053) 0.000 (0.000)3R013408 R10 8 0.646 (0.045) -0.355 (0.104) 0.196 (0.035)3R013409 R10 9A 0.688 (0.030) -1.218 (0.054) 0.000 (0.000)3R013410 R10 10 0.696 (0.055) -1.410 (0.178) 0.348 (0.054)3R013411 R10 11A 0.560 (0.028) -1.619 (0.081) 0.000 (0.000)3R013412 R10 12A 0.324 (0.025) -2.605 (0.200) 0.000 (0.000)3R013901 R11 1 1.181 (0.088) 0.220 (0.054) 0.355 (0.021)3R013902 R11 2A 0.776 (0.030) -0.977 (0.041) 0.000 (0.000)3R013903 R11 3A 0.964 (0.035) 0.381 (0.024) 0.000 (0.000)3R013904 R11 4A 0.742 (0.031) -1.476 (0.055) 0.000 (0.000)3R013905 R11 5 1.262 (0.156) 1.890 (0.102) 0.241 (0.010)3R013906 R11 6A 0.549 (0.027) 0.727 (0.045) 0.000 (0.000)3R013907 R11 7 0.909 (0.061) 0.206 (0.057) 0.210 (0.023)3R013908 R11 8A 0.485 (0.025) 0.638 (0.049) 0.000 (0.000)3R013909 R11 9 0.783 (0.058) 0.020 (0.080) 0.242 (0.029)3R013910 R11 10A 0.968 (0.035) 0.340 (0.024) 0.000 (0.000)3R013911 R11 11 0.673 (0.051) -1.236 (0.165) 0.334 (0.050)3R013912 R11 12 0.567 (0.075) 0.967 (0.114) 0.294 (0.031)3R013913 R11 13A 0.511 (0.035) -0.248 (0.061) 0.000 (0.000)
Table E-8 (continued)

IRT Parameters for the 1998 Reading ItemsReading to Perform a Task Scale, Grade 12

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.)3R013914 R11 14 0.513 (0.074) 0.396 (0.199) 0.428 (0.042)3R013915 R11 15A 0.349 (0.018) 2.073 (0.108) 0.000 (0.000) -3.845 (0.249) 3.845 (0.273)
Table E-9

IRT Parameters for the 1998 Writing Items, Grade 4

NAEP ID Block Item aj (s.e.) bj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 (s.e.) 6021W004002 W3 1 1.004 (0.034) -0.244 (0.022) 2.429 (0.144) 1.391 (0.059) -0.087 (0.038) -1.7

1W004102 W4 1 0.974 (0.032) 0.173 (0.021) 2.524 (0.095) 1.283 (0.045) -0.354 (0.040) -1.1

1W004202 W5 1 1.215 (0.043) 0.049 (0.018) 1.904 (0.071) 1.235 (0.043) -0.118 (0.034) -1.0

1W004302 W6 1 1.279 (0.044) -0.051 (0.018) 2.320 (0.095) 1.163 (0.043) 0.086 (0.032) -1.2

1W004402 W7 1 1.228 (0.041) -0.307 (0.018) 1.971 (0.103) 1.276 (0.055) 0.277 (0.034) -1.11W004502 W8 1 1.342 (0.045) -0.222 (0.018) 1.948 (0.088) 1.202 (0.049) 0.388 (0.032) -1.21W004602 W9 1 1.336 (0.045) -0.503 (0.017) 2.035 (0.118) 1.153 (0.055) 0.269 (0.034) -1.11W004702 W10 1 0.908 (0.031) -0.186 (0.021) 1.749 (0.111) 1.416 (0.071) 0.406 (0.044) -1.11W004802 W11 1 0.720 (0.025) 0.192 (0.024) 1.817 (0.101) 1.668 (0.069) -0.035 (0.052) -1.2

1W004902 W12 1 0.788 (0.024) 0.280 (0.025) 2.065 (0.109) 2.172 (0.069) -0.071 (0.044) -1.6

1W005002 W13 1 0.969 (0.032) 0.389 (0.021) 2.457 (0.085) 1.605 (0.047) -0.063 (0.038) -1.3

1W005102 W14 1 1.334 (0.042) -0.090 (0.018) 2.420 (0.104) 1.480 (0.046) 0.204 (0.029) -1.5

1W005202 W15 1 0.903 (0.030) 0.309 (0.022) 2.317 (0.091) 1.536 (0.053) 0.150 (0.040) -1.51W005302 W16 1 1.678 (0.055) -0.113 (0.016) 2.115 (0.086) 1.602 (0.045) -0.006 (0.024) -1.5

1W005402 W17 1 1.085 (0.034) 0.057 (0.020) 2.521 (0.105) 1.595 (0.050) 0.058 (0.033) -1.6

1W005502 W18 1 0.850 (0.028) -0.208 (0.024) 2.374 (0.145) 1.460 (0.072) 0.413 (0.045) -1.51W005602 W19 1 0.831 (0.028) 0.176 (0.022) 2.051 (0.086) 1.313 (0.057) 0.365 (0.045) -1.31W005702 W20 1 1.098 (0.037) 0.103 (0.020) 1.893 (0.080) 1.426 (0.052) 0.305 (0.036) -1.51W005802 W21 1 0.948 (0.031) 0.109 (0.024) 2.189 (0.111) 1.747 (0.064) 0.386 (0.039) -1.81W005902 W22 1 0.795 (0.026) 0.281 (0.025) 1.886 (0.110) 2.059 (0.075) 0.477 (0.045) -1.6
Table E-10

IRT Parameters for the 1998 Writing Items, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 (s.e.) 1W006002 W3 1 0.859 (0.029) -0.298 (0.021) 2.718 (0.145) 0.996 (0.056) -0.134 (0.043) -1.3

1W006102 W4 1 1.062 (0.035) -0.257 (0.019) 2.634 (0.127) 1.089 (0.048) -0.064 (0.035) -1.5

1W006202 W5 1 0.941 (0.030) -0.025 (0.020) 2.669 (0.111) 1.070 (0.046) -0.132 (0.038) -1.4

1W006302 W6 1 1.031 (0.034) -0.123 (0.018) 1.351 (0.080) 1.434 (0.062) 0.475 (0.039) -0.8

1W006402 W7 1 0.750 (0.025) -0.343 (0.022) 1.894 (0.114) 1.088 (0.073) 0.401 (0.052) -1.21W006502 W8 1 1.065 (0.034) -0.322 (0.020) 2.283 (0.126) 1.360 (0.061) 0.431 (0.038) -1.21W006602 W9 1 1.120 (0.035) -0.359 (0.018) 2.379 (0.115) 0.940 (0.050) 0.297 (0.036) -1.41W006802 W11 1 0.898 (0.029) -0.237 (0.022) 2.557 (0.144) 1.489 (0.062) 0.099 (0.039) -1.51W006902 W12 1 1.048 (0.032) -0.070 (0.021) 2.440 (0.112) 1.507 (0.054) 0.393 (0.034) -1.71W007002 W13 1 0.940 (0.028) -0.263 (0.022) 2.312 (0.126) 1.475 (0.063) 0.318 (0.038) -1.81W007102 W14 1 1.261 (0.039) -0.122 (0.018) 1.956 (0.086) 1.497 (0.052) 0.417 (0.031) -1.31W007202 W15 1 0.730 (0.023) 0.184 (0.025) 2.553 (0.119) 1.682 (0.062) -0.057 (0.046) -1.7

1W007302 W16 1 0.869 (0.027) 0.048 (0.022) 1.988 (0.099) 1.548 (0.062) 0.402 (0.041) -1.9

1W007402 W17 1 0.994 (0.031) -0.147 (0.019) 1.920 (0.094) 1.390 (0.057) 0.314 (0.036) -1.3

1W007602 W19 1 1.480 (0.047) 0.009 (0.015) 1.953 (0.060) 1.329 (0.037) 0.041 (0.026) -1.11W007702 W20 1 1.309 (0.042) 0.189 (0.017) 2.154 (0.070) 1.405 (0.040) 0.220 (0.029) -1.31W007802 W21 1 1.149 (0.037) 0.045 (0.017) 1.694 (0.069) 1.337 (0.047) 0.251 (0.033) -1.11W007902 W22 1 0.778 (0.024) 0.036 (0.025) 2.404 (0.131) 1.830 (0.069) 0.156 (0.043) -2.11W008002 W23 1 1.203 (0.038) 0.056 (0.018) 2.499 (0.098) 1.373 (0.043) 0.115 (0.030) -1.41W008102 W24 1 0.991 (0.032) 0.066 (0.019) 1.886 (0.081) 1.266 (0.053) 0.455 (0.038) -1.2
Table E-11

IRT Parameters for the 1998 Writing Items, Grade 12

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 6041W008302 W3 1 0.558 (0.017) 0.169 (0.030) 0.671 (0.165) 2.533 (0.145) 1.464 (0.072) -1.11W008402 W4 1 0.649 (0.021) -0.236 (0.029) 1.449 (0.165) 1.785 (0.126) 1.393 (0.071) -0.7

1W008502 W5 1 0.711 (0.024) -0.508 (0.028) 2.267 (0.201) 1.605 (0.106) 0.789 (0.062) -0.51W008602 W6 1 0.769 (0.025) -0.710 (0.027) 3.195 (0.357) 1.616 (0.101) 0.505 (0.052) -1.51W008702 W7 1 1.040 (0.033) -0.467 (0.020) 2.491 (0.162) 1.230 (0.062) 0.269 (0.038) -1.31W008902 W9 1 1.250 (0.041) -0.730 (0.019) 2.347 (0.173) 1.193 (0.067) 0.436 (0.038) -1.21W009002 W10 1 1.102 (0.037) -0.361 (0.019) 1.433 (0.101) 1.524 (0.070) 0.355 (0.038) -0.91W009102 W11 1 0.941 (0.029) -0.263 (0.022) 2.019 (0.125) 1.314 (0.070) 0.586 (0.044) -1.71W009202 W12 1 0.980 (0.033) -0.146 (0.021) 1.891 (0.099) 1.302 (0.064) 0.641 (0.044) -0.61W009302 W13 1 0.842 (0.029) -0.661 (0.022) 1.753 (0.139) 1.041 (0.083) 0.501 (0.053) -1.11W009402 W14 1 0.841 (0.029) 0.019 (0.022) 1.844 (0.101) 1.363 (0.069) 0.720 (0.048) -0.71W009502 W15 1 0.897 (0.029) -0.301 (0.022) 1.581 (0.109) 1.234 (0.077) 0.832 (0.050) -0.91W009702 W17 1 1.163 (0.038) -0.121 (0.018) 1.860 (0.083) 1.318 (0.050) 0.165 (0.033) -1.11W009802 W18 1 0.830 (0.026) -0.289 (0.022) 1.927 (0.117) 1.129 (0.072) 0.642 (0.048) -1.51W009902 W19 1 0.859 (0.027) -0.154 (0.021) 1.898 (0.103) 1.290 (0.063) 0.326 (0.043) -1.61W010002 W20 1 0.868 (0.028) -0.355 (0.022) 2.076 (0.122) 1.080 (0.069) 0.624 (0.048) -1.21W010102 W21 1 1.210 (0.040) 0.199 (0.017) 2.288 (0.066) 0.924 (0.036) 0.076 (0.033) -1.21W010202 W22 1 0.702 (0.023) -0.175 (0.025) 2.055 (0.129) 1.339 (0.078) 0.445 (0.054) -1.31W010302 W23 1 0.880 (0.029) -0.163 (0.020) 2.088 (0.101) 1.087 (0.058) 0.223 (0.043) -1.11W010402 W24 1 0.760 (0.026) -0.474 (0.022) 1.890 (0.143) 1.272 (0.081) 0.190 (0.052) -1.2
Table E-12

IRT Parameters for the 1998 Civics Items, Grade 4

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) P030001 C3 1 0.884 (0.080) -0.470 (0.104) 0.279 (0.038)P030002 C3 2 0.679 (0.070) -0.439 (0.141) 0.263 (0.044)P030003 C3 3 1.345 (0.118) 0.361 (0.048) 0.234 (0.023)P030004 C3 4A 0.581 (0.028) -0.707 (0.042) 0.000 (0.000) 0.864 (0.0 75) -0.864 (0.052)P030005 C3 5A 0.452 (0.026) 0.207 (0.042) 0.000 (0.000) 0.499 (0.0 72) -0.499 (0.075)P030006 C3 6 2.673 (0.231) 0.578 (0.027) 0.248 (0.016)P030007 C3 7A 0.531 (0.029) 1.006 (0.049) 0.000 (0.000) 0.771 (0.0 57) -0.771 (0.085)P030008 C3 8 1.006 (0.122) 0.643 (0.072) 0.269 (0.028)P030009 C3 9 0.594 (0.119) 1.680 (0.184) 0.208 (0.030)P030010 C3 10A 0.518 (0.029) 1.539 (0.051) 0.000 (0.000) 0.874 (0.0 64) 0.400 (0.084) -1.2P030011 C3 11 0.743 (0.080) -1.578 (0.219) 0.342 (0.060)P030012 C3 12 0.966 (0.144) 1.260 (0.087) 0.183 (0.022)P030013 C3 13 1.037 (0.091) -0.118 (0.076) 0.209 (0.033)P030014 C3 14 1.523 (0.203) 1.710 (0.103) 0.145 (0.011)P030015 C3 15 0.806 (0.074) -0.890 (0.132) 0.240 (0.045)P040101 C4 1 1.274 (0.090) -0.315 (0.055) 0.151 (0.027)P040102 C4 2A 0.348 (0.033) 0.330 (0.084) 0.000 (0.000)P040103 C4 3 0.944 (0.083) -0.086 (0.080) 0.213 (0.033)P040104 C4 4 1.578 (0.173) 0.939 (0.045) 0.220 (0.017)P040105 C4 5A 0.483 (0.029) 0.603 (0.044) 0.000 (0.000) 0.242 (0.0 69) -0.242 (0.080)P040106 C4 6 0.874 (0.092) 0.250 (0.088) 0.244 (0.034)P040107 C4 7 1.242 (0.147) 0.811 (0.058) 0.257 (0.024)P040108 C4 8 0.826 (0.079) -0.126 (0.099) 0.235 (0.037)P040109 C4 9A 0.587 (0.035) 1.700 (0.084) 0.000 (0.000) -0.545 (0.0 78) 0.545 (0.115)P040110 C4 10 1.685 (0.214) 1.662 (0.096) 0.244 (0.012)P040111 C4 11A 0.490 (0.023) 0.213 (0.037) 0.000 (0.000) -0.565 (0.0 81) 0.565 (0.081)P040112 C4 12 1.121 (0.150) 1.220 (0.070) 0.163 (0.020)P040113 C4 13 1.311 (0.118) 0.266 (0.056) 0.236 (0.027)P040114 C4 14 1.257 (0.103) -0.207 (0.064) 0.208 (0.032)P040115 C4 15 1.101 (0.154) 0.945 (0.072) 0.268 (0.026)P040201 C5 1 1.957 (0.155) 0.690 (0.029) 0.134 (0.014)P040202 C5 2 0.479 (0.060) -0.379 (0.215) 0.269 (0.052)P040203 C5 3A 0.614 (0.025) 1.444 (0.040) 0.000 (0.000) -0.737 (0.0 80) 1.404 (0.094) -0.6P040204 C5 4 1.145 (0.087) -0.545 (0.074) 0.223 (0.033)
Table E-12 (continued)

IRT Parameters for the 1998 Civics Items, Grade 4

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 606P040205 C5 5 1.321 (0.103) -0.228 (0.060) 0.239 (0.030)P040206 C5 6A 0.474 (0.014) -0.276 (0.067) 0.000 (0.000) 2.959 (0.1 09) -2.959 (0.089)P040207 C5 7 1.391 (0.155) 0.796 (0.052) 0.276 (0.021)P040208 C5 8 0.666 (0.177) 2.318 (0.320) 0.225 (0.024)P040209 C5 9A 0.350 (0.027) 1.859 (0.132) 0.000 (0.000) -0.254 (0.0 99) 0.254 (0.147)P040210 C5 10 0.659 (0.066) -2.391 (0.260) 0.272 (0.062)P040211 C5 11 0.989 (0.142) 1.251 (0.082) 0.204 (0.022)P040212 C5 12 0.634 (0.167) 2.304 (0.316) 0.214 (0.026)P040213 C5 13 2.028 (0.178) 0.417 (0.036) 0.254 (0.021)P040214 C5 14 1.623 (0.174) 0.765 (0.046) 0.256 (0.021)P040215 C5 15 1.140 (0.109) 0.215 (0.068) 0.243 (0.031)P040301 C6 1 0.865 (0.084) -0.152 (0.100) 0.264 (0.038)P040302 C6 2 0.613 (0.061) -0.695 (0.158) 0.232 (0.046)P040303 C6 3 0.957 (0.137) 1.028 (0.080) 0.259 (0.026)P040304 C6 4A 0.643 (0.035) 0.918 (0.042) 0.000 (0.000) 0.099 (0.0 53) -0.099 (0.070)P040305 C6 5 1.008 (0.111) 0.458 (0.075) 0.274 (0.030)P040306 C6 6 0.691 (0.077) -0.396 (0.153) 0.317 (0.046)P040307 C6 7 0.782 (0.167) 1.881 (0.189) 0.166 (0.022)P040308 C6 8 1.017 (0.097) 0.013 (0.082) 0.276 (0.034)P040309 C6 9 0.725 (0.066) -0.463 (0.117) 0.202 (0.040)P040310 C6 10A 0.551 (0.033) 1.441 (0.062) 0.000 (0.000) 0.796 (0.0 56) -0.796 (0.103)P040311 C6 11A 0.424 (0.033) 1.736 (0.109) 0.000 (0.000) 0.355 (0.0 77) -0.355 (0.130)P040312 C6 12 0.588 (0.057) -2.295 (0.250) 0.240 (0.057)P040313 C6 13 1.587 (0.179) 1.499 (0.075) 0.182 (0.012)P040314 C6 14 1.542 (0.119) -0.440 (0.058) 0.217 (0.032)P040315 C6 15 0.880 (0.078) -0.266 (0.093) 0.208 (0.037)P040401 C7 1 0.672 (0.066) -0.045 (0.108) 0.198 (0.036)P040402 C7 2A 0.464 (0.027) 1.161 (0.062) 0.000 (0.000) -0.245 (0.0 74) 0.245 (0.096)P040403 C7 3 0.499 (0.072) 0.388 (0.176) 0.257 (0.046)P040404 C7 4A 0.277 (0.014) -0.368 (0.049) 0.000 (0.000) 1.508 (0.1 47) -1.916 (0.142) 0.4P040405 C7 5 0.935 (0.075) -0.706 (0.096) 0.204 (0.037)P040406 C7 6 0.612 (0.119) 1.370 (0.150) 0.273 (0.035)P040407 C7 7 1.342 (0.186) 1.154 (0.061) 0.258 (0.019)P040408 C7 8 1.189 (0.112) 0.004 (0.075) 0.325 (0.033)P040409 C7 9A 0.623 (0.039) 1.684 (0.066) 0.000 (0.000) 0.707 (0.0 51) -0.707 (0.109)
Table E-12 (continued)

IRT Parameters for the 1998 Civics Items, Grade 4

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) P040410 C7 10 0.807 (0.085) -0.370 (0.132) 0.322 (0.044)P040411 C7 11 0.898 (0.108) 0.727 (0.078) 0.240 (0.029)P040412 C7 12A 0.432 (0.020) -0.539 (0.043) 0.000 (0.000) 1.401 (0.1 32) 0.189 (0.085) -1.5P040413 C7 13 0.752 (0.102) 0.876 (0.097) 0.220 (0.032)P040414 C7 14 0.998 (0.088) -0.009 (0.076) 0.214 (0.033)P040415 C7 15 0.790 (0.078) -0.563 (0.131) 0.260 (0.044)P040501 C8 1 1.491 (0.213) 1.193 (0.071) 0.400 (0.018)P040502 C8 2A 0.580 (0.026) 0.412 (0.028) 0.000 (0.000) 0.754 (0.0 67) 0.364 (0.062) -1.1P040503 C8 3 1.238 (0.110) 0.017 (0.065) 0.283 (0.031)P040504 C8 4 0.806 (0.104) 0.890 (0.088) 0.223 (0.029)P040505 C8 5 0.827 (0.152) 1.561 (0.130) 0.226 (0.024)P040507 C8 7A 0.444 (0.028) 2.758 (0.094) 0.000 (0.000) 1.702 (0.0 68) -1.702 (0.241)P040508 C8 8 0.486 (0.115) 2.108 (0.287) 0.230 (0.035)P040509 C8 9 0.930 (0.094) 0.019 (0.093) 0.284 (0.036)P040510 C8 10A 0.316 (0.017) -0.278 (0.057) 0.000 (0.000) -1.080 (0.1 28) 1.080 (0.120)P040511 C8 11 0.808 (0.084) 0.164 (0.096) 0.234 (0.036)P040512 C8 12 0.651 (0.088) -0.017 (0.170) 0.328 (0.049)P040513 C8 13 1.637 (0.168) 0.794 (0.043) 0.201 (0.019)P040514 C8 14 1.059 (0.134) 0.822 (0.072) 0.256 (0.028)P040515 C8 15 0.746 (0.074) -0.658 (0.139) 0.253 (0.046)
Table E-13

IRT Parameters for the 1998 Civics Items, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 608P040701 C3 1 1.394 (0.169) 1.414 (0.073) 0.255 (0.014)P040702 C3 2 0.512 (0.052) -0.110 (0.147) 0.182 (0.041)P040703 C3 3 0.700 (0.038) 1.424 (0.051) 0.000 (0.000) 0.634 (0.0 45) -0.634 (0.087)P040704 C3 4 1.945 (0.179) 0.416 (0.041) 0.333 (0.020)P040705 C3 5 0.928 (0.050) 1.438 (0.048) 0.000 (0.000) 0.019 (0.0 44) -0.019 (0.073)P040706 C3 6 0.956 (0.181) 1.408 (0.107) 0.316 (0.023)P040707 C3 7 1.720 (0.149) 1.112 (0.041) 0.187 (0.013)P040708 C3 8 0.736 (0.037) 1.054 (0.039) 0.000 (0.000) 0.576 (0.0 43) -0.576 (0.067)P040709 C3 9 1.095 (0.084) -0.619 (0.082) 0.230 (0.037)P040710 C3 10 0.550 (0.062) 0.419 (0.126) 0.178 (0.037)P040711 C3 11 0.911 (0.075) 0.085 (0.072) 0.169 (0.029)P040712 C3 12 0.608 (0.079) 0.370 (0.143) 0.261 (0.042)P040713 C3 13 1.205 (0.106) -0.587 (0.091) 0.337 (0.040)P040714 C3 14 1.049 (0.101) -0.101 (0.093) 0.334 (0.037)P040715 C3 15 0.702 (0.035) 0.733 (0.034) 0.000 (0.000) 0.370 (0.0 47) -0.370 (0.060)P040716 C3 16 1.225 (0.188) 1.732 (0.117) 0.203 (0.014)P040717 C3 17 0.965 (0.164) 1.111 (0.089) 0.328 (0.027)P040718 C3 18 0.652 (0.085) 0.698 (0.115) 0.231 (0.035)P040719 C3 19 1.119 (0.098) -0.506 (0.093) 0.296 (0.040)P040801 C4 1 1.562 (0.132) 0.149 (0.050) 0.293 (0.024)P040802 C4 2 1.937 (0.151) 0.269 (0.036) 0.225 (0.020)P040803 C4 3 0.648 (0.031) 0.401 (0.032) 0.000 (0.000) 0.525 (0.0 51) -0.525 (0.056)P040804 C4 4 0.543 (0.053) -1.719 (0.248) 0.285 (0.061)P040805 C4 5 1.385 (0.154) 1.038 (0.048) 0.193 (0.016)P040806 C4 6 1.511 (0.143) 1.174 (0.044) 0.105 (0.012)P040807 C4 7 0.448 (0.019) 0.822 (0.042) 0.000 (0.000) 1.650 (0.0 73) -0.921 (0.087) -0.7P040808 C4 8 0.710 (0.233) 2.960 (0.572) 0.183 (0.017)P040809 C4 9 1.386 (0.171) 1.052 (0.052) 0.237 (0.018)P040810 C4 10 1.140 (0.100) 0.148 (0.065) 0.250 (0.028)P040811 C4 11 1.393 (0.152) 0.872 (0.048) 0.239 (0.019)P040812 C4 12 0.826 (0.127) 1.080 (0.091) 0.270 (0.028)P040813 C4 13 0.689 (0.037) 1.588 (0.048) 0.000 (0.000) 0.647 (0.0 49) 0.051 (0.077) -0.6P040814 C4 14 1.323 (0.145) 1.198 (0.052) 0.124 (0.014)P040815 C4 15 1.624 (0.160) 0.910 (0.039) 0.173 (0.016)
Table E-13 (continued)

IRT Parameters for the 1998 Civics Items, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) P040816 C4 16 0.877 (0.110) 0.948 (0.073) 0.184 (0.025)P040817 C4 17 1.072 (0.093) -0.198 (0.082) 0.274 (0.034)P040818 C4 18 1.491 (0.132) 0.397 (0.048) 0.260 (0.022)P040819 C4 19 1.896 (0.150) 1.192 (0.042) 0.202 (0.012)P040901 C5 1 0.699 (0.065) -1.133 (0.170) 0.279 (0.052)P040902 C5 2 0.887 (0.107) 1.225 (0.075) 0.125 (0.019)P040903 C5 3 0.606 (0.028) 0.307 (0.036) 0.000 (0.000) 0.905 (0.0 56) -0.905 (0.060)P040904 C5 4 0.828 (0.088) 0.417 (0.088) 0.237 (0.032)P040906 C5 6 0.704 (0.035) 1.355 (0.051) 0.000 (0.000) -0.405 (0.0 61) 0.405 (0.084)P040907 C5 7 0.739 (0.097) 0.941 (0.092) 0.205 (0.029)P040908 C5 8 0.891 (0.073) -0.142 (0.082) 0.187 (0.033)P040909 C5 9 1.547 (0.118) -0.277 (0.055) 0.257 (0.029)P040910 C5 10 0.535 (0.028) 1.386 (0.054) 0.000 (0.000) 1.080 (0.0 56) -1.080 (0.103)P040911 C5 11 0.317 (0.061) 1.629 (0.312) 0.254 (0.044)P040912 C5 12 1.375 (0.179) 1.519 (0.085) 0.250 (0.014)P040913 C5 13 0.894 (0.041) 0.456 (0.026) 0.000 (0.000) 0.455 (0.0 39) -0.455 (0.044)P040914 C5 14 0.995 (0.095) 0.502 (0.064) 0.179 (0.026)P040915 C5 15 1.649 (0.195) 1.566 (0.084) 0.291 (0.013)P040916 C5 16 1.484 (0.167) 1.112 (0.053) 0.261 (0.017)P040917 C5 17 0.765 (0.092) 0.911 (0.083) 0.165 (0.027)P040918 C5 18 1.090 (0.109) 0.139 (0.082) 0.307 (0.033)P040919 C5 19 1.097 (0.113) 0.442 (0.069) 0.245 (0.029)P041001 C6 1 0.674 (0.075) 0.116 (0.128) 0.259 (0.040)P041002 C6 2 0.822 (0.139) 1.205 (0.105) 0.332 (0.027)P041003 C6 3 1.150 (0.062) 0.868 (0.035) 0.000 (0.000)P041004 C6 4 0.901 (0.080) -0.884 (0.128) 0.312 (0.046)P041005 C6 5 0.873 (0.065) -0.167 (0.074) 0.154 (0.029)P041006 C6 6 1.464 (0.193) 0.936 (0.056) 0.373 (0.020)P041007 C6 7 0.768 (0.038) 0.971 (0.035) 0.000 (0.000) 0.200 (0.0 44) -0.200 (0.060)P041008 C6 8 0.731 (0.065) -0.185 (0.105) 0.201 (0.037)P041009 C6 9 1.283 (0.112) 0.215 (0.058) 0.259 (0.027)P041010 C6 10 0.724 (0.105) 1.176 (0.100) 0.219 (0.027)P041011 C6 11 1.437 (0.140) 0.269 (0.060) 0.356 (0.026)P041012 C6 12 1.415 (0.114) -0.013 (0.056) 0.287 (0.027)P041013 C6 13 0.338 (0.015) 0.533 (0.063) 0.000 (0.000) 1.828 (0.0 94) -1.828 (0.110)
Table E-13 (continued)

IRT Parameters for the 1998 Civics Items, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 610P041014 C6 14 0.330 (0.018) -1.010 (0.071) 0.000 (0.000) 1.336 (0.1 25) -1.336 (0.089)P041015 C6 15 1.298 (0.105) 0.392 (0.047) 0.178 (0.022)P041016 C6 16 1.364 (0.137) 0.748 (0.048) 0.229 (0.020)P041017 C6 17 1.886 (0.190) 0.771 (0.042) 0.344 (0.018)P041018 C6 18 1.377 (0.119) -0.018 (0.063) 0.311 (0.029)P041019 C6 19 0.571 (0.083) 0.884 (0.130) 0.220 (0.037)P041101 C7 1 0.402 (0.054) -0.866 (0.342) 0.323 (0.066)P041102 C7 2 0.643 (0.032) 0.743 (0.035) 0.000 (0.000) 0.171 (0.0 52) -0.171 (0.063)P041103 C7 3 0.985 (0.080) 0.420 (0.055) 0.141 (0.022)P041104 C7 4 1.218 (0.169) 1.255 (0.064) 0.213 (0.018)P041105 C7 5 0.978 (0.148) 1.222 (0.080) 0.253 (0.023)P041106 C7 6 0.222 (0.008) 1.775 (0.065) 0.000 (0.000) -0.287 (0.1 82) 4.257 (0.178) -3.9P041107 C7 7 1.293 (0.129) 0.084 (0.075) 0.408 (0.029)P041108 C7 8 1.141 (0.088) -0.164 (0.066) 0.218 (0.029)P041109 C7 9 1.887 (0.261) 1.899 (0.121) 0.265 (0.011)P041110 C7 10 0.979 (0.142) 1.211 (0.078) 0.236 (0.022)P041111 C7 11 0.834 (0.036) 1.003 (0.032) 0.000 (0.000) -0.273 (0.0 49) 0.273 (0.060)P041112 C7 12 0.442 (0.051) -1.348 (0.302) 0.293 (0.064)P041113 C7 13 1.075 (0.141) 0.644 (0.082) 0.402 (0.027)P041114 C7 14 1.356 (0.235) 2.124 (0.179) 0.158 (0.011)P041115 C7 15 1.111 (0.189) 1.309 (0.087) 0.327 (0.021)P041116 C7 16 0.795 (0.043) 1.143 (0.040) 0.000 (0.000) 0.187 (0.0 46) -0.187 (0.066)P041117 C7 17 0.947 (0.083) -1.046 (0.126) 0.280 (0.045)P041118 C7 18 1.591 (0.158) 1.182 (0.052) 0.245 (0.015)P041119 C7 19 0.992 (0.104) 0.183 (0.088) 0.303 (0.033)P041201 C8 1 0.709 (0.074) 0.040 (0.117) 0.250 (0.038)P041202 C8 2 0.546 (0.028) 0.453 (0.037) 0.000 (0.000) 0.393 (0.0 59) -0.393 (0.066)P041203 C8 3 0.450 (0.045) -2.227 (0.296) 0.268 (0.062)P041205 C8 5 0.226 (0.016) -2.490 (0.184) 0.000 (0.000) -0.782 (0.2 15) 0.782 (0.150)P041206 C8 6 0.768 (0.064) -0.097 (0.088) 0.173 (0.032)P041207 C8 7 0.972 (0.123) 0.624 (0.085) 0.360 (0.028)P041208 C8 8 0.850 (0.094) 0.401 (0.091) 0.279 (0.032)P041209 C8 9 1.280 (0.147) 0.766 (0.057) 0.305 (0.022)
Table E-13 (continued)

IRT Parameters for the 1998 Civics Items, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) P041210 C8 10 0.512 (0.071) 0.472 (0.167) 0.249 (0.044)P041211 C8 11 1.185 (0.163) 1.088 (0.064) 0.274 (0.021)P041212 C8 12 0.960 (0.106) 0.354 (0.087) 0.327 (0.031)P041213 C8 13 0.557 (0.028) 1.314 (0.043) 0.000 (0.000) 0.487 (0.0 61) 0.375 (0.078) -0.8P041214 C8 14 1.620 (0.181) 1.411 (0.071) 0.334 (0.014)P041215 C8 15 1.040 (0.083) -0.367 (0.081) 0.230 (0.034)P041216 C8 16 1.959 (0.177) 0.735 (0.034) 0.212 (0.016)P041217 C8 17 0.675 (0.139) 1.635 (0.158) 0.256 (0.029)P041218 C8 18 2.069 (0.209) 0.610 (0.040) 0.339 (0.019)P041219 C8 19 0.881 (0.073) -0.708 (0.109) 0.238 (0.041)P041301 C9 1 0.975 (0.126) 0.602 (0.090) 0.371 (0.030)P041302 C9 2 1.170 (0.109) 0.408 (0.060) 0.246 (0.026)P041303 C9 3 1.460 (0.133) 0.285 (0.054) 0.320 (0.025)P041304 C9 4 1.637 (0.133) -0.436 (0.061) 0.328 (0.032)P041305 C9 5 1.115 (0.161) 0.901 (0.077) 0.392 (0.025)P041306 C9 6 1.186 (0.093) -0.702 (0.082) 0.266 (0.038)P041307 C9 7 0.823 (0.035) 1.018 (0.028) 0.000 (0.000) 0.851 (0.0 39) -0.612 (0.063) -0.2P041308 C9 8 1.699 (0.166) 1.535 (0.067) 0.216 (0.011)P041309 C9 9 0.522 (0.026) 0.808 (0.044) 0.000 (0.000) 0.881 (0.0 58) -0.881 (0.078)P041310 C9 10 1.666 (0.180) 0.795 (0.045) 0.300 (0.019)P041311 C9 11 0.301 (0.062) 1.997 (0.381) 0.315 (0.041)P041312 C9 12 1.283 (0.158) 1.441 (0.072) 0.183 (0.014)P041313 C9 13 1.146 (0.104) -1.226 (0.123) 0.342 (0.049)P041314 C9 14 1.370 (0.142) 0.816 (0.049) 0.235 (0.020)P041315 C9 15 0.560 (0.028) 0.017 (0.035) 0.000 (0.000) 0.524 (0.0 62) -0.524 (0.059)P041316 C9 16 1.223 (0.141) 0.746 (0.061) 0.291 (0.024)P041317 C9 17 1.819 (0.164) 0.869 (0.037) 0.214 (0.015)P041318 C9 18 0.841 (0.100) 0.574 (0.092) 0.278 (0.032)P041319 C9 19 0.730 (0.098) 0.607 (0.116) 0.310 (0.036)P040601 C10 1 0.564 (0.069) 0.189 (0.152) 0.247 (0.043)P040602 C10 2 0.533 (0.028) 0.361 (0.036) 0.000 (0.000) 0.374 (0.0 61) -0.374 (0.066)P040603 C10 3 2.097 (0.150) 1.369 (0.044) 0.150 (0.010)P040604 C10 4 0.853 (0.134) 1.111 (0.094) 0.299 (0.027)P040605 C10 5 0.939 (0.332) 2.533 (0.464) 0.290 (0.016)P040606 C10 6 1.385 (0.291) 2.065 (0.207) 0.296 (0.012)
Table E-13 (continued)

IRT Parameters for the 1998 Civics Items, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 612P040607 C10 7 0.706 (0.062) -0.091 (0.100) 0.176 (0.035)P040608 C10 8 0.567 (0.026) 0.700 (0.030) 0.000 (0.000) 0.310 (0.0 61) 0.021 (0.073) -0.3P040609 C10 9 0.972 (0.242) 1.895 (0.196) 0.285 (0.019)P040610 C10 10 1.624 (0.183) 1.697 (0.087) 0.180 (0.011)P040611 C10 11 0.631 (0.063) -0.457 (0.152) 0.257 (0.046)P040612 C10 12 1.160 (0.102) -0.818 (0.103) 0.343 (0.043)P040613 C10 13 0.667 (0.035) 1.103 (0.045) 0.000 (0.000) 0.076 (0.0 51) -0.076 (0.072)P040614 C10 14 1.116 (0.107) 0.484 (0.060) 0.229 (0.026)P040615 C10 15 1.045 (0.083) -0.559 (0.088) 0.255 (0.037)P040616 C10 16 1.297 (0.100) -0.402 (0.068) 0.243 (0.033)P040617 C10 17 0.766 (0.101) 0.739 (0.098) 0.260 (0.033)P040618 C10 18 0.980 (0.103) 0.658 (0.065) 0.188 (0.026)
Table E-14

IRT Parameters for the 1998 Civics Items, Grade 12

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) P041501 C3 1 0.833 (0.087) 0.354 (0.089) 0.243 (0.032)P041502 C3 2 0.726 (0.096) 0.721 (0.105) 0.278 (0.033)P041503 C3 3A 0.792 (0.038) 0.849 (0.033) 0.000 (0.000) 0.203 (0.044) -0.203 (0.057)P041504 C3 4 1.945 (0.167) 0.502 (0.035) 0.229 (0.018)P041505 C3 5A 0.582 (0.028) 0.200 (0.035) 0.000 (0.000) 0.664 (0.058) -0.664 (0.060)P041506 C3 6 1.377 (0.136) 0.392 (0.058) 0.324 (0.024)P041507 C3 7 1.125 (0.087) -0.510 (0.077) 0.252 (0.033)P041508 C3 8 1.181 (0.126) 0.492 (0.066) 0.322 (0.026)P041509 C3 9A 0.647 (0.029) 0.786 (0.030) 0.000 (0.000) 0.785 (0.052) -0.479 (0.068) -0.3P041510 C3 10 1.119 (0.122) 0.969 (0.057) 0.194 (0.019)P041511 C3 11A 0.726 (0.028) 0.073 (0.027) 0.000 (0.000) -0.210 (0.054) 0.210 (0.053)P041512 C3 12 1.182 (0.095) -0.428 (0.077) 0.275 (0.034)P041513 C3 13 1.008 (0.101) 0.275 (0.079) 0.303 (0.029)P041514 C3 14 1.065 (0.102) 0.797 (0.055) 0.165 (0.020)P041515 C3 15 2.040 (0.183) 0.619 (0.034) 0.225 (0.016)P041516 C3 16 1.258 (0.145) 0.689 (0.061) 0.317 (0.023)P041517 C3 17 1.115 (0.129) 0.202 (0.093) 0.430 (0.031)P041518 C3 18 1.736 (0.138) 0.566 (0.034) 0.132 (0.016)P041519 C3 19 1.938 (0.160) -0.074 (0.046) 0.293 (0.025)P041601 C4 1 0.816 (0.070) -1.108 (0.136) 0.261 (0.046)P041602 C4 2 0.806 (0.075) -0.591 (0.127) 0.276 (0.044)P041603 C4 3 2.163 (0.183) 0.717 (0.031) 0.222 (0.015)P041604 C4 4 1.352 (0.131) 0.399 (0.058) 0.307 (0.025)P041605 C4 5 1.206 (0.103) 0.035 (0.066) 0.278 (0.029)P041606 C4 6A 0.541 (0.027) 0.175 (0.035) 0.000 (0.000) 0.175 (0.064) -0.175 (0.065)P041607 C4 7 0.706 (0.097) 0.835 (0.108) 0.268 (0.033)P041608 C4 8 1.786 (0.155) 1.089 (0.042) 0.235 (0.014)P041609 C4 9 0.682 (0.071) 0.079 (0.118) 0.243 (0.038)P041610 C4 10 1.835 (0.150) 0.954 (0.035) 0.156 (0.013)P041611 C4 11 0.829 (0.088) -0.071 (0.119) 0.354 (0.038)P041612 C4 12 1.369 (0.167) 1.019 (0.055) 0.282 (0.019)P041613 C4 13A 0.418 (0.021) 0.135 (0.049) 0.000 (0.000) 1.175 (0.081) -1.175 (0.082)P041614 C4 14A 0.503 (0.025) -0.816 (0.051) 0.000 (0.000) 0.257 (0.086) -0.257 (0.064)P041615 C4 15 0.716 (0.096) 0.648 (0.115) 0.299 (0.035)
Table E-14 (continued)

IRT Parameters for the 1998 Civics Items, Grade 12

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 614P041616 C4 16 1.408 (0.241) 2.205 (0.176) 0.208 (0.011)P041617 C4 17 1.086 (0.101) 0.015 (0.080) 0.305 (0.032)P041618 C4 18 1.533 (0.149) 0.536 (0.048) 0.273 (0.022)P041619 C4 19 0.760 (0.094) 0.561 (0.104) 0.252 (0.035)P041701 C5 1 0.900 (0.083) -0.304 (0.103) 0.264 (0.039)P041702 C5 2 0.943 (0.086) 0.274 (0.072) 0.204 (0.029)P041703 C5 3 1.224 (0.156) 0.960 (0.060) 0.264 (0.021)P041704 C5 4 2.180 (0.206) 1.328 (0.057) 0.375 (0.013)P041705 C5 5A 0.707 (0.041) -0.106 (0.045) 0.000 (0.000)P041706 C5 6A 0.643 (0.032) 1.027 (0.035) 0.000 (0.000) 0.608 (0.054) -0.076 (0.071) -0.5P041707 C5 7 1.559 (0.174) 1.069 (0.051) 0.275 (0.017)P041708 C5 8 1.069 (0.125) 0.780 (0.065) 0.249 (0.024)P041709 C5 9 1.509 (0.158) 0.226 (0.065) 0.400 (0.027)P041710 C5 10 0.968 (0.087) -0.530 (0.105) 0.290 (0.041)P041711 C5 11A 0.712 (0.038) 1.046 (0.042) 0.000 (0.000) 0.175 (0.050) -0.175 (0.070)P041712 C5 12 0.441 (0.060) 0.155 (0.214) 0.250 (0.050)P041713 C5 13A 0.617 (0.026) 0.120 (0.032) 0.000 (0.000) -0.340 (0.066) 0.340 (0.065)P041714 C5 14 0.844 (0.137) 0.847 (0.109) 0.362 (0.033)P041715 C5 15 1.217 (0.137) 0.595 (0.063) 0.283 (0.025)P041716 C5 16 1.000 (0.103) 0.423 (0.073) 0.221 (0.029)P041717 C5 17 1.343 (0.116) 0.126 (0.057) 0.204 (0.027)P041718 C5 18 1.150 (0.115) 0.271 (0.070) 0.249 (0.029)P041719 C5 19 1.278 (0.123) 0.381 (0.058) 0.213 (0.025)P041801 C6 1 0.791 (0.091) 0.349 (0.105) 0.276 (0.036)P041802 C6 2 0.418 (0.044) -2.017 (0.297) 0.271 (0.060)P041803 C6 3 0.829 (0.084) -0.025 (0.111) 0.291 (0.039)P041804 C6 4A 0.592 (0.029) 0.058 (0.033) 0.000 (0.000) 0.277 (0.060) -0.277 (0.058)P041805 C6 5 0.628 (0.057) -2.018 (0.219) 0.278 (0.060)P041806 C6 6A 0.506 (0.027) 0.417 (0.038) 0.000 (0.000) 0.214 (0.066) -0.214 (0.071)P041807 C6 7 0.864 (0.076) -1.019 (0.137) 0.287 (0.048)P041808 C6 8 1.609 (0.175) 1.604 (0.078) 0.258 (0.013)P041809 C6 9 0.818 (0.102) 0.693 (0.093) 0.277 (0.031)P041811 C6 11 1.287 (0.143) 0.645 (0.060) 0.325 (0.024)P041812 C6 12 1.277 (0.118) 0.633 (0.048) 0.177 (0.021)P041813 C6 13 1.366 (0.150) 0.712 (0.054) 0.307 (0.022)
Table E-14 (continued)

IRT Parameters for the 1998 Civics Items, Grade 12

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) P041814 C6 14 1.052 (0.169) 1.465 (0.091) 0.237 (0.020)P041815 C6 15A 0.926 (0.041) 0.829 (0.027) 0.000 (0.000) 0.038 (0.040) -0.038 (0.050)P041816 C6 16 0.981 (0.124) 0.557 (0.091) 0.378 (0.030)P041817 C6 17 0.902 (0.088) 0.094 (0.093) 0.257 (0.035)P041818 C6 18 1.454 (0.160) 0.238 (0.073) 0.447 (0.028)P041819 C6 19 1.175 (0.121) 0.878 (0.052) 0.179 (0.020)P041901 C7 1 0.659 (0.066) -1.283 (0.202) 0.313 (0.058)P041902 C7 2A 0.584 (0.023) 0.687 (0.035) 0.000 (0.000) -0.854 (0.074) 0.854 (0.081)P041903 C7 3 1.113 (0.148) 0.846 (0.074) 0.351 (0.024)P041904 C7 4 1.569 (0.154) 0.435 (0.051) 0.316 (0.023)P041905 C7 5A 0.522 (0.019) -0.383 (0.040) 0.000 (0.000) 3.043 (0.131) -0.905 (0.062) -2.1P041906 C7 6 1.262 (0.185) 1.175 (0.071) 0.324 (0.020)P041907 C7 7A 0.853 (0.038) 1.152 (0.030) 0.000 (0.000) 0.193 (0.046) 0.173 (0.063) -0.3P041908 C7 8 1.160 (0.119) 0.467 (0.066) 0.287 (0.026)P041909 C7 9 1.164 (0.160) 1.238 (0.070) 0.229 (0.019)P041910 C7 10 0.860 (0.113) 0.731 (0.094) 0.312 (0.030)P041911 C7 11 0.890 (0.110) 0.832 (0.080) 0.233 (0.028)P041912 C7 12A 0.440 (0.025) 1.129 (0.060) 0.000 (0.000) 1.003 (0.070) -1.003 (0.105)P041913 C7 13 1.720 (0.165) 0.984 (0.042) 0.214 (0.015)P041914 C7 14 0.834 (0.082) -0.440 (0.125) 0.299 (0.044)P041915 C7 15 1.507 (0.191) 0.895 (0.055) 0.338 (0.020)P041916 C7 16 1.072 (0.128) 0.959 (0.065) 0.204 (0.022)P041917 C7 17 1.810 (0.172) 0.541 (0.041) 0.242 (0.020)P041918 C7 18 0.857 (0.110) 0.306 (0.119) 0.367 (0.038)P041919 C7 19 1.419 (0.136) -0.012 (0.069) 0.337 (0.032)P042001 C8 1 1.118 (0.226) 1.319 (0.107) 0.468 (0.021)P042002 C8 2A 0.838 (0.038) 0.382 (0.026) 0.000 (0.000) 0.290 (0.042) -0.290 (0.046)P042003 C8 3 0.940 (0.111) 0.122 (0.111) 0.394 (0.037)P042004 C8 4 1.195 (0.119) 0.320 (0.067) 0.298 (0.028)P042005 C8 5 0.202 (0.043) 2.503 (0.599) 0.328 (0.039)P042006 C8 6 1.590 (0.153) 0.375 (0.050) 0.316 (0.023)P042007 C8 7 2.079 (0.194) 0.615 (0.034) 0.246 (0.017)P042008 C8 8A 0.825 (0.038) 0.380 (0.027) 0.000 (0.000) 0.425 (0.042) -0.425 (0.046)P042009 C8 9A 0.589 (0.024) 0.188 (0.031) 0.000 (0.000) -0.415 (0.066) 0.415 (0.066)P042010 C8 10 0.554 (0.057) -1.394 (0.240) 0.302 (0.061)
Table E-14 (continued)

IRT Parameters for the 1998 Civics Items, Grade 12

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 616P042011 C8 11 0.629 (0.128) 1.454 (0.156) 0.306 (0.033)P042012 C8 12A 0.612 (0.026) -0.702 (0.041) 0.000 (0.000) -0.203 (0.075) 0.203 (0.060)P042013 C8 13 2.380 (0.166) 1.190 (0.038) 0.211 (0.012)P042014 C8 14 0.476 (0.070) 0.284 (0.213) 0.286 (0.051)P042015 C8 15 2.674 (0.177) 1.323 (0.040) 0.193 (0.011)P042016 C8 16 1.343 (0.180) 1.211 (0.063) 0.234 (0.018)P042017 C8 17 1.791 (0.187) 1.075 (0.052) 0.350 (0.016)P042018 C8 18 1.460 (0.185) 1.226 (0.065) 0.287 (0.017)P042019 C8 19 0.869 (0.158) 1.430 (0.118) 0.275 (0.025)P042101 C9 1 1.073 (0.133) 0.404 (0.090) 0.420 (0.030)P042102 C9 2A 0.952 (0.041) 0.892 (0.029) 0.000 (0.000) 0.819 (0.035) -0.819 (0.056)P042103 C9 3A 0.635 (0.028) 1.693 (0.041) 0.000 (0.000) 1.639 (0.049) -0.333 (0.080) -1.3P042104 C9 4 0.869 (0.083) -1.526 (0.177) 0.335 (0.059)P042105 C9 5 1.207 (0.130) 1.056 (0.051) 0.133 (0.017)P042106 C9 6 2.823 (0.258) 0.496 (0.029) 0.314 (0.017)P042107 C9 7 1.344 (0.166) 0.610 (0.066) 0.401 (0.025)P042108 C9 8 1.301 (0.131) 0.011 (0.079) 0.417 (0.031)P042109 C9 9 0.996 (0.124) 0.844 (0.071) 0.261 (0.025)P042110 C9 10A 0.458 (0.022) 0.139 (0.046) 0.000 (0.000) 1.183 (0.075) -1.183 (0.076)P042111 C9 11 1.239 (0.132) 0.623 (0.058) 0.276 (0.024)P042112 C9 12 1.833 (0.194) 1.426 (0.068) 0.346 (0.013)P042113 C9 13 1.476 (0.159) 0.294 (0.065) 0.410 (0.027)P042114 C9 14 0.636 (0.120) 1.654 (0.160) 0.222 (0.028)P042115 C9 15 2.110 (0.191) 0.356 (0.038) 0.294 (0.021)P042116 C9 16 2.792 (0.174) 1.247 (0.035) 0.226 (0.011)P042117 C9 17 0.931 (0.078) -0.415 (0.095) 0.233 (0.038)P042118 C9 18 0.964 (0.087) -0.479 (0.106) 0.303 (0.041)P042119 C9 19 0.911 (0.101) 0.675 (0.074) 0.220 (0.027)P041401 C10 1 0.488 (0.051) -0.747 (0.205) 0.244 (0.051)P041402 C10 2 0.822 (0.078) -0.343 (0.117) 0.288 (0.040)P041403 C10 3 1.072 (0.108) 0.338 (0.072) 0.294 (0.028)P041404 C10 4A 0.646 (0.023) 0.431 (0.033) 0.000 (0.000) 2.034 (0.067) -0.556 (0.054) -1.4P041405 C10 5 0.768 (0.087) -0.462 (0.164) 0.394 (0.048)P041406 C10 6 1.323 (0.142) -0.765 (0.116) 0.520 (0.041)P041407 C10 7 0.955 (0.207) 1.650 (0.141) 0.328 (0.022)
Table E-14 (continued)

IRT Parameters for the 1998 Civics Items, Grade 12

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) P041408 C10 8A 0.397 (0.023) -0.149 (0.047) 0.000 (0.000) 0.149 (0.090) -0.149 (0.084)P041409 C10 9 2.771 (0.211) 0.606 (0.027) 0.273 (0.015)P041410 C10 10 0.993 (0.121) 0.822 (0.070) 0.251 (0.025)P041411 C10 11 1.471 (0.131) 0.611 (0.042) 0.192 (0.019)P041412 C10 12A 1.037 (0.050) 1.424 (0.035) 0.000 (0.000) 0.784 (0.033) -0.784 (0.080)P041413 C10 13A 0.798 (0.037) 1.094 (0.036) 0.000 (0.000) 0.850 (0.040) -0.850 (0.070)P041414 C10 14 0.971 (0.098) -0.080 (0.101) 0.335 (0.037)P041415 C10 15 0.595 (0.093) 0.779 (0.145) 0.277 (0.041)P041416 C10 16 1.175 (0.105) -0.719 (0.099) 0.340 (0.041)P041417 C10 17 0.832 (0.081) -0.068 (0.102) 0.248 (0.037)P041418 C10 18 0.457 (0.048) -1.601 (0.261) 0.265 (0.058)P041419 C10 19 1.829 (0.156) 1.097 (0.042) 0.200 (0.014)
Table E-15

IRT Parameters for the 1998 State Reading Items

Reading for Literary Experience Scale, Grade 4

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 6181R017001 R3 1A 0.632 (0.020) -0.971 (0.038) 0.000 (0.000)1R017002 R3 2 1.368 (0.056) -0.517 (0.037) 0.214 (0.019)1R017003 R3 3A 0.461 (0.014) 0.480 (0.023) 0.000 (0.000) 0.102 (0.040) -0.102 (0.044)1R017004 R3 4A 0.895 (0.031) 0.985 (0.027) 0.000 (0.000)1R017005 R3 5 0.690 (0.071) 0.945 (0.067) 0.285 (0.022)1R017006 R3 6A 0.981 (0.036) 1.113 (0.029) 0.000 (0.000)1R017007 R3 7A 0.539 (0.014) 0.392 (0.016) 0.000 (0.000) 0.237 (0.042) 0.201 (0.043) -0.41R017008 R3 8 1.090 (0.056) 0.437 (0.031) 0.129 (0.014)1R017009 R3 9A 0.484 (0.011) -0.160 (0.033) 0.000 (0.000) 1.963 (0.055) -1.963 (0.046)1R012101 R4 1 1.841 (0.090) -1.000 (0.039) 0.293 (0.023)1R012102 R4 2A 0.619 (0.021) -0.051 (0.027) 0.000 (0.000)1R012103 R4 3 1.306 (0.052) -0.514 (0.036) 0.165 (0.018)1R012104 R4 4A 0.694 (0.022) -0.504 (0.029) 0.000 (0.000)1R012105 R4 5 0.873 (0.048) 0.046 (0.053) 0.198 (0.021)1R012106 R4 6A 0.857 (0.027) 0.253 (0.021) 0.000 (0.000)1R012107 R4 7 1.400 (0.072) 0.319 (0.030) 0.237 (0.014)1R012108 R4 8A 0.608 (0.022) -1.111 (0.046) 0.000 (0.000)1R012109 R4 9A 0.577 (0.022) -1.396 (0.057) 0.000 (0.000)1R012110 R4 10 0.935 (0.054) -0.880 (0.090) 0.330 (0.035)1R012111 R4 11A 0.965 (0.026) 1.324 (0.019) 0.000 (0.000) 0.953 (0.021) -0.953 (0.048)1R012112 R4 12A 0.689 (0.028) -0.668 (0.041) 0.000 (0.000)1R012601 R5 1A 0.855 (0.033) 1.231 (0.036) 0.000 (0.000)1R012602 R5 2 1.488 (0.086) 1.212 (0.029) 0.182 (0.008)1R012603 R5 3 1.383 (0.068) 0.162 (0.032) 0.261 (0.016)1R012604 R5 4A 1.195 (0.041) 1.040 (0.023) 0.000 (0.000)1R012605 R5 5 0.972 (0.086) 0.979 (0.045) 0.290 (0.016)1R012606 R5 6 1.716 (0.098) 0.494 (0.027) 0.321 (0.013)1R012607 R5 7A 0.954 (0.026) 1.278 (0.017) 0.000 (0.000) 0.838 (0.021) -0.055 (0.030) -0.71R012608 R5 8 0.619 (0.051) -0.504 (0.154) 0.350 (0.042)1R012609 R5 9 1.313 (0.096) 0.770 (0.034) 0.279 (0.015)1R012610 R5 10 2.155 (0.143) 0.491 (0.027) 0.420 (0.013)1R012611 R5 11A 0.719 (0.027) 0.302 (0.027) 0.000 (0.000)1R015801 R9 1 1.012 (0.048) -1.319 (0.075) 0.247 (0.032)1R015802 R9 2A 0.515 (0.020) -1.392 (0.056) 0.000 (0.000)
Table E-15 (continued)

IRT Parameters for the 1998 State Reading Items, Grade 4

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 1R015803 R9 3A 0.570 (0.012) -0.286 (0.025) 0.000 (0.000) 1.604 (0.042) -1.604 (0.034)1R015804 R9 4A 0.572 (0.012) 0.751 (0.021) 0.000 (0.000) 2.198 (0.040) -0.468 (0.035) -1.71R015805 R9 5 1.166 (0.069) 0.314 (0.040) 0.277 (0.017)1R015806 R9 6A 0.606 (0.015) 0.375 (0.023) 0.000 (0.000) 1.198 (0.034) -1.198 (0.038)1R015807 R9 7A 0.596 (0.015) -0.101 (0.025) 0.000 (0.000) 1.292 (0.041) -1.292 (0.036)1R015808 R9 8 0.709 (0.041) -1.145 (0.113) 0.218 (0.039)1R015809 R9 9A 0.616 (0.016) 0.032 (0.026) 0.000 (0.000) 1.294 (0.042) -1.294 (0.038)
Table E-16

IRT Parameters for the 1998 State Reading Items

Reading to Gain Information Scale, Grade 4

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 6202R012201 R6 1A 0.280 (0.016) -1.166 (0.085) 0.000 (0.000)2R012202 R6 2 1.211 (0.079) 0.594 (0.040) 0.347 (0.015)2R012203 R6 3 0.697 (0.048) 0.531 (0.065) 0.201 (0.022)2R012204 R6 4A 0.471 (0.012) 0.112 (0.019) 0.000 (0.000) 1.405 (0.048) -0.518 (0.043) -0.82R012205 R6 5 1.434 (0.088) 0.627 (0.033) 0.308 (0.013)2R012206 R6 6A 1.088 (0.033) 0.685 (0.021) 0.000 (0.000)2R012207 R6 7 0.634 (0.040) -0.687 (0.121) 0.245 (0.039)2R012208 R6 8A 0.848 (0.028) -0.616 (0.029) 0.000 (0.000)2R012209 R6 9 1.310 (0.066) 0.305 (0.033) 0.186 (0.016)2R012210 R6 10A 0.591 (0.026) -1.592 (0.068) 0.000 (0.000)2R012701 R7 1 1.207 (0.059) -0.035 (0.041) 0.285 (0.019)2R012702 R7 2A 0.509 (0.020) -1.318 (0.055) 0.000 (0.000)2R012703 R7 8A 1.127 (0.032) 0.637 (0.018) 0.000 (0.000)2R012704 R7 4 1.291 (0.063) 0.785 (0.025) 0.123 (0.010)2R012705 R7 5A 1.322 (0.046) 1.308 (0.026) 0.000 (0.000)2R012706 R7 6A 0.571 (0.025) 1.482 (0.057) 0.000 (0.000)2R012707 R7 3 2.009 (0.095) 0.301 (0.023) 0.246 (0.012)2R012708 R7 10A 0.647 (0.018) 1.653 (0.022) 0.000 (0.000) 1.285 (0.031) 0.534 (0.039) -1.82R012709 R7 9 0.582 (0.055) 0.118 (0.145) 0.332 (0.039)2R012710 R7 11A 0.990 (0.038) 0.995 (0.029) 0.000 (0.000)2R015701 R8 1 0.877 (0.051) -0.946 (0.098) 0.343 (0.037)2R015702 R8 2A 0.619 (0.011) 0.002 (0.024) 0.000 (0.000) 1.782 (0.037) -1.782 (0.035)2R015703 R8 3A 0.668 (0.013) 0.100 (0.022) 0.000 (0.000) 1.527 (0.033) -1.527 (0.032)2R015704 R8 4A 0.598 (0.016) -0.294 (0.020) 0.000 (0.000) 0.387 (0.036) -0.387 (0.031)2R015705 R8 5A 0.710 (0.018) 0.263 (0.018) 0.000 (0.000) 0.821 (0.028) -0.821 (0.030)2R015706 R8 6 0.876 (0.074) 1.304 (0.048) 0.183 (0.014)2R015707 R8 7A 0.540 (0.013) 0.452 (0.025) 0.000 (0.000) 1.306 (0.036) -1.306 (0.042)2R015708 R8 8 0.538 (0.032) -0.283 (0.098) 0.145 (0.031)2R015709 R8 9A 0.460 (0.018) 1.083 (0.037) 0.000 (0.000) 0.347 (0.044) -0.347 (0.058)2R012501 R10 1 1.343 (0.168) 2.266 (0.136) 0.311 (0.007)2R012502 R10 2 0.897 (0.047) -1.902 (0.113) 0.267 (0.049)2R012503 R10 3A 1.067 (0.029) -0.122 (0.018) 0.000 (0.000)2R012504 R10 4A 0.816 (0.024) -0.330 (0.023) 0.000 (0.000)2R012505 R10 5 1.094 (0.050) -0.829 (0.056) 0.231 (0.027)
Table E-16 (continued)

IRT Parameters for the 1998 State Reading Items

Reading to Gain Information Scale, Grade 4

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 2R012506 R10 6A 0.804 (0.024) -0.249 (0.023) 0.000 (0.000)2R012507 R10 7 1.103 (0.060) -0.664 (0.065) 0.343 (0.028)2R012508 R10 8A 0.978 (0.029) -0.482 (0.022) 0.000 (0.000)2R012509 R10 9 0.673 (0.040) -0.651 (0.108) 0.235 (0.037)2R012510 R10 10 1.143 (0.064) -0.253 (0.056) 0.353 (0.024)2R012511 R10 11A 1.050 (0.032) -0.573 (0.024) 0.000 (0.000)2R012512 R10 12A 0.377 (0.012) 0.416 (0.024) 0.000 (0.000) 0.737 (0.060) 0.119 (0.060) -0.8
Table E-17

IRT Parameters for the 1998 State Reading Items

Reading for Literary Experience Scale, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 6221R017101 R3 1A 1.247 (0.039) -0.374 (0.022) 0.000 (0.000)1R017102 R3 2A 0.587 (0.023) 1.192 (0.037) 0.000 (0.000) 0.235 (0.041) -0.235 (0.058)1R017103 R3 3 0.737 (0.047) 0.083 (0.069) 0.140 (0.025)1R017104 R3 4A 1.130 (0.035) 0.759 (0.017) 0.000 (0.000) 0.265 (0.023) -0.265 (0.029)1R017105 R3 5A 0.803 (0.022) 1.084 (0.019) 0.000 (0.000) 0.833 (0.032) 0.511 (0.035) -1.31R017106 R3 6 0.705 (0.121) 1.963 (0.147) 0.234 (0.018)1R017107 R3 7A 0.530 (0.021) 0.883 (0.034) 0.000 (0.000) 0.494 (0.044) -0.494 (0.058)1R017108 R3 8A 1.408 (0.060) 1.127 (0.027) 0.000 (0.000)1R017109 R3 9 0.737 (0.053) -0.787 (0.133) 0.272 (0.047)1R017110 R3 10A 1.111 (0.041) 0.111 (0.024) 0.000 (0.000)1R015901 R4 1A 0.430 (0.021) -0.528 (0.052) 0.000 (0.000)1R015902 R4 2A 0.618 (0.019) 0.139 (0.026) 0.000 (0.000) 1.092 (0.040) -1.092 (0.041)1R015903 R4 3 0.873 (0.068) 0.346 (0.066) 0.244 (0.024)1R015904 R4 4A 0.461 (0.016) 1.907 (0.044) 0.000 (0.000) 1.661 (0.045) -1.661 (0.105)1R015905 R4 5A 0.442 (0.017) 0.329 (0.032) 0.000 (0.000) 0.733 (0.053) -0.733 (0.057)1R015906 R4 6A 0.470 (0.014) 2.967 (0.035) 0.000 (0.000) 3.872 (0.049) 0.817 (0.062) -4.61R015907 R4 7A 0.489 (0.013) 0.100 (0.038) 0.000 (0.000) 1.820 (0.058) -1.820 (0.057)1R015908 R4 8A 0.557 (0.022) 1.054 (0.037) 0.000 (0.000) 1.032 (0.046) -1.032 (0.069)1R012601 R5 1A 0.736 (0.027) 0.026 (0.029) 0.000 (0.000)1R012602 R5 2 0.979 (0.058) 0.265 (0.045) 0.140 (0.019)1R012603 R5 3 1.191 (0.064) -0.960 (0.067) 0.225 (0.034)1R012604 R5 4A 0.737 (0.027) 0.022 (0.029) 0.000 (0.000)1R012605 R5 5 0.584 (0.046) -0.273 (0.134) 0.216 (0.041)1R012606 R5 6 1.513 (0.085) -0.783 (0.052) 0.269 (0.028)1R012607 R5 7A 0.560 (0.017) 0.316 (0.021) 0.000 (0.000) 1.126 (0.049) -0.057 (0.045) -1.01R012608 R5 8 0.527 (0.036) -2.010 (0.193) 0.238 (0.057)1R012609 R5 9 1.305 (0.082) -0.089 (0.051) 0.301 (0.022)1R012610 R5 10 1.247 (0.083) -0.613 (0.075) 0.376 (0.032)1R012611 R5 11A 0.627 (0.026) -0.800 (0.045) 0.000 (0.000)
Table E-18

IRT Parameters for the 1998 State Reading Items

Reading to Gain Information Scale, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 2R013201 R6 1A 0.609 (0.019) 0.782 (0.022) 0.000 (0.000) 1.108 (0.040) -0.069 (0.043) -1.02R013202 R6 2 0.713 (0.053) -0.265 (0.105) 0.270 (0.035)2R013203 R6 3A 1.221 (0.059) -1.924 (0.055) 0.000 (0.000)2R013204 R6 4 0.962 (0.065) -0.189 (0.073) 0.293 (0.029)2R013205 R6 5A 0.978 (0.039) -1.322 (0.043) 0.000 (0.000)2R013206 R6 6 0.781 (0.056) 0.116 (0.075) 0.214 (0.028)2R013207 R6 7A 0.717 (0.029) -0.836 (0.042) 0.000 (0.000)2R013208 R6 8 1.856 (0.106) -0.091 (0.033) 0.272 (0.019)2R013209 R6 9A 0.775 (0.036) 1.064 (0.042) 0.000 (0.000)2R013210 R6 10 1.360 (0.236) 2.338 (0.187) 0.311 (0.009)2R013211 R6 11A 0.414 (0.028) 1.221 (0.082) 0.000 (0.000)2R013212 R6 12A 0.549 (0.017) 1.904 (0.034) 0.000 (0.000) 2.069 (0.042) -0.541 (0.070) -1.52R012701 R7 1 1.063 (0.069) -1.438 (0.103) 0.346 (0.045)2R012702 R7 2A 0.598 (0.030) -2.244 (0.097) 0.000 (0.000)2R012707 R7 3 1.577 (0.093) -1.073 (0.056) 0.286 (0.032)2R012704 R7 4 1.084 (0.055) -0.486 (0.052) 0.170 (0.025)2R012705 R7 5A 0.871 (0.031) -0.010 (0.025) 0.000 (0.000)2R012706 R7 6A 0.409 (0.023) 0.486 (0.052) 0.000 (0.000)2R012711 R7 7 0.957 (0.053) -0.441 (0.063) 0.185 (0.028)2R012703 R7 8A 0.949 (0.033) -0.428 (0.027) 0.000 (0.000)2R012709 R7 9 0.755 (0.059) -0.712 (0.135) 0.365 (0.043)2R012708 R7 10A 0.579 (0.016) 0.520 (0.023) 0.000 (0.000) 1.579 (0.049) 0.354 (0.040) -1.92R012710 R7 11A 0.798 (0.031) -0.603 (0.034) 0.000 (0.000)2R012712 R7 12 0.729 (0.061) 0.163 (0.093) 0.259 (0.031)2R012713 R7 13A 1.191 (0.045) -0.647 (0.028) 0.000 (0.000)2R017201 R8 1 0.851 (0.060) -0.697 (0.108) 0.334 (0.039)2R017202 R8 2A 0.480 (0.018) -0.518 (0.033) 0.000 (0.000) 0.715 (0.057) -0.715 (0.046)2R017203 R8 3 1.109 (0.077) -0.153 (0.067) 0.347 (0.027)2R017204 R8 4A 0.735 (0.021) 0.659 (0.024) 0.000 (0.000) 1.086 (0.031) -1.086 (0.044)2R017205 R8 5A 0.596 (0.015) 0.464 (0.024) 0.000 (0.000) 2.107 (0.050) -0.329 (0.040) -1.72R017206 R8 6 0.956 (0.098) 0.866 (0.061) 0.332 (0.020)2R017207 R8 7A 0.315 (0.016) 1.733 (0.074) 0.000 (0.000) 1.097 (0.068) -1.097 (0.110)2R017208 R8 8A 0.607 (0.017) 1.197 (0.032) 0.000 (0.000) 1.728 (0.037) -1.728 (0.076)2R017209 R8 9 0.957 (0.071) 0.208 (0.064) 0.265 (0.025)
Table E-18 (continued)

IRT Parameters for the 1998 State Reading Items

Reading to Gain Information Scale, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 6242R017210 R8 10A 0.424 (0.035) 2.527 (0.186) 0.000 (0.000)2R016201 R13 1A 0.551 (0.028) -3.547 (0.134) 0.000 (0.000) 0.096 (0.206) -0.096 (0.079)2R016202 R13 2A 0.581 (0.013) -0.928 (0.042) 0.000 (0.000) 2.836 (0.093) -2.836 (0.046)2R016203 R13 3 0.539 (0.049) -0.646 (0.191) 0.344 (0.048)2R016204 R13 4A 0.531 (0.011) -0.415 (0.037) 0.000 (0.000) 3.169 (0.124) 0.887 (0.045) -4.02R016205 R13 5A 0.585 (0.015) 0.699 (0.031) 0.000 (0.000) 1.655 (0.040) -1.655 (0.057)2R016206 R13 6 1.057 (0.066) -0.831 (0.082) 0.326 (0.035)2R016207 R13 7A 0.546 (0.019) 0.313 (0.025) 0.000 (0.000) 0.157 (0.044) -0.157 (0.047)2R016208 R13 8 0.997 (0.065) -0.898 (0.095) 0.354 (0.038)2R016209 R13 9 1.396 (0.076) -0.694 (0.051) 0.253 (0.027)2R016210 R13 10A 0.664 (0.018) 0.810 (0.022) 0.000 (0.000) 1.787 (0.041) 0.249 (0.036) -2.02R016211 R13 11A 0.407 (0.014) -2.439 (0.047) 0.000 (0.000) 2.597 (0.158) -2.597 (0.049)2R016212 R13 12A 0.394 (0.017) -0.101 (0.034) 0.000 (0.000) 0.284 (0.063) -0.284 (0.059)2R016213 R13 13A 0.425 (0.019) -2.107 (0.069) 0.000 (0.000) 1.149 (0.115) -1.149 (0.051)
Table E-19

IRT Parameters for the 1998 State Reading Items

Reading to Perform a Task Scale, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 3R016101 R9 1A 0.609 (0.020) -0.112 (0.024) 0.000 (0.000) 0.236 (0.043) -0.236 (0.040)3R016102 R9 2 1.024 (0.066) -0.783 (0.085) 0.315 (0.035)3R016103 R9 3 1.172 (0.081) 0.335 (0.048) 0.283 (0.020)3R016104 R9 4A 0.740 (0.036) -2.166 (0.081) 0.000 (0.000)3R016105 R9 5 1.368 (0.078) -0.442 (0.049) 0.264 (0.025)3R016106 R9 6 0.847 (0.077) 1.123 (0.055) 0.161 (0.017)3R016107 R9 7A 0.715 (0.020) -0.187 (0.020) 0.000 (0.000) -0.218 (0.041) 0.218 (0.038)3R016108 R9 8A 0.354 (0.011) -0.127 (0.034) 0.000 (0.000) -1.233 (0.082) 1.233 (0.079)3R016109 R9 9A 0.416 (0.012) 0.850 (0.042) 0.000 (0.000) 1.909 (0.055) -1.909 (0.078)3R013401 R10 1 1.459 (0.091) 0.319 (0.036) 0.260 (0.017)3R013402 R10 2A 0.812 (0.031) 0.073 (0.027) 0.000 (0.000)3R013403 R10 3A 0.465 (0.011) 0.485 (0.026) 0.000 (0.000) -2.456 (0.093) 2.456 (0.095)3R013404 R10 4 0.936 (0.072) -0.101 (0.084) 0.367 (0.030)3R013405 R10 5A 0.896 (0.033) -0.341 (0.027) 0.000 (0.000)3R013406 R10 6A 0.622 (0.028) 0.402 (0.035) 0.000 (0.000)3R013407 R10 7A 0.589 (0.026) -0.627 (0.044) 0.000 (0.000)3R013408 R10 8 0.637 (0.059) 0.239 (0.112) 0.271 (0.034)3R013409 R10 9A 0.686 (0.029) -0.370 (0.035) 0.000 (0.000)3R013410 R10 10 0.666 (0.053) -0.585 (0.138) 0.291 (0.043)3R013411 R10 11A 0.464 (0.026) -1.301 (0.080) 0.000 (0.000)3R013412 R10 12A 0.388 (0.028) -2.743 (0.188) 0.000 (0.000)3R013001 R11 1A 0.909 (0.035) -1.154 (0.041) 0.000 (0.000)3R013002 R11 2 1.480 (0.076) -0.431 (0.039) 0.190 (0.022)3R013003 R11 3A 0.899 (0.033) -0.547 (0.030) 0.000 (0.000)3R013004 R11 4A 0.378 (0.017) 0.625 (0.039) 0.000 (0.000) 0.357 (0.060) -0.357 (0.069)3R013005 R11 5A 0.876 (0.035) -1.281 (0.046) 0.000 (0.000)3R013006 R11 6 0.955 (0.059) -0.381 (0.071) 0.242 (0.030)3R013007 R11 7A 0.599 (0.029) -1.580 (0.073) 0.000 (0.000)3R013008 R11 8A 0.780 (0.031) 0.095 (0.029) 0.000 (0.000)3R013009 R11 9A 0.891 (0.037) -1.165 (0.045) 0.000 (0.000)3R013010 R11 10A 0.749 (0.033) -1.104 (0.050) 0.000 (0.000)3R013011 R11 11A 0.393 (0.024) -0.358 (0.060) 0.000 (0.000)3R013012 R11 12 0.887 (0.064) -0.084 (0.078) 0.237 (0.031)
Table E-20

IRT Parameters for the 1998 State Writing Items, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 (s.e.) 6261W006002 W3 1 0.808 (0.026) -0.283 (0.021) 2.946 (0.160) 1.049 (0.056) -0.163 (0.043) -1.4

1W006102 W4 1 0.970 (0.030) -0.360 (0.019) 3.399 (0.216) 0.907 (0.047) -0.175 (0.035) -1.6

1W006202 W5 1 0.869 (0.027) -0.061 (0.021) 3.173 (0.151) 1.240 (0.048) -0.365 (0.039) -1.4

1W006302 W6 1 0.856 (0.026) -0.049 (0.019) 1.333 (0.088) 1.672 (0.069) 0.482 (0.042) -0.9

1W006402 W7 1 0.940 (0.028) -0.341 (0.018) 1.877 (0.091) 1.132 (0.055) 0.280 (0.038) -1.21W006502 W8 1 0.884 (0.026) -0.487 (0.022) 2.771 (0.205) 1.417 (0.071) 0.358 (0.040) -1.41W006602 W9 1 1.091 (0.032) -0.367 (0.018) 2.599 (0.127) 0.954 (0.048) 0.278 (0.034) -1.51W006802 W11 1 0.996 (0.029) -0.177 (0.019) 2.516 (0.117) 1.395 (0.051) 0.048 (0.033) -1.61W006902 W12 1 1.065 (0.031) -0.091 (0.020) 2.603 (0.116) 1.452 (0.050) 0.230 (0.032) -1.71W007002 W13 1 1.066 (0.031) -0.191 (0.020) 2.522 (0.123) 1.492 (0.054) 0.283 (0.033) -1.61W007102 W14 1 0.910 (0.025) -0.138 (0.021) 2.164 (0.118) 1.643 (0.066) 0.608 (0.038) -1.71W007202 W15 1 1.061 (0.031) 0.245 (0.019) 2.447 (0.086) 1.567 (0.043) -0.113 (0.031) -1.7

1W007302 W16 1 0.886 (0.026) 0.148 (0.022) 2.374 (0.096) 1.573 (0.054) 0.367 (0.037) -2.1

1W007402 W17 1 0.940 (0.029) -0.075 (0.019) 1.884 (0.090) 1.404 (0.056) 0.268 (0.037) -1.2

1W007602 W19 1 1.211 (0.037) -0.014 (0.016) 2.286 (0.074) 1.280 (0.038) -0.120 (0.029) -1.2

1W007702 W20 1 1.260 (0.038) 0.236 (0.018) 2.536 (0.080) 1.515 (0.038) 0.135 (0.028) -1.5

1W007802 W21 1 1.238 (0.038) 0.010 (0.016) 1.631 (0.064) 1.423 (0.045) 0.249 (0.029) -1.11W007902 W22 1 0.776 (0.022) 0.044 (0.024) 2.674 (0.129) 1.681 (0.060) 0.076 (0.040) -2.21W008002 W23 1 1.109 (0.033) 0.047 (0.019) 2.898 (0.120) 1.426 (0.044) 0.152 (0.031) -1.51W008102 W24 1 0.981 (0.030) 0.050 (0.018) 1.951 (0.076) 1.384 (0.049) 0.243 (0.035) -1.4
627

Appendix F

CONDITIONING VARIABLES

AND CONTRAST CODINGS

This appendix contains information about the conditioning variables used in scaling/plausible

value estimation for the 1998 NAEP assessment. The initial step in construction of conditioning variables

involves forming primary student-based vectors of response data from answers to student, teacher, and

school questionnaires, demographic and background data such as supplied by Westat, Inc., and other

student information known prior to scaling. The initial conditioning vectors concatenate this student

background information into a series of identifying “contrasts” comprising:

1. Categorical variables derived by expanding the response options of a questionnaire

variable into a binary series of one-degree-of-freedom “dummy” variables or contrasts,

(these form the majority of each student conditioning vector);

2. Questionnaire or demographic variables that possess ordinal response options, such as

number of hours spent watching television, which are included as linear and/or quadratic

multi-degree-of-freedom contrasts;

3. Continuous variables, such as student logit scores based on percent correct values,

included as contrasts in their original form or a transformation of their original form,

and;

4. Interactions of two or more categorical variables forming a set of orthogonal one-degreeof-freedom dummy variables or contrasts.

This appendix gives the specifications used for constructing the conditioning variables.

Table F-1 defines the information provided for each sample variable.

Table F-2 provides a summary of the reading conditioning variables specifications that are

contained in the remainder of this appendix.

Table F-3 provides a summary of the writing conditioning variables specifications that are

contained in the remainder of this appendix.

Table F-4 provides a summary of the civics conditioning variables specifications that are

contained in the remainder of this appendix.

Tables F-5, F-6, and F-7 contain conditioning variable data specific to each subject.

As described in Chapter 12, the linear conditioning model employed for the estimation of

plausible values did not directly use the conditioning variable specifications listed in this appendix. To

eliminate inherent instabilities in estimation encountered when using a large number of correlated

variables, a principal component transformation of the correlation matrix obtained from the conditioning

variable contrasts derived according to these primary specifications was performed. The principal

components scores based on this transformation were used as the predictor variables in estimating the

linear conditioning model. For the national assessment, the proportions of variance of the conditioning
628

contrast accounted for by the principal components are given for each grade level in Tables F-8, F-9, and

F-10 for reading, and Tables F-11, F-12, and F-13 for writing, and Tables F-14, F-15, and F-16 for civics.

Table F-1

Description of Specifications Provided for Each Conditioning Variable

Title Description

CONDITIONING ID A unique eight-character ID assigned to identify each conditioning variable

corresponding to a particular background or subject area question within the

entire pool of conditioning variables. The first four characters identify the

origin of the variable: BACK (background questionnaire), READ (student

reading questionnaire), SCHL (school questionnaire), TCHR (background

part of teacher questionnaire), and TSUB (subject classroom part of teacher

questionnaire). The second four digits represent the sequential position within

each origin group.

DESCRIPTION A short description of the conditioning variable.

GRADES/ASSESSMENTS Three characters identifying assessment (“S” for state, “N” for national) and

grade (04, 08, and 12) in which the conditioning variable was used.

CONDITIONING VAR LABEL A descriptive eight-character label identifying the conditioning variable.

NAEP ID The seven-character NAEP database identification for the conditioning

variable.

TYPE OF CONTRAST The type of conditioning variable. “CLASS” identifies a categorical

conditioning variable and “SCALE” identifies continuous or quasi-continuous

conditioning variables. “INTERACTION” identifies a set of orthogonal

contrasts formed from two or more “CLASS” variables. “OTHER”

conditioning variables do not fall into any of the above types.

TOTAL NUMBER OF

SPECIFIED CONTRASTS

Each conditioning variable forms a set of one or more contrasts. For each

valid response value of conditioning variable a contrast must be defined. One

or more response values may be collapsed together to form one contrast. The

number of response value “sets” of a conditioning variable forming a unique

contrast is the value given in this field.

NUMBER OF INDEPENDENT

CONTRASTS

The number of degree of freedom in a set of contrasts formed from a

conditioning variable. For a categorical conditioning variable this number

would be the number of response options minus one if each response option

formed its own unique contrast.
Table F-2

Summary Table of the 1998 Reading Conditioning Variable Specifications

Cond’ng.NAEP ID ID TDDC ID DESCRIPTION BACK0001 BKSER GRAND MEAN BACK0002 DSEX DERIVED SEX BACK0003 DRACE DERIVED RACE/ETHNICITY BACK0004 B003101 TB003101 IF HISPANIC, WHAT IS YOUR HISPANIC BACKGROUND? BACK0005 TOL7 TOL 7 - TYPE OF LOCATION BACK0006 TOL5 TYPE OF LOCALE (5 CATEGORIES) BACK0007 PARED PARENTS’ HIGHEST LEVEL OF EDUCATION, GRADES 8 AND 12 BACK0008 PARED2 PARENTS’ HIGHEST LEVEL OF EDUCATION, GRADE 4 BACK0009 REGION REGION OF THE COUNTRY BACK0010 SCHTYPE SCHOOL TYPE BACK0011 RACE RACE BACK0012 IEP INDIVIDUALIZED EDUCATION PLAN BACK0013 LEP LIMITED ENGLISH PROFICIENCY BACK0014 TITLE1 TITLE 1: (BOOK COVER) BACK0015 SLUNCH DO YOU RECEIVE A FREE OR REDUCED-PRICE LUNCH? BACK0016 B001801 TB001801 HOW MUCH TELEVISION DO YOU USUALLY WATCH EACH DAY? (LINEAR) BACK0017 B001801 TB001801 HOW MUCH TELEVISION DO YOU USUALLY WATCH EACH DAY? (QUADRATIC) BACK0018 B006601 TB006601 HOMEWORK ASSIGNED?: BASED ON TIME SPENT ON HOMEWORK EACH DAY. BACK0019 B006601 TB006601 HOW MUCH TIME DO YOU USUALLY SPEND ON HOMEWORK EACH DAY? (LINEAR) BACK0020 B006601 TB006601 HOW MUCH TIME DO YOU USUALLY SPEND ON HOMEWORK EACH DAY (QUADRATIC) BACK0021 HOMEEN2 NUMBER OF ITEMS IN THE HOME (NEWSPAPER, > 25 BOOKS, ENCYCLOPEDIA, MAGAZINES) (DERIVEDBACK0022 B001101 TB001101 ABOUT HOW MANY PAGES A DAY DO YOU HAVE TO READ FOR SCHOOL AND HOMEWORK? BACK0023 B001101 TB001101 ABOUT HOW MANY PAGES A DAY DO YOU HAVE TO READ FOR SCHOOL AND HOMEWORK? BACK0024 INTERACT INTERACTION: GENDER BY RACE/ETHNICITY BACK0025 INTERACT INTERACTION: GENDER BY TYPE OF LOCALE (7 CATEGORIES) BACK0026 INTERACT INTERACTION: GENDER BY PARENTS’ EDUCATION GRADES 8 & 12 BACK0027 INTERACT INTERACTION: GENDER BY PARENTS’ EDUCATION GRADE 4 BACK0028 INTERACT INTERACTION: GENDER BY SCHOOL TYPE BACK0029 INTERACT INTERACTION: RACE/ETHNICITY BY TYPE OF LOCALE (7 CATEGORIES) BACK0030 INTERACT INTERACTION: RACE/ETHNICITY BY PARENTS’ EDUCATION GRADES 8 & 12 BACK0031 INTERACT INTERACTION: RACE/ETHNICITY BY PARENTS’ EDUCATION GRADE 4 BACK0032 INTERACT INTERACTION: RACE/ETHNICITY BY SCHOOL TYPE BACK0033 INTERACT INTERACTION: PARENT’S EDUCATION GRADES 8 & 12 BY TYPE OF LOCALE (7 CATEGORIES) BACK0034 INTERACT INTERACTION: PARENT’S EDUCATION GRADE 4 BY TYPE OF LOCALE (7 CATEGORIES) BACK0035 INTERACT INTERACTION: TYPE OF LOCALE (7 CATEGORIES) BY SCHOOL TYPE BACK0036 INTERACT INTERACTION: PARENTS’ EDUCATION GRADES 8 & 12 BY SCHOOL TYPE BACK0037 INTERACT INTERACTION: PARENTS’ EDUCATION GRADE 4 BY SCHOOL TYPE BACK0038 MA96FLG MSA/NON-MSA BACK0039 MONSTUD STATE ADMINISTRATION MONITORED/UNMONITORED SESSION BACK0040 INTERACT INTERACTION: SCHOOL TYPE BY MONITORED/UNMONITORED SESSION BACK0041 SUBSAMP SAMPLE TYPE BACK0042 INTERACT INTERACTION: SAMPLE BY RACE/ETHNICITY BACK0043 RPTSAMP REPORTING SAMPLE BACK0044 DISTRPT STATE/DISTRICT BACK0045 B003001 TB003001 WHICH RACE/ETHNICITY BEST DESCRIBES YOU BACK0046 B014601 LC000006 HOW LONG LIVED IN UNITED STATES BACK0047 B003201 TB003201 HOW OFTEN OTHER THAN ENGLISH SPOKEN IN HOME BACK0048 B013201 ID100314 MOTHER GRADUATED HIGH SCHOOL BACK0049 B013301 ID100315 MOTHER HAD SOME EDUCATION AFTER HIGH SCHOOL BACK0050 B013401 ID100316 MOTHER GRADUATED COLLEGE BACK0051 B013501 ID100317 FATHER GRADUATED HIGH SCHOOL 
Table F-2 (continued)

Summary Table of the 1998 Reading Conditioning Variable Specifications

Cond’ng.NAEP ID ID TDDC ID DESCRIPTION 630BACK0052 B013601 ID100318 FATHER HAD SOME EDUCATION AFTER HIGH SCHOOL BACK0053 B013701 ID100319 FATHER GRADUATED COLLEGE BACK0054 B000901 TB000901 DOES YOUR FAMILY GET A NEWSPAPER REGULARLY BACK0055 B000903 TB000903 IS THERE AN ENCYCLOPEDIA IN YOUR HOME BACK0056 B000904 TB000904 ARE THERE MORE THAN 25 BOOKS IN YOUR HOME BACK0057 B000905 TB000905 DOES YOUR FAMILY GET MAGAZINES REGULARLY BACK0058 S004001 TS004001 HOW MANY DAYS OF SCHOOL MISSED LAST MONTH BACK0059 B007301 HE000712 TIMES CHANGED SCHOOLS IN PAST TWO YEARS BACK0060 B007401 HE000717 HOW OFTEN DISCUSS STUDIES AT HOME BACK0061 B014501 HE000713 HOW OFTEN USE COMPUTER FOR SCHOOLWORK SUBJ0001 R830301 ID100376 HOW HARD TRIED ON THIS READING TEST THAN ON OTHERS SUBJ0002 R830401 ID100377 HOW IMPORTANT TO DO WELL ON THIS READING TEST SUBJ0003 RM00501 HE000781 HOW OFTEN HAD TO WRITE LONG ANSWERS TO QSTS? SUBJ0004 R830501 ID100342 MY FRIENDS MAKE FUN OF PEOPLE WHO TRY TO DO WELL SUBJ0005 R830502 ID100343 I HAVE FRIENDS TO TALK TO IF NEED HELP W/SCHOOL SUBJ0006 R810801 HE000687 BOOKS READ OUTSIDE SCHOOL IN PAST MONTH SUBJ0007 R810201 TR810201 WHAT KIND OF READER ARE YOU SUBJ0008 R810901 TR810001 HOW OFTEN READ FOR FUN ON OWN SUBJ0009 R810902 TR810002 HOW OFTEN TALK W/FRIENDS ABOUT WHAT YOU READ SUBJ0010 R810903 TR810003 HOW OFTEN TAKE BOOKS FROM LIBRARY ON YOUR OWN SUBJ0011 R810904 HE000684 HOW OFTEN READ A STORY OR NOVEL SUBJ0012 R810905 HE000685 HOW OFTEN READ A NEWSPAPER SUBJ0013 R810906 HE000686 HOW OFTEN READ A MAGAZINE SUBJ0014 R811005 TR810105 ASKED TO DO GROUP PROJECT ABOUT WHAT YOU READ SUBJ0015 R811006 TR810402 ASKED TO READ ALOUD SUBJ0016 R811007 TR810412 ASKED TO READ SILENTLY SUBJ0017 R811009 TR810413 GIVEN TIME TO READ BOOKS YOU HAVE CHOSEN SUBJ0018 R811002 TR810102 ASKED TO TALK W/STUDENTS ABOUT WHAT YOU READ SUBJ0019 R811004 TR810104 ASKED TO WRITE ABOUT WHAT YOU READ SUBJ0020 R818101 ID100186 TEACHER HELPS YOU BREAK WORDS INTO PARTS SUBJ0021 R818102 ID100187 TEACHER HELPS YOU UNDERSTAND NEW WORDS SUBJ0022 R830001 ID100188 DO YOU AND TEACHER REVIEW PROGRESS IN READING SUBJ0023 R830101 ID100189 IS THERE A SCHOOL/PUBLIC LIBRARY AVAILABLE SUBJ0024 R811301 HE000695 USE LIBRARY TO DO RESEARCH FOR SCHOOL ASSIGNMENT SUBJ0025 R811302 HE000696 USE LIBRARY TO BORROW BOOKS FOR SCHOOL SUBJ0026 R811303 ID100190 USE LIBRARY TO USE A COMPUTER SUBJ0027 R811304 HE000698 USE LIBRARY AS A QUIET PLACE TO STUDY SCHL0001 C042501 ID100378 FOURTH GRADERS ASSIGNED TO CLASS BY ABILITY SCHL0002 C042601 ID100041 HOW OFTEN STUDENTS RECEIVE READING INSTRUCTION SCHL0003 C042602 ID100042 HOW OFTEN STUDENTS RECEIVE WRITING INSTRUCTION SCHL0004 C042603 ID100043 HOW OFTEN STUDENTS RECEIVE SOC STUDIES INSTRUCT SCHL0005 C042604 ID100044 HOW OFTEN STUDENTS RECEIVE COMPUTER USE INSTRUCT SCHL0006 C042701 ID100379 DOES SCHOOL USE BLOCK SCHEDULING SCHL0007 C042801 ID100380 ARE COMPUTERS AVAILABLE IN ALL CLASSROOMS SCHL0008 C042802 HE000864 ARE COMPUTERS AVAILABLE IN COMPUTER LAB SCHL0009 C042803 HE000866 ARE COMPUTERS AVAILABLE TO CLASSROOM WHEN NEEDED SCHL0010 C042901 ID100381 HOW MANY COMPUTERS AVAILABLE TO STUDENTS SCHL0011 C036601 LC000502 PRIMARY WAY LIBRARY IS STAFFED SCHL0012 C043001 ID100069 PARENTS PARTICIPATE-PARENT-TEACHER ORG SCHL0013 C043002 ID100070 PARENTS PARTICIPATE-OPEN HOUSE SCHL0014 C043003 ID100071 PARTICIPATE-PARENT-TEACHER CONFERENCE SCHL0015 C043004 ID100072 PARENTS PARTICIPATE-SCHOOL CURRICULUM DECISIONS SCHL0016 C043005 ID100073 PARENTS PARTICIPATE-VOLUNTEER PROGRAMS SCHL0017 C043006 ID100074 PARENTS PARTICIPATE-PARENTING-SKILLS PROGRAM 
Table F-2 (continued)

Summary Table of the 1998 Reading Conditioning Variable Specifications

Cond’ng.NAEP ID ID TDDC ID DESCRIPTION SCHL0018 C043007 ID100076 PARENTS PARTICIPATE-SCHOOL ADVISORY COMMITTEES SCHL0019 C043008 ID100077 PARENTS PARTICIPATE-CLASSROOM ASSISTANTS SCHL0020 C032402 HE000888 IS STUDENT ABSENTEEISM A PROBLEM IN YOUR SCHOOL SCHL0021 C032401 HE000887 IS STUDENT TARDINESS A PROBLEM IN YOUR SCHOOL SCHL0022 C032404 HE000890 ARE PHYSICAL CONFLICTS A PROBLEM IN YOUR SCHOOL SCHL0023 C032407 HE000893 ARE RACIAL/CULT. CONFLICTS A PROBLEM IN SCHOOL SCHL0024 C032408 HE000894 IS STUDENT HEALTH A PROBLEM IN YOUR SCHOOL SCHL0025 C032409 HE002121 IS LACK OF PARENT INVLVMNT A PROBLEM IN SCHOOL SCHL0026 C032410 HE002122 IS STUDENT ALCOHOL USE A PROBLEM IN YOUR SCHOOL SCHL0027 C032411 HE002123 IS STUDENT TOBACCO USE A PROBLEM IN YOUR SCHOOL SCHL0028 C032412 HE002124 IS STUDENT DRUG USE A PROBLEM IN YOUR SCHOOL SCHL0029 C032413 HE002125 ARE GANG ACTIVITIES A PROBLEM IN YOUR SCHOOL SCHL0030 C032414 HE002126 IS STUDENT MISBEHAVIOR A PROBLEM IN YOUR SCHOOL SCHL0031 C043101 ID100079 IS STUDENT CHEATING A PROBLEM IN YOUR SCHOOL SCHL0032 C043102 ID100077 IS TEACHER ABSENTEEISM A PROBLEM IN YOUR SCHOOL SCHL0033 C043103 ID100078 ARE PHYSICAL CONFLICTS BETWEEN STUDENTS/TEACHERS SCHL0034 C043104 ID100080 IS VANDALISM A PROBLEM IN YOUR SCHOOL SCHL0035 C032502 HE000897 TEACHER MORALE SCHL0036 C032503 HE000898 STUDENT ATTITUDES TOWARD ACADEMIC ACHIEVEMENT SCHL0037 C032505 HE000900 PARENT SUPPORT FOR STUDENT ACHIEVEMENT SCHL0038 C032506 HE000901 REGARD FOR SCHOOL PROPERTY SCHL0039 C043201 ID100081 TEACHERS’ EXPECTATIONS FOR STUDENT ACHIEVEMENT SCHL0040 C043301 ID100082 PERCENT STUDENT BODY ABSENT AVERAGE DAY SCHL0041 C043401 ID100389 PERCENT TEACHING STAFF ABSENT AVERAGE DAY SCHL0042 C043501 ID100390 ENROLLMENT LAST YEAR COMPARED TO END OF SCHOOL YR SCHL0043 C043601 HE002112 PERCENT STUDENTS HELD BACK AND REPEATING GRADE SCHL0044 C043701 ID100391 PERCENT TEACHING STAFF LEFT BEFORE END OF YEAR SCHL0045 C038301 HE002094 IS SCHOOL IN NATIONAL SCHOOL LUNCH PROGRAM SCHL0046 C043801 ID100392 PERCENT ELIGIBLE NATIONAL SCHOOL LUNCH PROGRAM SCHL0047 C043901 ID100393 DOES SCHOOL RECEIVE CHAPTER 1/TITLE I FUNDING SCHL0048 C044001 ID100395 PERCENT STUDENTS RECEIVE CHAPTER1/TITLE I FUNDING SCHL0049 C044002 ID100396 PERCENT STUDENTS RECEIVE REMEDIAL READING INSTRUCT SCHL0050 C044003 ID100397 PERCENT STUDENTS RECEIVE REMEDIAL WRITING INSTRUCT SCHL0051 C044004 ID100398 PERCENT STUDENTS IN GIFTED AND TALENTED PROGRAM BACK0062 B003501 TB003501 MOTHER’S EDUCATION LEVEL BACK0063 B003601 TB003601 FATHER’S EDUCATION LEVEL SUBJ0028 R811010 TR810408 ASKED TO EXPLAIN UNDERSTANDING OF WHAT YOU READ SUBJ0029 R811011 TR810409 ASKED TO DISCUSS INTERPRETATIONS OF WHAT YOU READ SUBJ0030 R830201 LC000035 DO YOU HAVE ACCESS TO A SCHOOL/PUBLIC LIBRARY SCHL0052 C044401 ID100400 8TH GRADE ASSIGNED TO ENGLISH CLASS BY ABILITY SCHL0053 C044402 ID100403 8TH GRADE ASSIGNED-HISTORY/SS BY ABILITY SCHL0054 C043105 ID100086 IS STUDENT DROPOUT A PROBLEM IN YOUR SCHOOL SCHL0055 C043106 ID100087 IS TEEN PREGNANCY A PROBLEM IN YOUR SCHOOL BACK0064 B005501 TB005501 MAIN ACTIVITY YEAR FOLLOWING HIGH SCHOOL SUBJ0031 R820201 WP000073 ENROLLED IN OR TOOK AN AP ENGLISH COURSE SCHL0056 C044301 ID100404 12TH GRADE ASSIGNED TO ENGLISH CLASS BY ABILITY SCHL0057 C044302 ID100405 12TH GR ASSIGNED- HISTORY/CIVICS/SS CLASS ABILITY SCHL0058 C044101 ID100408 PERCENT LAST YEAR’S TWELFTH-GRADE CLASS GRADUATED SCHL0059 C044201 ID100410 PERCENT GRADUATING CLASS-ATTEND TWO-YEAR COLLEGE SCHL0060 C044202 ID100411 PERCENT GRADUATING CLASS-ATTEND FOUR-YEAR COLLEGE TCHR0001 T067001 PJ000121 DO YOU TEACH READING TCHR0002 T067002 PJ000122 DO YOU TEACH WRITING TCHR0003 T067003 PJ000123 DO YOU TEACH LANGUAGE ARTS TCHR0004 T067004 PJ000124 DO YOU TEACH SOCIAL STUDIES 
Table F-2 (continued)

Summary Table of the 1998 Reading Conditioning Variable Specifications

Cond’ng.NAEP ID ID TDDC ID DESCRIPTION 632TCHR0005 T067101 PJ000126 YEARS TOTAL TAUGHT ELEMENTARY LEVEL TCHR0006 T067201 PJ000128 YEARS TOTAL TAUGHT READING TCHR0007 T067202 PJ000129 YEARS TOTAL TAUGHT WRITING TCHR0008 T067203 PJ000130 YEARS TOTAL TAUGHT LANGUAGE ARTS TCHR0009 T067204 PJ000131 YEARS TOTAL TAUGHT HISTORY TCHR0010 T067205 PJ000132 YEARS TOTAL TAUGHT SOCIAL STUDIES TCHR0011 T067206 PJ000133 YEARS TOTAL TAUGHT CIVICS TCHR0012 T067301 PJ000134 MAIN ASSIGNMENT FIELD TCHR0013 T056201 HE002551 TEACHING CERTIF IN THIS STATE IN MAIN FIELD TCHR0014 T056301 HE001012 HIGHEST ACADEMIC DEGREE YOU HOLD TCHR0015 T067501 PJ000138 UNDERGRAD MAJOR/MINOR-ELEMENTARY EDUCATION TCHR0016 T067502 PJ000139 UNDERGRAD MAJOR/MINOR-SECONDARY EDUCATION TCHR0017 T067503 PJ000140 UNDERGRAD MAJOR/MINOR-SPECIAL EDUCATION TCHR0018 T067504 PJ000141 UNDERGRAD MAJOR/MINOR-BILINGUAL EDUCATION/ESL TCHR0019 T067505 PJ000142 UNDERGRAD MAJOR/MINOR-ADMINISTRATION & SUPERVISION TCHR0020 T067506 PJ000143 UNDERGRAD MAJOR/MINOR-CURRICULUM & SUPERVISION TCHR0021 T067507 PJ000144 UNDERGRAD MAJOR/MINOR-COUNSELING TCHR0022 T067508 PJ000145 UNDERGRAD MAJOR/MINOR-ENGLISH TCHR0023 T067509 PJ000146 UNDERGRAD MAJOR/MINOR-READING AND/OR LANGUAGE ARTS TCHR0024 T067510 PJ000147 UNDERGRAD MAJOR/MINOR-HISTORY TCHR0025 T067511 PJ000148 UNDERGRAD MAJOR/MINOR-POLITICAL SCIENCE TCHR0026 T067512 PJ000149 UNDERGRAD MAJOR/MINOR-OTHER TCHR0027 T067601 PJ000151 GRAD MAJOR/MINOR-ELEMENTARY EDUCATION TCHR0028 T067602 PJ000152 GRAD MAJOR/MINOR-SECONDARY EDUCATION TCHR0029 T067603 PJ000153 GRAD MAJOR/MINOR-SPECIAL EDUCATION TCHR0030 T067604 PJ000154 GRAD MAJOR/MINOR-BILINGUAL EDUCATION/ESL TCHR0031 T067605 PJ000155 GRAD MAJOR/MINOR-ADMINSTRATION & SUPERVISION TCHR0032 T067606 PJ000156 GRAD MAJOR/MINOR-CURRICULUM AND INSTRUCTION TCHR0033 T067607 PJ000157 GRAD MAJOR/MINOR-COUNSELING TCHR0034 T067608 PJ000158 GRAD MAJOR/MINOR-ENGLISH TCHR0035 T067609 PJ000159 GRAD MAJOR/MINOR-READING AND/OR LANGUAGE ARTS TCHR0036 T067610 PJ000160 GRAD MAJOR/MINOR-HISTORY TCHR0037 T067611 PJ000161 GRAD MAJOR/MINOR-POLITICAL SCIENCE TCHR0038 T067612 PJ000162 GRAD MAJOR/MINOR-OTHER TCHR0039 T067701 ID100358 LAST 12 MOS, PROF DEV-READING AND WRITING TCHR0040 T067702 ID100147 LAST 12 MOS, PROF DEV-SOCIAL STUDIES TCHR0041 T067801 PJ000169 PREPARED IN THE USE OF TELECOMMUNICATIONS TCHR0042 T067802 ID100360 PREPARED IN THE USE OF COMPUTERS TCHR0043 T067803 PJ000171 PREPARED IN COOPERATIVE GROUP INSTRUCTION TCHR0044 T067804 PJ000176 PREPARED IN TEACHING STUDENTS-DIFFERENT CULTURES TCHR0045 T067805 PJ000177 PREPARED IN TEACHING STUDENTS WHO ARE LEP TCHR0046 T067806 PJ000178 PREPARED IN TEACHING STUDENTS WITH DISABILITIES TCHR0047 T067807 PJ000179 PREPARED IN CLASSROOM MANAGEMENT AND ORGANIZATION TCHR0048 T041201 HE001022 AVAILABILITY OF RESOURCES TCHR0049 T067901 ID100417 HOW WELL PREPARED TO TEACH READING TCHR0050 T067902 ID100418 HOW WELL PREPARED TO TEACH WRITING TCHR0051 T068001 PJ000182 PREPARED IN LIT-BASED READING INSTRUCTION TCHR0052 T068002 PJ000183 PREPARED IN CONTENT AREA READING TCHR0053 T068003 PJ000184 PREPARED IN COMBINING RDG AND WRITING TCHR0054 T068004 PJ000185 PREPARED IN WHOLE LANGUAGE APPROACH TO TEACH RDG TCHR0055 T068005 PJ000186 PREPARED IN PHONICS IN TEACHING READING TCHR0056 T068006 PJ000187 PREPARED IN TEACHING MULTICULTURAL LITERATURE TCHR0057 T068007 PJ000188 PREPARED IN COMPUTER SOFTWARE FOR TEACHING RDG TCHR0058 T068008 PJ000190 PREPARED IN WRITING ACROSS THE CURRICULUM 
Table F-2 (continued)

Summary Table of the 1998 Reading Conditioning Variable Specifications

Cond’ng.NAEP ID ID TDDC ID DESCRIPTION TCHR0059 T068009 PJ000191 PREPARED IN USING COMPUTER SOFTWARE TO TEACH WRTG TCHR0060 T068010 PJ000192 PREPARED IN TEACHING SPELLING, GRAMMAR, MECHANICS TCHR0061 T068101 ID100368 AVERAGE READING CLASS SIZE TCHR0062 T046101 HE001284 CLASS ASSIGNMENT BY ABILITY TCHR0063 T046201 HE001201 ABILITY LEVEL OF STUDENTS TCHR0064 T068201 PJ000196 HOW MUCH CLASS TIME PER DAY-READING INSTRUCTION TCHR0065 T068301 PJ000198 BASIS FOR CREATING READING INSTRUCTIONAL GROUPS TCHR0066 T068401 PJ000199 CLASS DIVIDED INTO HOW MANY INSTRUCTIONAL GROUPS TCHR0067 T068601 PJ000195 WRITING ABILITY LEVEL OF CLASS TCHR0068 T068701 PJ000197 EACH WEEK, TIME SPENT INSTRUCTING/HELPING-WRITING TCHR0069 T068801 PJ000202 HOW OFTEN USE CHILDREN’S NEWSPAPERS/MAGAZINES TCHR0070 T068802 PJ000203 HOW OFTEN USE READING KITS TO TEACH READING TCHR0071 T068803 PJ000204 HOW OFTEN USE COMPUTER SOFTWARE FOR READING INSTR TCHR0072 T068804 PJ000205 HOW OFTEN USE BOOKS (NOVELS, POETRY, NONFICTION) TCHR0073 T068805 PJ000206 HOW OFTEN USE MATERIALS FROM OTHER SUBJECTS TCHR0074 T068901 ID100374 WHAT TYPE OF MATERIALS FORM CORE READING PROGRAM TCHR0075 T069001 PJ000207 AVAILABILITY OF COMPUTERS FOR USE IN CLASS TCHR0076 T069101 PJ000208 PROPORTION TIME SPENT ON RDG FOR LIT EXPERIENCE TCHR0077 T069102 PJ000210 PROPORTION TIME SPENT ON RDG TO GAIN INFORMATION TCHR0078 T069103 PJ000211 PROPORTION TIME SPENT ON RDG TO PERFORM A TASK TCHR0079 T069201 PJ000213 PROPORTION TIME SPENT ON NARRATIVE WRITING TCHR0080 T069202 PJ000214 PROPORTION TIME SPENT ON INFORMATIVE WRITING TCHR0081 T069203 PJ000215 PROPORTION TIME SPENT ON PERSUASIVE WRITING TCHR0082 T069301 PJ000217 DO YOU USE GRAMMAR OR SKILL-BASED INSTRUCTION TCHR0083 T069302 PJ000218 DO YOU USE WRITING PROCESS INSTRUCTION TCHR0084 T069303 PJ000219 DO YOU INTEGRATE READING AND WRITING INSTRUCTION TCHR0085 T069304 PJ000220 DO YOU USE WRITING ABOUT LITERATURE TCHR0086 T069305 PJ000221 DO YOU USE WRITING ACROSS OTHER SUBJECT AREAS TCHR0087 T069401 PJ000223 HOW OFTEN STUDENTS DO SPELLING, PUNCTUATION, GRAMM TCHR0088 T069402 PJ000224 HOW OFTEN STUDENTS WORK ON WRITING PROCESS TCHR0089 T069403 PJ000225 HOW OFTEN STUDENTS WRITE IN A LOG/JOURNAL TCHR0090 T069404 PJ000226 HOW OFTEN PARENTS SIGN/REVIEW STUDENTS’ HOMEWORK TCHR0091 T069405 PJ000227 HOW OFTEN ASSIGN HOMEWORK TO DO WITH PARENTS TCHR0092 T069501 PJ000228 EXPECTED TIME SPENT ON WRITING ASSIGNMENTS/WEEK TCHR0093 T069601 PJ000193 THIS YEAR, PROJECTS TO DO/SHARE WITH PARENTS TCHR0094 T069701 PJ000231 HOW OFTEN ASK STUDENTS TO READ ALOUD TCHR0095 T069702 PJ000233 HOW OFTEN ASK STUDENTS-DISCUSS WHAT WAS READ TCHR0096 T069703 PJ000234 HOW OFTEN ASK STUDENTS- WRITE ABOUT WHAT WAS READ TCHR0097 T069704 PJ000235 HOW OFTEN ASK STUDENTS-WRITE IN WORKSHEET/BOOK TCHR0098 T069705 PJ000232 HOW OFTEN ASK STUDENTS-READ SILENTLY TCHR0099 T069706 PJ000236 HOW OFTEN GIVE STUDENTS TIME TO READ BOOKS CHOSEN TCHR0100 T069707 PJ000237 HOW OFTEN ASK STUDENTS-GROUP ACTIVITY/PROJECT TCHR0101 T069708 ID100371 HOW OFTEN ASK STUDENTS-DISCUSS INTERPRETATIONS TCHR0102 T069709 PJ000238 HOW OFTEN ASK STUDENTS-EXPLAIN/SUPPORT WHAT READ TCHR0103 T069710 ID100372 HOW OFTEN GIVE READING QUIZZES OR TESTS TCHR0104 T069711 PJ000239 HOW OFTEN WATCH MOVIES, VIDEOS, FILMSTRIPS, TV, CD TCHR0105 T069712 PJ000229 HOW OFTEN HELP STUDENTS UNDERSTAND NEW WORDS TCHR0106 T069713 PJ000240 HOW OFTEN ASK STUDENTS-ANSWER QUESTIONS IN WRITING TCHR0107 T069714 PJ000241 HOW OFTEN ASK STUDENTS-PREDICT OUTCOME OF READING TCHR0108 T069715 PJ000242 HOW OFTEN ASK STUDENTS-MAKE GENERALIZATIONS TCHR0109 T069716 PJ000243 HOW OFTEN ASK STUDENTS-DESCRIBE STYLE/STRUCTURE TCHR0110 T071801 PJ000245 HOW OFTEN STUDENTS CHOOSE WRITING TOPIC TCHR0111 T071802 PJ000246 HOW OFTEN STUDENTS PLAN THEIR WRITING TCHR0112 T071803 PJ000247 HOW OFTEN STUDENTS DEFINE PURPOSES AND AUDIENCE 
Table F-2 (continued)

Summary Table of the 1998 Reading Conditioning Variable Specifications

Cond’ng.NAEP ID ID TDDC ID DESCRIPTION 634TCHR0113 T071804 PJ000248 HOW OFTEN STUDENTS MAKE FORMAL OUTLINE TCHR0114 T071805 PJ000249 HOW OFTEN STUDENTS WRITE MORE THAN ONE DRAFT TCHR0115 T071806 PJ000250 HOW OFTEN STUDENTS USE RESOURCES OTHER THAN TEXT TCHR0116 T071807 PJ000251 HOW OFTEN STUDENTS DISCUSS WRITING WHILE WRITING TCHR0117 T071808 PJ000252 HOW OFTEN STUDENTS DISCUSS OTHERS’ WRITING TCHR0118 T071809 PJ000253 HOW OFTEN STUDENTS CHECK PROPER SPELLING, GRAMMAR TCHR0119 T071810 PJ000254 HOW OFTEN STUDENTS DISCUSS WRITING WITH FAMILY TCHR0120 T071811 PJ000255 HOW OFTEN STUDENTS CONTRIBUTE TO COLLECTION TCHR0121 T071812 PJ000256 HOW OFTEN STUDENTS WORK ON AN ASSIGNED TOPIC TCHR0122 T071813 PJ000257 HOW OFTEN STUDENTS FOLLOW ASSIGNED FORMAT TCHR0123 T069901 PJ000259 HOW OFTEN WRITING ASSIGNMENTS-LESS THAN ONE PAGE TCHR0124 T069902 PJ000260 HOW OFTEN WRITING ASSIGNMENTS-ONE TO TWO PAGES TCHR0125 T069903 PJ000261 HOW OFTEN WRITING ASSIGNMENTS-THREE OR MORE PAGES TCHR0126 T070001 PJ000263 HOW OFTEN STUDENTS USE COMPUTER-SPELL, PUNC, GRAM TCHR0127 T070002 PJ000264 HOW OFTEN STUDENTS USE COMPUTERS-WRITE DRAFTS TCHR0128 T070003 PJ000265 HOW OFTEN STUDENTS USE COMPUTERS-READ STORIES TCHR0129 T070101 ID100373 HOW OFTEN READING ASSESSED-MULTIPLE-CHOICE TESTS TCHR0130 T070102 ID100375 HOW OFTEN READING ASSESSED-SHORT-ANSWER TESTS TCHR0131 T070103 PJ000269 HOW OFTEN READ ASSESSED-PARAGRAPH WRITTEN RESPONSE TCHR0132 T070104 PJ000270 HOW OFTEN STUDENTS ASSESSED-INDIVIDUAL/GROUP PROJ TCHR0133 T070105 PJ000272 HOW OFTEN STUDENTS ASSESSED-READING PORTFOLIOS TCHR0134 T070106 PJ000271 HOW OFTEN STUDENTS ASSESSED-ESSAYS/PAPERS ASSIGNED TCHR0135 T070107 PJ000273 HOW OFTEN STUDENTS ASSESSED-ORAL READING TCHR0136 T070201 PJ000275 HOW OFTEN WRITING ASSESSED-MULTIPLE-CHOICE TESTS TCHR0137 T070202 PJ000276 HOW OFTEN WRITING ASSESSED-PARAGRAPH WRITTEN TCHR0138 T070203 PJ000277 HOW OFTEN WRITING ASSESSED-ESSAYS, REPORTS TCHR0139 T070204 PJ000278 HOW OFTEN WRITING ASSESSED-WRITING PORTFOLIOS TCHR0140 T070301 PJ000280 HOW IMPORTANT TO GRADE-SPELLING, GRAMMAR, PUNC TCHR0141 T070302 PJ000281 HOW IMPORTANT TO GRADE-ORGANIZATION/COHERENCE TCHR0142 T070303 PJ000282 HOW IMPORTANT TO GRADE-QUALITY/CREATIVITY OF IDEAS TCHR0143 T070304 PJ000283 HOW IMPORTANT TO GRADE-LENGTH OF PAPERS TCHR0144 T070305 PJ000284 HOW IMPORTANT TO GRADE-ACCOMPLISH WRITING PURPOSE TCHR0145 T071601 PJ000330 DO YOU TEACH READING TCHR0146 T071602 PJ000331 DO YOU TEACH WRITING TCHR0147 T071603 PJ000332 DO YOU TEACH ENGLISH TCHR0148 T071604 PJ000333 DO YOU TEACH-OTHER TCHR0149 T040301 HE001007 YEARS TOTAL TAUGHT ELEMENTARY OR SECONDARY TCHR0150 T071701 PJ000335 YEARS TOTAL TAUGHT READING TCHR0151 T071702 PJ000336 YEARS TOTAL TAUGHT WRITING TCHR0152 T071703 PJ000337 YEARS TOTAL TAUGHT ENGLISH TCHR0153 T071704 PJ000338 YEARS TOTAL TAUGHT- OTHER TCHR0154 T067703 PJ000167 LAST 12 MOS, PROF DEV-LITERATURE TCHR0155 T068501 ID100370 ARE STUDENTS ASSIGNED TO THIS CLASS BY ABILITY TCHR0156 T069801 PJ000245 HOW OFTEN STUDENTS CHOOSE WRITING TOPIC TCHR0157 T069802 PJ000246 HOW OFTEN STUDENTS PLAN THEIR WRITING TCHR0158 T069803 PJ000247 HOW OFTEN STUDENTS DEFINE PURPOSES AND AUDIENCE TCHR0159 T069804 PJ000248 HOW OFTEN STUDENTS MAKE FORMAL OUTLINE TCHR0160 T069805 PJ000249 HOW OFTEN STUDENTS WRITE MORE THAN ONE DRAFT TCHR0161 T069806 PJ000250 HOW OFTEN STUDENTS USE RESOURCES OTHER THAN TEXT TCHR0162 T069807 PJ000251 HOW OFTEN STUDENTS DISCUSS WRITING WHILE WRITING TCHR0163 T069808 PJ000252 HOW OFTEN STUDENTS DISCUSS OTHERS’ WRITING TCHR0164 T069809 PJ000253 HOW OFTEN STUDENTS CHECK PROPER SPELLING, GRAMMAR TCHR0165 T069810 PJ000254 HOW OFTEN STUDENTS DISCUSS WRITING WITH FAMILY TCHR0166 T069811 PJ000255 HOW OFTEN STUDENTS CONTRIBUTE TO COLLECTION 
Table F-2 (continued)

Summary Table of the 1998 Reading Conditioning Variable Specifications

Cond’ng.NAEP ID ID TDDC ID DESCRIPTION TCHR0167 T069812 PJ000256 HOW OFTEN STUDENTS WORK ON AN ASSIGNED TOPIC TCHR0168 T069813 PJ000257 HOW OFTEN STUDENTS FOLLOW ASSIGNED FORMAT TCHR0169 TCSIZE WHAT IS THE NUMBER OF STUDENTS IN EACH CLASS? (8TH GRADE) 
Table F-3

Summary Table of the 1998 Writing Conditioning Variable Specifications

Cond’ng.NAEP ID ID TDDC ID DESCRIPTION 

636

BACK0001 BKSER GRAND MEAN BACK0002 DSEX DERIVED SEX BACK0003 DRACE DERIVED RACE/ETHNICITY BACK0004 B003101 TB003101 IF HISPANIC, WHAT IS YOUR HISPANIC BACKGROUND? BACK0005 TOL7 TOL 7 - TYPE OF LOCATION BACK0006 TOL5 TYPE OF LOCALE (5 CATEGORIES) BACK0007 PARED2 PARENTS’ HIGHEST LEVEL OF EDUCATION BACK0008 REGION REGION OF THE COUNTRY BACK0009 SCHTYPE SCHOOL TYPE BACK0010 RACE RACE BACK0011 IEP INDIVIDUALIZED EDUCATION PLAN BACK0012 LEP LIMITED ENGLISH PROFICIENCY BACK0013 TITLE1 TITLE 1: (BOOK COVER) BACK0014 SLUNCH DO YOU RECEIVE A FREE OR REDUCED-PRICE LUNCH? BACK0015 B013901 ID100323 HOW MUCH TELEVISION DO YOU USUALLY WATCH EACH DAY? (LINEAR) BACK0016 B013901 ID100323 HOW MUCH TELEVISION DO YOU USUALLY WATCH EACH DAY? (QUADRATIC) BACK0017 B006601 TB006601 HOMEWORK ASSIGNED?: BASED ON TIME SPENT ON HOMEWORK EACH DAY. BACK0018 B006601 TB006601 HOW MUCH TIME DO YOU USUALLY SPEND ON HOMEWORK EACH DAY? (LINEAR) BACK0019 B006601 TB006601 HOW MUCH TIME DO YOU USUALLY SPEND ON HOMEWORK EACH DAY (QUADRATIC) BACK0020 HOMEEN3 NUMBER OF ITEMS IN THE HOME (NEWSPAPER, > 25 BOOKS, ENCYCLOPEDIA, MAGAZINES) (DERIVEDBACK0021 B001101 TB001101 ABOUT HOW MANY PAGES A DAY DO YOU HAVE TO READ FOR SCHOOL AND HOMEWORK? BACK0022 B001101 TB001101 ABOUT HOW MANY PAGES A DAY DO YOU HAVE TO READ FOR SCHOOL AND HOMEWORK? BACK0023 ACCOM STUDENTS ACCOMMODATION STATUS BACK0024 INTERACT INTERACTION: GENDER BY RACE/ETHNICITY BACK0025 INTERACT INTERACTION: GENDER BY TYPE OF LOCALE (7 CATEGORIES) BACK0026 INTERACT INTERACTION: GENDER BY PARENTS’ EDUCATION BACK0027 INTERACT INTERACTION: GENDER BY SCHOOL TYPE BACK0028 INTERACT INTERACTION: RACE/ETHNICITY BY TYPE OF LOCALE (7 CATEGORIES) BACK0029 INTERACT INTERACTION: RACE/ETHNICITY BY PARENTS’ EDUCATION BACK0030 INTERACT INTERACTION: RACE/ETHNICITY BY SCHOOL TYPE BACK0031 INTERACT INTERACTION: PARENT’S EDUCATION BY TYPE OF LOCALE (7 CATEGORIES) BACK0032 INTERACT INTERACTION: TYPE OF LOCALE (7 CATEGORIES) BY SCHOOL TYPE BACK0033 INTERACT INTERACTION: PARENTS’ EDUCATION BY SCHOOL TYPE BACK0034 INTERACT INTERACTION: ACCOMMODATED BY GENDER BACK0035 INTERACT INTERACTION: ACCOMMODATED BY RACE/ETHNICITY BACK0036 INTERACT INTERACTION: ACCOMMODATED BY TYPE OF LOCALE (7 CATEGORIES) BACK0037 INTERACT INTERACTION: ACCOMMODATED BY PARENTS’ EDUCATION ALL GRADES BACK0038 INTERACT INTERACTION: ACCOMMODATED BY SCHOOL TYPE BACK0039 INTERACT INTERACTION: ACCOMMODATED BY IEP BACK0040 INTERACT INTERACTION: ACCOMMODATED BY LEP BACK0041 MA96FLG MSA/NON-MSA BACK0042 MONSTUD STATE ADMINISTRATION MONITORED/UNMONITORED SESSION BACK0043 INTERACT INTERACTION: SCHOOL TYPE BY MONITORED/UNMONITORED SESSION BACK0044 RPTSAMP REPORTING SAMPLE BACK0045 DISTRPT STATE/DISTRICT BACK0046 B003001 TB003001 WHICH RACE/ETHNICITY BEST DESCRIBES YOU BACK0047 B003101 TB003101 IF HISPANIC, WHAT IS YOUR HISPANIC BACKGROUND BACK0048 B013001 ID100333 HOW LONG LIVED IN UNITED STATES BACK0049 B013101 ID100322 HOW OFTEN OTHER THAN ENGLISH SPOKEN AT HOME BACK0050 B013201 ID100314 MOTHER GRADUATED HIGH SCHOOL BACK0051 B013301 ID100315 MOTHER HAD SOME EDUCATION AFTER HIGH SCHOOL 
Table F-3 (continued)

Summary Table of the 1998 Writing Conditioning Variable Specifications

Cond’ng.NAEP ID ID TDDC ID DESCRIPTION BACK0052 B013401 ID100316 MOTHER GRADUATED COLLEGE BACK0053 B013501 ID100317 FATHER GRADUATED HIGH SCHOOL BACK0054 B013601 ID100318 FATHER HAD SOME EDUCATION AFTER HIGH SCHOOL BACK0055 B013701 ID100319 FATHER GRADUATED COLLEGE BACK0056 B000901 TB000901 DOES YOUR FAMILY GET A NEWSPAPER REGULARLY BACK0057 B000903 TB000903 IS THERE AN ENCYCLOPEDIA IN YOUR HOME BACK0058 B013801 ID100334 HOW MANY BOOKS ARE IN YOUR HOME BACK0059 B000905 TB000905 DOES YOUR FAMILY GET MAGAZINES REGULARLY BACK0060 B013901 ID100323 HOURS OF TV/VIDEO WATCHED ON SCHOOL DAYS BACK0061 B006601 TB006601 TIME SPENT ON HOMEWORK EACH DAY BACK0062 B001101 TB001101 HOW MANY PAGES READ IN SCHOOL AND FOR HOMEWORK BACK0063 B014001 ID100324 DAYS ABSENT FROM SCHOOL LAST MONTH BACK0064 B007301 HE000712 TIMES CHANGED SCHOOLS IN PAST TWO YEARS BACK0065 B007401 HE000717 HOW OFTEN DISCUSS STUDIES AT HOME BACK0066 B014101 ID100325 HOW OFTEN USE COMPUTER AT HOME FOR SCHOOLWORK SUBJ0001 W803001 HE000729 HOW HARD TRIED ON THIS WRITING TEST THAN ON OTHERS SUBJ0002 W803101 HE000730 HOW IMPORTANT TO DO WELL ON THIS WRITING TEST SUBJ0003 W803201 HE000731 HOW OFTEN TAKE ESSAY TEST FOR WHOLE CLASS PERIOD SUBJ0004 W803301 ID100342 MY FRIENDS MAKE FUN OF PEOPLE WHO TRY TO DO WELL SUBJ0005 W803302 ID100343 I HAVE FRIENDS TO TALK TO IF NEED HELP W/SCHOOL SUBJ0006 W801901 ID100003 I LIKE TO WRITE SUBJ0007 W801902 ID100004 I AM GOOD AT WRITING SUBJ0008 W802001 ID100335 TEACHER TALKS ABOUT WHAT YOU ARE WRITING SUBJ0009 W802101 ID100336 TEACHER ASKS TO WRITE MORE THAN ONE DRAFT OF PAPER SUBJ0010 W802201 ID100337 TEACHER ASKS TO CONTRIBUTE WRITING TO A COLLECTION SUBJ0011 W802301 HE000484 DO SPELLING, PUNCTUATION, GRAMMAR EXERCISES SUBJ0012 W802302 ID100011 HOW OFTEN WRITE A STORY OR REPORT SUBJ0013 W802303 HE000723 HOW OFTEN WORK IN PAIRS/SMALL GROUPS-WRITING SUBJ0014 W802304 HE000724 HOW OFTEN WRITE IN A LOG/JOURNAL SUBJ0015 W802401 ID100014 DO YOU/TEACHER SAVE WRITING-FOLDER/PORTFOLIO SUBJ0016 W802501 HE000488 GRADE/WRITING-SPELLING, PUNCTUATION, GRAMMAR SUBJ0017 W802502 HE000489 GRADE/WRITING-ORGANIZATION OF PAPER SUBJ0018 W802503 HE000490 GRADE/WRITING-QUALITY, CREATIVITY OF IDEAS SUBJ0019 W802504 HE000491 GRADE/WRITING-LENGTH OF PAPER SUBJ0020 W802601 ID100022 ON COMPUTER-DO SPELLING, PUNCTUATION, GRAMMAR SUBJ0021 W802602 ID100023 ON COMPUTER-WRITE IN A LOG/JOURNAL SUBJ0022 W802603 ID100024 ON COMPUTER-WRITE DRAFTS/FINAL VERSIONS OF PAPERS SCHL0001 C042501 ID100378 FOURTH GRADERS ASSIGNED TO CLASS BY ABILITY SCHL0002 C042601 ID100041 HOW OFTEN STUDENTS RECEIVE READING INSTRUCTION SCHL0003 C042602 ID100042 HOW OFTEN STUDENTS RECEIVE WRITING INSTRUCTION SCHL0004 C042603 ID100043 HOW OFTEN STUDENTS RECEIVE SOC STUDIES INSTRUCT SCHL0005 C042604 ID100044 HOW OFTEN STUDENTS RECEIVE COMPUTER USE INSTRUCT SCHL0006 C042701 ID100379 DOES SCHOOL USE BLOCK SCHEDULING SCHL0007 C042801 ID100380 ARE COMPUTERS AVAILABLE IN ALL CLASSROOMS SCHL0008 C042802 HE000864 ARE COMPUTERS AVAILABLE IN COMPUTER LAB SCHL0009 C042803 HE000866 ARE COMPUTERS AVAILABLE TO CLASSROOM WHEN NEEDED SCHL0010 C042901 ID100381 HOW MANY COMPUTERS AVAILABLE TO STUDENTS SCHL0011 C036601 LC000502 PRIMARY WAY LIBRARY IS STAFFED SCHL0012 C043001 ID100069 PARENTS PARTICIPATE-PARENT-TEACHER ORG SCHL0013 C043002 ID100070 PARENTS PARTICIPATE-OPEN HOUSE SCHL0014 C043003 ID100071 PARTICIPATE-PARENT-TEACHER CONFERENCE SCHL0015 C043004 ID100072 PARENTS PARTICIPATE-SCHOOL CURRICULUM DECISIONS SCHL0016 C043005 ID100073 PARENTS PARTICIPATE-VOLUNTEER PROGRAMS SCHL0017 C043006 ID100074 PARENTS PARTICIPATE-PARENTING-SKILLS PROGRAM 
Table F-3 (continued)

Summary Table of the 1998 Writing Conditioning Variable Specifications

Cond’ng.NAEP ID ID TDDC ID DESCRIPTION 638SCHL0018 C043007 ID100076 PARENTS PARTICIPATE-SCHOOL ADVISORY COMMITTEES SCHL0019 C043008 ID100077 PARENTS PARTICIPATE-CLASSROOM ASSISTANTS SCHL0020 C032402 HE000888 IS STUDENT ABSENTEEISM A PROBLEM IN YOUR SCHOOL SCHL0021 C032401 HE000887 IS STUDENT TARDINESS A PROBLEM IN YOUR SCHOOL SCHL0022 C032404 HE000890 ARE PHYSICAL CONFLICTS A PROBLEM IN YOUR SCHOOL SCHL0023 C032407 HE000893 ARE RACIAL/CULT. CONFLICTS A PROBLEM IN SCHOOL SCHL0024 C032408 HE000894 IS STUDENT HEALTH A PROBLEM IN YOUR SCHOOL SCHL0025 C032409 HE002121 IS LACK OF PARENT INVLVMNT A PROBLEM IN SCHOOL SCHL0026 C032410 HE002122 IS STUDENT ALCOHOL USE A PROBLEM IN YOUR SCHOOL SCHL0027 C032411 HE002123 IS STUDENT TOBACCO USE A PROBLEM IN YOUR SCHOOL SCHL0028 C032412 HE002124 IS STUDENT DRUG USE A PROBLEM IN YOUR SCHOOL SCHL0029 C032413 HE002125 ARE GANG ACTIVITIES A PROBLEM IN YOUR SCHOOL SCHL0030 C032414 HE002126 IS STUDENT MISBEHAVIOR A PROBLEM IN YOUR SCHOOL SCHL0031 C043101 ID100079 IS STUDENT CHEATING A PROBLEM IN YOUR SCHOOL SCHL0032 C043102 ID100077 IS TEACHER ABSENTEEISM A PROBLEM IN YOUR SCHOOL SCHL0033 C043103 ID100078 ARE PHYSICAL CONFLICTS BETWEEN STUDENTS/TEACHERS SCHL0034 C043104 ID100080 IS VANDALISM A PROBLEM IN YOUR SCHOOL SCHL0035 C032502 HE000897 TEACHER MORALE SCHL0036 C032503 HE000898 STUDENT ATTITUDES TOWARD ACADEMIC ACHIEVEMENT SCHL0037 C032505 HE000900 PARENT SUPPORT FOR STUDENT ACHIEVEMENT SCHL0038 C032506 HE000901 REGARD FOR SCHOOL PROPERTY SCHL0039 C043201 ID100081 TEACHERS’ EXPECTATIONS FOR STUDENT ACHIEVEMENT SCHL0040 C043301 ID100082 PERCENT STUDENT BODY ABSENT AVERAGE DAY SCHL0041 C043401 ID100389 PERCENT TEACHING STAFF ABSENT AVERAGE DAY SCHL0042 C043501 ID100390 ENROLLMENT LAST YEAR COMPARED TO END OF SCHOOL YR SCHL0043 C043601 HE002112 PERCENT STUDENTS HELD BACK AND REPEATING GRADE SCHL0044 C043701 ID100391 PERCENT TEACHING STAFF LEFT BEFORE END OF YEAR SCHL0045 C038301 HE002094 IS SCHOOL IN NATIONAL SCHOOL LUNCH PROGRAM SCHL0046 C043801 ID100392 PERCENT ELIGIBLE NATIONAL SCHOOL LUNCH PROGRAM SCHL0047 C043901 ID100393 DOES SCHOOL RECEIVE CHAPTER 1/TITLE I FUNDING SCHL0048 C044001 ID100395 PERCENT STUDENTS RECEIVE CHAPTER1/TITLE I FUNDING SCHL0049 C044002 ID100396 PERCENT STUDENTS RECEIVE REMEDIAL READING INSTRUCT SCHL0050 C044003 ID100397 PERCENT STUDENTS RECEIVE REMEDIAL WRITING INSTRUCT SCHL0051 C044004 ID100398 PERCENT STUDENTS IN GIFTED AND TALENTED PROGRAM BACK0067 B014201 ID100248 HOW MUCH EDUCATION DO YOU EXPECT TO RECEIVE SUBJ0023 W802701 TW800301 HOW OFTEN PAPERS ASSIGED-ONE TO TWO PARAGRAPHS SUBJ0024 W802702 TW800302 HOW OFTEN PAPERS ASSIGNED-ONE TO TWO PAGES SUBJ0025 W802703 TW800303 HOW OFTEN PAPERS ASSIGNED-THREE OR MORE PAGES SUBJ0026 W802801 HE000431 HOW OFTEN WRITING ASSIGNED-REPORT OR SUMMARY SUBJ0027 W802802 HE000432 HOW OFTEN WRITING ASSIGNED-ESSAY/THEME TO ANALYZE SUBJ0028 W802803 HE000512 HOW OFTEN WRITING ASSIGNED-ESSAY/LETTER- PERSUADE SUBJ0029 W802804 TW800503 HOW OFTEN WRITING ASSIGNED-STORY/NARRATIVE SUBJ0030 W802901 ID100035 HOW OFTEN ASKED TO PLAN YOUR WRITING SUBJ0031 W802902 ID100036 HOW OFTEN ASKED TO MAKE FORMAL OUTLINE FIRST SUBJ0032 W802903 ID100037 HOW OFTEN ASKED TO DEFINE PURPOSE AND AUDIENCE SUBJ0033 W802904 ID100038 HOW OFTEN ASKED TO USE SOURCES OTHER THAN TEXTBOOK SCHL0052 C044401 ID100400 8TH GRADE ASSIGNED TO ENGLISH CLASS BY ABILITY SCHL0053 C044402 ID100403 8TH GRADE ASSIGNED-HISTORY/SS BY ABILITY SCHL0054 C043105 ID100086 IS STUDENT DROPOUT A PROBLEM IN YOUR SCHOOL SCHL0055 C043106 ID100087 IS TEEN PREGNANCY A PROBLEM IN YOUR SCHOOL BACK0068 B005501 TB005501 MAIN ACTIVITY YEAR FOLLOWING HIGH SCHOOL BACK0069 B014301 ID100326 VOLUNTEER WORK IN YOUR COMMUNITY THIS YEAR BACK0070 B014401 ID100332 HOW MANY HOURS/WEEK WORK JOB FOR PAY SCHL0056 C044301 ID100404 12TH GRADE ASSIGNED TO ENGLISH CLASS BY ABILITY 
Table F-3 (continued)

Summary Table of the 1998 Writing Conditioning Variable Specifications

Cond’ng.NAEP ID ID TDDC ID DESCRIPTION SCHL0057 C044302 ID100405 12TH GR ASSIGNED- HISTORY/CIVICS/SS CLASS ABILITY SCHL0058 C044101 ID100408 PERCENT LAST YEAR’S TWELFTH-GRADE CLASS GRADUATED SCHL0059 C044201 ID100410 PERCENT GRADUATING CLASS-ATTEND TWO-YEAR COLLEGE SCHL0060 C044202 ID100411 PERCENT GRADUATING CLASS-ATTEND FOUR-YEAR COLLEGE TCHR0001 T067001 PJ000121 DO YOU TEACH READING TCHR0002 T067002 PJ000122 DO YOU TEACH WRITING TCHR0003 T067003 PJ000123 DO YOU TEACH LANGUAGE ARTS TCHR0004 T067004 PJ000124 DO YOU TEACH SOCIAL STUDIES TCHR0005 T067101 PJ000126 YEARS TOTAL TAUGHT ELEMENTARY LEVEL TCHR0006 T067201 PJ000128 YEARS TOTAL TAUGHT READING TCHR0007 T067202 PJ000129 YEARS TOTAL TAUGHT WRITING TCHR0008 T067203 PJ000130 YEARS TOTAL TAUGHT LANGUAGE ARTS TCHR0009 T067204 PJ000131 YEARS TOTAL TAUGHT HISTORY TCHR0010 T067205 PJ000132 YEARS TOTAL TAUGHT SOCIAL STUDIES TCHR0011 T067206 PJ000133 YEARS TOTAL TAUGHT CIVICS TCHR0012 T067301 PJ000134 MAIN ASSIGNMENT FIELD TCHR0013 T056201 HE002551 TEACHING CERTIF IN THIS STATE IN MAIN FIELD TCHR0014 T056301 HE001012 HIGHEST ACADEMIC DEGREE YOU HOLD TCHR0015 T067501 PJ000138 UNDERGRAD MAJOR/MINOR-ELEMENTARY EDUCATION TCHR0016 T067502 PJ000139 UNDERGRAD MAJOR/MINOR-SECONDARY EDUCATION TCHR0017 T067503 PJ000140 UNDERGRAD MAJOR/MINOR-SPECIAL EDUCATION TCHR0018 T067504 PJ000141 UNDERGRAD MAJOR/MINOR-BILINGUAL EDUCATION/ESL TCHR0019 T067505 PJ000142 UNDERGRAD MAJOR/MINOR-ADMINISTRATION & SUPERVISION TCHR0020 T067506 PJ000143 UNDERGRAD MAJOR/MINOR-CURRICULUM & SUPERVISION TCHR0021 T067507 PJ000144 UNDERGRAD MAJOR/MINOR-COUNSELING TCHR0022 T067508 PJ000145 UNDERGRAD MAJOR/MINOR-ENGLISH TCHR0023 T067509 PJ000146 UNDERGRAD MAJOR/MINOR-READING AND/OR LANGUAGE ARTS TCHR0024 T067510 PJ000147 UNDERGRAD MAJOR/MINOR-HISTORY TCHR0025 T067511 PJ000148 UNDERGRAD MAJOR/MINOR-POLITICAL SCIENCE TCHR0026 T067512 PJ000149 UNDERGRAD MAJOR/MINOR-OTHER TCHR0027 T067601 PJ000151 GRAD MAJOR/MINOR-ELEMENTARY EDUCATION TCHR0028 T067602 PJ000152 GRAD MAJOR/MINOR-SECONDARY EDUCATION TCHR0029 T067603 PJ000153 GRAD MAJOR/MINOR-SPECIAL EDUCATION TCHR0030 T067604 PJ000154 GRAD MAJOR/MINOR-BILINGUAL EDUCATION/ESL TCHR0031 T067605 PJ000155 GRAD MAJOR/MINOR-ADMINSTRATION & SUPERVISION TCHR0032 T067606 PJ000156 GRAD MAJOR/MINOR-CURRICULUM AND INSTRUCTION TCHR0033 T067607 PJ000157 GRAD MAJOR/MINOR-COUNSELING TCHR0034 T067608 PJ000158 GRAD MAJOR/MINOR-ENGLISH TCHR0035 T067609 PJ000159 GRAD MAJOR/MINOR-READING AND/OR LANGUAGE ARTS TCHR0036 T067610 PJ000160 GRAD MAJOR/MINOR-HISTORY TCHR0037 T067611 PJ000161 GRAD MAJOR/MINOR-POLITICAL SCIENCE TCHR0038 T067612 PJ000162 GRAD MAJOR/MINOR-OTHER TCHR0039 T067701 ID100358 LAST 12 MOS, PROF DEV-READING AND WRITING TCHR0040 T067702 ID100147 LAST 12 MOS, PROF DEV-SOCIAL STUDIES TCHR0041 T067801 PJ000169 PREPARED IN THE USE OF TELECOMMUNICATIONS TCHR0042 T067802 ID100360 PREPARED IN THE USE OF COMPUTERS TCHR0043 T067803 PJ000171 PREPARED IN COOPERATIVE GROUP INSTRUCTION TCHR0044 T067804 PJ000176 PREPARED IN TEACHING STUDENTS-DIFFERENT CULTURES TCHR0045 T067805 PJ000177 PREPARED IN TEACHING STUDENTS WHO ARE LEP TCHR0046 T067806 PJ000178 PREPARED IN TEACHING STUDENTS WITH DISABILITIES TCHR0047 T067807 PJ000179 PREPARED IN CLASSROOM MANAGEMENT AND ORGANIZATION TCHR0048 T041201 HE001022 AVAILABILITY OF RESOURCES TCHR0049 T067901 ID100417 HOW WELL PREPARED TO TEACH READING TCHR0050 T067902 ID100418 HOW WELL PREPARED TO TEACH WRITING 
Table F-3 (continued)

Summary Table of the 1998 Writing Conditioning Variable Specifications

Cond’ng.NAEP ID ID TDDC ID DESCRIPTION 640TCHR0051 T068001 PJ000182 PREPARED IN LIT-BASED READING INSTRUCTION TCHR0052 T068002 PJ000183 PREPARED IN CONTENT AREA READING TCHR0053 T068003 PJ000184 PREPARED IN COMBINING RDG AND WRITING TCHR0054 T068004 PJ000185 PREPARED IN WHOLE LANGUAGE APPROACH TO TEACH RDG TCHR0055 T068005 PJ000186 PREPARED IN PHONICS IN TEACHING READING TCHR0056 T068006 PJ000187 PREPARED IN TEACHING MULTICULTURAL LITERATURE TCHR0057 T068007 PJ000188 PREPARED IN COMPUTER SOFTWARE FOR TEACHING RDG TCHR0058 T068008 PJ000190 PREPARED IN WRITING ACROSS THE CURRICULUM TCHR0059 T068009 PJ000191 PREPARED IN USING COMPUTER SOFTWARE TO TEACH WRTG TCHR0060 T068010 PJ000192 PREPARED IN TEACHING SPELLING, GRAMMAR, MECHANICS TCHR0061 T068101 ID100368 AVERAGE READING CLASS SIZE TCHR0062 T046101 HE001284 CLASS ASSIGNMENT BY ABILITY TCHR0063 T046201 HE001201 ABILITY LEVEL OF STUDENTS TCHR0064 T068201 PJ000196 HOW MUCH CLASS TIME PER DAY-READING INSTRUCTION TCHR0065 T068301 PJ000198 BASIS FOR CREATING READING INSTRUCTIONAL GROUPS TCHR0066 T068401 PJ000199 CLASS DIVIDED INTO HOW MANY INSTRUCTIONAL GROUPS TCHR0067 T068601 PJ000195 WRITING ABILITY LEVEL OF CLASS TCHR0068 T068701 PJ000197 EACH WEEK, TIME SPENT INSTRUCTING/HELPING-WRITING TCHR0069 T068801 PJ000202 HOW OFTEN USE CHILDREN’S NEWSPAPERS/MAGAZINES TCHR0070 T068802 PJ000203 HOW OFTEN USE READING KITS TO TEACH READING TCHR0071 T068803 PJ000204 HOW OFTEN USE COMPUTER SOFTWARE FOR READING INSTR TCHR0072 T068804 PJ000205 HOW OFTEN USE BOOKS (NOVELS, POETRY, NONFICTION) TCHR0073 T068805 PJ000206 HOW OFTEN USE MATERIALS FROM OTHER SUBJECTS TCHR0074 T068901 ID100374 WHAT TYPE OF MATERIALS FORM CORE READING PROGRAM TCHR0075 T069001 PJ000207 AVAILABILITY OF COMPUTERS FOR USE IN CLASS TCHR0076 T069101 PJ000208 PROPORTION TIME SPENT ON RDG FOR LIT EXPERIENCE TCHR0077 T069102 PJ000210 PROPORTION TIME SPENT ON RDG TO GAIN INFORMATION TCHR0078 T069103 PJ000211 PROPORTION TIME SPENT ON RDG TO PERFORM A TASK TCHR0079 T069201 PJ000213 PROPORTION TIME SPENT ON NARRATIVE WRITING TCHR0080 T069202 PJ000214 PROPORTION TIME SPENT ON INFORMATIVE WRITING TCHR0081 T069203 PJ000215 PROPORTION TIME SPENT ON PERSUASIVE WRITING TCHR0082 T069301 PJ000217 DO YOU USE GRAMMAR OR SKILL-BASED INSTRUCTION TCHR0083 T069302 PJ000218 DO YOU USE WRITING PROCESS INSTRUCTION TCHR0084 T069303 PJ000219 DO YOU INTEGRATE READING AND WRITING INSTRUCTION TCHR0085 T069304 PJ000220 DO YOU USE WRITING ABOUT LITERATURE TCHR0086 T069305 PJ000221 DO YOU USE WRITING ACROSS OTHER SUBJECT AREAS TCHR0087 T069401 PJ000223 HOW OFTEN STUDENTS DO SPELLING, PUNCTUATION, GRAMM TCHR0088 T069402 PJ000224 HOW OFTEN STUDENTS WORK ON WRITING PROCESS TCHR0089 T069403 PJ000225 HOW OFTEN STUDENTS WRITE IN A LOG/JOURNAL TCHR0090 T069404 PJ000226 HOW OFTEN PARENTS SIGN/REVIEW STUDENTS’ HOMEWORK TCHR0091 T069405 PJ000227 HOW OFTEN ASSIGN HOMEWORK TO DO WITH PARENTS TCHR0092 T069501 PJ000228 EXPECTED TIME SPENT ON WRITING ASSIGNMENTS/WEEK TCHR0093 T069601 PJ000193 THIS YEAR, PROJECTS TO DO/SHARE WITH PARENTS TCHR0094 T069701 PJ000231 HOW OFTEN ASK STUDENTS TO READ ALOUD TCHR0095 T069702 PJ000233 HOW OFTEN ASK STUDENTS-DISCUSS WHAT WAS READ TCHR0096 T069703 PJ000234 HOW OFTEN ASK STUDENTS- WRITE ABOUT WHAT WAS READ TCHR0097 T069704 PJ000235 HOW OFTEN ASK STUDENTS-WRITE IN WORKSHEET/BOOK TCHR0098 T069705 PJ000232 HOW OFTEN ASK STUDENTS-READ SILENTLY TCHR0099 T069706 PJ000236 HOW OFTEN GIVE STUDENTS TIME TO READ BOOKS CHOSEN TCHR0100 T069707 PJ000237 HOW OFTEN ASK STUDENTS-GROUP ACTIVITY/PROJECT TCHR0101 T069708 ID100371 HOW OFTEN ASK STUDENTS-DISCUSS INTERPRETATIONS TCHR0102 T069709 PJ000238 HOW OFTEN ASK STUDENTS-EXPLAIN/SUPPORT WHAT READ TCHR0103 T069710 ID100372 HOW OFTEN GIVE READING QUIZZES OR TESTS TCHR0104 T069711 PJ000239 HOW OFTEN WATCH MOVIES, VIDEOS, FILMSTRIPS, TV, CD 
Table F-3 (continued)

Summary Table of the 1998 Writing Conditioning Variable Specifications

Cond’ng.NAEP ID ID TDDC ID DESCRIPTION TCHR0105 T069712 PJ000229 HOW OFTEN HELP STUDENTS UNDERSTAND NEW WORDS TCHR0106 T069713 PJ000240 HOW OFTEN ASK STUDENTS-ANSWER QUESTIONS IN WRITING TCHR0107 T069714 PJ000241 HOW OFTEN ASK STUDENTS-PREDICT OUTCOME OF READING TCHR0108 T069715 PJ000242 HOW OFTEN ASK STUDENTS-MAKE GENERALIZATIONS TCHR0109 T069716 PJ000243 HOW OFTEN ASK STUDENTS-DESCRIBE STYLE/STRUCTURE TCHR0110 T071801 PJ000245 HOW OFTEN STUDENTS CHOOSE WRITING TOPIC TCHR0111 T071802 PJ000246 HOW OFTEN STUDENTS PLAN THEIR WRITING TCHR0112 T071803 PJ000247 HOW OFTEN STUDENTS DEFINE PURPOSES AND AUDIENCE TCHR0113 T071804 PJ000248 HOW OFTEN STUDENTS MAKE FORMAL OUTLINE TCHR0114 T071805 PJ000249 HOW OFTEN STUDENTS WRITE MORE THAN ONE DRAFT TCHR0115 T071806 PJ000250 HOW OFTEN STUDENTS USE RESOURCES OTHER THAN TEXT TCHR0116 T071807 PJ000251 HOW OFTEN STUDENTS DISCUSS WRITING WHILE WRITING TCHR0117 T071808 PJ000252 HOW OFTEN STUDENTS DISCUSS OTHERS’ WRITING TCHR0118 T071809 PJ000253 HOW OFTEN STUDENTS CHECK PROPER SPELLING, GRAMMAR TCHR0119 T071810 PJ000254 HOW OFTEN STUDENTS DISCUSS WRITING WITH FAMILY TCHR0120 T071811 PJ000255 HOW OFTEN STUDENTS CONTRIBUTE TO COLLECTION TCHR0121 T071812 PJ000256 HOW OFTEN STUDENTS WORK ON AN ASSIGNED TOPIC TCHR0122 T071813 PJ000257 HOW OFTEN STUDENTS FOLLOW ASSIGNED FORMAT TCHR0123 T069901 PJ000259 HOW OFTEN WRITING ASSIGNMENTS-LESS THAN ONE PAGE TCHR0124 T069902 PJ000260 HOW OFTEN WRITING ASSIGNMENTS-ONE TO TWO PAGES TCHR0125 T069903 PJ000261 HOW OFTEN WRITING ASSIGNMENTS-THREE OR MORE PAGES TCHR0126 T070001 PJ000263 HOW OFTEN STUDENTS USE COMPUTER-SPELL, PUNC, GRAM TCHR0127 T070002 PJ000264 HOW OFTEN STUDENTS USE COMPUTERS-WRITE DRAFTS TCHR0128 T070003 PJ000265 HOW OFTEN STUDENTS USE COMPUTERS-READ STORIES TCHR0129 T070101 ID100373 HOW OFTEN READING ASSESSED-MULTIPLE-CHOICE TESTS TCHR0130 T070102 ID100375 HOW OFTEN READING ASSESSED-SHORT-ANSWER TESTS TCHR0131 T070103 PJ000269 HOW OFTEN READ ASSESSED-PARAGRAPH WRITTEN RESPONSE TCHR0132 T070104 PJ000270 HOW OFTEN STUDENTS ASSESSED-INDIVIDUAL/GROUP PROJ TCHR0133 T070105 PJ000272 HOW OFTEN STUDENTS ASSESSED-READING PORTFOLIOS TCHR0134 T070106 PJ000271 HOW OFTEN STUDENTS ASSESSED-ESSAYS/PAPERS ASSIGNED TCHR0135 T070107 PJ000273 HOW OFTEN STUDENTS ASSESSED-ORAL READING TCHR0136 T070201 PJ000275 HOW OFTEN WRITING ASSESSED-MULTIPLE-CHOICE TESTS TCHR0137 T070202 PJ000276 HOW OFTEN WRITING ASSESSED-PARAGRAPH WRITTEN TCHR0138 T070203 PJ000277 HOW OFTEN WRITING ASSESSED-ESSAYS, REPORTS TCHR0139 T070204 PJ000278 HOW OFTEN WRITING ASSESSED-WRITING PORTFOLIOS TCHR0140 T070301 PJ000280 HOW IMPORTANT TO GRADE-SPELLING, GRAMMAR, PUNC TCHR0141 T070302 PJ000281 HOW IMPORTANT TO GRADE-ORGANIZATION/COHERENCE TCHR0142 T070303 PJ000282 HOW IMPORTANT TO GRADE-QUALITY/CREATIVITY OF IDEAS TCHR0143 T070304 PJ000283 HOW IMPORTANT TO GRADE-LENGTH OF PAPERS TCHR0144 T070305 PJ000284 HOW IMPORTANT TO GRADE-ACCOMPLISH WRITING PURPOSE TCHR0145 T071601 PJ000330 DO YOU TEACH READING TCHR0146 T071602 PJ000331 DO YOU TEACH WRITING TCHR0147 T071603 PJ000332 DO YOU TEACH ENGLISH TCHR0148 T071604 PJ000333 DO YOU TEACH-OTHER TCHR0149 T040301 HE001007 YEARS TOTAL TAUGHT ELEMENTARY OR SECONDARY TCHR0150 T071701 PJ000335 YEARS TOTAL TAUGHT READING TCHR0151 T071702 PJ000336 YEARS TOTAL TAUGHT WRITING TCHR0152 T071703 PJ000337 YEARS TOTAL TAUGHT ENGLISH TCHR0153 T071704 PJ000338 YEARS TOTAL TAUGHT- OTHER TCHR0154 T067703 PJ000167 LAST 12 MOS, PROF DEV-LITERATURE TCHR0155 T068501 ID100370 ARE STUDENTS ASSIGNED TO THIS CLASS BY ABILITY TCHR0156 T069801 PJ000245 HOW OFTEN STUDENTS CHOOSE WRITING TOPIC TCHR0157 T069802 PJ000246 HOW OFTEN STUDENTS PLAN THEIR WRITING TCHR0158 T069803 PJ000247 HOW OFTEN STUDENTS DEFINE PURPOSES AND AUDIENCE 
Table F-3 (continued)

Summary Table of the 1998 Writing Conditioning Variable Specifications

Cond’ng.NAEP ID ID TDDC ID DESCRIPTION 642TCHR0159 T069804 PJ000248 HOW OFTEN STUDENTS MAKE FORMAL OUTLINE TCHR0160 T069805 PJ000249 HOW OFTEN STUDENTS WRITE MORE THAN ONE DRAFT TCHR0161 T069806 PJ000250 HOW OFTEN STUDENTS USE RESOURCES OTHER THAN TEXT TCHR0162 T069807 PJ000251 HOW OFTEN STUDENTS DISCUSS WRITING WHILE WRITING TCHR0163 T069808 PJ000252 HOW OFTEN STUDENTS DISCUSS OTHERS’ WRITING TCHR0164 T069809 PJ000253 HOW OFTEN STUDENTS CHECK PROPER SPELLING, GRAMMAR TCHR0165 T069810 PJ000254 HOW OFTEN STUDENTS DISCUSS WRITING WITH FAMILY TCHR0166 T069811 PJ000255 HOW OFTEN STUDENTS CONTRIBUTE TO COLLECTION TCHR0167 T069812 PJ000256 HOW OFTEN STUDENTS WORK ON AN ASSIGNED TOPIC TCHR0168 T069813 PJ000257 HOW OFTEN STUDENTS FOLLOW ASSIGNED FORMAT TCHR0169 TCSIZE WHAT IS THE NUMBER OF STUDENTS IN EACH CLASS? (8TH GRADE) 
Table F-4

Summary Table of the 1998 Civics Conditioning Variable Specifications

Cond’ng.NAEP ID ID TDDC ID DESCRIPTION BACK0001 BKSER GRAND MEAN BACK0002 DSEX DERIVED SEX BACK0003 DRACE DERIVED RACE/ETHNICITY BACK0004 B003101 TB003101 IF HISPANIC, WHAT IS YOUR HISPANIC BACKGROUND? BACK0005 TOL7 TOL 7 - TYPE OF LOCATION BACK0006 TOL5 TYPE OF LOCALE (5 CATEGORIES) BACK0007 PARED2 PARENTS’ HIGHEST LEVEL OF EDUCATION, GRADE 4 BACK0008 REGION REGION OF THE COUNTRY BACK0009 SCHTYPE SCHOOL TYPE BACK0010 RACE RACE BACK0011 IEP INDIVIDUALIZED EDUCATION PLAN BACK0012 LEP LIMITED ENGLISH PROFICIENCY BACK0013 TITLE1 TITLE 1: (BOOK COVER) BACK0014 SLUNCH DO YOU RECEIVE A FREE OR REDUCED-PRICE LUNCH? BACK0015 B013901 ID100323 HOW MUCH TELEVISION/VIDEO GAMES DO YOU USUALLY WATCH EACH DAY? (LINEAR) BACK0016 B013901 ID100323 HOW MUCH TELEVISION/VIDEO GAMES DO YOU USUALLY WATCH EACH DAY? (QUADRATIC) BACK0017 B006601 TB006601 HOMEWORK ASSIGNED?: BASED ON TIME SPENT ON HOMEWORK EACH DAY. BACK0018 B006601 TB006601 HOW MUCH TIME DO YOU USUALLY SPEND ON HOMEWORK EACH DAY? (LINEAR) BACK0019 B006601 TB006601 HOW MUCH TIME DO YOU USUALLY SPEND ON HOMEWORK EACH DAY (QUADRATIC) BACK0020 HOMEEN3 NUMBER OF ITEMS IN THE HOME (NEWSPAPER, > 25 BOOKS, ENCYCLOPEDIA, MAGAZINES) (DERIVEDBACK0021 B001101 TB001101 ABOUT HOW MANY PAGES A DAY DO YOU HAVE TO READ FOR SCHOOL AND HOMEWORK? BACK0022 B001101 TB001101 ABOUT HOW MANY PAGES A DAY DO YOU HAVE TO READ FOR SCHOOL AND HOMEWORK? BACK0023 ACCOM STUDENTS ACCOMMODATION STATUS BACK0024 NYRCIV NUMBER OF YEARS TAKING CIVICS COURSES IN HIGH SCHOOL BACK0025 NYRCIV2 CIVICS COURSES TAKING IN 11TH AND 12TH GRADES BACK0026 INTERACT INTERACTION: GENDER BY RACE/ETHNICITY BACK0027 INTERACT INTERACTION: GENDER BY TYPE OF LOCALE (7 CATEGORIES) BACK0028 INTERACT INTERACTION: GENDER BY PARENTS’ EDUCATION ALL GRADES BACK0029 INTERACT INTERACTION: GENDER BY SCHOOL TYPE BACK0030 INTERACT INTERACTION: RACE/ETHNICITY BY TYPE OF LOCALE (7 CATEGORIES) BACK0031 INTERACT INTERACTION: RACE/ETHNICITY BY PARENTS’ EDUCATION ALL GRADES BACK0032 INTERACT INTERACTION: RACE/ETHNICITY BY SCHOOL TYPE BACK0033 INTERACT INTERACTION: PARENT’S EDUCATION ALL GRADES BY TYPE OF LOCALE (7 CATEGORIES) BACK0034 INTERACT INTERACTION: TYPE OF LOCALE (7 CATEGORIES) BY SCHOOL TYPE BACK0035 INTERACT INTERACTION: PARENTS’ EDUCATION ALL GRADES BY SCHOOL TYPE BACK0036 INTERACT INTERACTION: ACCOMMODATED BY GENDER BACK0037 INTERACT INTERACTION: ACCOMMODATED BY RACE/ETHNICITY BACK0038 INTERACT INTERACTION: ACCOMMODATED BY TYPE OF LOCALE (7 CATEGORIES) BACK0039 INTERACT INTERACTION: ACCOMMODATED BY PARENTS’ EDUCATION ALL GRADES BACK0040 INTERACT INTERACTION: ACCOMMODATED BY SCHOOL TYPE BACK0041 INTERACT INTERACTION: ACCOMMODATED BY IEP BACK0042 INTERACT INTERACTION: ACCOMMODATED BY LEP BACK0043 INTERACT INTERACTION: GENDER BY YEARS TAKING CIVICS COURSES BACK0044 INTERACT INTERACTION: RACE/ETHNICITY BY YEARS TAKING CIVICS COURSES BACK0045 INTERACT INTERACTION: YEARS TAKING CIVICS COURSES BY TYPE OF LOCALE (7 CATEGORIES) BACK0046 INTERACT INTERACTION: PARENT’S EDUCATION BY YEARS TAKING CIVICS COURSES BACK0047 INTERACT INTERACTION: YEARS TAKING CIVICS COURSES BY SCHOOL TYPE BACK0048 INTERACT INTERACTION: ACCOMMODATED BY YEARS TAKING CIVICS COURSES BACK0049 INTERACT INTERACTION: GENDER BY CIVICS COURSES TAKING IN 11TH AND 12TH GRADES BACK0050 INTERACT INTERACTION: RACE/ETHNICITY BY CIVICS COURSES TAKING IN 11TH AND 12TH GRADES BACK0051 INTERACT INTERACTION: TYPE OF LOCALE (7 CATEGORIES) BY CIVICS COURSES TAKING IN 11TH AND 12TH 
Table F-4 (continued)

Summary Table of the 1998 Civics Conditioning Variable Specifications

Cond’ng.NAEP ID ID TDDC ID DESCRIPTION 644BACK0052 INTERACT INTERACTION: PARENTS’ EDUCATION ALL GRADES BY CIVICS COURSES TAKING IN 11TH AND 12TH BACK0053 INTERACT INTERACTION: SCHOOL TYPE BY CIVICS COURSES TAKING IN 11TH AND 12TH GRADES BACK0054 INTERACT INTERACTION: ACCOMMODATED BY CIVICS COURSES TAKING IN 11TH AND 12TH GRADES BACK0055 B003001 TB003001 WHICH RACE/ETHNICITY BEST DESCRIBES YOU BACK0056 B013001 ID100333 HOW LONG LIVED IN UNITED STATES BACK0057 B013101 ID100322 HOW OFTEN OTHER THAN ENGLISH SPOKEN AT HOME BACK0058 B013201 ID100314 MOTHER GRADUATED HIGH SCHOOL BACK0059 B013301 ID100315 MOTHER HAD SOME EDUCATION AFTER HIGH SCHOOL BACK0060 B013401 ID100316 MOTHER GRADUATED COLLEGE BACK0061 B013501 ID100317 FATHER GRADUATED HIGH SCHOOL BACK0062 B013601 ID100318 FATHER HAD SOME EDUCATION AFTER HIGH SCHOOL BACK0063 B013701 ID100319 FATHER GRADUATED COLLEGE BACK0064 B000901 TB000901 DOES YOUR FAMILY GET A NEWSPAPER REGULARLY BACK0065 B000903 TB000903 IS THERE AN ENCYCLOPEDIA IN YOUR HOME BACK0066 B013801 ID100334 HOW MANY BOOKS ARE IN YOUR HOME BACK0067 B000905 TB000905 DOES YOUR FAMILY GET MAGAZINES REGULARLY BACK0068 B006601 TB006601 TIME SPENT ON HOMEWORK EACH DAY BACK0069 B014001 ID100324 DAYS ABSENT FROM SCHOOL LAST MONTH BACK0070 B007301 HE000712 TIMES CHANGED SCHOOLS IN PAST TWO YEARS BACK0071 B007401 HE000717 HOW OFTEN DISCUSS STUDIES AT HOME BACK0072 B014101 ID100325 HOW OFTEN USE COMPUTER AT HOME FOR SCHOOLWORK SUBJ0001 P804001 ID100338 HOW HARD TRIED ON THIS SS TEST THAN ON OTHERS SUBJ0002 P804101 ID100339 HOW IMPORTANT TO DO WELL ON THIS SS TEST SUBJ0003 P804201 ID100340 HOW OFTEN WRITE LONG ANSWERS ON SS TESTS SUBJ0004 P804301 ID100342 MY FRIENDS MAKE FUN OF PEOPLE WHO TRY TO DO WELL SUBJ0005 P804302 ID100343 I HAVE FRIENDS TO TALK TO IF NEED HELP W/SCHOOL SUBJ0006 P803501 ID100191 HOW OFTEN STUDY SOCIAL STUDIES IN SCHOOL SUBJ0007 P803601 ID100193 THIS YEAR-STUDY HOW OUR GOVERNMENT WORKS SUBJ0008 P803602 ID100194 THIS YEAR-STUDY RULES/LAWS OF GOVERNMENT SUBJ0009 P803603 ID100196 THIS YEAR-STUDY ELECTIONS AND VOTING SUBJ0010 P803604 ID100197 THIS YEAR-STUDY THE PRESIDENT/LEADERS OF COUNTRY SUBJ0011 P803605 ID100198 THIS YEAR-STUDY YOUR COMMUNITY SUBJ0012 P803606 ID100199 THIS YEAR-STUDY RIGHTS/RESPONSIBILITIES-CITIZENS SUBJ0013 P803607 ID100200 THIS YEAR-STUDY HOW PEOPLE SOLVE DISAGREEMENTS SUBJ0014 P803701 ID100202 IN SOCIAL STUDIES-READ FROM TEXTBOOK SUBJ0015 P803702 ID100203 IN SOCIAL STUDIES-MEMORIZE READING MATERIAL SUBJ0016 P803703 ID100204 IN SOCIAL STUDIES-READ EXTRA MATERIAL SUBJ0017 P803704 ID100205 IN SOCIAL STUDIES-FILL OUT WORKSHEETS SUBJ0018 P803705 ID100206 IN SOCIAL STUDIES-WRITE REPORTS SUBJ0019 P803706 ID100207 IN SOCIAL STUDIES-DISCUSS CURRENT EVENTS SUBJ0020 P803707 ID100208 IN SOCIAL STUDIES-WATCH TV, VIDEOS, FILMSTRIPS SUBJ0021 P803708 ID100209 IN SOCIAL STUDIES-DISCUSS TV, VIDEOS, FILMSTRIP SUBJ0022 P803709 ID100210 IN SOCIAL STUDIES-TAKE PART IN DEBATES/PANEL DISC SUBJ0023 P803710 ID100211 IN SOCIAL STUDIES-ROLE PLAYING, MOCK TRIALS SUBJ0024 P803711 ID100212 IN SOCIAL STUDIES-WRITE LETTER FOR COMMUNITY SUBJ0025 P803712 ID100213 IN SOCIAL STUDIES-HAVE VISITORS FROM COMMUNITY SUBJ0026 P803801 ID100214 HOW OFTEN DO YOU HAVE SOCIAL STUDIES HOMEWORK SUBJ0027 P803901 ID100215 DO YOU HAVE A CLASSROOM GOVERNMENT SCHL0001 C042501 ID100378 FOURTH GRADERS ASSIGNED TO CLASS BY ABILITY SCHL0002 C042601 ID100041 HOW OFTEN STUDENTS RECEIVE READING INSTRUCTION SCHL0003 C042602 ID100042 HOW OFTEN STUDENTS RECEIVE WRITING INSTRUCTION SCHL0004 C042603 ID100043 HOW OFTEN STUDENTS RECEIVE SOC STUDIES INSTRUCT SCHL0005 C042604 ID100044 HOW OFTEN STUDENTS RECEIVE COMPUTER USE INSTRUCT SCHL0006 C042701 ID100379 DOES SCHOOL USE BLOCK SCHEDULING 
Table F-4 (continued)

Summary Table of the 1998 Civics Conditioning Variable Specifications

Cond’ng.NAEP ID ID TDDC ID DESCRIPTION SCHL0007 C042801 ID100380 ARE COMPUTERS AVAILABLE IN ALL CLASSROOMS SCHL0008 C042802 HE000864 ARE COMPUTERS AVAILABLE IN COMPUTER LAB SCHL0009 C042803 HE000866 ARE COMPUTERS AVAILABLE TO CLASSROOM WHEN NEEDED SCHL0010 C042901 ID100381 HOW MANY COMPUTERS AVAILABLE TO STUDENTS SCHL0011 C036601 LC000502 PRIMARY WAY LIBRARY IS STAFFED SCHL0012 C043001 ID100069 PARENTS PARTICIPATE-PARENT-TEACHER ORG SCHL0013 C043002 ID100070 PARENTS PARTICIPATE-OPEN HOUSE SCHL0014 C043003 ID100071 PARTICIPATE-PARENT-TEACHER CONFERENCE SCHL0015 C043004 ID100072 PARENTS PARTICIPATE-SCHOOL CURRICULUM DECISIONS SCHL0016 C043005 ID100073 PARENTS PARTICIPATE-VOLUNTEER PROGRAMS SCHL0017 C043006 ID100074 PARENTS PARTICIPATE-PARENTING-SKILLS PROGRAM SCHL0018 C043007 ID100076 PARENTS PARTICIPATE-SCHOOL ADVISORY COMMITTEES SCHL0019 C043008 ID100077 PARENTS PARTICIPATE-CLASSROOM ASSISTANTS SCHL0020 C032402 HE000888 IS STUDENT ABSENTEEISM A PROBLEM IN YOUR SCHOOL SCHL0021 C032401 HE000887 IS STUDENT TARDINESS A PROBLEM IN YOUR SCHOOL SCHL0022 C032404 HE000890 ARE PHYSICAL CONFLICTS A PROBLEM IN YOUR SCHOOL SCHL0023 C032407 HE000893 ARE RACIAL/CULT. CONFLICTS A PROBLEM IN SCHOOL SCHL0024 C032408 HE000894 IS STUDENT HEALTH A PROBLEM IN YOUR SCHOOL SCHL0025 C032409 HE002121 IS LACK OF PARENT INVLVMNT A PROBLEM IN SCHOOL SCHL0026 C032410 HE002122 IS STUDENT ALCOHOL USE A PROBLEM IN YOUR SCHOOL SCHL0027 C032411 HE002123 IS STUDENT TOBACCO USE A PROBLEM IN YOUR SCHOOL SCHL0028 C032412 HE002124 IS STUDENT DRUG USE A PROBLEM IN YOUR SCHOOL SCHL0029 C032413 HE002125 ARE GANG ACTIVITIES A PROBLEM IN YOUR SCHOOL SCHL0030 C032414 HE002126 IS STUDENT MISBEHAVIOR A PROBLEM IN YOUR SCHOOL SCHL0031 C043101 ID100079 IS STUDENT CHEATING A PROBLEM IN YOUR SCHOOL SCHL0032 C043102 ID100077 IS TEACHER ABSENTEEISM A PROBLEM IN YOUR SCHOOL SCHL0033 C043103 ID100078 ARE PHYSICAL CONFLICTS BETWEEN STUDENTS/TEACHERS SCHL0034 C043104 ID100080 IS VANDALISM A PROBLEM IN YOUR SCHOOL SCHL0035 C032502 HE000897 TEACHER MORALE SCHL0036 C032503 HE000898 STUDENT ATTITUDES TOWARD ACADEMIC ACHIEVEMENT SCHL0037 C032505 HE000900 PARENT SUPPORT FOR STUDENT ACHIEVEMENT SCHL0038 C032506 HE000901 REGARD FOR SCHOOL PROPERTY SCHL0039 C043201 ID100081 TEACHERS’ EXPECTATIONS FOR STUDENT ACHIEVEMENT SCHL0040 C043301 ID100082 PERCENT STUDENT BODY ABSENT AVERAGE DAY SCHL0041 C043401 ID100389 PERCENT TEACHING STAFF ABSENT AVERAGE DAY SCHL0042 C043501 ID100390 ENROLLMENT LAST YEAR COMPARED TO END OF SCHOOL YR SCHL0043 C043601 HE002112 PERCENT STUDENTS HELD BACK AND REPEATING GRADE SCHL0044 C043701 ID100391 PERCENT TEACHING STAFF LEFT BEFORE END OF YEAR SCHL0045 C038301 HE002094 IS SCHOOL IN NATIONAL SCHOOL LUNCH PROGRAM SCHL0046 C043801 ID100392 PERCENT ELIGIBLE NATIONAL SCHOOL LUNCH PROGRAM SCHL0047 C043901 ID100393 DOES SCHOOL RECEIVE CHAPTER 1/TITLE I FUNDING SCHL0048 C044001 ID100395 PERCENT STUDENTS RECEIVE CHAPTER1/TITLE I FUNDING SCHL0049 C044002 ID100396 PERCENT STUDENTS RECEIVE REMEDIAL READING INSTRUCT SCHL0050 C044003 ID100397 PERCENT STUDENTS RECEIVE REMEDIAL WRITING INSTRUCT SCHL0051 C044004 ID100398 PERCENT STUDENTS IN GIFTED AND TALENTED PROGRAM BACK0073 B014201 ID100248 HOW MUCH EDUCATION DO YOU EXPECT TO RECEIVE SUBJ0028 P804401 ID100217 THIS YEAR-STUDIED U. S. CONSTITUTION SUBJ0029 P804402 ID100218 THIS YEAR-STUDIED CONGRESS SUBJ0030 P804403 ID100219 THIS YEAR-STUDIED PRESIDENT AND CABINET SUBJ0031 P804404 ID100220 THIS YEAR-STUDIED HOW LAWS ARE MADE SUBJ0032 P804405 ID100221 THIS YEAR-STUDIED THE COURT SYSTEM SUBJ0033 P804406 ID100222 THIS YEAR-STUDIED POLIT PARTIES, ELECTIONS, VOTE SUBJ0034 P804407 ID100223 THIS YEAR-STUDIED STATE & LOCAL GOVERNMENT SUBJ0035 P804408 ID100224 THIS YEAR-STUDIED OTHER COUNTRIES’ GOVERNMENT 
Table F-4 (continued)

Summary Table of the 1998 Civics Conditioning Variable Specifications

Cond’ng.NAEP ID ID TDDC ID DESCRIPTION 646SUBJ0036 P804409 ID100225 THIS YEAR-STUDIED INTERNATIONAL ORGANIZATIONS SUBJ0037 P804501 ID100226 HOMEWORK HOURS/WEEK-SOCIAL STUDIES CLASS SCHL0052 C044401 ID100400 8TH GRADE ASSIGNED TO ENGLISH CLASS BY ABILITY SCHL0053 C044402 ID100403 8TH GRADE ASSIGNED-HISTORY/SS BY ABILITY SCHL0054 C043105 ID100086 IS STUDENT DROPOUT A PROBLEM IN YOUR SCHOOL SCHL0055 C043106 ID100087 IS TEEN PREGNANCY A PROBLEM IN YOUR SCHOOL BACK0074 B005501 TB005501 MAIN ACTIVITY YEAR FOLLOWING HIGH SCHOOL BACK0075 B014301 ID100326 VOLUNTEER WORK IN YOUR COMMUNITY THIS YEAR BACK0076 B014401 ID100332 HOW MANY HOURS/WEEK WORK JOB FOR PAY SUBJ0038 P802545 ID100344 HOW HARD TRIED ON THIS CIVICS TEST THAN ON OTHERS SUBJ0039 P802546 ID100345 HOW IMPORTANT TO DO WELL ON THIS CIVICS TEST SUBJ0040 P802547 ID100346 HOW OFTEN WRITE LONG ANSWERS ON CIVICS TESTS SUBJ0041 P804601 ID100228 GRADE 9 - STUDIED CIVICS OR GOVERNMENT SUBJ0042 P804602 ID100229 GRADE 10 - STUDIED CIVICS OR GOVERNMENT SUBJ0043 P804603 ID100230 GRADE 11 - STUDIED CIVICS OR GOVERNMENT SUBJ0044 P804604 ID100231 GRADE 12 - STUDIED CIVICS OR GOVERNMENT SUBJ0045 P804701 ID100247 HOMEWORK HOURS/WEEK CIVICS-GOVERNMENT CLASS SUBJ0046 P804801 ID100233 DO YOU HAVE A TEXTBOOK TO STUDY CIVICS/GOVERNMENT SUBJ0047 P804901 ID100232 ENROLLED IN OR TOOK AP U.S. GOV’T & POLITICS SCHL0056 C044301 ID100404 12TH GRADE ASSIGNED TO ENGLISH CLASS BY ABILITY SCHL0057 C044302 ID100405 12TH GR ASSIGNED- HISTORY/CIVICS/SS CLASS ABILITY SCHL0058 C044101 ID100408 PERCENT LAST YEAR’S TWELFTH-GRADE CLASS GRADUATED SCHL0059 C044201 ID100410 PERCENT GRADUATING CLASS-ATTEND TWO-YEAR COLLEGE SCHL0060 C044202 ID100411 PERCENT GRADUATING CLASS-ATTEND FOUR-YEAR COLLEGE TCHR0001 T067001 PJ000121 DO YOU TEACH READING TCHR0002 T067002 PJ000122 DO YOU TEACH WRITING TCHR0003 T067003 PJ000123 DO YOU TEACH LANGUAGE ARTS TCHR0004 T067004 PJ000124 DO YOU TEACH SOCIAL STUDIES TCHR0005 T067101 PJ000126 YEARS TOTAL TAUGHT ELEMENTARY LEVEL TCHR0006 T067201 PJ000128 YEARS TOTAL TAUGHT READING TCHR0007 T067202 PJ000129 YEARS TOTAL TAUGHT WRITING TCHR0008 T067203 PJ000130 YEARS TOTAL TAUGHT LANGUAGE ARTS TCHR0009 T067204 PJ000131 YEARS TOTAL TAUGHT HISTORY TCHR0010 T067205 PJ000132 YEARS TOTAL TAUGHT SOCIAL STUDIES TCHR0011 T067206 PJ000133 YEARS TOTAL TAUGHT CIVICS TCHR0012 T067301 PJ000134 MAIN ASSIGNMENT FIELD TCHR0013 T056201 HE002551 TEACHING CERTIF IN THIS STATE IN MAIN FIELD TCHR0014 T056301 HE001012 HIGHEST ACADEMIC DEGREE YOU HOLD TCHR0015 T067501 PJ000138 UNDERGRAD MAJOR/MINOR-ELEMENTARY EDUCATION TCHR0016 T067502 PJ000139 UNDERGRAD MAJOR/MINOR-SECONDARY EDUCATION TCHR0017 T067503 PJ000140 UNDERGRAD MAJOR/MINOR-SPECIAL EDUCATION TCHR0018 T067504 PJ000141 UNDERGRAD MAJOR/MINOR-BILINGUAL EDUCATION/ESL TCHR0019 T067505 PJ000142 UNDERGRAD MAJOR/MINOR-ADMINISTRATION & SUPERVISION TCHR0020 T067506 PJ000143 UNDERGRAD MAJOR/MINOR-CURRICULUM & SUPERVISION TCHR0021 T067507 PJ000144 UNDERGRAD MAJOR/MINOR-COUNSELING TCHR0022 T067508 PJ000145 UNDERGRAD MAJOR/MINOR-ENGLISH TCHR0023 T067509 PJ000146 UNDERGRAD MAJOR/MINOR-READING AND/OR LANGUAGE ARTS TCHR0024 T067510 PJ000147 UNDERGRAD MAJOR/MINOR-HISTORY TCHR0025 T067511 PJ000148 UNDERGRAD MAJOR/MINOR-POLITICAL SCIENCE TCHR0026 T067512 PJ000149 UNDERGRAD MAJOR/MINOR-OTHER TCHR0027 T067601 PJ000151 GRAD MAJOR/MINOR-ELEMENTARY EDUCATION TCHR0028 T067602 PJ000152 GRAD MAJOR/MINOR-SECONDARY EDUCATION TCHR0029 T067603 PJ000153 GRAD MAJOR/MINOR-SPECIAL EDUCATION TCHR0030 T067604 PJ000154 GRAD MAJOR/MINOR-BILINGUAL EDUCATION/ESL 
Table F-4 (continued)

Summary Table of the 1998 Civics Conditioning Variable Specifications

Cond’ng.NAEP ID ID TDDC ID DESCRIPTION TCHR0031 T067605 PJ000155 GRAD MAJOR/MINOR-ADMINSTRATION & SUPERVISION TCHR0032 T067606 PJ000156 GRAD MAJOR/MINOR-CURRICULUM AND INSTRUCTION TCHR0033 T067607 PJ000157 GRAD MAJOR/MINOR-COUNSELING TCHR0034 T067608 PJ000158 GRAD MAJOR/MINOR-ENGLISH TCHR0035 T067609 PJ000159 GRAD MAJOR/MINOR-READING AND/OR LANGUAGE ARTS TCHR0036 T067610 PJ000160 GRAD MAJOR/MINOR-HISTORY TCHR0037 T067611 PJ000161 GRAD MAJOR/MINOR-POLITICAL SCIENCE TCHR0038 T067612 PJ000162 GRAD MAJOR/MINOR-OTHER TCHR0039 T067701 ID100358 LAST 12 MOS, PROF DEV-READING AND WRITING TCHR0040 T067702 ID100147 LAST 12 MOS, PROF DEV-SOCIAL STUDIES TCHR0041 T067801 PJ000169 PREPARED IN THE USE OF TELECOMMUNICATIONS TCHR0042 T067802 ID100360 PREPARED IN THE USE OF COMPUTERS TCHR0043 T067803 PJ000171 PREPARED IN COOPERATIVE GROUP INSTRUCTION TCHR0044 T067804 PJ000176 PREPARED IN TEACHING STUDENTS-DIFFERENT CULTURES TCHR0045 T067805 PJ000177 PREPARED IN TEACHING STUDENTS WHO ARE LEP TCHR0046 T067806 PJ000178 PREPARED IN TEACHING STUDENTS WITH DISABILITIES TCHR0047 T067807 PJ000179 PREPARED IN CLASSROOM MANAGEMENT AND ORGANIZATION TCHR0048 T041201 HE001022 AVAILABILITY OF RESOURCES TCHR0049 T070401 PJ000286 PREPARED IN SOCIAL STUDIES INSTRUCTION TCHR0050 T070402 PJ000287 PREPARED IN PUBLIC SERVICE OPPORTUNITIES TCHR0051 T070403 PJ000288 PREPARED IN INSTRUCTIONAL MATERIALS IN SOC STUDIES TCHR0052 T070404 PJ000289 PREPARED IN USE OF COMMUNITY RESOURCES IN INSTRUC TCHR0053 T070405 PJ000290 PREPARED IN CLASSROOM CLIMATE AND GOVERNANCE TCHR0054 T070406 PJ000291 PREPARED IN USING NATL STANDARDS FOR CIVICS TCHR0055 T070407 PJ000292 PREPARED IN USING SOFTWARE FOR SOCIAL STUDIES TCHR0056 T070501 ID100367 WHAT IS YOUR AVERAGE SOCIAL STUDIES CLASS SIZE TCHR0057 T070601 PJ000294 ARE STUDENTS ASSIGNED TO THIS CLASS BY ABILITY TCHR0058 T070701 PJ000295 WHAT IS THE ABILITY LEVEL OF THE STUDENTS TCHR0059 T070801 PJ000296 CLASS TIME PER DAY-SOCIAL STUDIES INSTRUCTION TCHR0060 T070901 PJ000298 HOW OFTEN USE SOCIAL STUDIES TEXTBOOK TCHR0061 T070902 PJ000299 HOW OFTEN USE BOOKS, NEWSPAPER, MAGAZINES TCHR0062 T070903 PJ000300 HOW OFTEN USE PRIMARY DOCUMENTS TCHR0063 T070904 PJ000301 HOW OFTEN USE QUANTITATIVE DATA-CHARTS, GRAPHS TCHR0064 T070905 PJ000302 HOW OFTEN USE COMPUTER SOFTWARE TCHR0065 T070906 PJ000303 HOW OFTEN USE FILMS, VIDEOS, FILMSTRIPS TCHR0066 T070907 PJ000304 HOW OFTEN USE MATERIALS FROM OTHER SUBJECT AREAS TCHR0067 T071001 PJ000305 AVAILABILITY OF COMPUTERS IN SOCIAL STUDIES CLASS TCHR0068 T071101 PJ000307 HOW OFTEN STUDENTS COMPLETE A WORKSHEET TCHR0069 T071102 PJ000309 HOW OFTEN STUDENTS READ EXTRA MATERIAL TCHR0070 T071103 PJ000310 HOW OFTEN GIVE LECTURE ABOUT SOCIAL STUDIES TCHR0071 T071104 PJ000311 HOW OFTEN STUDENTS DO GROUP ACTIVITY OR PROJECT TCHR0072 T071105 PJ000312 HOW OFTEN STUDENTS WRITE THREE OR MORE PAGE REPORT TCHR0073 T071106 PJ000313 HOW OFTEN STUDENTS WATCH TELEVISION, VIDEOS, FILMS TCHR0074 T071107 PJ000314 HOW OFTEN STUDENTS PARTICIPATE-DEBATES TCHR0075 T071108 PJ000315 HOW OFTEN STUDENTS PARTICIPATE-MOCK TRIALS TCHR0076 T071109 PJ000316 HOW OFTEN STUDENTS WRITE LETTERS TCHR0077 T071110 PJ000317 HOW OFTEN VISITORS MEET/DISCUSS IMPORTANT EVENTS TCHR0078 T071111 PJ000318 HOW OFTEN STUDENTS VISIT GOVERNMENT/COMMUNITY TCHR0079 T071112 PJ000319 HOW OFTEN STUDENTS PARTICIPATE-VOLUNTEER PROJ/SERV TCHR0080 T071113 PJ000320 HOW OFTEN STUDENTS ACCESS INTERNET-CLASSROOM TCHR0081 T071114 PJ000321 HOW OFTEN STUDENTS DISCUSS CURRENT EVENTS TCHR0082 T071115 PJ000322 HOW OFTEN STUDENTS USE STUDENT GOVERNMENT TCHR0083 T071116 PJ000323 HOW OFTEN GIVE STUDENTS SOCIAL STUDIES HOMEWORK TCHR0084 T071201 PJ000325 HOW OFTEN USE MULTIPLE-CHOICE,TRUE/FALSE,MATCHING 
Table F-4 (continued)

Summary Table of the 1998 Civics Conditioning Variable Specifications

Cond’ng.NAEP ID ID TDDC ID DESCRIPTION 648TCHR0085 T071202 PJ000326 HOW OFTEN USE FILL-IN-THE BLANK QUESTIONS TCHR0086 T071203 PJ000327 HOW OFTEN USE PARAGRAPH WRITTEN RESPONSE TCHR0087 T071204 PJ000328 HOW OFTEN USE INDIVIDUAL/GROUP PROJECTS TCHR0088 T071205 ID100148 HOW OFTEN USE ESSAYS, PAPERS ASSIGNED TOPICS TCHR0089 T071301 PJ000305 AVAILABILITY OF COMPUTERS IN SOCIAL STUDIES CLASS TCHR0090 T071401 ID100150 DO YOU TEACH HISTORY TCHR0091 T071402 ID100151 DO YOU TEACH SOCIAL STUDIES TCHR0092 T071403 ID100152 DO YOU TEACH GOVERNMENT/CIVICS TCHR0093 T071404 ID100153 DO YOU TEACH-OTHER TCHR0094 T040301 HE001007 YEARS TOTAL TAUGHT ELEMENTARY OR SECONDARY TCHR0095 T071501 ID100362 YEARS TOTAL TAUGHT HISTORY TCHR0096 T071502 ID100363 YEARS TOTAL TAUGHT SOCIAL STUDIES TCHR0097 T071503 ID100364 YEARS TOTAL TAUGHT GOVERNMENT/CIVICS TCHR0098 T071504 ID100365 YEARS TOTAL TAUGHT-OTHER TCHR0099 TCSIZE WHAT IS THE NUMBER OF STUDENTS IN EACH CLASS? (8TH GRADE) 
Table F-5

 1998 Reading Conditioning Variable Specifications

649

 CONDITIONING VARIABLE ID: BACK0001

 DESCRIPTION: GRAND MEAN

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: OVERALL

 NAEP ID: BKSER TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: OTHER NUMBER OF INDEPENDENT CONTRASTS: 1

 001 OVERALL (@ ) 1 GRAND MEAN

 CONDITIONING VARIABLE ID: BACK0002

 DESCRIPTION: DERIVED SEX

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: GENDER

 NAEP ID: DSEX TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 MALE (1,M ) 0 MALE

 002 FEMALE (2 ) 1 FEMALE

 CONDITIONING VARIABLE ID: BACK0003

 DESCRIPTION: DERIVED RACE/ETHNICITY

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: RACE/ETH

 NAEP ID: DRACE TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 WHI/AI/O (1,5,6,M ) 000 RACE/ETHNICITY: WHITE, AMERICAN

INDIAN/ALASKAN NATIVE, OTHER,

MISSING, UNCLASSIFIED

 002 BLACK (2 ) 100 RACE/ETHNICITY: BLACK

 003 HISPANIC (3 ) 010 RACE/ETHNICITY: HISPANIC

 004 ASIAN (4 ) 001 RACE/ETHNICITY: ASIAN / PACIFIC ISLANDER

 CONDITIONING VARIABLE ID: BACK0004

 DESCRIPTION: IF HISPANIC, WHAT IS YOUR HISPANIC BACKGROUND?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: HISPANIC

 NAEP ID: B003101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 NOT HISP (1 ) 0000 HISPANIC: NOT HISPANIC

 002 MEXICAN (2 ) 1000 HISPANIC: MEXICAN, MEXICAN AMERICAN, CHICANO

 003 PUER RIC (3 ) 0100 HISPANIC: PUERTO RICAN

 004 CUBN,OTH (4,5 ) 0010 HISPANIC: CUBAN, OTHER

 005 HISP-? (M ) 0001 HISPANIC: MISSING

 CONDITIONING VARIABLE ID: BACK0005

 DESCRIPTION: TOL 7 - TYPE OF LOCATION

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: TOL7

 NAEP ID: TOL7 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 BIG CTY7 (1 ) 000000 TOL7: LARGE CITY

 002 MID CTY7 (2,M ) 100000 TOL7: MID-SIZE CITY

 003 FR/LCTY7 (3 ) 010000 TOL7: URBAN FRINGE OF LARGE CITY

 004 FR/MCTY7 (4 ) 001000 TOL7: URBAN FRINGE OF MID-SIZE CITY

 005 LAR TWN7 (5 ) 000100 TOL7: LARGE TOWN

 006 SML TWN7 (6 ) 000010 TOL7: SMALL TOWN

 007 OTHER (7 ) 000001 TOL7: OTHER

 CONDITIONING VARIABLE ID: BACK0006

 DESCRIPTION: TYPE OF LOCALE (5 CATEGORIES)

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: TOL5

 NAEP ID: TOL5 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 BIG CTY5 (1 ) 0000 TOL5: LARGE CITY

 002 MID CTY5 (2,M ) 1000 TOL5: MID-SIZE CITY

 003 FR/BTWN5 (3 ) 0100 TOL5: URBAN FRINGE AND LARGE TOWN

 004 SML TWN5 (4 ) 0010 TOL5: SMALL TOWN

 005 RURAL5 (5 ) 0001 TOL5: RURAL (MSA AND NON-MSA)

 CONDITIONING VARIABLE ID: BACK0007

 DESCRIPTION: PARENTS’ HIGHEST LEVEL OF EDUCATION, GRADES 8 AND 12

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL: PARED

 NAEP ID: PARED TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 < HS (1 ) 0000 PARED: LESS THAN HIGH SCHOOL

 002 HS GRAD (2 ) 1000 PARED: HIGH SCHOOL GRADUATE

 003 POST HS (3 ) 0100 PARED: POST HIGH SCHOOL

 004 COL GRAD (4 ) 0010 PARED: COLLEGE GRADUATE

 005 PARED-? (5,M ) 0001 PARED: MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0008

 DESCRIPTION: PARENTS’ HIGHEST LEVEL OF EDUCATION, GRADE 4

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL: PARED2

 NAEP ID: PARED2 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 < HS (1 ) 0000 PARED: LESS THAN HIGH SCHOOL

 002 HS GRAD (2 ) 1000 PARED: HIGH SCHOOL GRADUATE

 003 POST HS (3 ) 0100 PARED: POST HIGH SCHOOL

 004 COL GRAD (4 ) 0010 PARED: COLLEGE GRADUATE

 005 PARED-? (5,M ) 0001 PARED: MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0009

 DESCRIPTION: REGION OF THE COUNTRY

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: REGION

 NAEP ID: REGION TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 N EAST (1,M ) 000 REGION: NORTHEAST

 002 S EAST (2 ) 100 REGION: SOUTHEAST

 003 CENTRAL (3 ) 010 REGION: CENTRAL

 004 WEST (4,5 ) 001 REGION: WEST, TERRITORIES (NONE)
Table F-5 (continued)

1998 Reading Conditioning Variable Specifications

650

 CONDITIONING VARIABLE ID: BACK0010

 DESCRIPTION: SCHOOL TYPE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: SCHTYPE

 NAEP ID: SCHTYPE TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 PUBLIC (1 ) 00 SCHOOL TYPE: PUBLIC,CHARTER SCHOOLS

 002 PRIVATE (2,4,5,M ) 10 SCHOOL TYPE: PRIVATE, BIA, RELIGIOUS,

DEPARTMENT OF DEFENSE, MISSING

 003 CATHOLIC (3 ) 01 SCHOOL TYPE: CATHOLIC

 CONDITIONING VARIABLE ID: BACK0011

 DESCRIPTION: RACE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: RACE

 NAEP ID: RACE TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 WHI/AI/O (1,5,6,M ) 000 RACE: WHITE, AMERICAN INDIAN/ALASKAN NATIVE,

OTHER, MISSING, UNCLASSIFIED

 002 BLACK (2 ) 100 RACE: BLACK

 003 HISPANIC (3 ) 010 RACE: HISPANIC

 004 ASIAN (4 ) 001 RACE: ASIAN / PACIFIC ISLANDER

 CONDITIONING VARIABLE ID: BACK0012

 DESCRIPTION: INDIVIDUALIZED EDUCATION PLAN

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: IEP

 NAEP ID: IEP TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 IEP-YES (1 ) 0 IEP: YES

 002 IEP-NO (2,M ) 1 IEP: NO

 CONDITIONING VARIABLE ID: BACK0013

 DESCRIPTION: LIMITED ENGLISH PROFICIENCY

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: LEP

 NAEP ID: LEP TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 LEP-YES (1 ) 0 LEP: YES

 002 LEP-NO (2,M ) 1 LEP: NO

 CONDITIONING VARIABLE ID: BACK0014

 DESCRIPTION: TITLE 1: (BOOK COVER)

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: TITLE 1

 NAEP ID: TITLE1 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 TITLE-Y (1 ) 0 TITLE 1: YES

 002 TITLE-N (2,M ) 1 TITLE 1: NO

 CONDITIONING VARIABLE ID: BACK0015

 DESCRIPTION: DO YOU RECEIVE A FREE OR REDUCED-PRICE LUNCH?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: LUNCH

 NAEP ID: SLUNCH TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 NOT ELIG (1 ) 00000 LUNCH PROGRAM: NOT ELIGIBLE

 002 RED PRIC (2 ) 10000 LUNCH PROGRAM: REDUCED PRICE

 003 FREE (3 ) 01000 LUNCH PROGRAM: FREE

 004 INFO N/A (4,M ) 00100 LUNCH PROGRAM: INFO NOT AVAILABLE

 005 SCH/REF (5 ) 00010 LUNCH PROGRAM: SCHOOL REFUSAL

 006 SCH/NP (6 ) 00001 LUNCH PROGRAM: SCHOOL NOT PARTIPATE

 CONDITIONING VARIABLE ID: BACK0016

 DESCRIPTION: HOW MUCH TELEVISION DO YOU USUALLY WATCH EACH DAY? (LINEAR)

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: TVWATCHL

 NAEP ID: B001801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 TVLIN-0 (1 ) 0 TV WATCHING (LINEAR) (0 TO 6+ HOURS PER DAY)

 002 TVLIN-1 (2 ) 1 TV WATCHING (LINEAR)

 003 TVLIN-2 (3 ) 2 TV WATCHING (LINEAR)

 004 TVLIN-3 (4,M ) 3 TV WATCHING (LINEAR)

 005 TVLIN-4 (5 ) 4 TV WATCHING (LINEAR)

 006 TVLIN-5 (6 ) 5 TV WATCHING (LINEAR)

 007 TVLIN-6 (7 ) 6 TV WATCHING (LINEAR)

 CONDITIONING VARIABLE ID: BACK0017

 DESCRIPTION: HOW MUCH TELEVISION DO YOU USUALLY WATCH EACH DAY? (QUADRATIC)

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: TVWATCHQ

 NAEP ID: B001801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: QUADRATIC NUMBER OF INDEPENDENT CONTRASTS: 1

 001 TV-QUAD (1-7,M=4 ) 1.0 + -2.0*X + 1.0*X**2 TV WATCHING (QUADRATIC)

 CONDITIONING VARIABLE ID: BACK0018

 DESCRIPTION: HOMEWORK ASSIGNED?: BASED ON TIME SPENT ON HOMEWORK EACH DAY.

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: HWASSIGN

 NAEP ID: B006601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 HW-MISS (M ) 00 HOMEWORK ASSIGNED?: MISSING

 002 HW-NO (1 ) 10 HOMEWORK ASSIGNED?: NO

 003 HW-YES (2-5 ) 01 HOMEWORK ASSIGNED?: YES
Table F-5 (continued)

 1998 Reading Conditioning Variable Specification

651

 CONDITIONING VARIABLE ID: BACK0019

 DESCRIPTION: HOW MUCH TIME DO YOU USUALLY SPEND ON HOMEWORK EACH DAY? (LINEAR)

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: HOMEWRKL

 NAEP ID: B006601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 HWLIN-0 (1,2,M ) 0 HOMEWORK (LINEAR): DON’T HAVE ANY, DON’T DO

ANY, MISSING

 002 HWLIN-1 (3 ) 1 HOMEWORK (LINEAR): 1/2 HOUR OR LESS

 003 HWLIN-2 (4 ) 2 HOMEWORK (LINEAR): 1 HOUR

 004 HWLIN-3 (5 ) 3 HOMEWORK (LINEAR): MORE THAN 1 HOUR

 CONDITIONING VARIABLE ID: BACK0020

 DESCRIPTION: HOW MUCH TIME DO YOU USUALLY SPEND ON HOMEWORK EACH DAY (QUADRATIC)

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: HOMEWRKQ

 NAEP ID: B006601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: SCALE NUMBER OF INDEPENDENT CONTRASTS: 1

 001 HWQUAD-0 (1,2,M ) 0 HOMEWORK (QUADRATIC): DON’T HAVE ANY, DON’T

DO ANY, MISSING

 002 HWQUAD-1 (3 ) 1 HOMEWORK (QUADRATIC): 1/2 HOUR OR LESS

 003 HWQUAD-2 (4 ) 4 HOMEWORK (QUADRATIC): 1 HOUR

 004 HWQUAD-3 (5 ) 9 HOMEWORK (QUADRATIC): MORE THAN 1 HOUR

 CONDITIONING VARIABLE ID: BACK0021

 DESCRIPTION: NUMBER OF ITEMS IN THE HOME (NEWSPAPER, > 25 BOOKS, ENCYCLOPEDIA, MAGAZINES) (DERIVED)

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: HOMEITMS

 NAEP ID: HOMEEN2 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 HITEM<=2 (1,M ) 00 ITEMS IN HOME: ZERO TO TWO ITEMS, MISSING

 002 HITEM=3 (2 ) 10 ITEMS IN HOME: THREE ITEMS

 003 HITEM=4 (3 ) 01 ITEMS IN HOME: FOUR ITEMS

 CONDITIONING VARIABLE ID: BACK0022

 DESCRIPTION: ABOUT HOW MANY PAGES A DAY DO YOU HAVE TO READ FOR SCHOOL AND HOMEWORK?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: PGSREAD1

 NAEP ID: B001101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 PGS<6,? (5,M ) 0 PAGES READ: 5 OR FEWER A DAY, MISSING

 002 PGS>5 (1,2,3,4 ) 1 PAGES READ: 6-10, 11-15, 16-20, 20 OR MORE

 CONDITIONING VARIABLE ID: BACK0023

 DESCRIPTION: ABOUT HOW MANY PAGES A DAY DO YOU HAVE TO READ FOR SCHOOL AND HOMEWORK?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: PGSREAD2

 NAEP ID: B001101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 PGS<11,? (4,5,M ) 0 PAGES READ: 6-10, 5 OR FEWER A DAY, MISSING

 002 PGS>10 (1,2,3 ) 1 PAGES READ: 11-15, 16-20, 20 OR MORE

 CONDITIONING VARIABLE ID: BACK0024

 DESCRIPTION: INTERACTION: GENDER BY RACE/ETHNICITY

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: GEND/RAC

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 3

 001 G/R 11 (11 ) 010101 GEND/RAC INTACT: 1. MALE 1. WHI/AI/O

 002 G/R 12 (12 ) -10000 GEND/RAC INTACT: 1. MALE 2. BLACK

 003 G/R 13 (13 ) 00-100 GEND/RAC INTACT: 1. MALE 3. HISPANIC

 004 G/R 14 (14 ) 0000-1 GEND/RAC INTACT: 1. MALE 4. ASIAN

 005 G/R 21 (21 ) -1-1-1 GEND/RAC INTACT: 2. FEMALE 1. WHI/AI/O

 006 G/R 22 (22 ) 010000 GEND/RAC INTACT: 2. FEMALE 2. BLACK

 007 G/R 23 (23 ) 000100 GEND/RAC INTACT: 2. FEMALE 3. HISPANIC

 008 G/R 24 (24 ) 000001 GEND/RAC INTACT: 2. FEMALE 4. ASIAN

 CONDITIONING VARIABLE ID: BACK0025

 DESCRIPTION: INTERACTION: GENDER BY TYPE OF LOCALE (7 CATEGORIES)

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: GEND/TOL

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 14

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 6

 001 G/T 11 (11 ) 010101010101 GEND/TOL INTACT: 1. MALE 1. BIG CTY7

 002 G/T 12 (12 ) -10000000000 GEND/TOL INTACT: 1. MALE 2. MID CTY7

 003 G/T 13 (13 ) 00-100000000 GEND/TOL INTACT: 1. MALE 3. FR/LCTY7

 004 G/T 14 (14 ) 0000-1000000 GEND/TOL INTACT: 1. MALE 4. FR/MCTY7

 005 G/T 15 (15 ) 000000-10000 GEND/TOL INTACT: 1. MALE 5. LAR TWN7

 006 G/T 16 (16 ) 00000000-100 GEND/TOL INTACT: 1. MALE 6. SML TWN7

 007 G/T 17 (17 ) 0000000000-1 GEND/TOL INTACT: 1. MALE 7. OTHER

 008 G/T 21 (21 ) -1-1-1-1-1-1 GEND/TOL INTACT: 2. FEMALE 1. BIG CTY7

 009 G/T 22 (22 ) 010000000000 GEND/TOL INTACT: 2. FEMALE 2. MID CTY7

 010 G/T 23 (23 ) 000100000000 GEND/TOL INTACT: 2. FEMALE 3. FR/LCTY7

 011 G/T 24 (24 ) 000001000000 GEND/TOL INTACT: 2. FEMALE 4. FR/MCTY7

 012 G/T 25 (25 ) 000000010000 GEND/TOL INTACT: 2. FEMALE 5. LAR TWN7

 013 G/T 26 (26 ) 000000000100 GEND/TOL INTACT: 2. FEMALE 6. SML TWN7

 014 G/T 27 (27 ) 000000000001 GEND/TOL INTACT: 2. FEMALE 7. OTHER

 CONDITIONING VARIABLE ID: BACK0026

 DESCRIPTION: INTERACTION: GENDER BY PARENTS’ EDUCATION GRADES 8 & 12

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL: GEND/PAR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 10

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 4

 001 G/P 11 (11 ) 01010101 GEND/PAR INTACT: 1. MALE 1. < HS

 002 G/P 12 (12 ) -1000000 GEND/PAR INTACT: 1. MALE 2. HS GRAD

 003 G/P 13 (13 ) 00-10000 GEND/PAR INTACT: 1. MALE 3. POST HS

 004 G/P 14 (14 ) 0000-100 GEND/PAR INTACT: 1. MALE 4. COL GRAD

 005 G/P 15 (15 ) 000000-1 GEND/PAR INTACT: 1. MALE 5. PARED-?

 006 G/P 21 (21 ) -1-1-1-1 GEND/PAR INTACT: 2. FEMALE 1. < HS

 007 G/P 22 (22 ) 01000000 GEND/PAR INTACT: 2. FEMALE 2. HS GRAD

 008 G/P 23 (23 ) 00010000 GEND/PAR INTACT: 2. FEMALE 3. POST HS

 009 G/P 24 (24 ) 00000100 GEND/PAR INTACT: 2. FEMALE 4. COL GRAD

 010 G/P 25 (25 ) 00000001 GEND/PAR INTACT: 2. FEMALE 5. PARED-?
Table F-5 (continued)

1998 Reading Conditioning Variable Specifications

652

 CONDITIONING VARIABLE ID: BACK0027

 DESCRIPTION: INTERACTION: GENDER BY PARENTS’ EDUCATION GRADE 4

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL: GEND/PAR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 10

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 4

 001 G/P 11 (11 ) 01010101 GEND/PAR INTACT: 1. MALE 1. < HS

 002 G/P 12 (12 ) -1000000 GEND/PAR INTACT: 1. MALE 2. HS GRAD

 003 G/P 13 (13 ) 00-10000 GEND/PAR INTACT: 1. MALE 3. POST HS

 004 G/P 14 (14 ) 0000-100 GEND/PAR INTACT: 1. MALE 4. COL GRAD

 005 G/P 15 (15 ) 000000-1 GEND/PAR INTACT: 1. MALE 5. PARED-?

 006 G/P 21 (21 ) -1-1-1-1 GEND/PAR INTACT: 2. FEMALE 1. < HS

 007 G/P 22 (22 ) 01000000 GEND/PAR INTACT: 2. FEMALE 2. HS GRAD

 008 G/P 23 (23 ) 00010000 GEND/PAR INTACT: 2. FEMALE 3. POST HS

 009 G/P 24 (24 ) 00000100 GEND/PAR INTACT: 2. FEMALE 4. COL GRAD

 010 G/P 25 (25 ) 00000001 GEND/PAR INTACT: 2. FEMALE 5. PARED-?

 CONDITIONING VARIABLE ID: BACK0028

 DESCRIPTION: INTERACTION: GENDER BY SCHOOL TYPE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: GEND/SCH

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 2

 001 G/S 11 (11 ) 0101 GEND/SCH INTACT: 1. MALE 1. PUBLIC

 002 G/S 12 (12 ) -100 GEND/SCH INTACT: 1. MALE 2. PRIVATE

 003 G/S 13 (13 ) 00-1 GEND/SCH INTACT: 1. MALE 3. CATHOLIC

 004 G/S 21 (21 ) -1-1 GEND/SCH INTACT: 2. FEMALE 1. PUBLIC

 005 G/S 22 (22 ) 0100 GEND/SCH INTACT: 2. FEMALE 2. PRIVATE

 006 G/S 23 (23 ) 0001 GEND/SCH INTACT: 2. FEMALE 3. CATHOLIC

 CONDITIONING VARIABLE ID: BACK0029

 DESCRIPTION: INTERACTION: RACE/ETHNICITY BY TYPE OF LOCALE (7 CATEGORIES)

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: RACE/TOL

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 28

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 18

 001 R/T 11 (11 ) 010101010101010101010101010101010101 RACE/TOL INTACT: 1. WHI/AI/O 1. BIG CTY7

 002 R/T 12 (12 ) -10000000000-10000000000-10000000000 RACE/TOL INTACT: 1. WHI/AI/O 2. MID CTY7

 003 R/T 13 (13 ) 00-10000000000-10000000000-100000000 RACE/TOL INTACT: 1. WHI/AI/O 3. FR/LCTY7

 004 R/T 14 (14 ) 0000-10000000000-10000000000-1000000 RACE/TOL INTACT: 1. WHI/AI/O 4. FR/MCTY7

 005 R/T 15 (15 ) 000000-10000000000-10000000000-10000 RACE/TOL INTACT: 1. WHI/AI/O 5. LAR TWN7

 006 R/T 16 (16 ) 00000000-10000000000-10000000000-100 RACE/TOL INTACT: 1. WHI/AI/O 6. SML TWN7

 007 R/T 17 (17 ) 0000000000-10000000000-10000000000-1 RACE/TOL INTACT: 1. WHI/AI/O 7. OTHER

 008 R/T 21 (21 ) -1-1-1-1-1-1000000000000000000000000 RACE/TOL INTACT: 2. BLACK 1. BIG CTY7

 009 R/T 22 (22 ) 010000000000000000000000000000000000 RACE/TOL INTACT: 2. BLACK 2. MID CTY7

 010 R/T 23 (23 ) 000100000000000000000000000000000000 RACE/TOL INTACT: 2. BLACK 3. FR/LCTY7

 011 R/T 24 (24 ) 000001000000000000000000000000000000 RACE/TOL INTACT: 2. BLACK 4. FR/MCTY7

 012 R/T 25 (25 ) 000000010000000000000000000000000000 RACE/TOL INTACT: 2. BLACK 5. LAR TWN7

 013 R/T 26 (26 ) 000000000100000000000000000000000000 RACE/TOL INTACT: 2. BLACK 6. SML TWN7

 014 R/T 27 (27 ) 000000000001000000000000000000000000 RACE/TOL INTACT: 2. BLACK 7. OTHER

 015 R/T 31 (31 ) 000000000000-1-1-1-1-1-1000000000000 RACE/TOL INTACT: 3. HISPANIC 1. BIG CTY7

 016 R/T 32 (32 ) 000000000000010000000000000000000000 RACE/TOL INTACT: 3. HISPANIC 2. MID CTY7

 017 R/T 33 (33 ) 000000000000000100000000000000000000 RACE/TOL INTACT: 3. HISPANIC 3. FR/LCTY7

 018 R/T 34 (34 ) 000000000000000001000000000000000000 RACE/TOL INTACT: 3. HISPANIC 4. FR/MCTY7

 019 R/T 35 (35 ) 000000000000000000010000000000000000 RACE/TOL INTACT: 3. HISPANIC 5. LAR TWN7

 020 R/T 36 (36 ) 000000000000000000000100000000000000 RACE/TOL INTACT: 3. HISPANIC 6. SML TWN7

 021 R/T 37 (37 ) 000000000000000000000001000000000000 RACE/TOL INTACT: 3. HISPANIC 7. OTHER

 022 R/T 41 (41 ) 000000000000000000000000-1-1-1-1-1-1 RACE/TOL INTACT: 4. ASIAN 1. BIG CTY7

 023 R/T 42 (42 ) 000000000000000000000000010000000000 RACE/TOL INTACT: 4. ASIAN 2. MID CTY7

 024 R/T 43 (43 ) 000000000000000000000000000100000000 RACE/TOL INTACT: 4. ASIAN 3. FR/LCTY7

 025 R/T 44 (44 ) 000000000000000000000000000001000000 RACE/TOL INTACT: 4. ASIAN 4. FR/MCTY7

 026 R/T 45 (45 ) 000000000000000000000000000000010000 RACE/TOL INTACT: 4. ASIAN 5. LAR TWN7

 027 R/T 46 (46 ) 000000000000000000000000000000000100 RACE/TOL INTACT: 4. ASIAN 6. SML TWN7

 028 R/T 47 (47 ) 000000000000000000000000000000000001 RACE/TOL INTACT: 4. ASIAN 7. OTHER

 CONDITIONING VARIABLE ID: BACK0030

 DESCRIPTION: INTERACTION: RACE/ETHNICITY BY PARENTS’ EDUCATION GRADES 8 & 12

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL: RACE/PAR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 20

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 12

 001 R/P 11 (11 ) 010101010101010101010101 RACE/PAR INTACT: 1. WHI/AI/O 1. < HS

 002 R/P 12 (12 ) -1000000-1000000-1000000 RACE/PAR INTACT: 1. WHI/AI/O 2. HS GRAD

 003 R/P 13 (13 ) 00-1000000-1000000-10000 RACE/PAR INTACT: 1. WHI/AI/O 3. POST HS

 004 R/P 14 (14 ) 0000-1000000-1000000-100 RACE/PAR INTACT: 1. WHI/AI/O 4. COL GRAD

 005 R/P 15 (15 ) 000000-1000000-1000000-1 RACE/PAR INTACT: 1. WHI/AI/O 5. PARED-?

 006 R/P 21 (21 ) -1-1-1-10000000000000000 RACE/PAR INTACT: 2. BLACK 1. < HS

 007 R/P 22 (22 ) 010000000000000000000000 RACE/PAR INTACT: 2. BLACK 2. HS GRAD

 008 R/P 23 (23 ) 000100000000000000000000 RACE/PAR INTACT: 2. BLACK 3. POST HS

 009 R/P 24 (24 ) 000001000000000000000000 RACE/PAR INTACT: 2. BLACK 4. COL GRAD

 010 R/P 25 (25 ) 000000010000000000000000 RACE/PAR INTACT: 2. BLACK 5. PARED-?

 011 R/P 31 (31 ) 00000000-1-1-1-100000000 RACE/PAR INTACT: 3. HISPANIC 1. < HS

 012 R/P 32 (32 ) 000000000100000000000000 RACE/PAR INTACT: 3. HISPANIC 2. HS GRAD

 013 R/P 33 (33 ) 000000000001000000000000 RACE/PAR INTACT: 3. HISPANIC 3. POST HS

 014 R/P 34 (34 ) 000000000000010000000000 RACE/PAR INTACT: 3. HISPANIC 4. COL GRAD

 015 R/P 35 (35 ) 000000000000000100000000 RACE/PAR INTACT: 3. HISPANIC 5. PARED-?

 016 R/P 41 (41 ) 0000000000000000-1-1-1-1 RACE/PAR INTACT: 4. ASIAN 1. < HS

 017 R/P 42 (42 ) 000000000000000001000000 RACE/PAR INTACT: 4. ASIAN 2. HS GRAD

 018 R/P 43 (43 ) 000000000000000000010000 RACE/PAR INTACT: 4. ASIAN 3. POST HS

 019 R/P 44 (44 ) 000000000000000000000100 RACE/PAR INTACT: 4. ASIAN 4. COL GRAD

 020 R/P 45 (45 ) 000000000000000000000001 RACE/PAR INTACT: 4. ASIAN 5. PARED-?
Table F-5 (continued)

 1998 Reading Conditioning Variable Specification

653

 CONDITIONING VARIABLE ID: BACK0031

 DESCRIPTION: INTERACTION: RACE/ETHNICITY BY PARENTS’ EDUCATION GRADE 4

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL: RACE/PAR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 20

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 12

 001 R/P 11 (11 ) 010101010101010101010101 RACE/PAR INTACT: 1. WHI/AI/O 1. < HS

 002 R/P 12 (12 ) -1000000-1000000-1000000 RACE/PAR INTACT: 1. WHI/AI/O 2. HS GRAD

 003 R/P 13 (13 ) 00-1000000-1000000-10000 RACE/PAR INTACT: 1. WHI/AI/O 3. POST HS

 004 R/P 14 (14 ) 0000-1000000-1000000-100 RACE/PAR INTACT: 1. WHI/AI/O 4. COL GRAD

 005 R/P 15 (15 ) 000000-1000000-1000000-1 RACE/PAR INTACT: 1. WHI/AI/O 5. PARED-?

 006 R/P 21 (21 ) -1-1-1-10000000000000000 RACE/PAR INTACT: 2. BLACK 1. < HS

 007 R/P 22 (22 ) 010000000000000000000000 RACE/PAR INTACT: 2. BLACK 2. HS GRAD

 008 R/P 23 (23 ) 000100000000000000000000 RACE/PAR INTACT: 2. BLACK 3. POST HS

 009 R/P 24 (24 ) 000001000000000000000000 RACE/PAR INTACT: 2. BLACK 4. COL GRAD

 010 R/P 25 (25 ) 000000010000000000000000 RACE/PAR INTACT: 2. BLACK 5. PARED-?

 011 R/P 31 (31 ) 00000000-1-1-1-100000000 RACE/PAR INTACT: 3. HISPANIC 1. < HS

 012 R/P 32 (32 ) 000000000100000000000000 RACE/PAR INTACT: 3. HISPANIC 2. HS GRAD

 013 R/P 33 (33 ) 000000000001000000000000 RACE/PAR INTACT: 3. HISPANIC 3. POST HS

 014 R/P 34 (34 ) 000000000000010000000000 RACE/PAR INTACT: 3. HISPANIC 4. COL GRAD

 015 R/P 35 (35 ) 000000000000000100000000 RACE/PAR INTACT: 3. HISPANIC 5. PARED-?

 016 R/P 41 (41 ) 0000000000000000-1-1-1-1 RACE/PAR INTACT: 4. ASIAN 1. < HS

 017 R/P 42 (42 ) 000000000000000001000000 RACE/PAR INTACT: 4. ASIAN 2. HS GRAD

 018 R/P 43 (43 ) 000000000000000000010000 RACE/PAR INTACT: 4. ASIAN 3. POST HS

 019 R/P 44 (44 ) 000000000000000000000100 RACE/PAR INTACT: 4. ASIAN 4. COL GRAD

 020 R/P 45 (45 ) 000000000000000000000001 RACE/PAR INTACT: 4. ASIAN 5. PARED-?

 CONDITIONING VARIABLE ID: BACK0032

 DESCRIPTION: INTERACTION: RACE/ETHNICITY BY SCHOOL TYPE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: RACE/SCH

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 12

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 6

 001 R/S 11 (11 ) 010101010101 RACE/SCH INTACT: 1. WHI/AI/O 1. PUBLIC

 002 R/S 12 (12 ) -100-100-100 RACE/SCH INTACT: 1. WHI/AI/O 2. PRIVATE

 003 R/S 13 (13 ) 00-100-100-1 RACE/SCH INTACT: 1. WHI/AI/O 3. CATHOLIC

 004 R/S 21 (21 ) -1-100000000 RACE/SCH INTACT: 2. BLACK 1. PUBLIC

 005 R/S 22 (22 ) 010000000000 RACE/SCH INTACT: 2. BLACK 2. PRIVATE

 006 R/S 23 (23 ) 000100000000 RACE/SCH INTACT: 2. BLACK 3. CATHOLIC

 007 R/S 31 (31 ) 0000-1-10000 RACE/SCH INTACT: 3. HISPANIC 1. PUBLIC

 008 R/S 32 (32 ) 000001000000 RACE/SCH INTACT: 3. HISPANIC 2. PRIVATE

 009 R/S 33 (33 ) 000000010000 RACE/SCH INTACT: 3. HISPANIC 3. CATHOLIC

 010 R/S 41 (41 ) 00000000-1-1 RACE/SCH INTACT: 4. ASIAN 1. PUBLIC

 011 R/S 42 (42 ) 000000000100 RACE/SCH INTACT: 4. ASIAN 2. PRIVATE

 012 R/S 43 (43 ) 000000000001 RACE/SCH INTACT: 4. ASIAN 3. CATHOLIC

 CONDITIONING VARIABLE ID: BACK0033

 DESCRIPTION: INTERACTION: PARENT’S EDUCATION GRADES 8 & 12 BY TYPE OF LOCALE (7 CATEGORIES)

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL: PARE/TOL

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 35

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 24

 001 P/T 11 (11 ) 010101010101010101010101010101010101010101010101 PARE/TOL INTACT: 1. < HS 1. BIG CTY7

 002 P/T 12 (12 ) -10000000000-10000000000-10000000000-10000000000 PARE/TOL INTACT: 1. < HS 2. MID CTY7

 003 P/T 13 (13 ) 00-10000000000-10000000000-10000000000-100000000 PARE/TOL INTACT: 1. < HS 3. FR/LCTY7

 004 P/T 14 (14 ) 0000-10000000000-10000000000-10000000000-1000000 PARE/TOL INTACT: 1. < HS 4. FR/MCTY7

 005 P/T 15 (15 ) 000000-10000000000-10000000000-10000000000-10000 PARE/TOL INTACT: 1. < HS 5. LAR TWN7

 006 P/T 16 (16 ) 00000000-10000000000-10000000000-10000000000-100 PARE/TOL INTACT: 1. < HS 6. SML TWN7

 007 P/T 17 (17 ) 0000000000-10000000000-10000000000-10000000000-1 PARE/TOL INTACT: 1. < HS 7. OTHER

 008 P/T 21 (21 ) -1-1-1-1-1-1000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 1. BIG CTY7

 009 P/T 22 (22 ) 010000000000000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 2. MID CTY7

 010 P/T 23 (23 ) 000100000000000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 3. FR/LCTY7

 011 P/T 24 (24 ) 000001000000000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 4. FR/MCTY7

 012 P/T 25 (25 ) 000000010000000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 5. LAR TWN7

 013 P/T 26 (26 ) 000000000100000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 6. SML TWN7

 014 P/T 27 (27 ) 000000000001000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 7. OTHER

 015 P/T 31 (31 ) 000000000000-1-1-1-1-1-1000000000000000000000000 PARE/TOL INTACT: 3. POST HS 1. BIG CTY7

 016 P/T 32 (32 ) 000000000000010000000000000000000000000000000000 PARE/TOL INTACT: 3. POST HS 2. MID CTY7

 017 P/T 33 (33 ) 000000000000000100000000000000000000000000000000 PARE/TOL INTACT: 3. POST HS 3. FR/LCTY7

 018 P/T 34 (34 ) 000000000000000001000000000000000000000000000000 PARE/TOL INTACT: 3. POST HS 4. FR/MCTY7

 019 P/T 35 (35 ) 000000000000000000010000000000000000000000000000 PARE/TOL INTACT: 3. POST HS 5. LAR TWN7

 020 P/T 36 (36 ) 000000000000000000000100000000000000000000000000 PARE/TOL INTACT: 3. POST HS 6. SML TWN7

 021 P/T 37 (37 ) 000000000000000000000001000000000000000000000000 PARE/TOL INTACT: 3. POST HS 7. OTHER

 022 P/T 41 (41 ) 000000000000000000000000-1-1-1-1-1-1000000000000 PARE/TOL INTACT: 4. COL GRAD 1. BIG CTY7

 023 P/T 42 (42 ) 000000000000000000000000010000000000000000000000 PARE/TOL INTACT: 4. COL GRAD 2. MID CTY7

 024 P/T 43 (43 ) 000000000000000000000000000100000000000000000000 PARE/TOL INTACT: 4. COL GRAD 3. FR/LCTY7

 025 P/T 44 (44 ) 000000000000000000000000000001000000000000000000 PARE/TOL INTACT: 4. COL GRAD 4. FR/MCTY7

 026 P/T 45 (45 ) 000000000000000000000000000000010000000000000000 PARE/TOL INTACT: 4. COL GRAD 5. LAR TWN7

 027 P/T 46 (46 ) 000000000000000000000000000000000100000000000000 PARE/TOL INTACT: 4. COL GRAD 6. SML TWN7

 028 P/T 47 (47 ) 000000000000000000000000000000000001000000000000 PARE/TOL INTACT: 4. COL GRAD 7. OTHER

 029 P/T 51 (51 ) 000000000000000000000000000000000000-1-1-1-1-1-1 PARE/TOL INTACT: 5. PARED-? 1. BIG CTY7

 030 P/T 52 (52 ) 000000000000000000000000000000000000010000000000 PARE/TOL INTACT: 5. PARED-? 2. MID CTY7

 031 P/T 53 (53 ) 000000000000000000000000000100000000000100000000 PARE/TOL INTACT: 5. PARED-? 3. FR/LCTY7

 032 P/T 54 (54 ) 000000000000000000000000000000000000000001000000 PARE/TOL INTACT: 5. PARED-? 4. FR/MCTY7

 033 P/T 55 (55 ) 000000000000000000000000000000000000000000010000 PARE/TOL INTACT: 5. PARED-? 5. LAR TWN7

 034 P/T 56 (56 ) 000000000000000000000000000000000000000000000100 PARE/TOL INTACT: 5. PARED-? 6. SML TWN7

 035 P/T 57 (57 ) 000000000000000000000000000000000000000000000001 PARE/TOL INTACT: 5. PARED-? 7. OTHER
Table F-5 (continued)

1998 Reading Conditioning Variable Specifications

654

 CONDITIONING VARIABLE ID: BACK0034

 DESCRIPTION: INTERACTION: PARENT’S EDUCATION GRADE 4 BY TYPE OF LOCALE (7 CATEGORIES)

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL: PARE/TOL

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 35

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 24

 001 P/T 11 (11 ) 010101010101010101010101010101010101010101010101 PARE/TOL INTACT: 1. < HS 1. BIG CTY7

 002 P/T 12 (12 ) -10000000000-10000000000-10000000000-10000000000 PARE/TOL INTACT: 1. < HS 2. MID CTY7

 003 P/T 13 (13 ) 00-10000000000-10000000000-10000000000-100000000 PARE/TOL INTACT: 1. < HS 3. FR/LCTY7

 004 P/T 14 (14 ) 0000-10000000000-10000000000-10000000000-1000000 PARE/TOL INTACT: 1. < HS 4. FR/MCTY7

 005 P/T 15 (15 ) 000000-10000000000-10000000000-10000000000-10000 PARE/TOL INTACT: 1. < HS 5. LAR TWN7

 006 P/T 16 (16 ) 00000000-10000000000-10000000000-10000000000-100 PARE/TOL INTACT: 1. < HS 6. SML TWN7

 007 P/T 17 (17 ) 0000000000-10000000000-10000000000-10000000000-1 PARE/TOL INTACT: 1. < HS 7. OTHER

 008 P/T 21 (21 ) -1-1-1-1-1-1000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 1. BIG CTY7

 009 P/T 22 (22 ) 010000000000000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 2. MID CTY7

 010 P/T 23 (23 ) 000100000000000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 3. FR/LCTY7

 011 P/T 24 (24 ) 000001000000000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 4. FR/MCTY7

 012 P/T 25 (25 ) 000000010000000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 5. LAR TWN7

 013 P/T 26 (26 ) 000000000100000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 6. SML TWN7

 014 P/T 27 (27 ) 000000000001000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 7. OTHER

 015 P/T 31 (31 ) 000000000000-1-1-1-1-1-1000000000000000000000000 PARE/TOL INTACT: 3. POST HS 1. BIG CTY7

 016 P/T 32 (32 ) 000000000000010000000000000000000000000000000000 PARE/TOL INTACT: 3. POST HS 2. MID CTY7

 017 P/T 33 (33 ) 000000000000000100000000000000000000000000000000 PARE/TOL INTACT: 3. POST HS 3. FR/LCTY7

 018 P/T 34 (34 ) 000000000000000001000000000000000000000000000000 PARE/TOL INTACT: 3. POST HS 4. FR/MCTY7

 019 P/T 35 (35 ) 000000000000000000010000000000000000000000000000 PARE/TOL INTACT: 3. POST HS 5. LAR TWN7

 020 P/T 36 (36 ) 000000000000000000000100000000000000000000000000 PARE/TOL INTACT: 3. POST HS 6. SML TWN7

 021 P/T 37 (37 ) 000000000000000000000001000000000000000000000000 PARE/TOL INTACT: 3. POST HS 7. OTHER

 022 P/T 41 (41 ) 000000000000000000000000-1-1-1-1-1-1000000000000 PARE/TOL INTACT: 4. COL GRAD 1. BIG CTY7

 023 P/T 42 (42 ) 000000000000000000000000010000000000000000000000 PARE/TOL INTACT: 4. COL GRAD 2. MID CTY7

 024 P/T 43 (43 ) 000000000000000000000000000100000000000000000000 PARE/TOL INTACT: 4. COL GRAD 3. FR/LCTY7

 025 P/T 44 (44 ) 000000000000000000000000000001000000000000000000 PARE/TOL INTACT: 4. COL GRAD 4. FR/MCTY7

 026 P/T 45 (45 ) 000000000000000000000000000000010000000000000000 PARE/TOL INTACT: 4. COL GRAD 5. LAR TWN7

 027 P/T 46 (46 ) 000000000000000000000000000000000100000000000000 PARE/TOL INTACT: 4. COL GRAD 6. SML TWN7

 028 P/T 47 (47 ) 000000000000000000000000000000000001000000000000 PARE/TOL INTACT: 4. COL GRAD 7. OTHER

 029 P/T 51 (51 ) 000000000000000000000000000000000000-1-1-1-1-1-1 PARE/TOL INTACT: 5. PARED-? 1. BIG CTY7

 030 P/T 52 (52 ) 000000000000000000000000000000000000010000000000 PARE/TOL INTACT: 5. PARED-? 2. MID CTY7

 031 P/T 53 (53 ) 000000000000000000000000000100000000000100000000 PARE/TOL INTACT: 5. PARED-? 3. FR/LCTY7

 032 P/T 54 (54 ) 000000000000000000000000000000000000000001000000 PARE/TOL INTACT: 5. PARED-? 4. FR/MCTY7

 033 P/T 55 (55 ) 000000000000000000000000000000000000000000010000 PARE/TOL INTACT: 5. PARED-? 5. LAR TWN7

 034 P/T 56 (56 ) 000000000000000000000000000000000000000000000100 PARE/TOL INTACT: 5. PARED-? 6. SML TWN7

 035 P/T 57 (57 ) 000000000000000000000000000000000000000000000001 PARE/TOL INTACT: 5. PARED-? 7. OTHER

 CONDITIONING VARIABLE ID: BACK0035

 DESCRIPTION: INTERACTION: TYPE OF LOCALE (7 CATEGORIES) BY SCHOOL TYPE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: TOL7/SCH

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 21

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 12

 001 T/S 11 (11 ) 010101010101010101010101 TOL7/SCH INTACT: 1. BIG CTY7 1. PUBLIC

 002 T/S 12 (12 ) -100-100-100-100-100-100 TOL7/SCH INTACT: 1. BIG CTY7 2. PRIVATE

 003 T/S 13 (13 ) 00-100-100-100-100-100-1 TOL7/SCH INTACT: 1. BIG CTY7 3. CATHOLIC

 004 T/S 21 (21 ) -1-100000000000000000000 TOL7/SCH INTACT: 2. MID CTY7 1. PUBLIC

 005 T/S 22 (22 ) 010000000000000000000000 TOL7/SCH INTACT: 2. MID CTY7 2. PRIVATE

 006 T/S 23 (23 ) 000100000000000000000000 TOL7/SCH INTACT: 2. MID CTY7 3. CATHOLIC

 007 T/S 31 (31 ) 0000-1-10000000000000000 TOL7/SCH INTACT: 3. FR/LCTY7 1. PUBLIC

 008 T/S 32 (32 ) 000001000000000000000000 TOL7/SCH INTACT: 3. FR/LCTY7 2. PRIVATE

 009 T/S 33 (33 ) 000000010000000000000000 TOL7/SCH INTACT: 3. FR/LCTY7 3. CATHOLIC

 010 T/S 41 (41 ) 00000000-1-1000000000000 TOL7/SCH INTACT: 4. FR/MCTY7 1. PUBLIC

 011 T/S 42 (42 ) 000000000100000000000000 TOL7/SCH INTACT: 4. FR/MCTY7 2. PRIVATE

 012 T/S 43 (43 ) 000000000001000000000000 TOL7/SCH INTACT: 4. FR/MCTY7 3. CATHOLIC

 013 T/S 51 (51 ) 000000000000-1-100000000 TOL7/SCH INTACT: 5. LAR TWN7 1. PUBLIC

 014 T/S 52 (52 ) 000000000000010000000000 TOL7/SCH INTACT: 5. LAR TWN7 2. PRIVATE

 015 T/S 53 (53 ) 000000000000000100000000 TOL7/SCH INTACT: 5. LAR TWN7 3. CATHOLIC

 016 T/S 61 (61 ) 0000000000000000-1-10000 TOL7/SCH INTACT: 6. SML TWN7 1. PUBLIC

 017 T/S 62 (62 ) 000000000000000001000000 TOL7/SCH INTACT: 6. SML TWN7 2. PRIVATE

 018 T/S 63 (63 ) 000000000000000000010000 TOL7/SCH INTACT: 6. SML TWN7 3. CATHOLIC

 019 T/S 71 (71 ) 00000000000000000000-1-1 TOL7/SCH INTACT: 7. OTHER 1. PUBLIC

 020 T/S 72 (72 ) 000000000000000000000100 TOL7/SCH INTACT: 7. OTHER 2. PRIVATE

 021 T/S 73 (73 ) 000000000000000000000001 TOL7/SCH INTACT: 7. OTHER 3. CATHOLIC

 CONDITIONING VARIABLE ID: BACK0036

 DESCRIPTION: INTERACTION: PARENTS’ EDUCATION GRADES 8 & 12 BY SCHOOL TYPE

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL: PARE/SCH

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 15

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 8

 001 P/S 11 (11 ) 0101010101010101 PARE/SCH INTACT: 1. < HS 1. PUBLIC

 002 P/S 12 (12 ) -100-100-100-100 PARE/SCH INTACT: 1. < HS 2. PRIVATE

 003 P/S 13 (13 ) 00-100-100-100-1 PARE/SCH INTACT: 1. < HS 3. CATHOLIC

 004 P/S 21 (21 ) -1-1000000000000 PARE/SCH INTACT: 2. HS GRAD 1. PUBLIC

 005 P/S 22 (22 ) 0100000000000000 PARE/SCH INTACT: 2. HS GRAD 2. PRIVATE

 006 P/S 23 (23 ) 0001000000000000 PARE/SCH INTACT: 2. HS GRAD 3. CATHOLIC

 007 P/S 31 (31 ) 0000-1-100000000 PARE/SCH INTACT: 3. POST HS 1. PUBLIC

 008 P/S 32 (32 ) 0000010000000000 PARE/SCH INTACT: 3. POST HS 2. PRIVATE

 009 P/S 33 (33 ) 0000000100000000 PARE/SCH INTACT: 3. POST HS 3. CATHOLIC

 010 P/S 41 (41 ) 00000000-1-10000 PARE/SCH INTACT: 4. COL GRAD 1. PUBLIC

 011 P/S 42 (42 ) 0000000001000000 PARE/SCH INTACT: 4. COL GRAD 2. PRIVATE

 012 P/S 43 (43 ) 0000000000010000 PARE/SCH INTACT: 4. COL GRAD 3. CATHOLIC

 013 P/S 51 (51 ) 000000000000-1-1 PARE/SCH INTACT: 5. PARED-? 1. PUBLIC

 014 P/S 52 (52 ) 0000000000000100 PARE/SCH INTACT: 5. PARED-? 2. PRIVATE

 015 P/S 53 (53 ) 0000000000000001 PARE/SCH INTACT: 5. PARED-? 3. CATHOLIC
Table F-5 (continued)

 1998 Reading Conditioning Variable Specification

655

 CONDITIONING VARIABLE ID: BACK0037

 DESCRIPTION: INTERACTION: PARENTS’ EDUCATION GRADE 4 BY SCHOOL TYPE

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL: PARE/SCH

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 15

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 8

 001 P/S 11 (11 ) 0101010101010101 PARE/SCH INTACT: 1. < HS 1. PUBLIC

 002 P/S 12 (12 ) -100-100-100-100 PARE/SCH INTACT: 1. < HS 2. PRIVATE

 003 P/S 13 (13 ) 00-100-100-100-1 PARE/SCH INTACT: 1. < HS 3. CATHOLIC

 004 P/S 21 (21 ) -1-1000000000000 PARE/SCH INTACT: 2. HS GRAD 1. PUBLIC

 005 P/S 22 (22 ) 0100000000000000 PARE/SCH INTACT: 2. HS GRAD 2. PRIVATE

 006 P/S 23 (23 ) 0001000000000000 PARE/SCH INTACT: 2. HS GRAD 3. CATHOLIC

 007 P/S 31 (31 ) 0000-1-100000000 PARE/SCH INTACT: 3. POST HS 1. PUBLIC

 008 P/S 32 (32 ) 0000010000000000 PARE/SCH INTACT: 3. POST HS 2. PRIVATE

 009 P/S 33 (33 ) 0000000100000000 PARE/SCH INTACT: 3. POST HS 3. CATHOLIC

 010 P/S 41 (41 ) 00000000-1-10000 PARE/SCH INTACT: 4. COL GRAD 1. PUBLIC

 011 P/S 42 (42 ) 0000000001000000 PARE/SCH INTACT: 4. COL GRAD 2. PRIVATE

 012 P/S 43 (43 ) 0000000000010000 PARE/SCH INTACT: 4. COL GRAD 3. CATHOLIC

 013 P/S 51 (51 ) 000000000000-1-1 PARE/SCH INTACT: 5. PARED-? 1. PUBLIC

 014 P/S 52 (52 ) 0000000000000100 PARE/SCH INTACT: 5. PARED-? 2. PRIVATE

 015 P/S 53 (53 ) 0000000000000001 PARE/SCH INTACT: 5. PARED-? 3. CATHOLIC

 CONDITIONING VARIABLE ID: BACK0041

 DESCRIPTION: SAMPLE TYPE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: SAMPLE

 NAEP ID: SUBSAMP TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 SAMP S2 (02 ) 0 SAMPLE S2

 002 SAMP S3 (03 ) 1 SAMPLE S3

 CONDITIONING VARIABLE ID: BACK0042

 DESCRIPTION: INTERACTION: SAMPLE BY RACE/ETHNICITY

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: SAMP/RAC

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 3

 001 S/R 11 (11 ) 010101 SAMP/RAC INTACT: 1. SAMP S2 1. WHI/AI/O

 002 S/R 12 (12 ) -10000 SAMP/RAC INTACT: 1. SAMP S2 2. BLACK

 003 S/R 13 (13 ) 00-100 SAMP/RAC INTACT: 1. SAMP S2 3. HISPANIC

 004 S/R 14 (14 ) 0000-1 SAMP/RAC INTACT: 1. SAMP S2 4. ASIAN

 005 S/R 21 (21 ) -1-1-1 SAMP/RAC INTACT: 2. SAMP S3 1. WHI/AI/O

 006 S/R 22 (22 ) 010000 SAMP/RAC INTACT: 2. SAMP S3 2. BLACK

 007 S/R 23 (23 ) 000100 SAMP/RAC INTACT: 2. SAMP S3 3. HISPANIC

 008 S/R 24 (24 ) 000001 SAMP/RAC INTACT: 2. SAMP S3 4. ASIAN

 CONDITIONING VARIABLE ID: BACK0043

 DESCRIPTION: REPORTING SAMPLE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: RPTSAMP TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 RPTSAMP (01 ) 0 YES

 002 RPT NO (02 ) 1 NO

 CONDITIONING VARIABLE ID: BACK0045

 DESCRIPTION: WHICH RACE/ETHNICITY BEST DESCRIBES YOU

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B003001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 WHITE (01 ) 000000 WHITE

 002 BLACK (02 ) 100000 BLACK

 003 HISPANIC (03 ) 010000 HISPANIC

 004 ASIAN AM (04 ) 001000 ASIAN/PACIFIC ISLAND

 005 AMER IND (05 ) 000100 AMER IND/ALASKA NATV

 006 OTHER (06 ) 000010 OTHER

 007 B003001M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: BACK0046

 DESCRIPTION: HOW LONG LIVED IN UNITED STATES

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B014601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 B014601A (01 ) 000 MORE THAN 5 YEARS

 002 B014601B (02 ) 100 3-5 YEARS

 003 B014601C (03 ) 010 LESS THAN 3 YEARS

 004 B014601M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: BACK0047

 DESCRIPTION: HOW OFTEN OTHER THAN ENGLISH SPOKEN IN HOME

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B003201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 B003201A (01 ) 000 NEVER

 002 B003201B (02 ) 100 SOMETIMES

 003 B003201C (03 ) 010 ALWAYS

 004 B003201M (M ) 001 MISSING
Table F-5 (continued)

1998 Reading Conditioning Variable Specifications

656

 CONDITIONING VARIABLE ID: BACK0048

 DESCRIPTION: MOTHER GRADUATED HIGH SCHOOL

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: B013201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B013201Y (01 ) 000 YES

 002 B013201N (02 ) 100 NO

 003 B013201M (M,IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0049

 DESCRIPTION: MOTHER HAD SOME EDUCATION AFTER HIGH SCHOOL

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: B013301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B013301Y (01 ) 000 YES

 002 B013301N (02 ) 100 NO

 003 B013301M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0050

 DESCRIPTION: MOTHER GRADUATED COLLEGE

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: B013401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B013401Y (01 ) 000 YES

 002 B013401N (02 ) 100 NO

 003 B013401M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0051

 DESCRIPTION: FATHER GRADUATED HIGH SCHOOL

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: B013501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B013501Y (01 ) 000 YES

 002 B013501N (02 ) 100 NO

 003 B013501M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0052

 DESCRIPTION: FATHER HAD SOME EDUCATION AFTER HIGH SCHOOL

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: B013601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B013601Y (01 ) 000 YES

 002 B013601N (02 ) 100 NO

 003 B013601M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0053

 DESCRIPTION: FATHER GRADUATED COLLEGE

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: B013701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B013701Y (01 ) 000 YES

 002 B013701N (02 ) 100 NO

 003 B013701M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0054

 DESCRIPTION: DOES YOUR FAMILY GET A NEWSPAPER REGULARLY

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B000901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B000901Y (01 ) 000 YES

 002 B000901N (02 ) 100 NO

 003 B000901M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0055

 DESCRIPTION: IS THERE AN ENCYCLOPEDIA IN YOUR HOME

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B000903 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B000903Y (01 ) 000 YES

 002 B000903N (02 ) 100 NO

 003 B000903M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0056

 DESCRIPTION: ARE THERE MORE THAN 25 BOOKS IN YOUR HOME

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B000904 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B000904Y (01 ) 000 YES

 002 B000904N (02 ) 100 NO

 003 B000904M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0057

 DESCRIPTION: DOES YOUR FAMILY GET MAGAZINES REGULARLY

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B000905 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B000905Y (01 ) 000 YES

 002 B000905N (02 ) 100 NO

 003 B000905M (M, IDK ) 001 MISSING, I DON’T KNOW
Table F-5 (continued)

 1998 Reading Conditioning Variable Specification

657

 CONDITIONING VARIABLE ID: BACK0058

 DESCRIPTION: HOW MANY DAYS OF SCHOOL MISSED LAST MONTH

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: S004001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 S004001A (01 ) 00000 NONE

 002 S004001B (02 ) 10000 1 OR 2 DAYS

 003 S004001C (03 ) 01000 3 OR 4 DAYS

 004 S004001D (04 ) 00100 5 TO 10 DAYS

 005 S004001E (05 ) 00010 MORE THAN 10 DAYS

 006 S004001M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: BACK0059

 DESCRIPTION: TIMES CHANGED SCHOOLS IN PAST TWO YEARS

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B007301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 B007301N (01 ) 0000 NONE

 002 B007301B (02 ) 1000 1

 003 B007301C (03 ) 0100 2

 004 B007301D (04 ) 0010 3 OR MORE

 005 B007301M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0060

 DESCRIPTION: HOW OFTEN DISCUSS STUDIES AT HOME

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B007401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 B007401A (01 ) 0000 ALMOST EVERY DAY

 002 B007401B (02 ) 1000 ONCE/TWICE A WEEK

 003 B007401C (03 ) 0100 ONCE/TWICE A MONTH

 004 B007401D (04 ) 0010 NEVER OR HARDLY EVER

 005 B007401M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0061

 DESCRIPTION: HOW OFTEN USE COMPUTER FOR SCHOOLWORK

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B014501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 B014501A (01 ) 0000 ALMOST EVERY DAY

 002 B014501B (02 ) 1000 ONCE/TWICE A WEEK

 003 B014501C (03 ) 0100 ONCE/TWICE A MONTH

 004 B014501D (04 ) 0010 NEVER OR HARDLY EVER

 005 B014501M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0001

 DESCRIPTION: HOW HARD TRIED ON THIS READING TEST THAN ON OTHERS

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R830301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 R830301A (01 ) 0000 TRIED MUCH HARDER

 002 R830301B (02 ) 1000 TRIED HARDER

 003 R830301C (03 ) 0100 TRIED ABOUT AS HARD

 004 R830301N (04 ) 0010 TRIED NOT AS HARD

 005 R830301M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0002

 DESCRIPTION: HOW IMPORTANT TO DO WELL ON THIS READING TEST

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R830401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 R830401A (01 ) 0000 VERY IMPORTANT

 002 R830401B (02 ) 1000 IMPORTANT

 003 R830401C (03 ) 0100 SOMEWHAT IMPORTANT

 004 R830401N (04 ) 0010 NOT VERY IMPORTANT

 005 R830401M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0003

 DESCRIPTION: HOW OFTEN HAD TO WRITE LONG ANSWERS TO QSTS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: RM00501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 RM00501A (01 ) 0000 AT LEAST ONCE A WEEK

 002 RM00501B (02 ) 1000 ONCE OR TWICE A MNTH

 003 RM00501C (03 ) 0100 ONCE OR TWICE A YEAR

 004 RM00501D (04 ) 0010 NEVER

 005 RM00501M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0004

 DESCRIPTION: MY FRIENDS MAKE FUN OF PEOPLE WHO TRY TO DO WELL

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R830501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 R830501A (01 ) 0000 STRONGLY AGREE

 002 R830501B (02 ) 1000 AGREE

 003 R830501C (03 ) 0100 DISAGREE

 004 R830501D (04 ) 0010 STRONGLY DISAGREE

 005 R830501M (M ) 0001 MISSING
Table F-5 (continued)

1998 Reading Conditioning Variable Specifications

658

 CONDITIONING VARIABLE ID: SUBJ0005

 DESCRIPTION: I HAVE FRIENDS TO TALK TO IF NEED HELP W/SCHOOL

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R830502 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 R830502A (01 ) 0000 STRONGLY AGREE

 002 R830502B (02 ) 1000 AGREE

 003 R830502C (03 ) 0100 DISAGREE

 004 R830502D (04 ) 0010 STRONGLY DISAGREE

 005 R830502M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0006

 DESCRIPTION: BOOKS READ OUTSIDE SCHOOL IN PAST MONTH

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R810801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 R810801N (01 ) 0000 NONE

 002 R810801B (02 ) 1000 ONE OR TWO

 003 R810801C (03 ) 0100 THREE OR FOUR

 004 R810801D (04 ) 0010 FIVE OR MORE

 005 R810801M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0007

 DESCRIPTION: WHAT KIND OF READER ARE YOU

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R810201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 R810201A (01 ) 0000 A VERY GOOD READER

 002 R810201B (02 ) 1000 A GOOD READER

 003 R810201C (03 ) 0100 AN AVERAGE READER

 004 R810201D (04 ) 0010 A POOR READER

 005 R810201M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0008

 DESCRIPTION: HOW OFTEN READ FOR FUN ON OWN

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R810901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 R810901A (01 ) 0000 ALMOST EVERY DAY

 002 R810901B (02 ) 1000 ONCE/TWICE A WEEK

 003 R810901C (03 ) 0100 ONCE/TWICE A MONTH

 004 R810901D (04 ) 0010 NEVER OR HARDLY EVER

 005 R810901M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0009

 DESCRIPTION: HOW OFTEN TALK W/FRIENDS ABOUT WHAT YOU READ

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R810902 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 R810902A (01 ) 0000 ALMOST EVERY DAY

 002 R810902B (02 ) 1000 ONCE/TWICE A WEEK

 003 R810902C (03 ) 0100 ONCE/TWICE A MONTH

 004 R810902D (04 ) 0010 NEVER OR HARDLY EVER

 005 R810902M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0010

 DESCRIPTION: HOW OFTEN TAKE BOOKS FROM LIBRARY ON YOUR OWN

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R810903 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 R810903A (01 ) 0000 ALMOST EVERY DAY

 002 R810903B (02 ) 1000 ONCE/TWICE A WEEK

 003 R810903C (03 ) 0100 ONCE/TWICE A MONTH

 004 R810903D (04 ) 0010 NEVER OR HARDLY EVER

 005 R810903M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0011

 DESCRIPTION: HOW OFTEN READ A STORY OR NOVEL

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R810904 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 R810904A (01 ) 0000 ALMOST EVERY DAY

 002 R810904B (02 ) 1000 ONCE/TWICE A WEEK

 003 R810904C (03 ) 0100 ONCE/TWICE A MONTH

 004 R810904D (04 ) 0010 NEVER OR HARDLY EVER

 005 R810904M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0012

 DESCRIPTION: HOW OFTEN READ A NEWSPAPER

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R810905 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 R810905A (01 ) 0000 ALMOST EVERY DAY

 002 R810905B (02 ) 1000 ONCE/TWICE A WEEK

 003 R810905C (03 ) 0100 ONCE/TWICE A MONTH

 004 R810905D (04 ) 0010 NEVER OR HARDLY EVER

 005 R810905M (M ) 0001 MISSING
Table F-5 (continued)

 1998 Reading Conditioning Variable Specification

659

 CONDITIONING VARIABLE ID: SUBJ0013

 DESCRIPTION: HOW OFTEN READ A MAGAZINE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R810906 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 R810906A (01 ) 0000 ALMOST EVERY DAY

 002 R810906B (02 ) 1000 ONCE/TWICE A WEEK

 003 R810906C (03 ) 0100 ONCE/TWICE A MONTH

 004 R810906D (04 ) 0010 NEVER OR HARDLY EVER

 005 R810906M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0014

 DESCRIPTION: ASKED TO DO GROUP PROJECT ABOUT WHAT YOU READ

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R811005 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 R811005A (01 ) 0000 ALMOST EVERY DAY

 002 R811005B (02 ) 1000 ONCE/TWICE A WEEK

 003 R811005C (03 ) 0100 ONCE/TWICE A MONTH

 004 R811005D (04 ) 0010 NEVER OR HARDLY EVER

 005 R811005M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0015

 DESCRIPTION: ASKED TO READ ALOUD

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R811006 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 R811006A (01 ) 0000 ALMOST EVERY DAY

 002 R811006B (02 ) 1000 ONCE/TWICE A WEEK

 003 R811006C (03 ) 0100 ONCE/TWICE A MONTH

 004 R811006D (04 ) 0010 NEVER OR HARDLY EVER

 005 R811006M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0016

 DESCRIPTION: ASKED TO READ SILENTLY

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R811007 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 R811007A (01 ) 0000 ALMOST EVERY DAY

 002 R811007B (02 ) 1000 ONCE/TWICE A WEEK

 003 R811007C (03 ) 0100 ONCE/TWICE A MONTH

 004 R811007D (04 ) 0010 NEVER OR HARDLY EVER

 005 R811007M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0017

 DESCRIPTION: GIVEN TIME TO READ BOOKS YOU HAVE CHOSEN

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R811009 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 R811009A (01 ) 0000 ALMOST EVERY DAY

 002 R811009B (02 ) 1000 ONCE/TWICE A WEEK

 003 R811009C (03 ) 0100 ONCE/TWICE A MONTH

 004 R811009D (04 ) 0010 NEVER OR HARDLY EVER

 005 R811009M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0018

 DESCRIPTION: ASKED TO TALK W/STUDENTS ABOUT WHAT YOU READ

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R811002 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 R811002A (01 ) 0000 ALMOST EVERY DAY

 002 R811002B (02 ) 1000 ONCE/TWICE A WEEK

 003 R811002C (03 ) 0100 ONCE/TWICE A MONTH

 004 R811002D (04 ) 0010 NEVER OR HARDLY EVER

 005 R811002M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0019

 DESCRIPTION: ASKED TO WRITE ABOUT WHAT YOU READ

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R811004 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 R811004A (01 ) 0000 ALMOST EVERY DAY

 002 R811004B (02 ) 1000 ONCE/TWICE A WEEK

 003 R811004C (03 ) 0100 ONCE/TWICE A MONTH

 004 R811004D (04 ) 0010 NEVER OR HARDLY EVER

 005 R811004M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0020

 DESCRIPTION: TEACHER HELPS YOU BREAK WORDS INTO PARTS

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R818101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 R818101A (01 ) 0000 ALMOST EVERY DAY

 002 R818101B (02 ) 1000 ONCE/TWICEA WEEK

 003 R818101C (03 ) 0100 ONCE/TWICE A MONTH

 004 R818101D (04 ) 0010 NEVER OR HARDLY EVER

 005 R818101M (M ) 0001 MISSING
Table F-5 (continued)

1998 Reading Conditioning Variable Specifications

660

 CONDITIONING VARIABLE ID: SUBJ0021

 DESCRIPTION: TEACHER HELPS YOU UNDERSTAND NEW WORDS

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R818102 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 R818102A (01 ) 0000 ALMOST EVERY DAY

 002 R818102B (02 ) 1000 ONCE/TWICE A WEEK

 003 R818102C (03 ) 0100 ONCE/TWICE A MONTH

 004 R818102D (04 ) 0010 NEVER OR HARDLY EVER

 005 R818102M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0022

 DESCRIPTION: DO YOU AND TEACHER REVIEW PROGRESS IN READING

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R830001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 R830001Y (01 ) 00 YES

 002 R830001N (02 ) 10 NO

 003 R830001M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SUBJ0023

 DESCRIPTION: IS THERE A SCHOOL/PUBLIC LIBRARY AVAILABLE

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: R830101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 R830101Y (01 ) 00 YES

 002 R830101N (02 ) 10 NO

 003 R830101M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SUBJ0024

 DESCRIPTION: USE LIBRARY TO DO RESEARCH FOR SCHOOL ASSIGNMENT

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R811301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 R811301A (01 ) 00000 ALMOST EVERY DAY

 002 R811301B (02 ) 10000 ONCE/TWICE A WEEK

 003 R811301C (03 ) 01000 ONCE/TWICE A MONTH

 004 R811301D (04 ) 00100 ONCE/TWICE A YEAR

 005 R811301E (05 ) 00010 NEVER OR HARDLY EVER

 006 R811301M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0025

 DESCRIPTION: USE LIBRARY TO BORROW BOOKS FOR SCHOOL

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R811302 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 R811302A (01 ) 00000 ALMOST EVERY DAY

 002 R811302B (02 ) 10000 ONCE/TWICE A WEEK

 003 R811302C (03 ) 01000 ONCE/TWICE A MONTH

 004 R811302D (04 ) 00100 ONCE/TWICE A YEAR

 005 R811302E (05 ) 00010 NEVER OR HARDLY EVER

 006 R811302M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0026

 DESCRIPTION: USE LIBRARY TO USE A COMPUTER

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R811303 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 R811303A (01 ) 00000 ALMOST EVERY DAY

 002 R811303B (02 ) 10000 ONCE/TWICE A WEEK

 003 R811303C (03 ) 01000 ONCE/TWICE A MONTH

 004 R811303D (04 ) 00100 ONCE/TWICE A YEAR

 005 R811303E (05 ) 00010 NEVER OR HARDLY EVER

 006 R811303M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0027

 DESCRIPTION: USE LIBRARY AS A QUIET PLACE TO STUDY

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R811304 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 R811304A (01 ) 00000 ALMOST EVERY DAY

 002 R811304B (02 ) 10000 ONCE/TWICE A WEEK

 003 R811304C (03 ) 01000 ONCE/TWICE A MONTH

 004 R811304D (04 ) 00100 ONCE/TWICE A YEAR

 005 R811304E (05 ) 00010 NEVER OR HARDLY EVER

 006 R811304M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0001

 DESCRIPTION: FOURTH GRADERS ASSIGNED TO CLASS BY ABILITY

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C042501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C042501Y (01 ) 00 YES

 002 C042501N (02 ) 10 NO

 003 C042501M (M ) 01 MISSING
Table F-5 (continued)

 1998 Reading Conditioning Variable Specification

661

 CONDITIONING VARIABLE ID: SCHL0002

 DESCRIPTION: HOW OFTEN STUDENTS RECEIVE READING INSTRUCTION

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C042601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C042601A (01 ) 00000 EVERY DAY

 002 C042601B (02 ) 10000 3-4 TIMES A WEEK

 003 C042601C (03 ) 01000 ONCE OR TWICE A WEEK

 004 C042601D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C042601N (05 ) 00010 SUBJECT NOT TAUGHT

 006 C042601M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0003

 DESCRIPTION: HOW OFTEN STUDENTS RECEIVE WRITING INSTRUCTION

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C042602 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C042602A (01 ) 00000 EVERY DAY

 002 C042602B (02 ) 10000 3-4 TIMES A WEEK

 003 C042602C (03 ) 01000 ONCE OR TWICE A WEEK

 004 C042602D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C042602N (05 ) 00010 SUBJECT NOT TAUGHT

 006 C042602M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0004

 DESCRIPTION: HOW OFTEN STUDENTS RECEIVE SOC STUDIES INSTRUCT

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C042603 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C042603A (01 ) 00000 EVERY DAY

 002 C042603B (02 ) 10000 3-4 TIMES A WEEK

 003 C042603C (03 ) 01000 ONCE OR TWICE A WEEK

 004 C042603D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C042603N (05 ) 00010 SUBJECT NOT TAUGHT

 006 C042603M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0005

 DESCRIPTION: HOW OFTEN STUDENTS RECEIVE COMPUTER USE INSTRUCT

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C042604 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C042604A (01 ) 00000 EVERY DAY

 002 C042604B (02 ) 10000 3-4 TIMES A WEEK

 003 C042604C (03 ) 01000 ONCE OR TWICE A WEEK

 004 C042604D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C042604N (05 ) 00010 SUBJECT NOT TAUGHT

 006 C042604M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0006

 DESCRIPTION: DOES SCHOOL USE BLOCK SCHEDULING

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C042701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 C042701Y (01 ) 000 YES-ALL SUBJECTS

 002 C042701Y (02 ) 100 YES-SOME SUBJECTS

 003 C042701N (03 ) 010 NO

 004 C042701M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SCHL0007

 DESCRIPTION: ARE COMPUTERS AVAILABLE IN ALL CLASSROOMS

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C042801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C042801Y (01 ) 00 YES

 002 C042801N (02 ) 10 NO

 003 C042801M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0008

 DESCRIPTION: ARE COMPUTERS AVAILABLE IN COMPUTER LAB

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C042802 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C042802Y (01 ) 00 YES

 002 C042802N (02 ) 10 NO

 003 C042802M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0009

 DESCRIPTION: ARE COMPUTERS AVAILABLE TO CLASSROOM WHEN NEEDED

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C042803 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C042803Y (01 ) 00 YES

 002 C042803N (02 ) 10 NO

 003 C042803M (M ) 01 MISSING
Table F-5 (continued)

1998 Reading Conditioning Variable Specifications

662

 CONDITIONING VARIABLE ID: SCHL0010

 DESCRIPTION: HOW MANY COMPUTERS AVAILABLE TO STUDENTS

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C042901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 C042901N (01 ) 0000000 NONE

 002 C042901B (02 ) 1000000 1-10

 003 C042901C (03 ) 0100000 11-25

 004 C042901D (04 ) 0010000 26-50

 005 C042901E (05 ) 0001000 51-75

 006 C042901F (06 ) 0000100 76-100

 007 C042901G (07 ) 0000010 MORE THAN 100

 008 C042901M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0011

 DESCRIPTION: PRIMARY WAY LIBRARY IS STAFFED

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C036601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C036601N (01 ) 0000 NO LIBRARY IN SCHOOL

 002 C036601N (02 ) 1000 LIBRARY-NO/VOL STAFF

 003 C036601C (03 ) 0100 PART-TIME STAFF

 004 C036601D (04 ) 0010 FULL-TIME STAFF

 005 C036601M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0012

 DESCRIPTION: PARENTS PARTICIPATE-PARENT-TEACHER ORG

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043001A (01 ) 00000 NOT AVAILABLE

 002 C043001B (02 ) 10000 0-10%

 003 C043001C (03 ) 01000 11-25%

 004 C043001D (04 ) 00100 26-50%

 005 C043001E (05 ) 00010 51-100%

 006 C043001M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0013

 DESCRIPTION: PARENTS PARTICIPATE-OPEN HOUSE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043002 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043002A (01 ) 00000 NOT AVAILABLE

 002 C043002B (02 ) 10000 0-10%

 003 C043002C (03 ) 01000 11-25%

 004 C043002D (04 ) 00100 26-50%

 005 C043002E (05 ) 00010 51-100%

 006 C043002M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0014

 DESCRIPTION: PARTICIPATE-PARENT-TEACHER CONFERENCE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043003 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043003A (01 ) 00000 NOT AVAILABLE

 002 C043003B (02 ) 10000 0-10%

 003 C043003C (03 ) 01000 11-25%

 004 C043003D (04 ) 00100 26-50%

 005 C043003E (05 ) 00010 51-100%

 006 C043003M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0015

 DESCRIPTION: PARENTS PARTICIPATE-SCHOOL CURRICULUM DECISIONS

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043004 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043004A (01 ) 00000 NOT AVAILABLE

 002 C043004B (02 ) 10000 0-10%

 003 C043004C (03 ) 01000 11-25%

 004 C043004D (04 ) 00100 26-50%

 005 C043004E (05 ) 00010 51-100%

 006 C043004M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0016

 DESCRIPTION: PARENTS PARTICIPATE-VOLUNTEER PROGRAMS

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043005 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043005A (01 ) 00000 NOT AVAILABLE

 002 C043005B (02 ) 10000 0-10%

 003 C043005C (03 ) 01000 11-25%

 004 C043005D (04 ) 00100 26-50%

 005 C043005E (05 ) 00010 51-100%

 006 C043005M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0017

 DESCRIPTION: PARENTS PARTICIPATE-PARENTING-SKILLS PROGRAM

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043006 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043006A (01 ) 00000 NOT AVAILABLE

 002 C043006B (02 ) 10000 0-10%

 003 C043006C (03 ) 01000 11-25%

 004 C043006D (04 ) 00100 26-50%

 005 C043006E (05 ) 00010 51-100%

 006 C043006M (M ) 00001 MISSING
Table F-5 (continued)

 1998 Reading Conditioning Variable Specification

663

 CONDITIONING VARIABLE ID: SCHL0018

 DESCRIPTION: PARENTS PARTICIPATE-SCHOOL ADVISORY COMMITTEES

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043007 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043007A (01 ) 00000 NOT AVAILABLE

 002 C043007B (02 ) 10000 0-10%

 003 C043007C (03 ) 01000 11-25%

 004 C043007D (04 ) 00100 26-50%

 005 C043007E (05 ) 00010 51-100%

 006 C043007M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0019

 DESCRIPTION: PARENTS PARTICIPATE-CLASSROOM ASSISTANTS

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043008 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043008A (01 ) 00000 NOT AVAILABLE

 002 C043008B (02 ) 10000 0-10%

 003 C043008C (03 ) 01000 11-25%

 004 C043008D (04 ) 00100 26-50%

 005 C043008E (05 ) 00010 51-100%

 006 C043008M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0020

 DESCRIPTION: IS STUDENT ABSENTEEISM A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032402 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032402A (01 ) 0000 SERIOUS

 002 C032402B (02 ) 1000 MODERATE

 003 C032402C (03 ) 0100 MINOR

 004 C032402N (04 ) 0010 NOT A PROBLEM

 005 C032402M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0021

 DESCRIPTION: IS STUDENT TARDINESS A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032401A (01 ) 0000 SERIOUS

 002 C032401B (02 ) 1000 MODERATE

 003 C032401C (03 ) 0100 MINOR

 004 C032401N (04 ) 0010 NOT A PROBLEM

 005 C032401M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0022

 DESCRIPTION: ARE PHYSICAL CONFLICTS A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032404 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032404A (01 ) 0000 SERIOUS

 002 C032404B (02 ) 1000 MODERATE

 003 C032404C (03 ) 0100 MINOR

 004 C032404N (04 ) 0010 NOT A PROBLEM

 005 C032404M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0023

 DESCRIPTION: ARE RACIAL/CULT. CONFLICTS A PROBLEM IN SCHOOL

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032407 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032407A (01 ) 0000 SERIOUS

 002 C032407B (02 ) 1000 MODERATE

 003 C032407C (03 ) 0100 MINOR

 004 C032407N (04 ) 0010 NOT A PROBLEM

 005 C032407M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0024

 DESCRIPTION: IS STUDENT HEALTH A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032408 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032408A (01 ) 0000 SERIOUS

 002 C032408B (02 ) 1000 MODERATE

 003 C032408C (03 ) 0100 MINOR

 004 C032408N (04 ) 0010 NOT A PROBLEM

 005 C032408M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0025

 DESCRIPTION: IS LACK OF PARENT INVLVMNT A PROBLEM IN SCHOOL

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032409 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032409A (01 ) 0000 SERIOUS

 002 C032409B (02 ) 1000 MODERATE

 003 C032409C (03 ) 0100 MINOR

 004 C032409N (04 ) 0010 NOT A PROBLEM

 005 C032409M (M ) 0001 MISSING
Table F-5 (continued)

1998 Reading Conditioning Variable Specifications

664

 CONDITIONING VARIABLE ID: SCHL0026

 DESCRIPTION: IS STUDENT ALCOHOL USE A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032410 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032410A (01 ) 0000 SERIOUS

 002 C032410B (02 ) 1000 MODERATE

 003 C032410C (03 ) 0100 MINOR

 004 C032410N (04 ) 0010 NOT A PROBLEM

 005 C032410M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0027

 DESCRIPTION: IS STUDENT TOBACCO USE A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032411 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032411A (01 ) 0000 SERIOUS

 002 C032411B (02 ) 1000 MODERATE

 003 C032411C (03 ) 0100 MINOR

 004 C032411N (04 ) 0010 NOT A PROBLEM

 005 C032411M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0028

 DESCRIPTION: IS STUDENT DRUG USE A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032412 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032412A (01 ) 0000 SERIOUS

 002 C032412B (02 ) 1000 MODERATE

 003 C032412C (03 ) 0100 MINOR

 004 C032412N (04 ) 0010 NOT A PROBLEM

 005 C032412M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0029

 DESCRIPTION: ARE GANG ACTIVITIES A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032413 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032413A (01 ) 0000 SERIOUS

 002 C032413B (02 ) 1000 MODERATE

 003 C032413C (03 ) 0100 MINOR

 004 C032413N (04 ) 0010 NOT A PROBLEM

 005 C032413M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0030

 DESCRIPTION: IS STUDENT MISBEHAVIOR A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032414 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032414A (01 ) 0000 SERIOUS

 002 C032414B (02 ) 1000 MODERATE

 003 C032414C (03 ) 0100 MINOR

 004 C032414N (04 ) 0010 NOT A PROBLEM

 005 C032414M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0031

 DESCRIPTION: IS STUDENT CHEATING A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C043101A (01 ) 0000 SERIOUS

 002 C043101B (02 ) 1000 MODERATE

 003 C043101C (03 ) 0100 MINOR

 004 C043101N (04 ) 0010 NOT A PROBLEM

 005 C043101M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0032

 DESCRIPTION: IS TEACHER ABSENTEEISM A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043102 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C043102A (01 ) 0000 SERIOUS

 002 C043102B (02 ) 1000 MODERATE

 003 C043102C (03 ) 0100 MINOR

 004 C043102N (04 ) 0010 NOT A PROBLEM

 005 C043102M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0033

 DESCRIPTION: ARE PHYSICAL CONFLICTS BETWEEN STUDENTS/TEACHERS

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043103 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C043103A (01 ) 0000 SERIOUS

 002 C043103B (02 ) 1000 MODERATE

 003 C043103C (03 ) 0100 MINOR

 004 C043103N (04 ) 0010 NOT A PROBLEM

 005 C043103M (M ) 0001 MISSING
Table F-5 (continued)

 1998 Reading Conditioning Variable Specification

665

 CONDITIONING VARIABLE ID: SCHL0034

 DESCRIPTION: IS VANDALISM A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043104 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C043104A (01 ) 0000 SERIOUS

 002 C043104B (02 ) 1000 MODERATE

 003 C043104C (03 ) 0100 MINOR

 004 C043104N (04 ) 0010 NOT A PROBLEM

 005 C043104M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0035

 DESCRIPTION: TEACHER MORALE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032502 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032502A (01 ) 0000 VERY POSITIVE

 002 C032502B (02 ) 1000 SOMEWHAT POSITIVE

 003 C032502C (03 ) 0100 SOMEWHAT NEGATIVE

 004 C032502D (04 ) 0010 VERY NEGATIVE

 005 C032502M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0036

 DESCRIPTION: STUDENT ATTITUDES TOWARD ACADEMIC ACHIEVEMENT

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032503 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032503A (01 ) 0000 VERY POSITIVE

 002 C032503B (02 ) 1000 SOMEWHAT POSITIVE

 003 C032503C (03 ) 0100 SOMEWHAT NEGATIVE

 004 C032503D (04 ) 0010 VERY NEGATIVE

 005 C032503M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0037

 DESCRIPTION: PARENT SUPPORT FOR STUDENT ACHIEVEMENT

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032505 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032505A (01 ) 0000 VERY POSITIVE

 002 C032505B (02 ) 1000 SOMEWHAT POSITIVE

 003 C032505C (03 ) 0100 SOMEWHAT NEGATIVE

 004 C032505D (04 ) 0010 VERY NEGATIVE

 005 C032505M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0038

 DESCRIPTION: REGARD FOR SCHOOL PROPERTY

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032506 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032506A (01 ) 0000 VERY POSITIVE

 002 C032506B (02 ) 1000 SOMEWHAT POSITIVE

 003 C032506C (03 ) 0100 SOMEWHAT NEGATIVE

 004 C032506D (04 ) 0010 VERY NEGATIVE

 005 C032506M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0039

 DESCRIPTION: TEACHERS’ EXPECTATIONS FOR STUDENT ACHIEVEMENT

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C043201A (01 ) 0000 VERY POSITIVE

 002 C043201B (02 ) 1000 SOMEWHAT POSITIVE

 003 C043201C (03 ) 0100 SOMEWHAT NEGATIVE

 004 C043201D (04 ) 0010 VERY NEGATIVE

 005 C043201M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0040

 DESCRIPTION: PERCENT STUDENT BODY ABSENT AVERAGE DAY

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 C043301A (01 ) 000000 0-2%

 002 C043301B (02 ) 100000 3-5%

 003 C043301C (03 ) 010000 6-10%

 004 C043301D (04 ) 001000 11-25%

 005 C043301E (05 ) 000100 26-50%

 006 C043301F (06 ) 000010 MORE THAN 50%

 007 C043301M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0041

 DESCRIPTION: PERCENT TEACHING STAFF ABSENT AVERAGE DAY

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 C043401A (01 ) 000000 0-2%

 002 C043401B (02 ) 100000 3-5%

 003 C043401C (03 ) 010000 6-10%

 004 C043401D (04 ) 001000 11-25%

 005 C043401E (05 ) 000100 26-50%

 006 C043401F (06 ) 000010 MORE THAN 50%

 007 C043401M (M ) 000001 MISSING
Table F-5 (continued)

1998 Reading Conditioning Variable Specifications

666

 CONDITIONING VARIABLE ID: SCHL0042

 DESCRIPTION: ENROLLMENT LAST YEAR COMPARED TO END OF SCHOOL YR

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 C043501A (01 ) 000000 98-100%

 002 C043501B (02 ) 100000 95-97%

 003 C043501C (03 ) 010000 90-94%

 004 C043501D (04 ) 001000 80-89%

 005 C043501E (05 ) 000100 70-79%

 006 C043501F (06 ) 000010 LESS THAN 70%

 007 C043501M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0043

 DESCRIPTION: PERCENT STUDENTS HELD BACK AND REPEATING GRADE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043601A (01 ) 00000 0%

 002 C043601B (02 ) 10000 1-2%

 003 C043601C (03 ) 01000 3-5%

 004 C043601D (04 ) 00100 6-10%

 005 C043601E (05 ) 00010 MORE THAN 10%

 006 C043601M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0044

 DESCRIPTION: PERCENT TEACHING STAFF LEFT BEFORE END OF YEAR

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043701A (01 ) 00000 0%

 002 C043701B (02 ) 10000 1-2%

 003 C043701C (03 ) 01000 3-5%

 004 C043701D (04 ) 00100 6-10%

 005 C043701E (05 ) 00010 MORE THAN 10%

 006 C043701M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0045

 DESCRIPTION: IS SCHOOL IN NATIONAL SCHOOL LUNCH PROGRAM

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C038301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C038301Y (01 ) 00 YES

 002 C038301N (02 ) 10 NO

 003 C038301M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0046

 DESCRIPTION: PERCENT ELIGIBLE NATIONAL SCHOOL LUNCH PROGRAM

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 8

 001 C043801A (01 ) 00000000 0%

 002 C043801B (02 ) 10000000 1-5%

 003 C043801C (03 ) 01000000 6-10%

 004 C043801D (04 ) 00100000 11-25%

 005 C043801E (05 ) 00010000 26-50%

 006 C043801F (06 ) 00001000 51-75%

 007 C043801G (07 ) 00000100 76-99%

 008 C043801H (08 ) 00000010 100%

 009 C043801M (M ) 00000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0047

 DESCRIPTION: DOES SCHOOL RECEIVE CHAPTER 1/TITLE I FUNDING

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C043901Y (01 ) 00 YES

 002 C043901N (02 ) 10 NO

 003 C043901M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0048

 DESCRIPTION: PERCENT STUDENTS RECEIVE CHAPTER1/TITLE I FUNDING

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 8

 001 C044001N (01 ) 00000000 NONE

 002 C044001B (02 ) 10000000 1-5%

 003 C044001C (03 ) 01000000 6-10%

 004 C044001D (04 ) 00100000 11-25%

 005 C044001E (05 ) 00010000 26-50%

 006 C044001F (06 ) 00001000 51-75%

 007 C044001G (07 ) 00000100 76-90%

 008 C044001H (08 ) 00000010 OVER 90%

 009 C044001M (M ) 00000001 MISSING
Table F-5 (continued)

 1998 Reading Conditioning Variable Specification

667

 CONDITIONING VARIABLE ID: SCHL0049

 DESCRIPTION: PERCENT STUDENTS RECEIVE REMEDIAL READING INSTRUCT

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044002 TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 8

 001 C044002N (01 ) 00000000 NONE

 002 C044002B (02 ) 10000000 1-5%

 003 C044002C (03 ) 01000000 6-10%

 004 C044002D (04 ) 00100000 11-25%

 005 C044002E (05 ) 00010000 26-50%

 006 C044002F (06 ) 00001000 51-75%

 007 C044002G (07 ) 00000100 76-90%

 008 C044002H (08 ) 00000010 OVER 90%

 009 C044002M (M ) 00000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0050

 DESCRIPTION: PERCENT STUDENTS RECEIVE REMEDIAL WRITING INSTRUCT

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044003 TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 8

 001 C044003N (01 ) 00000000 NONE

 002 C044003B (02 ) 10000000 1-5%

 003 C044003C (03 ) 01000000 6-10%

 004 C044003D (04 ) 00100000 11-25%

 005 C044003E (05 ) 00010000 26-50%

 006 C044003F (06 ) 00001000 51-75%

 007 C044003G (07 ) 00000100 76-90%

 008 C044003H (08 ) 00000010 OVER 90%

 009 C044003M (M ) 00000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0051

 DESCRIPTION: PERCENT STUDENTS IN GIFTED AND TALENTED PROGRAM

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044004 TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 8

 001 C044004N (01 ) 00000000 NONE

 002 C044004B (02 ) 10000000 1-5%

 003 C044004C (03 ) 01000000 6-10%

 004 C044004D (04 ) 00100000 11-25%

 005 C044004E (05 ) 00010000 26-50%

 006 C044004F (06 ) 00001000 51-75%

 007 C044004G (07 ) 00000100 76-90%

 008 C044004H (08 ) 00000010 OVER 90%

 009 C044004M (M ) 00000001 MISSING

 CONDITIONING VARIABLE ID: BACK0062

 DESCRIPTION: MOTHER’S EDUCATION LEVEL

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B003501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 B003501A (01 ) 00000 DID NOT FINISH HS

 002 B003501B (02 ) 10000 GRADUATED HS

 003 B003501C (03 ) 01000 SOME ED AFTER HS

 004 B003501D (04 ) 00100 GRADUATED COLLEGE

 005 B003501M (M, IDK ) 00001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0063

 DESCRIPTION: FATHER’S EDUCATION LEVEL

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B003601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 B003601A (01 ) 00000 DID NOT FINISH HS

 002 B003601B (02 ) 10000 GRADUATED HS

 003 B003601C (03 ) 01000 SOME ED AFTER HS

 004 B003601D (04 ) 00100 GRADUATED COLLEGE

 005 B003601M (M, IDK ) 00001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0028

 DESCRIPTION: ASKED TO EXPLAIN UNDERSTANDING OF WHAT YOU READ

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R811010 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 R811010A (01 ) 0000 ALMOST EVERY DAY

 002 R811010B (02 ) 1000 ONCE/TWICE A WEEK

 003 R811010C (03 ) 0100 ONCE/TWICE A MONTH

 004 R811010D (04 ) 0010 NEVER OR HARDLY EVER

 005 R811010M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0029

 DESCRIPTION: ASKED TO DISCUSS INTERPRETATIONS OF WHAT YOU READ

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R811011 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 R811011A (01 ) 0000 ALMOST EVERY DAY

 002 R811011B (02 ) 1000 ONCE/TWICE A WEEK

 003 R811011C (03 ) 0100 ONCE/TWICE A MONTH

 004 R811011D (04 ) 0010 NEVER OR HARDLY EVER

 005 R811011M (M ) 0001 MISSING
Table F-5 (continued)

1998 Reading Conditioning Variable Specifications

668

 CONDITIONING VARIABLE ID: SUBJ0030

 DESCRIPTION: DO YOU HAVE ACCESS TO A SCHOOL/PUBLIC LIBRARY

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: R830201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 R830201Y (01 ) 00 YES

 002 R830201N (02 ) 10 NO

 003 R830201M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0052

 DESCRIPTION: 8TH GRADE ASSIGNED TO ENGLISH CLASS BY ABILITY

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C044401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C044401Y (01 ) 00 YES

 002 C044401N (02 ) 10 NO

 003 C044401M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0053

 DESCRIPTION: 8TH GRADE ASSIGNED-HISTORY/SS BY ABILITY

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C044402 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C044402Y (01 ) 00 YES

 002 C044402N (02 ) 10 NO

 003 C044402M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0054

 DESCRIPTION: IS STUDENT DROPOUT A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043105 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C043105A (01 ) 0000 SERIOUS

 002 C043105B (02 ) 1000 MODERATE

 003 C043105C (03 ) 0100 MINOR

 004 C043105N (04 ) 0010 NOT A PROBLEM

 005 C043105M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0055

 DESCRIPTION: IS TEEN PREGNANCY A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043106 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C043106A (01 ) 0000 SERIOUS

 002 C043106B (02 ) 1000 MODERATE

 003 C043106C (03 ) 0100 MINOR

 004 C043106N (04 ) 0010 NOT A PROBLEM

 005 C043106M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0064

 DESCRIPTION: MAIN ACTIVITY YEAR FOLLOWING HIGH SCHOOL

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: B005501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 B005501A (01 ) 000000 WORK FULL-TIME

 002 B005501B (02 ) 100000 VOCA/TECH/BUSINESS

 003 B005501C (03 ) 010000 ATTEND 2 YR COLLEGE

 004 B005501D (04 ) 001000 ATTEND 4 YR COLLEGE

 005 B005501E (05 ) 000100 SERVE IN MILITARY

 006 B005501F (06 ) 000010 OTHER

 007 B005501M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0031

 DESCRIPTION: ENROLLED IN OR TOOK AN AP ENGLISH COURSE

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: R820201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 R820201Y (01 ) 00 YES

 002 R820201N (02 ) 10 NO

 003 R820201M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0056

 DESCRIPTION: 12TH GRADE ASSIGNED TO ENGLISH CLASS BY ABILITY

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C044301Y (01 ) 00 YES

 002 C044301N (02 ) 10 NO

 003 C044301M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0057

 DESCRIPTION: 12TH GR ASSIGNED- HISTORY/CIVICS/SS CLASS ABILITY

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044302 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C044302Y (01 ) 00 YES

 002 C044302N (02 ) 10 NO

 003 C044302M (M ) 01 MISSING
Table F-5 (continued)

 1998 Reading Conditioning Variable Specification

669

 CONDITIONING VARIABLE ID: SCHL0058

 DESCRIPTION: PERCENT LAST YEAR’S TWELFTH-GRADE CLASS GRADUATED

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C044101A (01 ) 00000 99-100%

 002 C044101B (02 ) 10000 95-98%

 003 C044101C (03 ) 01000 90-94%

 004 C044101D (04 ) 00100 75-89%

 005 C044101E (05 ) 00010 LESS THAN 75%

 006 C044101M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0059

 DESCRIPTION: PERCENT GRADUATING CLASS-ATTEND TWO-YEAR COLLEGE

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 8

 001 C044201N (01 ) 00000000 NONE

 002 C044201B (02 ) 10000000 1-5%

 003 C044201C (03 ) 01000000 6-10%

 004 C044201D (04 ) 00100000 11-25%

 005 C044201E (05 ) 00010000 26-50%

 006 C044201F (06 ) 00001000 51-75%

 007 C044201G (07 ) 00000100 76-90%

 008 C044201H (08 ) 00000010 OVER 100%

 009 C044201M (M ) 00000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0060

 DESCRIPTION: PERCENT GRADUATING CLASS-ATTEND FOUR-YEAR COLLEGE

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044202 TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 8

 001 C044202N (01 ) 00000000 NONE

 002 C044202B (02 ) 10000000 1-5%

 003 C044202C (03 ) 01000000 6-10%

 004 C044202D (04 ) 00100000 11-25%

 005 C044202E (05 ) 00010000 26-50%

 006 C044202F (06 ) 00001000 51-75%

 007 C044202G (07 ) 00000100 76-90%

 008 C044202H (08 ) 00000010 OVER 100%

 009 C044202M (M ) 00000001 MISSING

 009 C044202M (M ) 00000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0001

 DESCRIPTION: DO YOU TEACH READING

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T067001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T067001Y (01 ) 0 YES

 002 T067001M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0002

 DESCRIPTION: DO YOU TEACH WRITING

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T067002 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T067002Y (01 ) 0 YES

 002 T067002M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0003

 DESCRIPTION: DO YOU TEACH LANGUAGE ARTS

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T067003 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T067003Y (01 ) 0 YES

 002 T067003M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0004

 DESCRIPTION: DO YOU TEACH SOCIAL STUDIES

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T067004 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T067004Y (01 ) 0 YES

 002 T067004M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0005

 DESCRIPTION: YEARS TOTAL TAUGHT ELEMENTARY LEVEL

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T067101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067101A (01 ) 00000 2 YEARS OR LESS

 002 T067101B (02 ) 10000 3-5 YEARS

 003 T067101C (03 ) 01000 6-10 YEARS

 004 T067101D (04 ) 00100 11-24 YEARS

 005 T067101E (05 ) 00010 25 YEARS OR MORE

 006 T067101M (M ) 00001 MISSING
Table F-5 (continued)

1998 Reading Conditioning Variable Specifications

670

 CONDITIONING VARIABLE ID: TCHR0006

 DESCRIPTION: YEARS TOTAL TAUGHT READING

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T067201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067201A (01 ) 00000 2 YEARS OR LESS

 002 T067201B (02 ) 10000 3-5 YEARS

 003 T067201C (03 ) 01000 6-10 YEARS

 004 T067201D (04 ) 00100 11-24 YEARS

 005 T067201E (05 ) 00010 25 YEARS OR MORE

 006 T067201M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0007

 DESCRIPTION: YEARS TOTAL TAUGHT WRITING

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T067202 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067202A (01 ) 00000 2 YEARS OR LESS

 002 T067202B (02 ) 10000 3-5 YEARS

 003 T067202C (03 ) 01000 6-10 YEARS

 004 T067202D (04 ) 00100 11-24 YEARS

 005 T067202E (05 ) 00010 25 YEARS OR MORE

 006 T067202M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0008

 DESCRIPTION: YEARS TOTAL TAUGHT LANGUAGE ARTS

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T067203 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067203A (01 ) 00000 2 YEARS OR LESS

 002 T067203B (02 ) 10000 3-5 YEARS

 003 T067203C (03 ) 01000 6-10 YEARS

 004 T067203D (04 ) 00100 11-24 YEARS

 005 T067203E (05 ) 00010 25 YEARS OR MORE

 006 T067203M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0009

 DESCRIPTION: YEARS TOTAL TAUGHT HISTORY

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T067204 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067204A (01 ) 00000 2 YEARS OR LESS

 002 T067204B (02 ) 10000 3-5 YEARS

 003 T067204C (03 ) 01000 6-10 YEARS

 004 T067204D (04 ) 00100 11-24 YEARS

 005 T067204E (05 ) 00010 25 YEARS OR MORE

 006 T067204M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0010

 DESCRIPTION: YEARS TOTAL TAUGHT SOCIAL STUDIES

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T067205 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067205A (01 ) 00000 2 YEARS OR LESS

 002 T067205B (02 ) 10000 3-5 YEARS

 003 T067205C (03 ) 01000 6-10 YEARS

 004 T067205D (04 ) 00100 11-24 YEARS

 005 T067205E (05 ) 00010 25 YEARS OR MORE

 006 T067205M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0011

 DESCRIPTION: YEARS TOTAL TAUGHT CIVICS

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T067206 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067206A (01 ) 00000 2 YEARS OR LESS

 002 T067206B (02 ) 10000 3-5 YEARS

 003 T067206C (03 ) 01000 6-10 YEARS

 004 T067206D (04 ) 00100 11-24 YEARS

 005 T067206E (05 ) 00010 25 YEARS OR MORE

 006 T067206M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0012

 DESCRIPTION: MAIN ASSIGNMENT FIELD

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T067301A (01 ) 0000 REGULAR CLASSROOM

 002 T067301B (02 ) 1000 SPECIAL CLASSROOM

 003 T067301C (03 ) 0100 ESL/BILINGUAL ED

 004 T067301D (04 ) 0010 OTHER

 005 T067301M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0013

 DESCRIPTION: TEACHING CERTIF IN THIS STATE IN MAIN FIELD

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 T056201A (01 ) 000000 ADVANCED PROFESSIONL

 002 T056201B (02 ) 100000 REGULAR/STANDARD ST

 003 T056201C (03 ) 010000 PROBATIONARY STATE

 004 T056201D (04 ) 001000 TEMPORARY/PROVISIONL

 005 T056201E (05 ) 000100 OTHER THAN STATE CRT

 006 T056201F (06 ) 000010 NOT HAVE CERT MAIN

 007 T056201M (M ) 000001 MISSING
Table F-5 (continued)

 1998 Reading Conditioning Variable Specification

671

 CONDITIONING VARIABLE ID: TCHR0014

 DESCRIPTION: HIGHEST ACADEMIC DEGREE YOU HOLD

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 T056301A (01 ) 0000000 HIGH SCHOOL DIPLOMA

 002 T056301B (02 ) 1000000 ASSOCIATES/VOCATIONL

 003 T056301C (03 ) 0100000 BACHELOR’S DEGREE

 004 T056301D (04 ) 0010000 MASTER’S DEGREE

 005 T056301E (05 ) 0001000 EDUCATION SPECIALIST

 006 T056301F (06 ) 0000100 DOCTORATE

 007 T056301G (07 ) 0000010 PROFESSIONAL DEGREE

 008 T056301M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0015

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-ELEMENTARY EDUCATION

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067501A (01 ) 000 MAJOR

 002 T067501B (02 ) 100 MINOR

 003 T067501C (03 ) 010 NOT IN THIS SUBJECT

 004 T067501M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0016

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-SECONDARY EDUCATION

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067502 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067502A (01 ) 000 MAJOR

 002 T067502B (02 ) 100 MINOR

 003 T067502C (03 ) 010 NOT IN THIS SUBJECT

 004 T067502M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0017

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-SPECIAL EDUCATION

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067503 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067503A (01 ) 000 MAJOR

 002 T067503B (02 ) 100 MINOR

 003 T067503C (03 ) 010 NOT IN THIS SUBJECT

 004 T067503M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0018

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-BILINGUAL EDUCATION/ESL

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067504 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067504A (01 ) 000 MAJOR

 002 T067504B (02 ) 100 MINOR

 003 T067504C (03 ) 010 NOT IN THIS SUBJECT

 004 T067504M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0019

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-ADMINISTRATION & SUPERVISION

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067505 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067505A (01 ) 000 MAJOR

 002 T067505B (02 ) 100 MINOR

 003 T067505C (03 ) 010 NOT IN THIS SUBJECT

 004 T067505M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0020

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-CURRICULUM & SUPERVISION

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067506 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067506A (01 ) 000 MAJOR

 002 T067506B (02 ) 100 MINOR

 003 T067506C (03 ) 010 NOT IN THIS SUBJECT

 004 T067506M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0021

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-COUNSELING

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067507 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067507A (01 ) 000 MAJOR

 002 T067507B (02 ) 100 MINOR

 003 T067507C (03 ) 010 NOT IN THIS SUBJECT

 004 T067507M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0022

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-ENGLISH

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067508 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067508A (01 ) 000 MAJOR

 002 T067508B (02 ) 100 MINOR

 003 T067508C (03 ) 010 NOT IN THIS SUBJECT

 004 T067508M (M ) 001 MISSING
Table F-5 (continued)

1998 Reading Conditioning Variable Specifications

672

 CONDITIONING VARIABLE ID: TCHR0023

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-READING AND/OR LANGUAGE ARTS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067509 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067509A (01 ) 000 MAJOR

 002 T067509B (02 ) 100 MINOR

 003 T067509C (03 ) 010 NOT IN THIS SUBJECT

 004 T067509M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0024

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-HISTORY

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067510 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067510A (01 ) 000 MAJOR

 002 T067510B (02 ) 100 MINOR

 003 T067510C (03 ) 010 NOT IN THIS SUBJECT

 004 T067510M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0025

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-POLITICAL SCIENCE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067511 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067511A (01 ) 000 MAJOR

 002 T067511B (02 ) 100 MINOR

 003 T067511C (03 ) 010 NOT IN THIS SUBJECT

 004 T067511M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0026

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-OTHER

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067512 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067512A (01 ) 000 MAJOR

 002 T067512B (02 ) 100 MINOR

 003 T067512C (03 ) 010 NOT IN THIS SUBJECT

 004 T067512M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0027

 DESCRIPTION: GRAD MAJOR/MINOR-ELEMENTARY EDUCATION

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067601A (01 ) 000 MAJOR

 002 T067601B (02 ) 100 MINOR

 003 T067601C (03 ) 010 NOT IN THIS SUBJECT

 004 T067601M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0028

 DESCRIPTION: GRAD MAJOR/MINOR-SECONDARY EDUCATION

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067602 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067602A (01 ) 000 MAJOR

 002 T067602B (02 ) 100 MINOR

 003 T067602C (03 ) 010 NOT IN THIS SUBJECT

 004 T067602M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0029

 DESCRIPTION: GRAD MAJOR/MINOR-SPECIAL EDUCATION

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067603 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067603A (01 ) 000 MAJOR

 002 T067603B (02 ) 100 MINOR

 003 T067603C (03 ) 010 NOT IN THIS SUBJECT

 004 T067603M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0030

 DESCRIPTION: GRAD MAJOR/MINOR-BILINGUAL EDUCATION/ESL

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067604 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067604A (01 ) 000 MAJOR

 002 T067604B (02 ) 100 MINOR

 003 T067604C (03 ) 010 NOT IN THIS SUBJECT

 004 T067604M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0031

 DESCRIPTION: GRAD MAJOR/MINOR-ADMINSTRATION & SUPERVISION

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067605 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067605A (01 ) 000 MAJOR

 002 T067605B (02 ) 100 MINOR

 003 T067605C (03 ) 010 NOT IN THIS SUBJECT

 004 T067605M (M ) 001 MISSING
Table F-5 (continued)

 1998 Reading Conditioning Variable Specification

673

 CONDITIONING VARIABLE ID: TCHR0032

 DESCRIPTION: GRAD MAJOR/MINOR-CURRICULUM AND INSTRUCTION

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067606 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067606A (01 ) 000 MAJOR

 002 T067606B (02 ) 100 MINOR

 003 T067606C (03 ) 010 NOT IN THIS SUBJECT

 004 T067606M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0033

 DESCRIPTION: GRAD MAJOR/MINOR-COUNSELING

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067607 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067607A (01 ) 000 MAJOR

 002 T067607B (02 ) 100 MINOR

 003 T067607C (03 ) 010 NOT IN THIS SUBJECT

 004 T067607M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0034

 DESCRIPTION: GRAD MAJOR/MINOR-ENGLISH

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067608 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067608A (01 ) 000 MAJOR

 002 T067608B (02 ) 100 MINOR

 003 T067608C (03 ) 010 NOT IN THIS SUBJECT

 004 T067608M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0035

 DESCRIPTION: GRAD MAJOR/MINOR-READING AND/OR LANGUAGE ARTS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067609 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067609A (01 ) 000 MAJOR

 002 T067609B (02 ) 100 MINOR

 003 T067609C (03 ) 010 NOT IN THIS SUBJECT

 004 T067609M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0036

 DESCRIPTION: GRAD MAJOR/MINOR-HISTORY

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067610 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067610A (01 ) 000 MAJOR

 002 T067610B (02 ) 100 MINOR

 003 T067610C (03 ) 010 NOT IN THIS SUBJECT

 004 T067610M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0037

 DESCRIPTION: GRAD MAJOR/MINOR-POLITICAL SCIENCE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067611 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067611A (01 ) 000 MAJOR

 002 T067611B (02 ) 100 MINOR

 003 T067611C (03 ) 010 NOT IN THIS SUBJECT

 004 T067611M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0038

 DESCRIPTION: GRAD MAJOR/MINOR-OTHER

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067612 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067612A (01 ) 000 MAJOR

 002 T067612B (02 ) 100 MINOR

 003 T067612C (03 ) 010 NOT IN THIS SUBJECT

 004 T067612M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0039

 DESCRIPTION: LAST 12 MOS, PROF DEV-READING AND WRITING

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067701A (01 ) 00000 NONE

 002 T067701B (02 ) 10000 LESS THAN 6 HOURS

 003 T067701C (03 ) 01000 6 - 15 HOURS

 004 T067701D (04 ) 00100 16 - 35 HOURS

 005 T067701E (05 ) 00010 MORE THAN 35 HOURS

 006 T067701M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0040

 DESCRIPTION: LAST 12 MOS, PROF DEV-SOCIAL STUDIES

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067702 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067702A (01 ) 00000 NONE

 002 T067702B (02 ) 10000 LESS THAN 6 HOURS

 003 T067702C (03 ) 01000 6 - 15 HOURS

 004 T067702D (04 ) 00100 16 - 35 HOURS

 005 T067702E (05 ) 00010 MORE THAN 35 HOURS

 006 T067702M (M ) 00001 MISSING
Table F-5 (continued)

1998 Reading Conditioning Variable Specifications

674

 CONDITIONING VARIABLE ID: TCHR0041

 DESCRIPTION: PREPARED IN THE USE OF TELECOMMUNICATIONS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067801A (01 ) 000 WELL PREPARED

 002 T067801B (02 ) 100 MODERATELY PREPARED

 003 T067801C (03 ) 010 NOT WELL PREPARED

 004 T067801M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0042

 DESCRIPTION: PREPARED IN THE USE OF COMPUTERS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067802 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067802A (01 ) 000 WELL PREPARED

 002 T067802B (02 ) 100 MODERATELY PREPARED

 003 T067802C (03 ) 010 NOT WELL PREPARED

 004 T067802M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0043

 DESCRIPTION: PREPARED IN COOPERATIVE GROUP INSTRUCTION

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067803 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067803A (01 ) 000 WELL PREPARED

 002 T067803B (02 ) 100 MODERATELY PREPARED

 003 T067803C (03 ) 010 NOT WELL PREPARED

 004 T067803M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0044

 DESCRIPTION: PREPARED IN TEACHING STUDENTS-DIFFERENT CULTURES

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067804 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067804A (01 ) 000 WELL PREPARED

 002 T067804B (02 ) 100 MODERATELY PREPARED

 003 T067804C (03 ) 010 NOT WELL PREPARED

 004 T067804M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0045

 DESCRIPTION: PREPARED IN TEACHING STUDENTS WHO ARE LEP

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067805 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067805A (01 ) 000 WELL PREPARED

 002 T067805B (02 ) 100 MODERATELY PREPARED

 003 T067805C (03 ) 010 NOT WELL PREPARED

 004 T067805M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0046

 DESCRIPTION: PREPARED IN TEACHING STUDENTS WITH DISABILITIES

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067806 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067806A (01 ) 000 WELL PREPARED

 002 T067806B (02 ) 100 MODERATELY PREPARED

 003 T067806C (03 ) 010 NOT WELL PREPARED

 004 T067806M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0047

 DESCRIPTION: PREPARED IN CLASSROOM MANAGEMENT AND ORGANIZATION

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067807 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067807A (01 ) 000 WELL PREPARED

 002 T067807B (02 ) 100 MODERATELY PREPARED

 003 T067807C (03 ) 010 NOT WELL PREPARED

 004 T067807M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0048

 DESCRIPTION: AVAILABILITY OF RESOURCES

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T041201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T041201A (01 ) 0000 GET ALL RESOURCES

 002 T041201B (02 ) 1000 GET MOST RESOURCES

 003 T041201C (03 ) 0100 GET SOME RESOURCES

 004 T041201D (04 ) 0010 DON’T GET RESOURCES

 005 T041201M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0049

 DESCRIPTION: HOW WELL PREPARED TO TEACH READING

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067901A (01 ) 000 WELL PREPARED

 002 T067901B (02 ) 100 MODERATELY PREPARED

 003 T067901C (03 ) 010 NOT WELL PREPARED

 004 T067901M (M ) 001 MISSING
Table F-5 (continued)

 1998 Reading Conditioning Variable Specification

675

 CONDITIONING VARIABLE ID: TCHR0050

 DESCRIPTION: HOW WELL PREPARED TO TEACH WRITING

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067902 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067902A (01 ) 000 WELL PREPARED

 002 T067902B (02 ) 100 MODERATELY PREPARED

 003 T067902C (03 ) 010 NOT WELL PREPARED

 004 T067902M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0051

 DESCRIPTION: PREPARED IN LIT-BASED READING INSTRUCTION

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T068001A (01 ) 000 WELL PREPARED

 002 T068001B (02 ) 100 MODERATELY PREPARED

 003 T068001C (03 ) 010 NOT WELL PREPARED

 004 T068001M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0052

 DESCRIPTION: PREPARED IN CONTENT AREA READING

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068002 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T068002A (01 ) 000 WELL PREPARED

 002 T068002B (02 ) 100 MODERATELY PREPARED

 003 T068002C (03 ) 010 NOT WELL PREPARED

 004 T068002M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0053

 DESCRIPTION: PREPARED IN COMBINING RDG AND WRITING

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068003 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T068003A (01 ) 000 WELL PREPARED

 002 T068003B (02 ) 100 MODERATELY PREPARED

 003 T068003C (03 ) 010 NOT WELL PREPARED

 004 T068003M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0054

 DESCRIPTION: PREPARED IN WHOLE LANGUAGE APPROACH TO TEACH RDG

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068004 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T068004A (01 ) 000 WELL PREPARED

 002 T068004B (02 ) 100 MODERATELY PREPARED

 003 T068004C (03 ) 010 NOT WELL PREPARED

 004 T068004M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0055

 DESCRIPTION: PREPARED IN PHONICS IN TEACHING READING

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068005 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T068005A (01 ) 000 WELL PREPARED

 002 T068005B (02 ) 100 MODERATELY PREPARED

 003 T068005C (03 ) 010 NOT WELL PREPARED

 004 T068005M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0056

 DESCRIPTION: PREPARED IN TEACHING MULTICULTURAL LITERATURE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068006 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T068006A (01 ) 000 WELL PREPARED

 002 T068006B (02 ) 100 MODERATELY PREPARED

 003 T068006C (03 ) 010 NOT WELL PREPARED

 004 T068006M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0057

 DESCRIPTION: PREPARED IN COMPUTER SOFTWARE FOR TEACHING RDG

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068007 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T068007A (01 ) 000 WELL PREPARED

 002 T068007B (02 ) 100 MODERATELY PREPARED

 003 T068007C (03 ) 010 NOT WELL PREPARED

 004 T068007M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0058

 DESCRIPTION: PREPARED IN WRITING ACROSS THE CURRICULUM

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068008 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T068008A (01 ) 000 WELL PREPARED

 002 T068008B (02 ) 100 MODERATELY PREPARED

 003 T068008C (03 ) 010 NOT WELL PREPARED

 004 T068008M (M ) 001 MISSING
Table F-5 (continued)

1998 Reading Conditioning Variable Specifications

676

 CONDITIONING VARIABLE ID: TCHR0059

 DESCRIPTION: PREPARED IN USING COMPUTER SOFTWARE TO TEACH WRTG

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068009 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T068009A (01 ) 000 WELL PREPARED

 002 T068009B (02 ) 100 MODERATELY PREPARED

 003 T068009C (03 ) 010 NOT WELL PREPARED

 004 T068009M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0060

 DESCRIPTION: PREPARED IN TEACHING SPELLING, GRAMMAR, MECHANICS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068010 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T068010A (01 ) 000 WELL PREPARED

 002 T068010B (02 ) 100 MODERATELY PREPARED

 003 T068010C (03 ) 010 NOT WELL PREPARED

 004 T068010M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0061

 DESCRIPTION: AVERAGE READING CLASS SIZE

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T068101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T068101A (01 ) 00000 1-20 STUDENTS

 002 T068101B (02 ) 10000 21-25 STUDENTS

 003 T068101C (03 ) 01000 26-30 STUDENTS

 004 T068101D (04 ) 00100 31-35 STUDENTS

 005 T068101E (05 ) 00010 36 OR MORE STUDENTS

 006 T068101M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0062

 DESCRIPTION: CLASS ASSIGNMENT BY ABILITY

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T046101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T046101Y (01 ) 00 YES

 002 T046101N (02 ) 10 NO

 003 T046101M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TCHR0063

 DESCRIPTION: ABILITY LEVEL OF STUDENTS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T046201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T046201A (01 ) 0000 MOSTLY HIGH ABILITY

 002 T046201B (02 ) 1000 MOSTLY AVERAGE ABLTY

 003 T046201C (03 ) 0100 MOSTLY LOW ABILITY

 004 T046201D (04 ) 0010 MIXED ABILITY LEVELS

 005 T046201M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0064

 DESCRIPTION: HOW MUCH CLASS TIME PER DAY-READING INSTRUCTION

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T068201A (01 ) 00000 LESS THAN 30 MINUTES

 002 T068201B (02 ) 10000 30-44 MINUTES

 003 T068201C (03 ) 01000 45-59 MINUTES

 004 T068201D (04 ) 00100 60-90 MINUTES

 005 T068201E (05 ) 00010 MORE THAN 90 MINUTES

 006 T068201M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0065

 DESCRIPTION: BASIS FOR CREATING READING INSTRUCTIONAL GROUPS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T068301A (01 ) 00000 ABILITY

 002 T068301B (02 ) 10000 INTEREST

 003 T068301C (03 ) 01000 DIVERSITY

 004 T068301D (04 ) 00100 OTHER

 005 T068301E (05 ) 00010 NOT CREATED

 006 T068301M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0066

 DESCRIPTION: CLASS DIVIDED INTO HOW MANY INSTRUCTIONAL GROUPS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 T068401A (01 ) 0000000 WHOLE CLASS

 002 T068401B (02 ) 1000000 WHOLE W/FLEX GROUP

 003 T068401C (03 ) 0100000 2 GROUPS

 004 T068401D (04 ) 0010000 3 GROUPS

 005 T068401E (05 ) 0001000 4 GROUPS

 006 T068401F (06 ) 0000100 5 OR MORE GROUPS

 007 T068401G (07 ) 0000010 INDIVIDUALIZED

 008 T068401M (M ) 0000001 MISSING
Table F-5 (continued)

 1998 Reading Conditioning Variable Specification

677

 CONDITIONING VARIABLE ID: TCHR0067

 DESCRIPTION: WRITING ABILITY LEVEL OF CLASS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T068601A (01 ) 0000 PRIMARILY HIGH

 002 T068601B (02 ) 1000 PRIMARILY AVERAGE

 003 T068601C (03 ) 0100 PRIMARILY LOW

 004 T068601D (04 ) 0010 WIDELY MIXED

 005 T068601M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0068

 DESCRIPTION: EACH WEEK, TIME SPENT INSTRUCTING/HELPING-WRITING

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T068701A (01 ) 00000 LESS THAN 30 MINUTES

 002 T068701B (02 ) 10000 30-44 MINUTES

 003 T068701C (03 ) 01000 45-59 MINUTES

 004 T068701D (04 ) 00100 60-90 MINUTES

 005 T068701E (05 ) 00010 MORE THAN 90 MINUTES

 006 T068701M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0069

 DESCRIPTION: HOW OFTEN USE CHILDREN’S NEWSPAPERS/MAGAZINES

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T068801A (01 ) 0000 ALMOST EVERY DAY

 002 T068801B (02 ) 1000 ONCE/TWICE A WEEK

 003 T068801C (03 ) 0100 ONCE/TWICE A MONTH

 004 T068801D (04 ) 0010 NEVER OR HARDLY EVER

 005 T068801M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0070

 DESCRIPTION: HOW OFTEN USE READING KITS TO TEACH READING

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068802 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T068802A (01 ) 0000 ALMOST EVERY DAY

 002 T068802B (02 ) 1000 ONCE/TWICE A WEEK

 003 T068802C (03 ) 0100 ONCE/TWICE A MONTH

 004 T068802D (04 ) 0010 NEVER OR HARDLY EVER

 005 T068802M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0071

 DESCRIPTION: HOW OFTEN USE COMPUTER SOFTWARE FOR READING INSTR

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068803 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T068803A (01 ) 0000 ALMOST EVERY DAY

 002 T068803B (02 ) 1000 ONCE/TWICE A WEEK

 003 T068803C (03 ) 0100 ONCE/TWICE A MONTH

 004 T068803D (04 ) 0010 NEVER OR HARDLY EVER

 005 T068803M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0072

 DESCRIPTION: HOW OFTEN USE BOOKS (NOVELS, POETRY, NONFICTION)

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068804 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T068804A (01 ) 0000 ALMOST EVERY DAY

 002 T068804B (02 ) 1000 ONCE/TWICE A WEEK

 003 T068804C (03 ) 0100 ONCE/TWICE A MONTH

 004 T068804D (04 ) 0010 NEVER OR HARDLY EVER

 005 T068804M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0073

 DESCRIPTION: HOW OFTEN USE MATERIALS FROM OTHER SUBJECTS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068805 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T068805A (01 ) 0000 ALMOST EVERY DAY

 002 T068805B (02 ) 1000 ONCE/TWICE A WEEK

 003 T068805C (03 ) 0100 ONCE/TWICE A MONTH

 004 T068805D (04 ) 0010 NEVER OR HARDLY EVER

 005 T068805M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0074

 DESCRIPTION: WHAT TYPE OF MATERIALS FORM CORE READING PROGRAM

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T068901A (01 ) 0000 PRIMARILY BASAL

 002 T068901B (02 ) 1000 PRIMARILY TRADE BOOK

 003 T068901C (03 ) 0100 BOTH BASAL AND TRADE

 004 T068901D (04 ) 0010 OTHER

 005 T068901M (M ) 0001 MISSING
Table F-5 (continued)

1998 Reading Conditioning Variable Specifications

678

 CONDITIONING VARIABLE ID: TCHR0075

 DESCRIPTION: AVAILABILITY OF COMPUTERS FOR USE IN CLASS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T069001A (01 ) 00000 NOT AVAILABLE

 002 T069001B (02 ) 10000 LIMITED ACCESS

 003 T069001C (03 ) 01000 LAB OR LIBRARY

 004 T069001D (04 ) 00100 ONE IN CLASSROOM

 005 T069001E (05 ) 00010 SEVERAL IN CLASSROOM

 006 T069001M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0076

 DESCRIPTION: PROPORTION TIME SPENT ON RDG FOR LIT EXPERIENCE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069101A (01 ) 0000 ALMOST ALL TIME

 002 T069101B (02 ) 1000 TWO-THIRDS OF TIME

 003 T069101C (03 ) 0100 AT LEAST ONE-THIRD

 004 T069101D (04 ) 0010 LITTLE OR NO TIME

 005 T069101M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0077

 DESCRIPTION: PROPORTION TIME SPENT ON RDG TO GAIN INFORMATION

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069102 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069102A (01 ) 0000 ALMOST ALL TIME

 002 T069102B (02 ) 1000 TWO-THIRDS OF TIME

 003 T069102C (03 ) 0100 AT LEAST ONE-THIRD

 004 T069102D (04 ) 0010 LITTLE OR NO TIME

 005 T069102M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0078

 DESCRIPTION: PROPORTION TIME SPENT ON RDG TO PERFORM A TASK

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069103 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069103A (01 ) 0000 ALMOST ALL TIME

 002 T069103B (02 ) 1000 TWO-THIRDS OF TIME

 003 T069103C (03 ) 0100 AT LEAST ONE-THIRD

 004 T069103D (04 ) 0010 LITTLE OR NO TIME

 005 T069103M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0079

 DESCRIPTION: PROPORTION TIME SPENT ON NARRATIVE WRITING

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069201A (01 ) 0000 ALMOST ALL TIME

 002 T069201B (02 ) 1000 TWO-THIRDS OF TIME

 003 T069201C (03 ) 0100 AT LEAST ONE-THIRD

 004 T069201D (04 ) 0010 LITTLE OR NO TIME

 005 T069201M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0080

 DESCRIPTION: PROPORTION TIME SPENT ON INFORMATIVE WRITING

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069202 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069202A (01 ) 0000 ALMOST ALL TIME

 002 T069202B (02 ) 1000 TWO-THIRDS OF TIME

 003 T069202C (03 ) 0100 AT LEAST ONE-THIRD

 004 T069202D (04 ) 0010 LITTLE OR NO TIME

 005 T069202M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0081

 DESCRIPTION: PROPORTION TIME SPENT ON PERSUASIVE WRITING

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069203 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069203A (01 ) 0000 ALMOST ALL TIME

 002 T069203B (02 ) 1000 TWO-THIRDS OF TIME

 003 T069203C (03 ) 0100 AT LEAST ONE-THIRD

 004 T069203D (04 ) 0010 LITTLE OR NO TIME

 005 T069203M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0082

 DESCRIPTION: DO YOU USE GRAMMAR OR SKILL-BASED INSTRUCTION

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069301A (01 ) 000 YES, CENTRAL PART

 002 T069301B (02 ) 100 YES, SUPPLEMENT PART

 003 T069301N (03 ) 010 NO

 004 T069301M (M ) 001 MISSING
Table F-5 (continued)

 1998 Reading Conditioning Variable Specification

679

 CONDITIONING VARIABLE ID: TCHR0083

 DESCRIPTION: DO YOU USE WRITING PROCESS INSTRUCTION

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069302 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069302A (01 ) 000 YES, CENTRAL PART

 002 T069302B (02 ) 100 YES, SUPPLEMENT PART

 003 T069302N (03 ) 010 NO

 004 T069302M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0084

 DESCRIPTION: DO YOU INTEGRATE READING AND WRITING INSTRUCTION

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069303 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069303A (01 ) 000 YES, CENTRAL PART

 002 T069303B (02 ) 100 YES, SUPPLEMENT PART

 003 T069303N (03 ) 010 NO

 004 T069303M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0085

 DESCRIPTION: DO YOU USE WRITING ABOUT LITERATURE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069304 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069304A (01 ) 000 YES, CENTRAL PART

 002 T069304B (02 ) 100 YES, SUPPLEMENT PART

 003 T069304N (03 ) 010 NO

 004 T069304M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0086

 DESCRIPTION: DO YOU USE WRITING ACROSS OTHER SUBJECT AREAS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069305 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069305A (01 ) 000 YES, CENTRAL PART

 002 T069305B (02 ) 100 YES, SUPPLEMENT PART

 003 T069305N (03 ) 010 NO

 004 T069305M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0087

 DESCRIPTION: HOW OFTEN STUDENTS DO SPELLING, PUNCTUATION, GRAMM

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069401A (01 ) 0000 ALMOST EVERY DAY

 002 T069401B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069401C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069401D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069401M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0088

 DESCRIPTION: HOW OFTEN STUDENTS WORK ON WRITING PROCESS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069402 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069402A (01 ) 0000 ALMOST EVERY DAY

 002 T069402B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069402C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069402D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069402M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0089

 DESCRIPTION: HOW OFTEN STUDENTS WRITE IN A LOG/JOURNAL

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069403 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069403A (01 ) 0000 ALMOST EVERY DAY

 002 T069403B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069403C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069403D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069403M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0090

 DESCRIPTION: HOW OFTEN PARENTS SIGN/REVIEW STUDENTS’ HOMEWORK

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069404 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069404A (01 ) 0000 ALMOST EVERY DAY

 002 T069404B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069404C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069404D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069404M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0091

 DESCRIPTION: HOW OFTEN ASSIGN HOMEWORK TO DO WITH PARENTS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069405 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069405A (01 ) 0000 ALMOST EVERY DAY

 002 T069405B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069405C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069405D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069405M (M ) 0001 MISSING
Table F-5 (continued)

1998 Reading Conditioning Variable Specifications

680

 CONDITIONING VARIABLE ID: TCHR0092

 DESCRIPTION: EXPECTED TIME SPENT ON WRITING ASSIGNMENTS/WEEK

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T069501A (01 ) 00000 NONE

 002 T069501B (02 ) 10000 LESS THAN 1 HOUR

 003 T069501C (03 ) 01000 1 HOUR

 004 T069501D (04 ) 00100 2 HOURS

 005 T069501E (05 ) 00010 3 HOURS OR MORE

 006 T069501M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0093

 DESCRIPTION: THIS YEAR, PROJECTS TO DO/SHARE WITH PARENTS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069601A (01 ) 0000 NEVER

 002 T069601B (02 ) 1000 ONCE

 003 T069601C (03 ) 0100 TWICE

 004 T069601D (04 ) 0010 THREE OR MORE TIMES

 005 T069601M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0094

 DESCRIPTION: HOW OFTEN ASK STUDENTS TO READ ALOUD

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069701A (01 ) 0000 ALMOST EVERY DAY

 002 T069701B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069701C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069701D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069701M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0095

 DESCRIPTION: HOW OFTEN ASK STUDENTS-DISCUSS WHAT WAS READ

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069702 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069702A (01 ) 0000 ALMOST EVERY DAY

 002 T069702B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069702C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069702D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069702M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0096

 DESCRIPTION: HOW OFTEN ASK STUDENTS- WRITE ABOUT WHAT WAS READ

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069703 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069703A (01 ) 0000 ALMOST EVERY DAY

 002 T069703B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069703C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069703D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069703M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0097

 DESCRIPTION: HOW OFTEN ASK STUDENTS-WRITE IN WORKSHEET/BOOK

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069704 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069704A (01 ) 0000 ALMOST EVERY DAY

 002 T069704B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069704C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069704D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069704M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0098

 DESCRIPTION: HOW OFTEN ASK STUDENTS-READ SILENTLY

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069705 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069705A (01 ) 0000 ALMOST EVERY DAY

 002 T069705B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069705C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069705D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069705M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0099

 DESCRIPTION: HOW OFTEN GIVE STUDENTS TIME TO READ BOOKS CHOSEN

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069706 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069706A (01 ) 0000 ALMOST EVERY DAY

 002 T069706B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069706C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069706D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069706M (M ) 0001 MISSING
Table F-5 (continued)

 1998 Reading Conditioning Variable Specification

681

 CONDITIONING VARIABLE ID: TCHR0100

 DESCRIPTION: HOW OFTEN ASK STUDENTS-GROUP ACTIVITY/PROJECT

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069707 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069707A (01 ) 0000 ALMOST EVERY DAY

 002 T069707B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069707C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069707D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069707M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0101

 DESCRIPTION: HOW OFTEN ASK STUDENTS-DISCUSS INTERPRETATIONS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069708 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069708A (01 ) 0000 ALMOST EVERY DAY

 002 T069708B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069708C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069708D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069708M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0102

 DESCRIPTION: HOW OFTEN ASK STUDENTS-EXPLAIN/SUPPORT WHAT READ

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069709 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069709A (01 ) 0000 ALMOST EVERY DAY

 002 T069709B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069709C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069709D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069709M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0103

 DESCRIPTION: HOW OFTEN GIVE READING QUIZZES OR TESTS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069710 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069710A (01 ) 0000 ALMOST EVERY DAY

 002 T069710B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069710C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069710D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069710M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0104

 DESCRIPTION: HOW OFTEN WATCH MOVIES, VIDEOS, FILMSTRIPS, TV, CD

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069711 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069711A (01 ) 0000 ALMOST EVERY DAY

 002 T069711B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069711C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069711D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069711M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0105

 DESCRIPTION: HOW OFTEN HELP STUDENTS UNDERSTAND NEW WORDS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069712 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069712A (01 ) 0000 ALMOST EVERY DAY

 002 T069712B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069712C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069712D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069712M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0106

 DESCRIPTION: HOW OFTEN ASK STUDENTS-ANSWER QUESTIONS IN WRITING

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069713 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069713A (01 ) 0000 ALMOST EVERY DAY

 002 T069713B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069713C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069713D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069713M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0107

 DESCRIPTION: HOW OFTEN ASK STUDENTS-PREDICT OUTCOME OF READING

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069714 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069714A (01 ) 0000 ALMOST EVERY DAY

 002 T069714B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069714C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069714D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069714M (M ) 0001 MISSING
Table F-5 (continued)

1998 Reading Conditioning Variable Specifications

682

 CONDITIONING VARIABLE ID: TCHR0108

 DESCRIPTION: HOW OFTEN ASK STUDENTS-MAKE GENERALIZATIONS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069715 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069715A (01 ) 0000 ALMOST EVERY DAY

 002 T069715B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069715C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069715D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069715M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0109

 DESCRIPTION: HOW OFTEN ASK STUDENTS-DESCRIBE STYLE/STRUCTURE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069716 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069716A (01 ) 0000 ALMOST EVERY DAY

 002 T069716B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069716C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069716D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069716M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0110

 DESCRIPTION: HOW OFTEN STUDENTS CHOOSE WRITING TOPIC

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T071801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071801A (01 ) 0000 ALMOST EVERY DAY

 002 T071801B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071801C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071801D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071801M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0111

 DESCRIPTION: HOW OFTEN STUDENTS PLAN THEIR WRITING

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T071802 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071802A (01 ) 0000 ALMOST EVERY DAY

 002 T071802B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071802C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071802D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071802M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0112

 DESCRIPTION: HOW OFTEN STUDENTS DEFINE PURPOSES AND AUDIENCE

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T071803 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071803A (01 ) 0000 ALMOST EVERY DAY

 002 T071803B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071803C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071803D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071803M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0113

 DESCRIPTION: HOW OFTEN STUDENTS MAKE FORMAL OUTLINE

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T071804 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071804A (01 ) 0000 ALMOST EVERY DAY

 002 T071804B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071804C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071804D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071804M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0114

 DESCRIPTION: HOW OFTEN STUDENTS WRITE MORE THAN ONE DRAFT

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T071805 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071805A (01 ) 0000 ALMOST EVERY DAY

 002 T071805B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071805C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071805D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071805M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0115

 DESCRIPTION: HOW OFTEN STUDENTS USE RESOURCES OTHER THAN TEXT

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T071806 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071806A (01 ) 0000 ALMOST EVERY DAY

 002 T071806B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071806C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071806D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071806M (M ) 0001 MISSING
Table F-5 (continued)

 1998 Reading Conditioning Variable Specification

683

 CONDITIONING VARIABLE ID: TCHR0116

 DESCRIPTION: HOW OFTEN STUDENTS DISCUSS WRITING WHILE WRITING

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T071807 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071807A (01 ) 0000 ALMOST EVERY DAY

 002 T071807B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071807C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071807D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071807M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0117

 DESCRIPTION: HOW OFTEN STUDENTS DISCUSS OTHERS’ WRITING

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T071808 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071808A (01 ) 0000 ALMOST EVERY DAY

 002 T071808B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071808C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071808D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071808M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0118

 DESCRIPTION: HOW OFTEN STUDENTS CHECK PROPER SPELLING, GRAMMAR

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T071809 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071809A (01 ) 0000 ALMOST EVERY DAY

 002 T071809B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071809C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071809D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071809M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0119

 DESCRIPTION: HOW OFTEN STUDENTS DISCUSS WRITING WITH FAMILY

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T071810 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071810A (01 ) 0000 ALMOST EVERY DAY

 002 T071810B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071810C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071810D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071810M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0120

 DESCRIPTION: HOW OFTEN STUDENTS CONTRIBUTE TO COLLECTION

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T071811 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071811A (01 ) 0000 ALMOST EVERY DAY

 002 T071811B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071811C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071811D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071811M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0121

 DESCRIPTION: HOW OFTEN STUDENTS WORK ON AN ASSIGNED TOPIC

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T071812 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071812A (01 ) 0000 ALMOST EVERY DAY

 002 T071812B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071812C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071812D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071812M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0122

 DESCRIPTION: HOW OFTEN STUDENTS FOLLOW ASSIGNED FORMAT

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T071813 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071813A (01 ) 0000 ALMOST EVERY DAY

 002 T071813B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071813C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071813D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071813M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0123

 DESCRIPTION: HOW OFTEN WRITING ASSIGNMENTS-LESS THAN ONE PAGE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069901A (01 ) 0000 ALMOST EVERY DAY

 002 T069901B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069901C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069901D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069901M (M ) 0001 MISSING
Table F-5 (continued)

1998 Reading Conditioning Variable Specifications

684

 CONDITIONING VARIABLE ID: TCHR0124

 DESCRIPTION: HOW OFTEN WRITING ASSIGNMENTS-ONE TO TWO PAGES

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069902 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069902A (01 ) 0000 ALMOST EVERY DAY

 002 T069902B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069902C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069902D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069902M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0125

 DESCRIPTION: HOW OFTEN WRITING ASSIGNMENTS-THREE OR MORE PAGES

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069903 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069903A (01 ) 0000 ALMOST EVERY DAY

 002 T069903B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069903C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069903D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069903M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0126

 DESCRIPTION: HOW OFTEN STUDENTS USE COMPUTER-SPELL, PUNC, GRAM

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070001A (01 ) 0000 ALMOST EVERY DAY

 002 T070001B (02 ) 1000 ONCE/TWICE A WEEK

 003 T070001C (03 ) 0100 ONCE/TWICE A MONTH

 004 T070001D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070001M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0127

 DESCRIPTION: HOW OFTEN STUDENTS USE COMPUTERS-WRITE DRAFTS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070002 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070002A (01 ) 0000 ALMOST EVERY DAY

 002 T070002B (02 ) 1000 ONCE/TWICE A WEEK

 003 T070002C (03 ) 0100 ONCE/TWICE A MONTH

 004 T070002D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070002M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0128

 DESCRIPTION: HOW OFTEN STUDENTS USE COMPUTERS-READ STORIES

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070003 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070003A (01 ) 0000 ALMOST EVERY DAY

 002 T070003B (02 ) 1000 ONCE/TWICE A WEEK

 003 T070003C (03 ) 0100 ONCE/TWICE A MONTH

 004 T070003D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070003M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0129

 DESCRIPTION: HOW OFTEN READING ASSESSED-MULTIPLE-CHOICE TESTS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070101A (01 ) 0000 ONCE/TWICE A WEEK

 002 T070101B (02 ) 1000 ONCE/TWICE A MONTH

 003 T070101C (03 ) 0100 ONCE/TWICE A YEAR

 004 T070101D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070101M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0130

 DESCRIPTION: HOW OFTEN READING ASSESSED-SHORT-ANSWER TESTS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070102 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070102A (01 ) 0000 ONCE/TWICE A WEEK

 002 T070102B (02 ) 1000 ONCE/TWICE A MONTH

 003 T070102C (03 ) 0100 ONCE/TWICE A YEAR

 004 T070102D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070102M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0131

 DESCRIPTION: HOW OFTEN READ ASSESSED-PARAGRAPH WRITTEN RESPONSE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070103 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070103A (01 ) 0000 ONCE/TWICE A WEEK

 002 T070103B (02 ) 1000 ONCE/TWICE A MONTH

 003 T070103C (03 ) 0100 ONCE/TWICE A YEAR

 004 T070103D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070103M (M ) 0001 MISSING
Table F-5 (continued)

 1998 Reading Conditioning Variable Specification

685

 CONDITIONING VARIABLE ID: TCHR0132

 DESCRIPTION: HOW OFTEN STUDENTS ASSESSED-INDIVIDUAL/GROUP PROJ

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070104 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070104A (01 ) 0000 ONCE/TWICE A WEEK

 002 T070104B (02 ) 1000 ONCE/TWICE A MONTH

 003 T070104C (03 ) 0100 ONCE/TWICE A YEAR

 004 T070104D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070104M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0133

 DESCRIPTION: HOW OFTEN STUDENTS ASSESSED-READING PORTFOLIOS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070105 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070105A (01 ) 0000 ONCE/TWICE A WEEK

 002 T070105B (02 ) 1000 ONCE/TWICE A MONTH

 003 T070105C (03 ) 0100 ONCE/TWICE A YEAR

 004 T070105D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070105M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0134

 DESCRIPTION: HOW OFTEN STUDENTS ASSESSED-ESSAYS/PAPERS ASSIGNED

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070106 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070106A (01 ) 0000 ONCE/TWICE A WEEK

 002 T070106B (02 ) 1000 ONCE/TWICE A MONTH

 003 T070106C (03 ) 0100 ONCE/TWICE A YEAR

 004 T070106D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070106M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0135

 DESCRIPTION: HOW OFTEN STUDENTS ASSESSED-ORAL READING

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070107 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070107A (01 ) 0000 ONCE/TWICE A WEEK

 002 T070107B (02 ) 1000 ONCE/TWICE A MONTH

 003 T070107C (03 ) 0100 ONCE/TWICE A YEAR

 004 T070107D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070107M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0136

 DESCRIPTION: HOW OFTEN WRITING ASSESSED-MULTIPLE-CHOICE TESTS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070201A (01 ) 0000 ONCE/TWICE A WEEK

 002 T070201B (02 ) 1000 ONCE/TWICE A MONTH

 003 T070201C (03 ) 0100 ONCE/TWICE A YEAR

 004 T070201D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070201M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0137

 DESCRIPTION: HOW OFTEN WRITING ASSESSED-PARAGRAPH WRITTEN

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070202 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070202A (01 ) 0000 ONCE/TWICE A WEEK

 002 T070202B (02 ) 1000 ONCE/TWICE A MONTH

 003 T070202C (03 ) 0100 ONCE/TWICE A YEAR

 004 T070202D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070202M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0138

 DESCRIPTION: HOW OFTEN WRITING ASSESSED-ESSAYS, REPORTS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070203 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070203A (01 ) 0000 ONCE/TWICE A WEEK

 002 T070203B (02 ) 1000 ONCE/TWICE A MONTH

 003 T070203C (03 ) 0100 ONCE/TWICE A YEAR

 004 T070203D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070203M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0139

 DESCRIPTION: HOW OFTEN WRITING ASSESSED-WRITING PORTFOLIOS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070204 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070204A (01 ) 0000 ONCE/TWICE A WEEK

 002 T070204B (02 ) 1000 ONCE/TWICE A MONTH

 003 T070204C (03 ) 0100 ONCE/TWICE A YEAR

 004 T070204D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070204M (M ) 0001 MISSING
Table F-5 (continued)

1998 Reading Conditioning Variable Specifications

686

 CONDITIONING VARIABLE ID: TCHR0140

 DESCRIPTION: HOW IMPORTANT TO GRADE-SPELLING, GRAMMAR, PUNC

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T070301A (01 ) 000 VERY IMPORTANT

 002 T070301B (02 ) 100 MODERATELY IMPORTANT

 003 T070301C (03 ) 010 UNIMPORTANT

 004 T070301M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0141

 DESCRIPTION: HOW IMPORTANT TO GRADE-ORGANIZATION/COHERENCE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070302 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T070302A (01 ) 000 VERY IMPORTANT

 002 T070302B (02 ) 100 MODERATELY IMPORTANT

 003 T070302C (03 ) 010 UNIMPORTANT

 004 T070302M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0142

 DESCRIPTION: HOW IMPORTANT TO GRADE-QUALITY/CREATIVITY OF IDEAS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070303 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T070303A (01 ) 000 VERY IMPORTANT

 002 T070303B (02 ) 100 MODERATELY IMPORTANT

 003 T070303C (03 ) 010 UNIMPORTANT

 004 T070303M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0143

 DESCRIPTION: HOW IMPORTANT TO GRADE-LENGTH OF PAPERS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070304 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T070304A (01 ) 000 VERY IMPORTANT

 002 T070304B (02 ) 100 MODERATELY IMPORTANT

 003 T070304C (03 ) 010 UNIMPORTANT

 004 T070304M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0144

 DESCRIPTION: HOW IMPORTANT TO GRADE-ACCOMPLISH WRITING PURPOSE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070305 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T070305A (01 ) 000 VERY IMPORTANT

 002 T070305B (02 ) 100 MODERATELY IMPORTANT

 003 T070305C (03 ) 010 UNIMPORTANT

 004 T070305M (M ) 001 MISSING

 004 T070305M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0145

 DESCRIPTION: DO YOU TEACH READING

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T071601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T071601Y (01 ) 0 YES

 002 T071601M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0146

 DESCRIPTION: DO YOU TEACH WRITING

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T071602 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T071602Y (01 ) 0 YES

 002 T071602M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0147

 DESCRIPTION: DO YOU TEACH ENGLISH

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T071603 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T071603Y (01 ) 0 YES

 002 T071603M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0148

 DESCRIPTION: DO YOU TEACH-OTHER

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T071604 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T071604Y (01 ) 0 YES

 002 T071604M (M ) 1 MISSING
Table F-5 (continued)

 1998 Reading Conditioning Variable Specification

687

 CONDITIONING VARIABLE ID: TCHR0149

 DESCRIPTION: YEARS TOTAL TAUGHT ELEMENTARY OR SECONDARY

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T040301A (01 ) 00000 2 YEARS OR LESS

 002 T040301B (02 ) 10000 3-5 YEARS

 003 T040301C (03 ) 01000 6-10 YEARS

 004 T040301D (04 ) 00100 11-24 YEARS

 005 T040301E (05 ) 00010 25 YEARS OR MORE

 006 T040301M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0150

 DESCRIPTION: YEARS TOTAL TAUGHT READING

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T071701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 T071701A (01 ) 000000 NOT TAUGHT

 002 T071701B (02 ) 100000 2 YEARS OR LESS

 003 T071701C (03 ) 010000 3-5 YEARS

 004 T071701D (04 ) 001000 6-10 YEARS

 005 T071701E (05 ) 000100 11-24 YEARS

 006 T071701F (06 ) 000010 25 YEARS OR MORE

 007 T071701M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0151

 DESCRIPTION: YEARS TOTAL TAUGHT WRITING

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T071702 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 T071702A (01 ) 000000 NOT TAUGHT

 002 T071702B (02 ) 100000 2 YEARS OR LESS

 003 T071702C (03 ) 010000 3-5 YEARS

 004 T071702D (04 ) 001000 6-10 YEARS

 005 T071702E (05 ) 000100 11-24 YEARS

 006 T071702F (06 ) 000010 25 YEARS OR MORE

 007 T071702M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0152

 DESCRIPTION: YEARS TOTAL TAUGHT ENGLISH

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T071703 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 T071703A (01 ) 000000 NOT TAUGHT

 002 T071703B (02 ) 100000 2 YEARS OR LESS

 003 T071703C (03 ) 010000 3-5 YEARS

 004 T071703D (04 ) 001000 6-10 YEARS

 005 T071703E (05 ) 000100 11-24 YEARS

 006 T071703F (06 ) 000010 25 YEARS OR MORE

 007 T071703M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0153

 DESCRIPTION: YEARS TOTAL TAUGHT- OTHER

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T071704 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 T071704A (01 ) 000000 NOT TAUGHT

 002 T071704B (02 ) 100000 2 YEARS OR LESS

 003 T071704C (03 ) 010000 3-5 YEARS

 004 T071704D (04 ) 001000 6-10 YEARS

 005 T071704E (05 ) 000100 11-24 YEARS

 006 T071704F (06 ) 000010 25 YEARS OR MORE

 007 T071704M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0154

 DESCRIPTION: LAST 12 MOS, PROF DEV-LITERATURE

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067703 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067703A (01 ) 00000 NONE

 002 T067703B (02 ) 10000 LESS THAN 6 HOURS

 003 T067703C (03 ) 01000 6 - 15 HOURS

 004 T067703D (04 ) 00100 16 - 35 HOURS

 005 T067703E (05 ) 00010 MORE THAN 35 HOURS

 006 T067703M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0155

 DESCRIPTION: ARE STUDENTS ASSIGNED TO THIS CLASS BY ABILITY

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T068501Y (01 ) 00 YES

 002 T068501N (02 ) 10 NO

 003 T068501M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TCHR0156

 DESCRIPTION: HOW OFTEN STUDENTS CHOOSE WRITING TOPIC

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069801A (01 ) 000 ALWAYS

 002 T069801B (02 ) 100 SOMETIMES

 003 T069801C (03 ) 010 NEVER

 004 T069801M (M ) 001 MISSING
Table F-5 (continued)

1998 Reading Conditioning Variable Specifications

688

 CONDITIONING VARIABLE ID: TCHR0157

 DESCRIPTION: HOW OFTEN STUDENTS PLAN THEIR WRITING

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069802 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069802A (01 ) 000 ALWAYS

 002 T069802B (02 ) 100 SOMETIMES

 003 T069802C (03 ) 010 NEVER

 004 T069802M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0158

 DESCRIPTION: HOW OFTEN STUDENTS DEFINE PURPOSES AND AUDIENCE

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069803 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069803A (01 ) 000 ALWAYS

 002 T069803B (02 ) 100 SOMETIMES

 003 T069803C (03 ) 010 NEVER

 004 T069803M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0159

 DESCRIPTION: HOW OFTEN STUDENTS MAKE FORMAL OUTLINE

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069804 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069804A (01 ) 000 ALWAYS

 002 T069804B (02 ) 100 SOMETIMES

 003 T069804C (03 ) 010 NEVER

 004 T069804M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0160

 DESCRIPTION: HOW OFTEN STUDENTS WRITE MORE THAN ONE DRAFT

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069805 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069805A (01 ) 000 ALWAYS

 002 T069805B (02 ) 100 SOMETIMES

 003 T069805C (03 ) 010 NEVER

 004 T069805M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0161

 DESCRIPTION: HOW OFTEN STUDENTS USE RESOURCES OTHER THAN TEXT

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069806 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069806A (01 ) 000 ALWAYS

 002 T069806B (02 ) 100 SOMETIMES

 003 T069806C (03 ) 010 NEVER

 004 T069806M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0162

 DESCRIPTION: HOW OFTEN STUDENTS DISCUSS WRITING WHILE WRITING

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069807 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069807A (01 ) 000 ALWAYS

 002 T069807B (02 ) 100 SOMETIMES

 003 T069807C (03 ) 010 NEVER

 004 T069807M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0163

 DESCRIPTION: HOW OFTEN STUDENTS DISCUSS OTHERS’ WRITING

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069808 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069808A (01 ) 000 ALWAYS

 002 T069808B (02 ) 100 SOMETIMES

 003 T069808C (03 ) 010 NEVER

 004 T069808M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0164

 DESCRIPTION: HOW OFTEN STUDENTS CHECK PROPER SPELLING, GRAMMAR

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069809 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069809A (01 ) 000 ALWAYS

 002 T069809B (02 ) 100 SOMETIMES

 003 T069809C (03 ) 010 NEVER

 004 T069809M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0165

 DESCRIPTION: HOW OFTEN STUDENTS DISCUSS WRITING WITH FAMILY

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069810 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069810A (01 ) 000 ALWAYS

 002 T069810B (02 ) 100 SOMETIMES

 003 T069810C (03 ) 010 NEVER

 004 T069810M (M ) 001 MISSING
Table F-5 (continued)

 1998 Reading Conditioning Variable Specification

689

 CONDITIONING VARIABLE ID: TCHR0166

 DESCRIPTION: HOW OFTEN STUDENTS CONTRIBUTE TO COLLECTION

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069811 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069811A (01 ) 000 ALWAYS

 002 T069811B (02 ) 100 SOMETIMES

 003 T069811C (03 ) 010 NEVER

 004 T069811M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0167

 DESCRIPTION: HOW OFTEN STUDENTS WORK ON AN ASSIGNED TOPIC

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069812 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069812A (01 ) 000 ALWAYS

 002 T069812B (02 ) 100 SOMETIMES

 003 T069812C (03 ) 010 NEVER

 004 T069812M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0168

 DESCRIPTION: HOW OFTEN STUDENTS FOLLOW ASSIGNED FORMAT

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069813 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069813A (01 ) 000 ALWAYS

 002 T069813B (02 ) 100 SOMETIMES

 003 T069813C (03 ) 010 NEVER

 004 T069813M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0169

 DESCRIPTION: WHAT IS THE NUMBER OF STUDENTS IN EACH CLASS? (8TH GRADE)

 GRADES/ASSESSMENTS: N08

 CONDITIONING VAR LABEL: CLASSIZ8

 NAEP ID: TCSIZE TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 CLASIZ-1 (1 ) 00000 AVERAGE CLASS SIZE: 1-20 STUDENTS

 002 CLASIZ-2 (2 ) 10000 AVERAGE CLASS SIZE: 21-25 STUDENTS

 003 CLASIZ-3 (3 ) 01000 AVERAGE CLASS SIZE: 26-30 STUDENTS

 004 CLASIZ-4 (4 ) 00100 AVERAGE CLASS SIZE: 31-35 STUDENTS

 005 CLASIZ-5 (5 ) 00010 AVERAGE CLASS SIZE: 36 OR MORE STUDENTS

 006 CLASIZ-? (M ) 00001 AVERAGE CLASS SIZE: MISSING, DOES NOT APPLY
Table F-6

1998 Writing Conditioning Variable Specifications

690

 CONDITIONING VARIABLE ID: BACK0001

 DESCRIPTION: GRAND MEAN

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: OVERALL

 NAEP ID: BKSER TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: OTHER NUMBER OF INDEPENDENT CONTRASTS: 1

 001 OVERALL (@ ) 1 GRAND MEAN

 CONDITIONING VARIABLE ID: BACK0002

 DESCRIPTION: DERIVED SEX

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: GENDER

 NAEP ID: DSEX TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 MALE (1,M ) 0 MALE

 002 FEMALE (2 ) 1 FEMALE

 CONDITIONING VARIABLE ID: BACK0003

 DESCRIPTION: DERIVED RACE/ETHNICITY

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: RACE/ETH

 NAEP ID: DRACE TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 WHI/AI/O (1,5,6,M ) 000 RACE/ETHNICITY: WHITE, AMERICAN

INDIAN/ALASKAN NATIVE, OTHER,

MISSING, UNCLASSIFIED

 002 BLACK (2 ) 100 RACE/ETHNICITY: BLACK

 003 HISPANIC (3 ) 010 RACE/ETHNICITY: HISPANIC

 004 ASIAN (4 ) 001 RACE/ETHNICITY: ASIAN / PACIFIC ISLANDER

 CONDITIONING VARIABLE ID: BACK0004

 DESCRIPTION: IF HISPANIC, WHAT IS YOUR HISPANIC BACKGROUND?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: HISPANIC

 NAEP ID: B003101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 NOT HISP (1 ) 0000 HISPANIC: NOT HISPANIC

 002 MEXICAN (2 ) 1000 HISPANIC: MEXICAN, MEXICAN AMERICAN, CHICANO

 003 PUER RIC (3 ) 0100 HISPANIC: PUERTO RICAN

 004 CUBN,OTH (4,5 ) 0010 HISPANIC: CUBAN, OTHER

 005 HISP-? (M ) 0001 HISPANIC: MISSING

 CONDITIONING VARIABLE ID: BACK0005

 DESCRIPTION: TOL 7 - TYPE OF LOCATION

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: TOL7

 NAEP ID: TOL7 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 BIG CTY7 (1 ) 000000 TOL7: LARGE CITY

 002 MID CTY7 (2,M ) 100000 TOL7: MID-SIZE CITY

 003 FR/LCTY7 (3 ) 010000 TOL7: URBAN FRINGE OF LARGE CITY

 004 FR/MCTY7 (4 ) 001000 TOL7: URBAN FRINGE OF MID-SIZE CITY

 005 LAR TWN7 (5 ) 000100 TOL7: LARGE TOWN

 006 SML TWN7 (6 ) 000010 TOL7: SMALL TOWN

 007 OTHER (7 ) 000001 TOL7: OTHER

 CONDITIONING VARIABLE ID: BACK0006

 DESCRIPTION: TYPE OF LOCALE (5 CATEGORIES)

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: TOL5

 NAEP ID: TOL5 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 BIG CTY5 (1 ) 0000 TOL5: LARGE CITY

 002 MID CTY5 (2,M ) 1000 TOL5: MID-SIZE CITY

 003 FR/BTWN5 (3 ) 0100 TOL5: URBAN FRINGE AND LARGE TOWN

 004 SML TWN5 (4 ) 0010 TOL5: SMALL TOWN

 005 RURAL5 (5 ) 0001 TOL5: RURAL (MSA AND NON-MSA)

 CONDITIONING VARIABLE ID: BACK0007

 DESCRIPTION: PARENTS’ HIGHEST LEVEL OF EDUCATION

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: PARED2

 NAEP ID: PARED2 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 < HS (1 ) 0000 PARED: LESS THAN HIGH SCHOOL

 002 HS GRAD (2 ) 1000 PARED: HIGH SCHOOL GRADUATE

 003 POST HS (3 ) 0100 PARED: POST HIGH SCHOOL

 004 COL GRAD (4 ) 0010 PARED: COLLEGE GRADUATE

 005 PARED-? (5,M ) 0001 PARED: MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0008

 DESCRIPTION: REGION OF THE COUNTRY

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: REGION

 NAEP ID: REGION TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 N EAST (1,M ) 000 REGION: NORTHEAST

 002 S EAST (2 ) 100 REGION: SOUTHEAST

 003 CENTRAL (3 ) 010 REGION: CENTRAL

 004 WEST (4,5 ) 001 REGION: WEST, TERRITORIES (NONE)

 CONDITIONING VARIABLE ID: BACK0009

 DESCRIPTION: SCHOOL TYPE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: SCHTYPE

 NAEP ID: SCHTYPE TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 PUBLIC (1 ) 00 SCHOOL TYPE: PUBLIC,

 002 PRIVATE (2,4,5,M ) 10 SCHOOL TYPE: PRIVATE, BIA, DEPARTMENT OF

DEFENSE, MISSING

 003 CATHOLIC (3 ) 01 SCHOOL TYPE: CATHOLIC
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

691

 CONDITIONING VARIABLE ID: BACK0010

 DESCRIPTION: RACE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: RACE

 NAEP ID: RACE TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 WHI/AI/O (1,5,6,M ) 000 RACE: WHITE, AMERICAN INDIAN/ALASKAN NATIVE,

OTHER, MISSING, UNCLASSIFIED

 002 BLACK (2 ) 100 RACE: BLACK

 003 HISPANIC (3 ) 010 RACE: HISPANIC

 004 ASIAN (4 ) 001 RACE: ASIAN / PACIFIC ISLANDER

 CONDITIONING VARIABLE ID: BACK0011

 DESCRIPTION: INDIVIDUALIZED EDUCATION PLAN

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: IEP

 NAEP ID: IEP TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 IEP-YES (1 ) 0 IEP: YES

 002 IEP-NO (2,M ) 1 IEP: NO

 CONDITIONING VARIABLE ID: BACK0012

 DESCRIPTION: LIMITED ENGLISH PROFICIENCY

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: LEP

 NAEP ID: LEP TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 LEP-YES (1 ) 0 LEP: YES

 002 LEP-NO (2,M ) 1 LEP: NO

 CONDITIONING VARIABLE ID: BACK0013

 DESCRIPTION: TITLE 1: (BOOK COVER)

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: TITLE 1

 NAEP ID: TITLE1 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 TITLE-Y (1 ) 0 TITLE 1: YES

 002 TITLE-N (2,M ) 1 TITLE 1: NO

 CONDITIONING VARIABLE ID: BACK0014

 DESCRIPTION: DO YOU RECEIVE A FREE OR REDUCED-PRICE LUNCH?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: LUNCH

 NAEP ID: SLUNCH TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 NOT ELIG (1 ) 00000 LUNCH PROGRAM: NOT ELIGIBLE

 002 RED PRIC (2 ) 10000 LUNCH PROGRAM: REDUCED PRICE

 003 FREE (3 ) 01000 LUNCH PROGRAM: FREE

 004 INFO N/A (4,M ) 00100 LUNCH PROGRAM: INFO NOT AVAILABLE

 005 SCH/REF (5 ) 00010 LUNCH PROGRAM: SCHOOL REFUSAL

 006 SCH/NP (6 ) 00001 LUNCH PROGRAM: SCHOOL NOT PARTIPATE

 CONDITIONING VARIABLE ID: BACK0015

 DESCRIPTION: HOW MUCH TELEVISION DO YOU USUALLY WATCH EACH DAY? (LINEAR)

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: TVWATCHL

 NAEP ID: B013901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 TVLIN-0 (1 ) 0 TV WATCHING (LINEAR) (0 TO 6+ HOURS PER DAY)

 002 TVLIN-1 (2 ) 1 TV WATCHING (LINEAR)

 003 TVLIN-2 (3 ) 2 TV WATCHING (LINEAR)

 004 TVLIN-3 (4,M ) 3 TV WATCHING (LINEAR)

 005 TVLIN-4 (5 ) 4 TV WATCHING (LINEAR)

 006 TVLIN-5 (6 ) 5 TV WATCHING (LINEAR)

 007 TVLIN-6 (7 ) 6 TV WATCHING (LINEAR)

 CONDITIONING VARIABLE ID: BACK0016

 DESCRIPTION: HOW MUCH TELEVISION DO YOU USUALLY WATCH EACH DAY? (QUADRATIC)

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: TVWATCHQ

 NAEP ID: B013901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: QUADRATIC NUMBER OF INDEPENDENT CONTRASTS: 1

 001 TV-QUAD (1-7,M=4 ) 1.0 + -2.0*X + 1.0*X**2 TV WATCHING (QUADRATIC)

 CONDITIONING VARIABLE ID: BACK0017

 DESCRIPTION: HOMEWORK ASSIGNED?: BASED ON TIME SPENT ON HOMEWORK EACH DAY.

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: HWASSIGN

 NAEP ID: B006601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 HW-MISS (M ) 00 HOMEWORK ASSIGNED?: MISSING

 002 HW-NO (1 ) 10 HOMEWORK ASSIGNED?: NO

 003 HW-YES (2-5 ) 01 HOMEWORK ASSIGNED?: YES

 CONDITIONING VARIABLE ID: BACK0018

 DESCRIPTION: HOW MUCH TIME DO YOU USUALLY SPEND ON HOMEWORK EACH DAY? (LINEAR)

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: HOMEWRKL

 NAEP ID: B006601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 HWLIN-0 (1,2,M ) 0 HOMEWORK (LINEAR): DON’T HAVE ANY, DON’T DO

ANY, MISSING

 002 HWLIN-1 (3 ) 1 HOMEWORK (LINEAR): 1/2 HOUR OR LESS

 003 HWLIN-2 (4 ) 2 HOMEWORK (LINEAR): 1 HOUR

 004 HWLIN-3 (5 ) 3 HOMEWORK (LINEAR): MORE THAN 1 HOUR
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

692

 CONDITIONING VARIABLE ID: BACK0019

 DESCRIPTION: HOW MUCH TIME DO YOU USUALLY SPEND ON HOMEWORK EACH DAY (QUADRATIC)

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: HOMEWRKQ

 NAEP ID: B006601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: SCALE NUMBER OF INDEPENDENT CONTRASTS: 1

 001 HWQUAD-0 (1,2,M ) 0 HOMEWORK (QUADRATIC): DON’T HAVE ANY, DON’T

DO ANY, MISSING

 002 HWQUAD-1 (3 ) 1 HOMEWORK (QUADRATIC): 1/2 HOUR OR LESS

 003 HWQUAD-2 (4 ) 4 HOMEWORK (QUADRATIC): 1 HOUR

 004 HWQUAD-3 (5 ) 9 HOMEWORK (QUADRATIC): MORE THAN 1 HOUR

 CONDITIONING VARIABLE ID: BACK0020

 DESCRIPTION: NUMBER OF ITEMS IN THE HOME (NEWSPAPER, > 25 BOOKS, ENCYCLOPEDIA, MAGAZINES) (DERIVED)

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: HOMEITMS

 NAEP ID: HOMEEN3 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 HITEM<=2 (1,M ) 00 ITEMS IN HOME: ZERO TO TWO ITEMS, MISSING

 002 HITEM=3 (2 ) 10 ITEMS IN HOME: THREE ITEMS

 003 HITEM=4 (3 ) 01 ITEMS IN HOME: FOUR ITEMS

 CONDITIONING VARIABLE ID: BACK0021

 DESCRIPTION: ABOUT HOW MANY PAGES A DAY DO YOU HAVE TO READ FOR SCHOOL AND HOMEWORK?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: PGSREAD1

 NAEP ID: B001101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 PGS<6,? (5,M ) 0 PAGES READ: 5 OR FEWER A DAY, MISSING

 002 PGS>5 (1,2,3,4 ) 1 PAGES READ: 6-10, 11-15, 16-20, 20 OR MORE

 CONDITIONING VARIABLE ID: BACK0022

 DESCRIPTION: ABOUT HOW MANY PAGES A DAY DO YOU HAVE TO READ FOR SCHOOL AND HOMEWORK?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: PGSREAD2

 NAEP ID: B001101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 PGS<11,? (4,5,M ) 0 PAGES READ: 6-10, 5 OR FEWER A DAY, MISSING

 002 PGS>10 (1,2,3 ) 1 PAGES READ: 11-15, 16-20, 20 OR MORE

 CONDITIONING VARIABLE ID: BACK0023

 DESCRIPTION: STUDENTS ACCOMMODATION STATUS

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: ACCOM

 NAEP ID: ACCOM TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 ACC R/W (1,2 ) 0 ACCOMMODATED WITH APPROPRIATE BOOK OR WRONG

BOOK

 002 NO ACCOM (3 ) 1 NON ACCOMMODATED

 CONDITIONING VARIABLE ID: BACK0024

 DESCRIPTION: INTERACTION: GENDER BY RACE/ETHNICITY

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: GEND/RAC

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 3

 001 G/R 11 (11 ) 010101 GEND/RAC INTACT: 1. MALE 1. WHI/AI/O

 002 G/R 12 (12 ) -10000 GEND/RAC INTACT: 1. MALE 2. BLACK

 003 G/R 13 (13 ) 00-100 GEND/RAC INTACT: 1. MALE 3. HISPANIC

 004 G/R 14 (14 ) 0000-1 GEND/RAC INTACT: 1. MALE 4. ASIAN

 005 G/R 21 (21 ) -1-1-1 GEND/RAC INTACT: 2. FEMALE 1. WHI/AI/O

 006 G/R 22 (22 ) 010000 GEND/RAC INTACT: 2. FEMALE 2. BLACK

 007 G/R 23 (23 ) 000100 GEND/RAC INTACT: 2. FEMALE 3. HISPANIC

 008 G/R 24 (24 ) 000001 GEND/RAC INTACT: 2. FEMALE 4. ASIAN

 CONDITIONING VARIABLE ID: BACK0025

 DESCRIPTION: INTERACTION: GENDER BY TYPE OF LOCALE (7 CATEGORIES)

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: GEND/TOL

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 14

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 6

 001 G/T 11 (11 ) 010101010101 GEND/TOL INTACT: 1. MALE 1. BIG CTY7

 002 G/T 12 (12 ) -10000000000 GEND/TOL INTACT: 1. MALE 2. MID CTY7

 003 G/T 13 (13 ) 00-100000000 GEND/TOL INTACT: 1. MALE 3. FR/LCTY7

 004 G/T 14 (14 ) 0000-1000000 GEND/TOL INTACT: 1. MALE 4. FR/MCTY7

 005 G/T 15 (15 ) 000000-10000 GEND/TOL INTACT: 1. MALE 5. LAR TWN7

 006 G/T 16 (16 ) 00000000-100 GEND/TOL INTACT: 1. MALE 6. SML TWN7

 007 G/T 17 (17 ) 0000000000-1 GEND/TOL INTACT: 1. MALE 7. OTHER

 008 G/T 21 (21 ) -1-1-1-1-1-1 GEND/TOL INTACT: 2. FEMALE 1. BIG CTY7

 009 G/T 22 (22 ) 010000000000 GEND/TOL INTACT: 2. FEMALE 2. MID CTY7

 010 G/T 23 (23 ) 000100000000 GEND/TOL INTACT: 2. FEMALE 3. FR/LCTY7

 011 G/T 24 (24 ) 000001000000 GEND/TOL INTACT: 2. FEMALE 4. FR/MCTY7

 012 G/T 25 (25 ) 000000010000 GEND/TOL INTACT: 2. FEMALE 5. LAR TWN7

 013 G/T 26 (26 ) 000000000100 GEND/TOL INTACT: 2. FEMALE 6. SML TWN7

 014 G/T 27 (27 ) 000000000001 GEND/TOL INTACT: 2. FEMALE 7. OTHER

 CONDITIONING VARIABLE ID: BACK0026

 DESCRIPTION: INTERACTION: GENDER BY PARENTS’ EDUCATION

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: GEND/PAR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 10

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 4

 001 G/P 11 (11 ) 01010101 GEND/PAR INTACT: 1. MALE 1. < HS

 002 G/P 12 (12 ) -1000000 GEND/PAR INTACT: 1. MALE 2. HS GRAD

 003 G/P 13 (13 ) 00-10000 GEND/PAR INTACT: 1. MALE 3. POST HS

 004 G/P 14 (14 ) 0000-100 GEND/PAR INTACT: 1. MALE 4. COL GRAD

 005 G/P 15 (15 ) 000000-1 GEND/PAR INTACT: 1. MALE 5. PARED-?

 006 G/P 21 (21 ) -1-1-1-1 GEND/PAR INTACT: 2. FEMALE 1. < HS

 007 G/P 22 (22 ) 01000000 GEND/PAR INTACT: 2. FEMALE 2. HS GRAD

 008 G/P 23 (23 ) 00010000 GEND/PAR INTACT: 2. FEMALE 3. POST HS

 009 G/P 24 (24 ) 00000100 GEND/PAR INTACT: 2. FEMALE 4. COL GRAD

 010 G/P 25 (25 ) 00000001 GEND/PAR INTACT: 2. FEMALE 5. PARED-?
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

693

 CONDITIONING VARIABLE ID: BACK0027

 DESCRIPTION: INTERACTION: GENDER BY SCHOOL TYPE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: GEND/SCH

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 2

 001 G/S 11 (11 ) 0101 GEND/SCH INTACT: 1. MALE 1. PUBLIC

 002 G/S 12 (12 ) -100 GEND/SCH INTACT: 1. MALE 2. PRIVATE

 003 G/S 13 (13 ) 00-1 GEND/SCH INTACT: 1. MALE 3. CATHOLIC

 004 G/S 21 (21 ) -1-1 GEND/SCH INTACT: 2. FEMALE 1. PUBLIC

 005 G/S 22 (22 ) 0100 GEND/SCH INTACT: 2. FEMALE 2. PRIVATE

 006 G/S 23 (23 ) 0001 GEND/SCH INTACT: 2. FEMALE 3. CATHOLIC

 CONDITIONING VARIABLE ID: BACK0028

 DESCRIPTION: INTERACTION: RACE/ETHNICITY BY TYPE OF LOCALE (7 CATEGORIES)

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: RACE/TOL

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 28

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 18

 001 R/T 11 (11 ) 010101010101010101010101010101010101 RACE/TOL INTACT: 1. WHI/AI/O 1. BIG CTY7

 002 R/T 12 (12 ) -10000000000-10000000000-10000000000 RACE/TOL INTACT: 1. WHI/AI/O 2. MID CTY7

 003 R/T 13 (13 ) 00-10000000000-10000000000-100000000 RACE/TOL INTACT: 1. WHI/AI/O 3. FR/LCTY7

 004 R/T 14 (14 ) 0000-10000000000-10000000000-1000000 RACE/TOL INTACT: 1. WHI/AI/O 4. FR/MCTY7

 005 R/T 15 (15 ) 000000-10000000000-10000000000-10000 RACE/TOL INTACT: 1. WHI/AI/O 5. LAR TWN7

 006 R/T 16 (16 ) 00000000-10000000000-10000000000-100 RACE/TOL INTACT: 1. WHI/AI/O 6. SML TWN7

 007 R/T 17 (17 ) 0000000000-10000000000-10000000000-1 RACE/TOL INTACT: 1. WHI/AI/O 7. OTHER

 008 R/T 21 (21 ) -1-1-1-1-1-1000000000000000000000000 RACE/TOL INTACT: 2. BLACK 1. BIG CTY7

 009 R/T 22 (22 ) 010000000000000000000000000000000000 RACE/TOL INTACT: 2. BLACK 2. MID CTY7

 010 R/T 23 (23 ) 000100000000000000000000000000000000 RACE/TOL INTACT: 2. BLACK 3. FR/LCTY7

 011 R/T 24 (24 ) 000001000000000000000000000000000000 RACE/TOL INTACT: 2. BLACK 4. FR/MCTY7

 012 R/T 25 (25 ) 000000010000000000000000000000000000 RACE/TOL INTACT: 2. BLACK 5. LAR TWN7

 013 R/T 26 (26 ) 000000000100000000000000000000000000 RACE/TOL INTACT: 2. BLACK 6. SML TWN7

 014 R/T 27 (27 ) 000000000001000000000000000000000000 RACE/TOL INTACT: 2. BLACK 7. OTHER

 015 R/T 31 (31 ) 000000000000-1-1-1-1-1-1000000000000 RACE/TOL INTACT: 3. HISPANIC 1. BIG CTY7

 016 R/T 32 (32 ) 000000000000010000000000000000000000 RACE/TOL INTACT: 3. HISPANIC 2. MID CTY7

 017 R/T 33 (33 ) 000000000000000100000000000000000000 RACE/TOL INTACT: 3. HISPANIC 3. FR/LCTY7

 018 R/T 34 (34 ) 000000000000000001000000000000000000 RACE/TOL INTACT: 3. HISPANIC 4. FR/MCTY7

 019 R/T 35 (35 ) 000000000000000000010000000000000000 RACE/TOL INTACT: 3. HISPANIC 5. LAR TWN7

 020 R/T 36 (36 ) 000000000000000000000100000000000000 RACE/TOL INTACT: 3. HISPANIC 6. SML TWN7

 021 R/T 37 (37 ) 000000000000000000000001000000000000 RACE/TOL INTACT: 3. HISPANIC 7. OTHER

 022 R/T 41 (41 ) 000000000000000000000000-1-1-1-1-1-1 RACE/TOL INTACT: 4. ASIAN 1. BIG CTY7

 023 R/T 42 (42 ) 000000000000000000000000010000000000 RACE/TOL INTACT: 4. ASIAN 2. MID CTY7

 024 R/T 43 (43 ) 000000000000000000000000000100000000 RACE/TOL INTACT: 4. ASIAN 3. FR/LCTY7

 025 R/T 44 (44 ) 000000000000000000000000000001000000 RACE/TOL INTACT: 4. ASIAN 4. FR/MCTY7

 026 R/T 45 (45 ) 000000000000000000000000000000010000 RACE/TOL INTACT: 4. ASIAN 5. LAR TWN7

 027 R/T 46 (46 ) 000000000000000000000000000000000100 RACE/TOL INTACT: 4. ASIAN 6. SML TWN7

 028 R/T 47 (47 ) 000000000000000000000000000000000001 RACE/TOL INTACT: 4. ASIAN 7. OTHER

 CONDITIONING VARIABLE ID: BACK0029

 DESCRIPTION: INTERACTION: RACE/ETHNICITY BY PARENTS’ EDUCATION

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: RACE/PAR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 20

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 12

 001 R/P 11 (11 ) 010101010101010101010101 RACE/PAR INTACT: 1. WHI/AI/O 1. < HS

 002 R/P 12 (12 ) -1000000-1000000-1000000 RACE/PAR INTACT: 1. WHI/AI/O 2. HS GRAD

 003 R/P 13 (13 ) 00-1000000-1000000-10000 RACE/PAR INTACT: 1. WHI/AI/O 3. POST HS

 004 R/P 14 (14 ) 0000-1000000-1000000-100 RACE/PAR INTACT: 1. WHI/AI/O 4. COL GRAD

 005 R/P 15 (15 ) 000000-1000000-1000000-1 RACE/PAR INTACT: 1. WHI/AI/O 5. PARED-?

 006 R/P 21 (21 ) -1-1-1-10000000000000000 RACE/PAR INTACT: 2. BLACK 1. < HS

 007 R/P 22 (22 ) 010000000000000000000000 RACE/PAR INTACT: 2. BLACK 2. HS GRAD

 008 R/P 23 (23 ) 000100000000000000000000 RACE/PAR INTACT: 2. BLACK 3. POST HS

 009 R/P 24 (24 ) 000001000000000000000000 RACE/PAR INTACT: 2. BLACK 4. COL GRAD

 010 R/P 25 (25 ) 000000010000000000000000 RACE/PAR INTACT: 2. BLACK 5. PARED-?

 011 R/P 31 (31 ) 00000000-1-1-1-100000000 RACE/PAR INTACT: 3. HISPANIC 1. < HS

 012 R/P 32 (32 ) 000000000100000000000000 RACE/PAR INTACT: 3. HISPANIC 2. HS GRAD

 013 R/P 33 (33 ) 000000000001000000000000 RACE/PAR INTACT: 3. HISPANIC 3. POST HS

 014 R/P 34 (34 ) 000000000000010000000000 RACE/PAR INTACT: 3. HISPANIC 4. COL GRAD

 015 R/P 35 (35 ) 000000000000000100000000 RACE/PAR INTACT: 3. HISPANIC 5. PARED-?

 016 R/P 41 (41 ) 0000000000000000-1-1-1-1 RACE/PAR INTACT: 4. ASIAN 1. < HS

 017 R/P 42 (42 ) 000000000000000001000000 RACE/PAR INTACT: 4. ASIAN 2. HS GRAD

 018 R/P 43 (43 ) 000000000000000000010000 RACE/PAR INTACT: 4. ASIAN 3. POST HS

 019 R/P 44 (44 ) 000000000000000000000100 RACE/PAR INTACT: 4. ASIAN 4. COL GRAD

 020 R/P 45 (45 ) 000000000000000000000001 RACE/PAR INTACT: 4. ASIAN 5. PARED-?

 CONDITIONING VARIABLE ID: BACK0030

 DESCRIPTION: INTERACTION: RACE/ETHNICITY BY SCHOOL TYPE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: RACE/SCH

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 12

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 6

 001 R/S 11 (11 ) 010101010101 RACE/SCH INTACT: 1. WHI/AI/O 1. PUBLIC

 002 R/S 12 (12 ) -100-100-100 RACE/SCH INTACT: 1. WHI/AI/O 2. PRIVATE

 003 R/S 13 (13 ) 00-100-100-1 RACE/SCH INTACT: 1. WHI/AI/O 3. CATHOLIC

 004 R/S 21 (21 ) -1-100000000 RACE/SCH INTACT: 2. BLACK 1. PUBLIC

 005 R/S 22 (22 ) 010000000000 RACE/SCH INTACT: 2. BLACK 2. PRIVATE

 006 R/S 23 (23 ) 000100000000 RACE/SCH INTACT: 2. BLACK 3. CATHOLIC

 007 R/S 31 (31 ) 0000-1-10000 RACE/SCH INTACT: 3. HISPANIC 1. PUBLIC

 008 R/S 32 (32 ) 000001000000 RACE/SCH INTACT: 3. HISPANIC 2. PRIVATE

 009 R/S 33 (33 ) 000000010000 RACE/SCH INTACT: 3. HISPANIC 3. CATHOLIC

 010 R/S 41 (41 ) 00000000-1-1 RACE/SCH INTACT: 4. ASIAN 1. PUBLIC

 011 R/S 42 (42 ) 000000000100 RACE/SCH INTACT: 4. ASIAN 2. PRIVATE

 012 R/S 43 (43 ) 000000000001 RACE/SCH INTACT: 4. ASIAN 3. CATHOLIC
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

694

 CONDITIONING VARIABLE ID: BACK0031

 DESCRIPTION: INTERACTION: PARENT’S EDUCATION BY TYPE OF LOCALE (7 CATEGORIES)

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: PARE/TOL

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 35

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 24

 001 P/T 11 (11 ) 010101010101010101010101010101010101010101010101 PARE/TOL INTACT: 1. < HS 1. BIG CTY7

 002 P/T 12 (12 ) -10000000000-10000000000-10000000000-10000000000 PARE/TOL INTACT: 1. < HS 2. MID CTY7

 003 P/T 13 (13 ) 00-10000000000-10000000000-10000000000-100000000 PARE/TOL INTACT: 1. < HS 3. FR/LCTY7

 004 P/T 14 (14 ) 0000-10000000000-10000000000-10000000000-1000000 PARE/TOL INTACT: 1. < HS 4. FR/MCTY7

 005 P/T 15 (15 ) 000000-10000000000-10000000000-10000000000-10000 PARE/TOL INTACT: 1. < HS 5. LAR TWN7

 006 P/T 16 (16 ) 00000000-10000000000-10000000000-10000000000-100 PARE/TOL INTACT: 1. < HS 6. SML TWN7

 007 P/T 17 (17 ) 0000000000-10000000000-10000000000-10000000000-1 PARE/TOL INTACT: 1. < HS 7. OTHER

 008 P/T 21 (21 ) -1-1-1-1-1-1000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 1. BIG CTY7

 009 P/T 22 (22 ) 010000000000000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 2. MID CTY7

 010 P/T 23 (23 ) 000100000000000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 3. FR/LCTY7

 011 P/T 24 (24 ) 000001000000000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 4. FR/MCTY7

 012 P/T 25 (25 ) 000000010000000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 5. LAR TWN7

 013 P/T 26 (26 ) 000000000100000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 6. SML TWN7

 014 P/T 27 (27 ) 000000000001000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 7. OTHER

 015 P/T 31 (31 ) 000000000000-1-1-1-1-1-1000000000000000000000000 PARE/TOL INTACT: 3. POST HS 1. BIG CTY7

 016 P/T 32 (32 ) 000000000000010000000000000000000000000000000000 PARE/TOL INTACT: 3. POST HS 2. MID CTY7

 017 P/T 33 (33 ) 000000000000000100000000000000000000000000000000 PARE/TOL INTACT: 3. POST HS 3. FR/LCTY7

 018 P/T 34 (34 ) 000000000000000001000000000000000000000000000000 PARE/TOL INTACT: 3. POST HS 4. FR/MCTY7

 019 P/T 35 (35 ) 000000000000000000010000000000000000000000000000 PARE/TOL INTACT: 3. POST HS 5. LAR TWN7

 020 P/T 36 (36 ) 000000000000000000000100000000000000000000000000 PARE/TOL INTACT: 3. POST HS 6. SML TWN7

 021 P/T 37 (37 ) 000000000000000000000001000000000000000000000000 PARE/TOL INTACT: 3. POST HS 7. OTHER

 022 P/T 41 (41 ) 000000000000000000000000-1-1-1-1-1-1000000000000 PARE/TOL INTACT: 4. COL GRAD 1. BIG CTY7

 023 P/T 42 (42 ) 000000000000000000000000010000000000000000000000 PARE/TOL INTACT: 4. COL GRAD 2. MID CTY7

 024 P/T 43 (43 ) 000000000000000000000000000100000000000000000000 PARE/TOL INTACT: 4. COL GRAD 3. FR/LCTY7

 025 P/T 44 (44 ) 000000000000000000000000000001000000000000000000 PARE/TOL INTACT: 4. COL GRAD 4. FR/MCTY7

 026 P/T 45 (45 ) 000000000000000000000000000000010000000000000000 PARE/TOL INTACT: 4. COL GRAD 5. LAR TWN7

 027 P/T 46 (46 ) 000000000000000000000000000000000100000000000000 PARE/TOL INTACT: 4. COL GRAD 6. SML TWN7

 028 P/T 47 (47 ) 000000000000000000000000000000000001000000000000 PARE/TOL INTACT: 4. COL GRAD 7. OTHER

 029 P/T 51 (51 ) 000000000000000000000000000000000000-1-1-1-1-1-1 PARE/TOL INTACT: 5. PARED-? 1. BIG CTY7

 030 P/T 52 (52 ) 000000000000000000000000000000000000010000000000 PARE/TOL INTACT: 5. PARED-? 2. MID CTY7

 031 P/T 53 (53 ) 000000000000000000000000000000000000000100000000 PARE/TOL INTACT: 5. PARED-? 3. FR/LCTY7

 032 P/T 54 (54 ) 000000000000000000000000000000000000000001000000 PARE/TOL INTACT: 5. PARED-? 4. FR/MCTY7

 033 P/T 55 (55 ) 000000000000000000000000000000000000000000010000 PARE/TOL INTACT: 5. PARED-? 5. LAR TWN7

 034 P/T 56 (56 ) 000000000000000000000000000000000000000000000100 PARE/TOL INTACT: 5. PARED-? 6. SML TWN7

 035 P/T 57 (57 ) 000000000000000000000000000000000000000000000001 PARE/TOL INTACT: 5. PARED-? 7. OTHER

 CONDITIONING VARIABLE ID: BACK0032

 DESCRIPTION: INTERACTION: TYPE OF LOCALE (7 CATEGORIES) BY SCHOOL TYPE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: TOL7/SCH

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 21

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 12

 001 T/S 11 (11 ) 010101010101010101010101 TOL7/SCH INTACT: 1. BIG CTY7 1. PUBLIC

 002 T/S 12 (12 ) -100-100-100-100-100-100 TOL7/SCH INTACT: 1. BIG CTY7 2. PRIVATE

 003 T/S 13 (13 ) 00-100-100-100-100-100-1 TOL7/SCH INTACT: 1. BIG CTY7 3. CATHOLIC

 004 T/S 21 (21 ) -1-100000000000000000000 TOL7/SCH INTACT: 2. MID CTY7 1. PUBLIC

 005 T/S 22 (22 ) 010000000000000000000000 TOL7/SCH INTACT: 2. MID CTY7 2. PRIVATE

 006 T/S 23 (23 ) 000100000000000000000000 TOL7/SCH INTACT: 2. MID CTY7 3. CATHOLIC

 007 T/S 31 (31 ) 0000-1-10000000000000000 TOL7/SCH INTACT: 3. FR/LCTY7 1. PUBLIC

 008 T/S 32 (32 ) 000001000000000000000000 TOL7/SCH INTACT: 3. FR/LCTY7 2. PRIVATE

 009 T/S 33 (33 ) 000000010000000000000000 TOL7/SCH INTACT: 3. FR/LCTY7 3. CATHOLIC

 010 T/S 41 (41 ) 00000000-1-1000000000000 TOL7/SCH INTACT: 4. FR/MCTY7 1. PUBLIC

 011 T/S 42 (42 ) 000000000100000000000000 TOL7/SCH INTACT: 4. FR/MCTY7 2. PRIVATE

 012 T/S 43 (43 ) 000000000001000000000000 TOL7/SCH INTACT: 4. FR/MCTY7 3. CATHOLIC

 013 T/S 51 (51 ) 000000000000-1-100000000 TOL7/SCH INTACT: 5. LAR TWN7 1. PUBLIC

 014 T/S 52 (52 ) 000000000000010000000000 TOL7/SCH INTACT: 5. LAR TWN7 2. PRIVATE

 015 T/S 53 (53 ) 000000000000000100000000 TOL7/SCH INTACT: 5. LAR TWN7 3. CATHOLIC

 016 T/S 61 (61 ) 0000000000000000-1-10000 TOL7/SCH INTACT: 6. SML TWN7 1. PUBLIC

 017 T/S 62 (62 ) 000000000000000001000000 TOL7/SCH INTACT: 6. SML TWN7 2. PRIVATE

 018 T/S 63 (63 ) 000000000000000000010000 TOL7/SCH INTACT: 6. SML TWN7 3. CATHOLIC

 019 T/S 71 (71 ) 00000000000000000000-1-1 TOL7/SCH INTACT: 7. OTHER 1. PUBLIC

 020 T/S 72 (72 ) 000000000000000000000100 TOL7/SCH INTACT: 7. OTHER 2. PRIVATE

 021 T/S 73 (73 ) 000000000000000000000001 TOL7/SCH INTACT: 7. OTHER 3. CATHOLIC

 CONDITIONING VARIABLE ID: BACK0033

 DESCRIPTION: INTERACTION: PARENTS’ EDUCATION BY SCHOOL TYPE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: PARE/SCH

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 15

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 8

 001 P/S 11 (11 ) 0101010101010101 PARE/SCH INTACT: 1. < HS 1. PUBLIC

 002 P/S 12 (12 ) -100-100-100-100 PARE/SCH INTACT: 1. < HS 2. PRIVATE

 003 P/S 13 (13 ) 00-100-100-100-1 PARE/SCH INTACT: 1. < HS 3. CATHOLIC

 004 P/S 21 (21 ) -1-1000000000000 PARE/SCH INTACT: 2. HS GRAD 1. PUBLIC

 005 P/S 22 (22 ) 0100000000000000 PARE/SCH INTACT: 2. HS GRAD 2. PRIVATE

 006 P/S 23 (23 ) 0001000000000000 PARE/SCH INTACT: 2. HS GRAD 3. CATHOLIC

 007 P/S 31 (31 ) 0000-1-100000000 PARE/SCH INTACT: 3. POST HS 1. PUBLIC

 008 P/S 32 (32 ) 0000010000000000 PARE/SCH INTACT: 3. POST HS 2. PRIVATE

 009 P/S 33 (33 ) 0000000100000000 PARE/SCH INTACT: 3. POST HS 3. CATHOLIC

 010 P/S 41 (41 ) 00000000-1-10000 PARE/SCH INTACT: 4. COL GRAD 1. PUBLIC

 011 P/S 42 (42 ) 0000000001000000 PARE/SCH INTACT: 4. COL GRAD 2. PRIVATE

 012 P/S 43 (43 ) 0000000000010000 PARE/SCH INTACT: 4. COL GRAD 3. CATHOLIC

 013 P/S 51 (51 ) 000000000000-1-1 PARE/SCH INTACT: 5. PARED-? 1. PUBLIC

 014 P/S 52 (52 ) 0000000000000100 PARE/SCH INTACT: 5. PARED-? 2. PRIVATE

 015 P/S 53 (53 ) 0000000000000001 PARE/SCH INTACT: 5. PARED-? 3. CATHOLIC

 CONDITIONING VARIABLE ID: BACK0034

 DESCRIPTION: INTERACTION: ACCOMMODATED BY GENDER

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: ACCO/GEN

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 1

 001 A/G 11 (11 ) 01 ACCO/GEN INTACT: 1. ACC R/W 1. MALE

 002 A/G 12 (12 ) -1 ACCO/GEN INTACT: 1. ACC R/W 2. FEMALE

 003 A/G 21 (21 ) -1 ACCO/GEN INTACT: 2. NO ACCOM 1. MALE

 004 A/G 22 (22 ) 01 ACCO/GEN INTACT: 2. NO ACCOM 2. FEMALE
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

695

 CONDITIONING VARIABLE ID: BACK0035

 DESCRIPTION: INTERACTION: ACCOMMODATED BY RACE/ETHNICITY

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: ACCO/RAC

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 3

 001 A/R 11 (11 ) 010101 ACCO/RAC INTACT: 1. ACC R/W 1. WHI/AI/O

 002 A/R 12 (12 ) -10000 ACCO/RAC INTACT: 1. ACC R/W 2. BLACK

 003 A/R 13 (13 ) 00-100 ACCO/RAC INTACT: 1. ACC R/W 3. HISPANIC

 004 A/R 14 (14 ) 0000-1 ACCO/RAC INTACT: 1. ACC R/W 4. ASIAN

 005 A/R 21 (21 ) -1-1-1 ACCO/RAC INTACT: 2. NO ACCOM 1. WHI/AI/O

 006 A/R 22 (22 ) 010000 ACCO/RAC INTACT: 2. NO ACCOM 2. BLACK

 007 A/R 23 (23 ) 000100 ACCO/RAC INTACT: 2. NO ACCOM 3. HISPANIC

 008 A/R 24 (24 ) 000001 ACCO/RAC INTACT: 2. NO ACCOM 4. ASIAN

 CONDITIONING VARIABLE ID: BACK0036

 DESCRIPTION: INTERACTION: ACCOMMODATED BY TYPE OF LOCALE (7 CATEGORIES)

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: ACCO/TOL

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 14

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 6

 001 A/T 11 (11 ) 010101010101 ACCO/TOL INTACT: 1. ACC R/W 1. BIG CTY7

 002 A/T 12 (12 ) -10000000000 ACCO/TOL INTACT: 1. ACC R/W 2. MID CTY7

 003 A/T 13 (13 ) 00-100000000 ACCO/TOL INTACT: 1. ACC R/W 3. FR/LCTY7

 004 A/T 14 (14 ) 0000-1000000 ACCO/TOL INTACT: 1. ACC R/W 4. FR/MCTY7

 005 A/T 15 (15 ) 000000-10000 ACCO/TOL INTACT: 1. ACC R/W 5. LAR TWN7

 006 A/T 16 (16 ) 00000000-100 ACCO/TOL INTACT: 1. ACC R/W 6. SML TWN7

 007 A/T 17 (17 ) 0000000000-1 ACCO/TOL INTACT: 1. ACC R/W 7. OTHER

 008 A/T 21 (21 ) -1-1-1-1-1-1 ACCO/TOL INTACT: 2. NO ACCOM 1. BIG CTY7

 009 A/T 22 (22 ) 010000000000 ACCO/TOL INTACT: 2. NO ACCOM 2. MID CTY7

 010 A/T 23 (23 ) 000100000000 ACCO/TOL INTACT: 2. NO ACCOM 3. FR/LCTY7

 011 A/T 24 (24 ) 000001000000 ACCO/TOL INTACT: 2. NO ACCOM 4. FR/MCTY7

 012 A/T 25 (25 ) 000000010000 ACCO/TOL INTACT: 2. NO ACCOM 5. LAR TWN7

 013 A/T 26 (26 ) 000000000100 ACCO/TOL INTACT: 2. NO ACCOM 6. SML TWN7

 014 A/T 27 (27 ) 000000000001 ACCO/TOL INTACT: 2. NO ACCOM 7. OTHER

 CONDITIONING VARIABLE ID: BACK0037

 DESCRIPTION: INTERACTION: ACCOMMODATED BY PARENTS’ EDUCATION ALL GRADES

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: ACCO/PAR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 10

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 4

 001 A/P 11 (11 ) 01010101 ACCO/PAR INTACT: 1. ACC R/W 1. < HS

 002 A/P 12 (12 ) -1000000 ACCO/PAR INTACT: 1. ACC R/W 2. HS GRAD

 003 A/P 13 (13 ) 00-10000 ACCO/PAR INTACT: 1. ACC R/W 3. POST HS

 004 A/P 14 (14 ) 0000-100 ACCO/PAR INTACT: 1. ACC R/W 4. COL GRAD

 005 A/P 15 (15 ) 000000-1 ACCO/PAR INTACT: 1. ACC R/W 5. PARED-?

 006 A/P 21 (21 ) -1-1-1-1 ACCO/PAR INTACT: 2. NO ACCOM 1. < HS

 007 A/P 22 (22 ) 01000000 ACCO/PAR INTACT: 2. NO ACCOM 2. HS GRAD

 008 A/P 23 (23 ) 00010000 ACCO/PAR INTACT: 2. NO ACCOM 3. POST HS

 009 A/P 24 (24 ) 00000100 ACCO/PAR INTACT: 2. NO ACCOM 4. COL GRAD

 010 A/P 25 (25 ) 00000001 ACCO/PAR INTACT: 2. NO ACCOM 5. PARED-?

 CONDITIONING VARIABLE ID: BACK0038

 DESCRIPTION: INTERACTION: ACCOMMODATED BY SCHOOL TYPE

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: ACCO/SCH

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 2

 001 A/S 11 (11 ) 0101 ACCO/SCH INTACT: 1. ACC R/W 1. PUBLIC

 002 A/S 12 (12 ) -100 ACCO/SCH INTACT: 1. ACC R/W 2. PRIVATE

 003 A/S 13 (13 ) 00-1 ACCO/SCH INTACT: 1. ACC R/W 3. CATHOLIC

 004 A/S 21 (21 ) -1-1 ACCO/SCH INTACT: 2. NO ACCOM 1. PUBLIC

 005 A/S 22 (22 ) 0100 ACCO/SCH INTACT: 2. NO ACCOM 2. PRIVATE

 006 A/S 23 (23 ) 0001 ACCO/SCH INTACT: 2. NO ACCOM 3. CATHOLIC

 CONDITIONING VARIABLE ID: BACK0039

 DESCRIPTION: INTERACTION: ACCOMMODATED BY IEP

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: ACCO/IEP

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 1

 001 A/I 11 (11 ) 01 ACCO/IEP INTACT: 1. ACC R/W 1. IEP-YES

 002 A/I 12 (12 ) -1 ACCO/IEP INTACT: 1. ACC R/W 2. IEP-NO

 003 A/I 21 (21 ) -1 ACCO/IEP INTACT: 2. NO ACCOM 1. IEP-YES

 004 A/I 22 (22 ) 01 ACCO/IEP INTACT: 2. NO ACCOM 2. IEP-NO

 CONDITIONING VARIABLE ID: BACK0040

 DESCRIPTION: INTERACTION: ACCOMMODATED BY LEP

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: ACCO/LEP

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 1

 001 A/L 11 (11 ) 01 ACCO/LEP INTACT: 1. ACC R/W 1. LEP-YES

 002 A/L 12 (12 ) -1 ACCO/LEP INTACT: 1. ACC R/W 2. LEP-NO

 003 A/L 21 (21 ) -1 ACCO/LEP INTACT: 2. NO ACCOM 1. LEP-YES

 004 A/L 22 (22 ) 01 ACCO/LEP INTACT: 2. NO ACCOM 2. LEP-NO

 CONDITIONING VARIABLE ID: BACK0044

 DESCRIPTION: REPORTING SAMPLE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: RPTSAMP TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 RPTSAMP (01 ) 0 YES

 002 RPT NO (02 ) 1 NO
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

696

 CONDITIONING VARIABLE ID: BACK0046

 DESCRIPTION: WHICH RACE/ETHNICITY BEST DESCRIBES YOU

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B003001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 WHITE (01 ) 000000 WHITE

 002 BLACK (02 ) 100000 BLACK

 003 HISPANIC (03 ) 010000 HISPANIC

 004 ASIAN AM (04 ) 001000 ASIAN/PACIFIC ISLAND

 005 AMER IND (05 ) 000100 AMER IND/ALASKA NATV

 006 OTHER (06 ) 000010 OTHER

 007 B003001M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: BACK0047

 DESCRIPTION: IF HISPANIC, WHAT IS YOUR HISPANIC BACKGROUND

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B003101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 NOT HISP (01 ) 00000 NOT HISPANIC

 002 MEXICAN (02 ) 10000 MEX,MEX AMER,CHICANO

 003 PUER RIC (03 ) 01000 PUERTO RICAN

 004 CUBAN (04 ) 00100 CUBAN

 005 OTHER (05 ) 00010 OTHER HISPANIC

 006 B003101M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: BACK0048

 DESCRIPTION: HOW LONG LIVED IN UNITED STATES

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B013001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 B013001A (01 ) 0000 ALL MY LIFE

 002 B013001B (02 ) 1000 MORE THAN 5 YEARS

 003 B013001C (03 ) 0100 3-5 YEARS

 004 B013001D (04 ) 0010 LESS THAN 3 YEARS

 005 B013001M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0049

 DESCRIPTION: HOW OFTEN OTHER THAN ENGLISH SPOKEN AT HOME

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B013101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 B013101A (01 ) 0000 ALL OR MOST OF TIME

 002 B013101B (02 ) 1000 ABOUT HALF OF TIME

 003 B013101C (03 ) 0100 LESS THAN HALF TIME

 004 B013101D (04 ) 0010 NEVER

 005 B013101M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0050

 DESCRIPTION: MOTHER GRADUATED HIGH SCHOOL

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B013201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B013201Y (01 ) 000 YES

 002 B013201N (02 ) 100 NO

 003 B013201M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0051

 DESCRIPTION: MOTHER HAD SOME EDUCATION AFTER HIGH SCHOOL

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B013301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B013301Y (01 ) 000 YES

 002 B013301N (02 ) 100 NO

 003 B013301M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0052

 DESCRIPTION: MOTHER GRADUATED COLLEGE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B013401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B013401Y (01 ) 000 YES

 002 B013401N (02 ) 100 NO

 003 B013401M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0053

 DESCRIPTION: FATHER GRADUATED HIGH SCHOOL

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B013501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B013501Y (01 ) 000 YES

 002 B013501N (02 ) 100 NO

 003 B013501M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0054

 DESCRIPTION: FATHER HAD SOME EDUCATION AFTER HIGH SCHOOL

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B013601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B013601Y (01 ) 000 YES

 002 B013601N (02 ) 100 NO

 003 B013601M (M, IDK ) 001 MISSING, I DON’T KNOW
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

697

 CONDITIONING VARIABLE ID: BACK0055

 DESCRIPTION: FATHER GRADUATED COLLEGE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B013701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B013701Y (01 ) 000 YES

 002 B013701N (02 ) 100 NO

 003 B013701M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0056

 DESCRIPTION: DOES YOUR FAMILY GET A NEWSPAPER REGULARLY

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B000901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B000901Y (01 ) 000 YES

 002 B000901N (02 ) 100 NO

 003 B000901M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0057

 DESCRIPTION: IS THERE AN ENCYCLOPEDIA IN YOUR HOME

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B000903 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B000903Y (01 ) 000 YES

 002 B000903N (02 ) 100 NO

 003 B000903M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0058

 DESCRIPTION: HOW MANY BOOKS ARE IN YOUR HOME

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B013801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 B013801A (01 ) 0000 0-10 (FEW)

 002 B013801B (02 ) 1000 11-25 (1 SHELF)

 003 B013801C (03 ) 0100 26-100 (1 BOOKCASE)

 004 B013801D (04 ) 0010 >100 (>1 BOOKCASE)

 005 B013801M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0059

 DESCRIPTION: DOES YOUR FAMILY GET MAGAZINES REGULARLY

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B000905 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B000905Y (01 ) 000 YES

 002 B000905N (02 ) 100 NO

 003 B000905M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0060

 DESCRIPTION: HOURS OF TV/VIDEO WATCHED ON SCHOOL DAYS

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B013901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 B013901N (01 ) 0000000 NONE

 002 B013901B (02 ) 1000000 ONE HOUR OR LESS

 003 B013901C (03 ) 0100000 2 HOURS

 004 B013901D (04 ) 0010000 3 HOURS

 005 B013901E (05 ) 0001000 4 HOURS

 006 B013901F (06 ) 0000100 5 HOURS

 007 B013901G (07 ) 0000010 6+ HOURS

 008 B013901M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: BACK0061

 DESCRIPTION: TIME SPENT ON HOMEWORK EACH DAY

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B006601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 B006601N (01 ) 00000 DON’T USUALLY HAVE

 002 B006601B (02 ) 10000 HAVE BUT DON’T DO

 003 B006601C (03 ) 01000 1/2 HOUR OR LESS

 004 B006601D (04 ) 00100 1 HOUR

 005 B006601E (05 ) 00010 MORE THAN 1 HOUR

 006 B006601M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: BACK0062

 DESCRIPTION: HOW MANY PAGES READ IN SCHOOL AND FOR HOMEWORK

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B001101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 B001101A (01 ) 00000 MORE THAN 20

 002 B001101B (02 ) 10000 16-20

 003 B001101C (03 ) 01000 11-15

 004 B001101D (04 ) 00100 6-10

 005 B001101E (05 ) 00010 5 OR FEWER

 006 B001101M (M ) 00001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

698

 CONDITIONING VARIABLE ID: BACK0063

 DESCRIPTION: DAYS ABSENT FROM SCHOOL LAST MONTH

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B014001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 B014001N (01 ) 00000 NONE

 002 B014001B (02 ) 10000 1 OR 2 DAYS

 003 B014001C (03 ) 01000 3 OR 4 DAYS

 004 B014001D (04 ) 00100 5 TO 9 DAYS

 005 B014001E (05 ) 00010 10 OR MORE DAYS

 006 B014001M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: BACK0064

 DESCRIPTION: TIMES CHANGED SCHOOLS IN PAST TWO YEARS

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B007301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 B007301N (01 ) 0000 NONE

 002 B007301B (02 ) 1000 1

 003 B007301C (03 ) 0100 2

 004 B007301D (04 ) 0010 3 OR MORE

 005 B007301M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0065

 DESCRIPTION: HOW OFTEN DISCUSS STUDIES AT HOME

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B007401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 B007401A (01 ) 0000 ALMOST EVERY DAY

 002 B007401B (02 ) 1000 ONCE/TWICE A WEEK

 003 B007401C (03 ) 0100 ONCE/TWICE A MONTH

 004 B007401D (04 ) 0010 NEVER OR HARDLY EVER

 005 B007401M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0066

 DESCRIPTION: HOW OFTEN USE COMPUTER AT HOME FOR SCHOOLWORK

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B014101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 B014101A (01 ) 00000 NO COMPUTER AT HOME

 002 B014101B (02 ) 10000 NEVER OR HARDLY EVER

 003 B014101C (03 ) 01000 ONCE/TWICE A MONTH

 004 B014101D (04 ) 00100 ONCE/TWICE A WEEK

 005 B014101E (05 ) 00010 ALMOST EVERY DAY

 006 B014101M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0001

 DESCRIPTION: HOW HARD TRIED ON THIS WRITING TEST THAN ON OTHERS

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W803001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 W803001A (01 ) 0000 TRIED MUCH HARDER

 002 W803001B (02 ) 1000 TRIED HARDER

 003 W803001C (03 ) 0100 TRIED ABOUT AS HARD

 004 W803001N (04 ) 0010 TRIED NOT AS HARD

 005 W803001M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0002

 DESCRIPTION: HOW IMPORTANT TO DO WELL ON THIS WRITING TEST

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W803101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 W803101A (01 ) 0000 VERY IMPORTANT

 002 W803101B (02 ) 1000 IMPORTANT

 003 W803101C (03 ) 0100 SOMEWHAT IMPORTANT

 004 W803101N (04 ) 0010 NOT VERY IMPORTANT

 005 W803101M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0003

 DESCRIPTION: HOW OFTEN TAKE ESSAY TEST FOR WHOLE CLASS PERIOD

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W803201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 W803201A (01 ) 0000 AT LEAST ONCE A WEEK

 002 W803201B (02 ) 1000 ONCE/TWICE A MONTH

 003 W803201C (03 ) 0100 ONCE/TWICE A YEAR

 004 W803201D (04 ) 0010 NEVER

 005 W803201M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0004

 DESCRIPTION: MY FRIENDS MAKE FUN OF PEOPLE WHO TRY TO DO WELL

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W803301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 W803301A (01 ) 0000 STRONGLY AGREE

 002 W803301B (02 ) 1000 AGREE

 003 W803301C (03 ) 0100 DISAGREE

 004 W803301D (04 ) 0010 STRONGLY DISAGREE

 005 W803301M (M ) 0001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

699

 CONDITIONING VARIABLE ID: SUBJ0005

 DESCRIPTION: I HAVE FRIENDS TO TALK TO IF NEED HELP W/SCHOOL

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W803302 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 W803302A (01 ) 0000 STRONGLY AGREE

 002 W803302B (02 ) 1000 AGREE

 003 W803302C (03 ) 0100 DISAGREE

 004 W803302D (04 ) 0010 STRONGLY DISAGREE

 005 W803302M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0006

 DESCRIPTION: I LIKE TO WRITE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W801901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 W801901A (01 ) 00000 STRONGLY AGREE

 002 W801901B (02 ) 10000 AGREE

 003 W801901C (03 ) 01000 UNDECIDED

 004 W801901D (04 ) 00100 DISAGREE

 005 W801901E (05 ) 00010 STRONGLY DISAGREE

 006 W801901M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0007

 DESCRIPTION: I AM GOOD AT WRITING

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W801902 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 W801902A (01 ) 00000 STRONGLY AGREE

 002 W801902B (02 ) 10000 AGREE

 003 W801902C (03 ) 01000 UNDECIDED

 004 W801902D (04 ) 00100 DISAGREE

 005 W801902E (05 ) 00010 STRONGLY DISAGREE

 006 W801902M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0008

 DESCRIPTION: TEACHER TALKS ABOUT WHAT YOU ARE WRITING

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 W802001A (01 ) 000 ALWAYS

 002 W802001B (02 ) 100 SOMETIMES

 003 W802001C (03 ) 010 NEVER

 004 W802001M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0009

 DESCRIPTION: TEACHER ASKS TO WRITE MORE THAN ONE DRAFT OF PAPER

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 W802101A (01 ) 000 ALWAYS

 002 W802101B (02 ) 100 SOMETIMES

 003 W802101C (03 ) 010 NEVER

 004 W802101M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0010

 DESCRIPTION: TEACHER ASKS TO CONTRIBUTE WRITING TO A COLLECTION

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 W802201A (01 ) 000 ALWAYS

 002 W802201B (02 ) 100 SOMETIMES

 003 W802201C (03 ) 010 NEVER

 004 W802201M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0011

 DESCRIPTION: DO SPELLING, PUNCTUATION, GRAMMAR EXERCISES

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 W802301A (01 ) 0000 ALMOST EVERY DAY

 002 W802301B (02 ) 1000 ONCE/TWICE A WEEK

 003 W802301C (03 ) 0100 ONCE/TWICE A MONTH

 004 W802301D (04 ) 0010 NEVER OR HARDLY EVER

 005 W802301M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0012

 DESCRIPTION: HOW OFTEN WRITE A STORY OR REPORT

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802302 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 W802302A (01 ) 0000 ALMOST EVERY DAY

 002 W802302B (02 ) 1000 ONCE/TWICE A WEEK

 003 W802302C (03 ) 0100 ONCE/TWICE A MONTH

 004 W802302D (04 ) 0010 NEVER OR HARDLY EVER

 005 W802302M (M ) 0001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

700

 CONDITIONING VARIABLE ID: SUBJ0013

 DESCRIPTION: HOW OFTEN WORK IN PAIRS/SMALL GROUPS-WRITING

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802303 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 W802303A (01 ) 0000 ALMOST EVERY DAY

 002 W802303B (02 ) 1000 ONCE/TWICE A WEEK

 003 W802303C (03 ) 0100 ONCE/TWICE A MONTH

 004 W802303D (04 ) 0010 NEVER OR HARDLY EVER

 005 W802303M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0014

 DESCRIPTION: HOW OFTEN WRITE IN A LOG/JOURNAL

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802304 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 W802304A (01 ) 0000 ALMOST EVERY DAY

 002 W802304B (02 ) 1000 ONCE/TWICE A WEEK

 003 W802304C (03 ) 0100 ONCE/TWICE A MONTH

 004 W802304D (04 ) 0010 NEVER OR HARDLY EVER

 005 W802304M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0015

 DESCRIPTION: DO YOU/TEACHER SAVE WRITING-FOLDER/PORTFOLIO

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 W802401Y (01 ) 00 YES

 002 W802401N (02 ) 10 NO

 003 W802401M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SUBJ0016

 DESCRIPTION: GRADE/WRITING-SPELLING, PUNCTUATION, GRAMMAR

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 W802501A (01 ) 000 VERY IMPORTANT

 002 W802501B (02 ) 100 MODERATELY IMPORTANT

 003 W802501N (03 ) 010 NOT VERY IMPORTANT

 004 W802501M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0017

 DESCRIPTION: GRADE/WRITING-ORGANIZATION OF PAPER

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802502 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 W802502A (01 ) 000 VERY IMPORTANT

 002 W802502B (02 ) 100 MODERATELY IMPORTANT

 003 W802502N (03 ) 010 NOT VERY IMPORTANT

 004 W802502M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0018

 DESCRIPTION: GRADE/WRITING-QUALITY, CREATIVITY OF IDEAS

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802503 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 W802503A (01 ) 000 VERY IMPORTANT

 002 W802503B (02 ) 100 MODERATELY IMPORTANT

 003 W802503N (03 ) 010 NOT VERY IMPORTANT

 004 W802503M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0019

 DESCRIPTION: GRADE/WRITING-LENGTH OF PAPER

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802504 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 W802504A (01 ) 000 VERY IMPORTANT

 002 W802504B (02 ) 100 MODERATELY IMPORTANT

 003 W802504N (03 ) 010 NOT VERY IMPORTANT

 004 W802504M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0020

 DESCRIPTION: ON COMPUTER-DO SPELLING, PUNCTUATION, GRAMMAR

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 W802601A (01 ) 0000 ALMOST EVERY DAY

 002 W802601B (02 ) 1000 ONCE/TWICE A WEEK

 003 W802601C (03 ) 0100 ONCE/TWICE A MONTH

 004 W802601D (04 ) 0010 NEVER OR HARDLY EVER

 005 W802601M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0021

 DESCRIPTION: ON COMPUTER-WRITE IN A LOG/JOURNAL

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802602 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 W802602A (01 ) 0000 ALMOST EVERY DAY

 002 W802602B (02 ) 1000 ONCE/TWICE A WEEK

 003 W802602C (03 ) 0100 ONCE/TWICE A MONTH

 004 W802602D (04 ) 0010 NEVER OR HARDLY EVER

 005 W802602M (M ) 0001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

701

 CONDITIONING VARIABLE ID: SUBJ0022

 DESCRIPTION: ON COMPUTER-WRITE DRAFTS/FINAL VERSIONS OF PAPERS

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802603 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 W802603A (01 ) 0000 ALMOST EVERY DAY

 002 W802603B (02 ) 1000 ONCE/TWICE A WEEK

 003 W802603C (03 ) 0100 ONCE/TWICE A MONTH

 004 W802603D (04 ) 0010 NEVER OR HARDLY EVER

 005 W802603M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0001

 DESCRIPTION: FOURTH GRADERS ASSIGNED TO CLASS BY ABILITY

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C042501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C042501Y (01 ) 00 YES

 002 C042501N (02 ) 10 NO

 003 C042501M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0002

 DESCRIPTION: HOW OFTEN STUDENTS RECEIVE READING INSTRUCTION

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C042601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C042601A (01 ) 00000 EVERY DAY

 002 C042601B (02 ) 10000 3-4 TIMES A WEEK

 003 C042601C (03 ) 01000 ONCE OR TWICE A WEEK

 004 C042601D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C042601N (05 ) 00010 SUBJECT NOT TAUGHT

 006 C042601M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0003

 DESCRIPTION: HOW OFTEN STUDENTS RECEIVE WRITING INSTRUCTION

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C042602 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C042602A (01 ) 00000 EVERY DAY

 002 C042602B (02 ) 10000 3-4 TIMES A WEEK

 003 C042602C (03 ) 01000 ONCE OR TWICE A WEEK

 004 C042602D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C042602N (05 ) 00010 SUBJECT NOT TAUGHT

 006 C042602M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0004

 DESCRIPTION: HOW OFTEN STUDENTS RECEIVE SOC STUDIES INSTRUCT

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C042603 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C042603A (01 ) 00000 EVERY DAY

 002 C042603B (02 ) 10000 3-4 TIMES A WEEK

 003 C042603C (03 ) 01000 ONCE OR TWICE A WEEK

 004 C042603D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C042603N (05 ) 00010 SUBJECT NOT TAUGHT

 006 C042603M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0005

 DESCRIPTION: HOW OFTEN STUDENTS RECEIVE COMPUTER USE INSTRUCT

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C042604 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C042604A (01 ) 00000 EVERY DAY

 002 C042604B (02 ) 10000 3-4 TIMES A WEEK

 003 C042604C (03 ) 01000 ONCE OR TWICE A WEEK

 004 C042604D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C042604N (05 ) 00010 SUBJECT NOT TAUGHT

 006 C042604M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0006

 DESCRIPTION: DOES SCHOOL USE BLOCK SCHEDULING

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C042701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 C042701Y (01 ) 000 YES-ALL SUBJECTS

 002 C042701Y (02 ) 100 YES-SOME SUBJECTS

 003 C042701N (03 ) 010 NO

 004 C042701M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SCHL0007

 DESCRIPTION: ARE COMPUTERS AVAILABLE IN ALL CLASSROOMS

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C042801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C042801Y (01 ) 00 YES

 002 C042801N (02 ) 10 NO

 003 C042801M (M ) 01 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

702

 CONDITIONING VARIABLE ID: SCHL0008

 DESCRIPTION: ARE COMPUTERS AVAILABLE IN COMPUTER LAB

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C042802 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C042802Y (01 ) 00 YES

 002 C042802N (02 ) 10 NO

 003 C042802M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0009

 DESCRIPTION: ARE COMPUTERS AVAILABLE TO CLASSROOM WHEN NEEDED

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C042803 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C042803Y (01 ) 00 YES

 002 C042803N (02 ) 10 NO

 003 C042803M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0010

 DESCRIPTION: HOW MANY COMPUTERS AVAILABLE TO STUDENTS

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C042901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 C042901N (01 ) 0000000 NONE

 002 C042901B (02 ) 1000000 1-10

 003 C042901C (03 ) 0100000 11-25

 004 C042901D (04 ) 0010000 26-50

 005 C042901E (05 ) 0001000 51-75

 006 C042901F (06 ) 0000100 76-100

 007 C042901G (07 ) 0000010 MORE THAN 100

 008 C042901M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0011

 DESCRIPTION: PRIMARY WAY LIBRARY IS STAFFED

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C036601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C036601N (01 ) 0000 NO LIBRARY IN SCHOOL

 002 C036601N (02 ) 1000 LIBRARY-NO/VOL STAFF

 003 C036601C (03 ) 0100 PART-TIME STAFF

 004 C036601D (04 ) 0010 FULL-TIME STAFF

 005 C036601M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0012

 DESCRIPTION: PARENTS PARTICIPATE-PARENT-TEACHER ORG

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043001A (01 ) 00000 NOT AVAILABLE

 002 C043001B (02 ) 10000 0-10%

 003 C043001C (03 ) 01000 11-25%

 004 C043001D (04 ) 00100 26-50%

 005 C043001E (05 ) 00010 51-100%

 006 C043001M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0013

 DESCRIPTION: PARENTS PARTICIPATE-OPEN HOUSE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043002 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043002A (01 ) 00000 NOT AVAILABLE

 002 C043002B (02 ) 10000 0-10%

 003 C043002C (03 ) 01000 11-25%

 004 C043002D (04 ) 00100 26-50%

 005 C043002E (05 ) 00010 51-100%

 006 C043002M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0014

 DESCRIPTION: PARTICIPATE-PARENT-TEACHER CONFERENCE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043003 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043003A (01 ) 00000 NOT AVAILABLE

 002 C043003B (02 ) 10000 0-10%

 003 C043003C (03 ) 01000 11-25%

 004 C043003D (04 ) 00100 26-50%

 005 C043003E (05 ) 00010 51-100%

 006 C043003M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0015

 DESCRIPTION: PARENTS PARTICIPATE-SCHOOL CURRICULUM DECISIONS

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043004 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043004A (01 ) 00000 NOT AVAILABLE

 002 C043004B (02 ) 10000 0-10%

 003 C043004C (03 ) 01000 11-25%

 004 C043004D (04 ) 00100 26-50%

 005 C043004E (05 ) 00010 51-100%

 006 C043004M (M ) 00001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

703

 CONDITIONING VARIABLE ID: SCHL0016

 DESCRIPTION: PARENTS PARTICIPATE-VOLUNTEER PROGRAMS

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043005 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043005A (01 ) 00000 NOT AVAILABLE

 002 C043005B (02 ) 10000 0-10%

 003 C043005C (03 ) 01000 11-25%

 004 C043005D (04 ) 00100 26-50%

 005 C043005E (05 ) 00010 51-100%

 006 C043005M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0017

 DESCRIPTION: PARENTS PARTICIPATE-PARENTING-SKILLS PROGRAM

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043006 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043006A (01 ) 00000 NOT AVAILABLE

 002 C043006B (02 ) 10000 0-10%

 003 C043006C (03 ) 01000 11-25%

 004 C043006D (04 ) 00100 26-50%

 005 C043006E (05 ) 00010 51-100%

 006 C043006M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0018

 DESCRIPTION: PARENTS PARTICIPATE-SCHOOL ADVISORY COMMITTEES

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043007 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043007A (01 ) 00000 NOT AVAILABLE

 002 C043007B (02 ) 10000 0-10%

 003 C043007C (03 ) 01000 11-25%

 004 C043007D (04 ) 00100 26-50%

 005 C043007E (05 ) 00010 51-100%

 006 C043007M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0019

 DESCRIPTION: PARENTS PARTICIPATE-CLASSROOM ASSISTANTS

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043008 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043008A (01 ) 00000 NOT AVAILABLE

 002 C043008B (02 ) 10000 0-10%

 003 C043008C (03 ) 01000 11-25%

 004 C043008D (04 ) 00100 26-50%

 005 C043008E (05 ) 00010 51-100%

 006 C043008M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0020

 DESCRIPTION: IS STUDENT ABSENTEEISM A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032402 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032402A (01 ) 0000 SERIOUS

 002 C032402B (02 ) 1000 MODERATE

 003 C032402C (03 ) 0100 MINOR

 004 C032402N (04 ) 0010 NOT A PROBLEM

 005 C032402M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0021

 DESCRIPTION: IS STUDENT TARDINESS A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032401A (01 ) 0000 SERIOUS

 002 C032401B (02 ) 1000 MODERATE

 003 C032401C (03 ) 0100 MINOR

 004 C032401N (04 ) 0010 NOT A PROBLEM

 005 C032401M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0022

 DESCRIPTION: ARE PHYSICAL CONFLICTS A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032404 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032404A (01 ) 0000 SERIOUS

 002 C032404B (02 ) 1000 MODERATE

 003 C032404C (03 ) 0100 MINOR

 004 C032404N (04 ) 0010 NOT A PROBLEM

 005 C032404M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0023

 DESCRIPTION: ARE RACIAL/CULT. CONFLICTS A PROBLEM IN SCHOOL

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032407 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032407A (01 ) 0000 SERIOUS

 002 C032407B (02 ) 1000 MODERATE

 003 C032407C (03 ) 0100 MINOR

 004 C032407N (04 ) 0010 NOT A PROBLEM

 005 C032407M (M ) 0001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

704

 CONDITIONING VARIABLE ID: SCHL0024

 DESCRIPTION: IS STUDENT HEALTH A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032408 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032408A (01 ) 0000 SERIOUS

 002 C032408B (02 ) 1000 MODERATE

 003 C032408C (03 ) 0100 MINOR

 004 C032408N (04 ) 0010 NOT A PROBLEM

 005 C032408M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0025

 DESCRIPTION: IS LACK OF PARENT INVLVMNT A PROBLEM IN SCHOOL

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032409 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032409A (01 ) 0000 SERIOUS

 002 C032409B (02 ) 1000 MODERATE

 003 C032409C (03 ) 0100 MINOR

 004 C032409N (04 ) 0010 NOT A PROBLEM

 005 C032409M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0026

 DESCRIPTION: IS STUDENT ALCOHOL USE A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032410 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032410A (01 ) 0000 SERIOUS

 002 C032410B (02 ) 1000 MODERATE

 003 C032410C (03 ) 0100 MINOR

 004 C032410N (04 ) 0010 NOT A PROBLEM

 005 C032410M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0027

 DESCRIPTION: IS STUDENT TOBACCO USE A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032411 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032411A (01 ) 0000 SERIOUS

 002 C032411B (02 ) 1000 MODERATE

 003 C032411C (03 ) 0100 MINOR

 004 C032411N (04 ) 0010 NOT A PROBLEM

 005 C032411M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0028

 DESCRIPTION: IS STUDENT DRUG USE A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032412 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032412A (01 ) 0000 SERIOUS

 002 C032412B (02 ) 1000 MODERATE

 003 C032412C (03 ) 0100 MINOR

 004 C032412N (04 ) 0010 NOT A PROBLEM

 005 C032412M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0029

 DESCRIPTION: ARE GANG ACTIVITIES A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032413 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032413A (01 ) 0000 SERIOUS

 002 C032413B (02 ) 1000 MODERATE

 003 C032413C (03 ) 0100 MINOR

 004 C032413N (04 ) 0010 NOT A PROBLEM

 005 C032413M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0030

 DESCRIPTION: IS STUDENT MISBEHAVIOR A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032414 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032414A (01 ) 0000 SERIOUS

 002 C032414B (02 ) 1000 MODERATE

 003 C032414C (03 ) 0100 MINOR

 004 C032414N (04 ) 0010 NOT A PROBLEM

 005 C032414M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0031

 DESCRIPTION: IS STUDENT CHEATING A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C043101A (01 ) 0000 SERIOUS

 002 C043101B (02 ) 1000 MODERATE

 003 C043101C (03 ) 0100 MINOR

 004 C043101N (04 ) 0010 NOT A PROBLEM

 005 C043101M (M ) 0001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

705

 CONDITIONING VARIABLE ID: SCHL0032

 DESCRIPTION: IS TEACHER ABSENTEEISM A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043102 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C043102A (01 ) 0000 SERIOUS

 002 C043102B (02 ) 1000 MODERATE

 003 C043102C (03 ) 0100 MINOR

 004 C043102N (04 ) 0010 NOT A PROBLEM

 005 C043102M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0033

 DESCRIPTION: ARE PHYSICAL CONFLICTS BETWEEN STUDENTS/TEACHERS

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043103 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C043103A (01 ) 0000 SERIOUS

 002 C043103B (02 ) 1000 MODERATE

 003 C043103C (03 ) 0100 MINOR

 004 C043103N (04 ) 0010 NOT A PROBLEM

 005 C043103M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0034

 DESCRIPTION: IS VANDALISM A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043104 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C043104A (01 ) 0000 SERIOUS

 002 C043104B (02 ) 1000 MODERATE

 003 C043104C (03 ) 0100 MINOR

 004 C043104N (04 ) 0010 NOT A PROBLEM

 005 C043104M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0035

 DESCRIPTION: TEACHER MORALE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032502 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032502A (01 ) 0000 VERY POSITIVE

 002 C032502B (02 ) 1000 SOMEWHAT POSITIVE

 003 C032502C (03 ) 0100 SOMEWHAT NEGATIVE

 004 C032502D (04 ) 0010 VERY NEGATIVE

 005 C032502M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0036

 DESCRIPTION: STUDENT ATTITUDES TOWARD ACADEMIC ACHIEVEMENT

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032503 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032503A (01 ) 0000 VERY POSITIVE

 002 C032503B (02 ) 1000 SOMEWHAT POSITIVE

 003 C032503C (03 ) 0100 SOMEWHAT NEGATIVE

 004 C032503D (04 ) 0010 VERY NEGATIVE

 005 C032503M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0037

 DESCRIPTION: PARENT SUPPORT FOR STUDENT ACHIEVEMENT

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032505 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032505A (01 ) 0000 VERY POSITIVE

 002 C032505B (02 ) 1000 SOMEWHAT POSITIVE

 003 C032505C (03 ) 0100 SOMEWHAT NEGATIVE

 004 C032505D (04 ) 0010 VERY NEGATIVE

 005 C032505M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0038

 DESCRIPTION: REGARD FOR SCHOOL PROPERTY

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032506 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032506A (01 ) 0000 VERY POSITIVE

 002 C032506B (02 ) 1000 SOMEWHAT POSITIVE

 003 C032506C (03 ) 0100 SOMEWHAT NEGATIVE

 004 C032506D (04 ) 0010 VERY NEGATIVE

 005 C032506M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0039

 DESCRIPTION: TEACHERS’ EXPECTATIONS FOR STUDENT ACHIEVEMENT

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C043201A (01 ) 0000 VERY POSITIVE

 002 C043201B (02 ) 1000 SOMEWHAT POSITIVE

 003 C043201C (03 ) 0100 SOMEWHAT NEGATIVE

 004 C043201D (04 ) 0010 VERY NEGATIVE

 005 C043201M (M ) 0001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

706

 CONDITIONING VARIABLE ID: SCHL0040

 DESCRIPTION: PERCENT STUDENT BODY ABSENT AVERAGE DAY

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 C043301A (01 ) 000000 0-2%

 002 C043301B (02 ) 100000 3-5%

 003 C043301C (03 ) 010000 6-10%

 004 C043301D (04 ) 001000 11-25%

 005 C043301E (05 ) 000100 26-50%

 006 C043301F (06 ) 000010 MORE THAN 50%

 007 C043301M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0041

 DESCRIPTION: PERCENT TEACHING STAFF ABSENT AVERAGE DAY

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 C043401A (01 ) 000000 0-2%

 002 C043401B (02 ) 100000 3-5%

 003 C043401C (03 ) 010000 6-10%

 004 C043401D (04 ) 001000 11-25%

 005 C043401E (05 ) 000100 26-50%

 006 C043401F (06 ) 000010 MORE THAN 50%

 007 C043401M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0042

 DESCRIPTION: ENROLLMENT LAST YEAR COMPARED TO END OF SCHOOL YR

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 C043501A (01 ) 000000 98-100%

 002 C043501B (02 ) 100000 95-97%

 003 C043501C (03 ) 010000 90-94%

 004 C043501D (04 ) 001000 80-89%

 005 C043501E (05 ) 000100 70-79%

 006 C043501F (06 ) 000010 LESS THAN 70%

 007 C043501M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0043

 DESCRIPTION: PERCENT STUDENTS HELD BACK AND REPEATING GRADE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043601A (01 ) 00000 0%

 002 C043601B (02 ) 10000 1-2%

 003 C043601C (03 ) 01000 3-5%

 004 C043601D (04 ) 00100 6-10%

 005 C043601E (05 ) 00010 MORE THAN 10%

 006 C043601M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0044

 DESCRIPTION: PERCENT TEACHING STAFF LEFT BEFORE END OF YEAR

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043701A (01 ) 00000 0%

 002 C043701B (02 ) 10000 1-2%

 003 C043701C (03 ) 01000 3-5%

 004 C043701D (04 ) 00100 6-10%

 005 C043701E (05 ) 00010 MORE THAN 10%

 006 C043701M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0045

 DESCRIPTION: IS SCHOOL IN NATIONAL SCHOOL LUNCH PROGRAM

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C038301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C038301Y (01 ) 00 YES

 002 C038301N (02 ) 10 NO

 003 C038301M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0046

 DESCRIPTION: PERCENT ELIGIBLE NATIONAL SCHOOL LUNCH PROGRAM

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 8

 001 C043801A (01 ) 00000000 0%

 002 C043801B (02 ) 10000000 1-5%

 003 C043801C (03 ) 01000000 6-10%

 004 C043801D (04 ) 00100000 11-25%

 005 C043801E (05 ) 00010000 26-50%

 006 C043801F (06 ) 00001000 51-75%

 007 C043801G (07 ) 00000100 76-99%

 008 C043801H (08 ) 00000010 100%

 009 C043801M (M ) 00000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0047

 DESCRIPTION: DOES SCHOOL RECEIVE CHAPTER 1/TITLE I FUNDING

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C043901Y (01 ) 00 YES

 002 C043901N (02 ) 10 NO

 003 C043901M (M ) 01 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

707

 CONDITIONING VARIABLE ID: SCHL0048

 DESCRIPTION: PERCENT STUDENTS RECEIVE CHAPTER1/TITLE I FUNDING

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 8

 001 C044001N (01 ) 00000000 NONE

 002 C044001B (02 ) 10000000 1-5%

 003 C044001C (03 ) 01000000 6-10%

 004 C044001D (04 ) 00100000 11-25%

 005 C044001E (05 ) 00010000 26-50%

 006 C044001F (06 ) 00001000 51-75%

 007 C044001G (07 ) 00000100 76-90%

 008 C044001H (08 ) 00000010 OVER 90%

 009 C044001M (M ) 00000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0049

 DESCRIPTION: PERCENT STUDENTS RECEIVE REMEDIAL READING INSTRUCT

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044002 TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 8

 001 C044002N (01 ) 00000000 NONE

 002 C044002B (02 ) 10000000 1-5%

 003 C044002C (03 ) 01000000 6-10%

 004 C044002D (04 ) 00100000 11-25%

 005 C044002E (05 ) 00010000 26-50%

 006 C044002F (06 ) 00001000 51-75%

 007 C044002G (07 ) 00000100 76-90%

 008 C044002H (08 ) 00000010 OVER 90%

 009 C044002M (M ) 00000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0050

 DESCRIPTION: PERCENT STUDENTS RECEIVE REMEDIAL WRITING INSTRUCT

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044003 TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 8

 001 C044003N (01 ) 00000000 NONE

 002 C044003B (02 ) 10000000 1-5%

 003 C044003C (03 ) 01000000 6-10%

 004 C044003D (04 ) 00100000 11-25%

 005 C044003E (05 ) 00010000 26-50%

 006 C044003F (06 ) 00001000 51-75%

 007 C044003G (07 ) 00000100 76-90%

 008 C044003H (08 ) 00000010 OVER 90%

 009 C044003M (M ) 00000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0051

 DESCRIPTION: PERCENT STUDENTS IN GIFTED AND TALENTED PROGRAM

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044004 TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 8

 001 C044004N (01 ) 00000000 NONE

 002 C044004B (02 ) 10000000 1-5%

 003 C044004C (03 ) 01000000 6-10%

 004 C044004D (04 ) 00100000 11-25%

 005 C044004E (05 ) 00010000 26-50%

 006 C044004F (06 ) 00001000 51-75%

 007 C044004G (07 ) 00000100 76-90%

 008 C044004H (08 ) 00000010 OVER 90%

 009 C044004M (M ) 00000001 MISSING

 CONDITIONING VARIABLE ID: BACK0067

 DESCRIPTION: HOW MUCH EDUCATION DO YOU EXPECT TO RECEIVE

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: B014201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 B014201N (01 ) 000000 WILL NOT FINISH HS

 002 B014201B (02 ) 100000 WILL GRADUATE HS

 003 B014201C (03 ) 010000 SOME ED AFTER HS

 004 B014201D (04 ) 001000 GRADUATE COLLEGE

 005 B014201E (05 ) 000100 GO TO GRAD SCHOOL

 006 B014201M (M, IDK ) 000001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0023

 DESCRIPTION: HOW OFTEN PAPERS ASSIGED-ONE TO TWO PARAGRAPHS

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 W802701A (01 ) 0000 ALMOST EVERY DAY

 002 W802701B (02 ) 1000 ONCE/TWICE A WEEK

 003 W802701C (03 ) 0100 ONCE/TWICE A MONTH

 004 W802701D (04 ) 0010 NEVER OR HARDLY EVER

 005 W802701M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0024

 DESCRIPTION: HOW OFTEN PAPERS ASSIGNED-ONE TO TWO PAGES

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802702 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 W802702A (01 ) 0000 ALMOST EVERY DAY

 002 W802702B (02 ) 1000 ONCE/TWICE A WEEK

 003 W802702C (03 ) 0100 ONCE/TWICE A MONTH

 004 W802702D (04 ) 0010 NEVER OR HARDLY EVER

 005 W802702M (M ) 0001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

708

 CONDITIONING VARIABLE ID: SUBJ0025

 DESCRIPTION: HOW OFTEN PAPERS ASSIGNED-THREE OR MORE PAGES

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802703 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 W802703A (01 ) 0000 ALMOST EVERY DAY

 002 W802703B (02 ) 1000 ONCE/TWICE A WEEK

 003 W802703C (03 ) 0100 ONCE/TWICE A MONTH

 004 W802703D (04 ) 0010 NEVER OR HARDLY EVER

 005 W802703M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0026

 DESCRIPTION: HOW OFTEN WRITING ASSIGNED-REPORT OR SUMMARY

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 W802801A (01 ) 0000 ALMOST EVERY DAY

 002 W802801B (02 ) 1000 ONCE/TWICE A WEEK

 003 W802801C (03 ) 0100 ONCE/TWICE A MONTH

 004 W802801D (04 ) 0010 NEVER OR HARDLY EVER

 005 W802801M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0027

 DESCRIPTION: HOW OFTEN WRITING ASSIGNED-ESSAY/THEME TO ANALYZE

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802802 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 W802802A (01 ) 0000 ALMOST EVERY DAY

 002 W802802B (02 ) 1000 ONCE/TWICE A WEEK

 003 W802802C (03 ) 0100 ONCE/TWICE A MONTH

 004 W802802D (04 ) 0010 NEVER OR HARDLY EVER

 005 W802802M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0028

 DESCRIPTION: HOW OFTEN WRITING ASSIGNED-ESSAY/LETTER- PERSUADE

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802803 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 W802803A (01 ) 0000 ALMOST EVERY DAY

 002 W802803B (02 ) 1000 ONCE/TWICE A WEEK

 003 W802803C (03 ) 0100 ONCE/TWICE A MONTH

 004 W802803D (04 ) 0010 NEVER OR HARDLY EVER

 005 W802803M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0029

 DESCRIPTION: HOW OFTEN WRITING ASSIGNED-STORY/NARRATIVE

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802804 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 W802804A (01 ) 0000 ALMOST EVERY DAY

 002 W802804B (02 ) 1000 ONCE/TWICE A WEEK

 003 W802804C (03 ) 0100 ONCE/TWICE A MONTH

 004 W802804D (04 ) 0010 NEVER OR HARDLY EVER

 005 W802804M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0030

 DESCRIPTION: HOW OFTEN ASKED TO PLAN YOUR WRITING

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 W802901A (01 ) 0000 ALMOST EVERY DAY

 002 W802901B (02 ) 1000 ONCE/TWICE A WEEK

 003 W802901C (03 ) 0100 ONCE/TWICE A MONTH

 004 W802901D (04 ) 0010 NEVER OR HARDLY EVER

 005 W802901M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0031

 DESCRIPTION: HOW OFTEN ASKED TO MAKE FORMAL OUTLINE FIRST

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802902 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 W802902A (01 ) 0000 ALMOST EVERY DAY

 002 W802902B (02 ) 1000 ONCE/TWICE A WEEK

 003 W802902C (03 ) 0100 ONCE/TWICE A MONTH

 004 W802902D (04 ) 0010 NEVER OR HARDLY EVER

 005 W802902M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0032

 DESCRIPTION: HOW OFTEN ASKED TO DEFINE PURPOSE AND AUDIENCE

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802903 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 W802903A (01 ) 0000 ALMOST EVERY DAY

 002 W802903B (02 ) 1000 ONCE/TWICE A WEEK

 003 W802903C (03 ) 0100 ONCE/TWICE A MONTH

 004 W802903D (04 ) 0010 NEVER OR HARDLY EVER

 005 W802903M (M ) 0001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

709

 CONDITIONING VARIABLE ID: SUBJ0033

 DESCRIPTION: HOW OFTEN ASKED TO USE SOURCES OTHER THAN TEXTBOOK

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: W802904 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 W802904A (01 ) 0000 ALMOST EVERY DAY

 002 W802904B (02 ) 1000 ONCE/TWICE A WEEK

 003 W802904C (03 ) 0100 ONCE/TWICE A MONTH

 004 W802904D (04 ) 0010 NEVER OR HARDLY EVER

 005 W802904M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0052

 DESCRIPTION: 8TH GRADE ASSIGNED TO ENGLISH CLASS BY ABILITY

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C044401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C044401Y (01 ) 00 YES

 002 C044401N (02 ) 10 NO

 003 C044401M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0053

 DESCRIPTION: 8TH GRADE ASSIGNED-HISTORY/SS BY ABILITY

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C044402 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C044402Y (01 ) 00 YES

 002 C044402N (02 ) 10 NO

 003 C044402M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0054

 DESCRIPTION: IS STUDENT DROPOUT A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043105 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C043105A (01 ) 0000 SERIOUS

 002 C043105B (02 ) 1000 MODERATE

 003 C043105C (03 ) 0100 MINOR

 004 C043105N (04 ) 0010 NOT A PROBLEM

 005 C043105M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0055

 DESCRIPTION: IS TEEN PREGNANCY A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043106 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C043106A (01 ) 0000 SERIOUS

 002 C043106B (02 ) 1000 MODERATE

 003 C043106C (03 ) 0100 MINOR

 004 C043106N (04 ) 0010 NOT A PROBLEM

 005 C043106M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0068

 DESCRIPTION: MAIN ACTIVITY YEAR FOLLOWING HIGH SCHOOL

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: B005501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 B005501A (01 ) 000000 WORK FULL-TIME

 002 B005501B (02 ) 100000 VOCA/TECH/BUSINESS

 003 B005501C (03 ) 010000 ATTEND 2 YR COLLEGE

 004 B005501D (04 ) 001000 ATTEND 4 YR COLLEGE

 005 B005501E (05 ) 000100 SERVE IN MILITARY

 006 B005501F (06 ) 000010 OTHER

 007 B005501M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: BACK0069

 DESCRIPTION: VOLUNTEER WORK IN YOUR COMMUNITY THIS YEAR

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: B014301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 B014301Y (01 ) 000 YES, WITH MY SCHOOL

 002 B014301Y (02 ) 100 YES, ON MY OWN

 003 B014301N (03 ) 010 NO

 004 B014301M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: BACK0070

 DESCRIPTION: HOW MANY HOURS/WEEK WORK JOB FOR PAY

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: B014401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 B014401N (01 ) 000000 NONE

 002 B014401B (02 ) 100000 1-5 HOURS

 003 B014401C (03 ) 010000 6-10 HOURS

 004 B014401D (04 ) 001000 11-15 HOURS

 005 B014401E (05 ) 000100 16-20 HOURS

 006 B014401F (06 ) 000010 21 OR MORE HOURS

 007 B014401M (M ) 000001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

710

 CONDITIONING VARIABLE ID: SCHL0056

 DESCRIPTION: 12TH GRADE ASSIGNED TO ENGLISH CLASS BY ABILITY

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C044301Y (01 ) 00 YES

 002 C044301N (02 ) 10 NO

 003 C044301M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0057

 DESCRIPTION: 12TH GR ASSIGNED- HISTORY/CIVICS/SS CLASS ABILITY

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044302 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C044302Y (01 ) 00 YES

 002 C044302N (02 ) 10 NO

 003 C044302M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0058

 DESCRIPTION: PERCENT LAST YEAR’S TWELFTH-GRADE CLASS GRADUATED

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C044101A (01 ) 00000 99-100%

 002 C044101B (02 ) 10000 95-98%

 003 C044101C (03 ) 01000 90-94%

 004 C044101D (04 ) 00100 75-89%

 005 C044101E (05 ) 00010 LESS THAN 75%

 006 C044101M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0059

 DESCRIPTION: PERCENT GRADUATING CLASS-ATTEND TWO-YEAR COLLEGE

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 8

 001 C044201N (01 ) 00000000 NONE

 002 C044201B (02 ) 10000000 1-5%

 003 C044201C (03 ) 01000000 6-10%

 004 C044201D (04 ) 00100000 11-25%

 005 C044201E (05 ) 00010000 26-50%

 006 C044201F (06 ) 00001000 51-75%

 007 C044201G (07 ) 00000100 76-90%

 008 C044201H (08 ) 00000010 OVER 100%

 009 C044201M (M ) 00000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0060

 DESCRIPTION: PERCENT GRADUATING CLASS-ATTEND FOUR-YEAR COLLEGE

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044202 TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 8

 001 C044202N (01 ) 00000000 NONE

 002 C044202B (02 ) 10000000 1-5%

 003 C044202C (03 ) 01000000 6-10%

 004 C044202D (04 ) 00100000 11-25%

 005 C044202E (05 ) 00010000 26-50%

 006 C044202F (06 ) 00001000 51-75%

 007 C044202G (07 ) 00000100 76-90%

 008 C044202H (08 ) 00000010 OVER 100%

 009 C044202M (M ) 00000001 MISSING

 009 C044202M (M ) 00000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0001

 DESCRIPTION: DO YOU TEACH READING

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T067001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T067001Y (01 ) 0 YES

 002 T067001M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0002

 DESCRIPTION: DO YOU TEACH WRITING

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T067002 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T067002Y (01 ) 0 YES

 002 T067002M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0003

 DESCRIPTION: DO YOU TEACH LANGUAGE ARTS

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T067003 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T067003Y (01 ) 0 YES

 002 T067003M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0004

 DESCRIPTION: DO YOU TEACH SOCIAL STUDIES

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T067004 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T067004Y (01 ) 0 YES

 002 T067004M (M ) 1 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

711

 CONDITIONING VARIABLE ID: TCHR0005

 DESCRIPTION: YEARS TOTAL TAUGHT ELEMENTARY LEVEL

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T067101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067101A (01 ) 00000 2 YEARS OR LESS

 002 T067101B (02 ) 10000 3-5 YEARS

 003 T067101C (03 ) 01000 6-10 YEARS

 004 T067101D (04 ) 00100 11-24 YEARS

 005 T067101E (05 ) 00010 25 YEARS OR MORE

 006 T067101M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0006

 DESCRIPTION: YEARS TOTAL TAUGHT READING

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T067201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067201A (01 ) 00000 2 YEARS OR LESS

 002 T067201B (02 ) 10000 3-5 YEARS

 003 T067201C (03 ) 01000 6-10 YEARS

 004 T067201D (04 ) 00100 11-24 YEARS

 005 T067201E (05 ) 00010 25 YEARS OR MORE

 006 T067201M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0007

 DESCRIPTION: YEARS TOTAL TAUGHT WRITING

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T067202 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067202A (01 ) 00000 2 YEARS OR LESS

 002 T067202B (02 ) 10000 3-5 YEARS

 003 T067202C (03 ) 01000 6-10 YEARS

 004 T067202D (04 ) 00100 11-24 YEARS

 005 T067202E (05 ) 00010 25 YEARS OR MORE

 006 T067202M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0008

 DESCRIPTION: YEARS TOTAL TAUGHT LANGUAGE ARTS

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T067203 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067203A (01 ) 00000 2 YEARS OR LESS

 002 T067203B (02 ) 10000 3-5 YEARS

 003 T067203C (03 ) 01000 6-10 YEARS

 004 T067203D (04 ) 00100 11-24 YEARS

 005 T067203E (05 ) 00010 25 YEARS OR MORE

 006 T067203M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0009

 DESCRIPTION: YEARS TOTAL TAUGHT HISTORY

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T067204 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067204A (01 ) 00000 2 YEARS OR LESS

 002 T067204B (02 ) 10000 3-5 YEARS

 003 T067204C (03 ) 01000 6-10 YEARS

 004 T067204D (04 ) 00100 11-24 YEARS

 005 T067204E (05 ) 00010 25 YEARS OR MORE

 006 T067204M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0010

 DESCRIPTION: YEARS TOTAL TAUGHT SOCIAL STUDIES

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T067205 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067205A (01 ) 00000 2 YEARS OR LESS

 002 T067205B (02 ) 10000 3-5 YEARS

 003 T067205C (03 ) 01000 6-10 YEARS

 004 T067205D (04 ) 00100 11-24 YEARS

 005 T067205E (05 ) 00010 25 YEARS OR MORE

 006 T067205M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0011

 DESCRIPTION: YEARS TOTAL TAUGHT CIVICS

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T067206 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067206A (01 ) 00000 2 YEARS OR LESS

 002 T067206B (02 ) 10000 3-5 YEARS

 003 T067206C (03 ) 01000 6-10 YEARS

 004 T067206D (04 ) 00100 11-24 YEARS

 005 T067206E (05 ) 00010 25 YEARS OR MORE

 006 T067206M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0012

 DESCRIPTION: MAIN ASSIGNMENT FIELD

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T067301A (01 ) 0000 REGULAR CLASSROOM

 002 T067301B (02 ) 1000 SPECIAL CLASSROOM

 003 T067301C (03 ) 0100 ESL/BILINGUAL ED

 004 T067301D (04 ) 0010 OTHER

 005 T067301M (M ) 0001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

712

 CONDITIONING VARIABLE ID: TCHR0013

 DESCRIPTION: TEACHING CERTIF IN THIS STATE IN MAIN FIELD

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 T056201A (01 ) 000000 ADVANCED PROFESSIONL

 002 T056201B (02 ) 100000 REGULAR/STANDARD ST

 003 T056201C (03 ) 010000 PROBATIONARY STATE

 004 T056201D (04 ) 001000 TEMPORARY/PROVISIONL

 005 T056201E (05 ) 000100 OTHER THAN STATE CRT

 006 T056201F (06 ) 000010 NOT HAVE CERT MAIN

 007 T056201M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0014

 DESCRIPTION: HIGHEST ACADEMIC DEGREE YOU HOLD

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 T056301A (01 ) 0000000 HIGH SCHOOL DIPLOMA

 002 T056301B (02 ) 1000000 ASSOCIATES/VOCATIONL

 003 T056301C (03 ) 0100000 BACHELOR’S DEGREE

 004 T056301D (04 ) 0010000 MASTER’S DEGREE

 005 T056301E (05 ) 0001000 EDUCATION SPECIALIST

 006 T056301F (06 ) 0000100 DOCTORATE

 007 T056301G (07 ) 0000010 PROFESSIONAL DEGREE

 008 T056301M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0015

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-ELEMENTARY EDUCATION

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067501A (01 ) 000 MAJOR

 002 T067501B (02 ) 100 MINOR

 003 T067501C (03 ) 010 NOT IN THIS SUBJECT

 004 T067501M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0016

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-SECONDARY EDUCATION

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067502 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067502A (01 ) 000 MAJOR

 002 T067502B (02 ) 100 MINOR

 003 T067502C (03 ) 010 NOT IN THIS SUBJECT

 004 T067502M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0017

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-SPECIAL EDUCATION

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067503 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067503A (01 ) 000 MAJOR

 002 T067503B (02 ) 100 MINOR

 003 T067503C (03 ) 010 NOT IN THIS SUBJECT

 004 T067503M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0018

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-BILINGUAL EDUCATION/ESL

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067504 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067504A (01 ) 000 MAJOR

 002 T067504B (02 ) 100 MINOR

 003 T067504C (03 ) 010 NOT IN THIS SUBJECT

 004 T067504M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0019

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-ADMINISTRATION & SUPERVISION

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067505 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067505A (01 ) 000 MAJOR

 002 T067505B (02 ) 100 MINOR

 003 T067505C (03 ) 010 NOT IN THIS SUBJECT

 004 T067505M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0020

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-CURRICULUM & SUPERVISION

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067506 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067506A (01 ) 000 MAJOR

 002 T067506B (02 ) 100 MINOR

 003 T067506C (03 ) 010 NOT IN THIS SUBJECT

 004 T067506M (M ) 001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

713

 CONDITIONING VARIABLE ID: TCHR0021

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-COUNSELING

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067507 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067507A (01 ) 000 MAJOR

 002 T067507B (02 ) 100 MINOR

 003 T067507C (03 ) 010 NOT IN THIS SUBJECT

 004 T067507M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0022

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-ENGLISH

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067508 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067508A (01 ) 000 MAJOR

 002 T067508B (02 ) 100 MINOR

 003 T067508C (03 ) 010 NOT IN THIS SUBJECT

 004 T067508M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0023

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-READING AND/OR LANGUAGE ARTS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067509 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067509A (01 ) 000 MAJOR

 002 T067509B (02 ) 100 MINOR

 003 T067509C (03 ) 010 NOT IN THIS SUBJECT

 004 T067509M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0024

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-HISTORY

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067510 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067510A (01 ) 000 MAJOR

 002 T067510B (02 ) 100 MINOR

 003 T067510C (03 ) 010 NOT IN THIS SUBJECT

 004 T067510M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0025

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-POLITICAL SCIENCE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067511 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067511A (01 ) 000 MAJOR

 002 T067511B (02 ) 100 MINOR

 003 T067511C (03 ) 010 NOT IN THIS SUBJECT

 004 T067511M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0026

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-OTHER

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067512 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067512A (01 ) 000 MAJOR

 002 T067512B (02 ) 100 MINOR

 003 T067512C (03 ) 010 NOT IN THIS SUBJECT

 004 T067512M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0027

 DESCRIPTION: GRAD MAJOR/MINOR-ELEMENTARY EDUCATION

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067601A (01 ) 000 MAJOR

 002 T067601B (02 ) 100 MINOR

 003 T067601C (03 ) 010 NOT IN THIS SUBJECT

 004 T067601M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0028

 DESCRIPTION: GRAD MAJOR/MINOR-SECONDARY EDUCATION

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067602 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067602A (01 ) 000 MAJOR

 002 T067602B (02 ) 100 MINOR

 003 T067602C (03 ) 010 NOT IN THIS SUBJECT

 004 T067602M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0029

 DESCRIPTION: GRAD MAJOR/MINOR-SPECIAL EDUCATION

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067603 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067603A (01 ) 000 MAJOR

 002 T067603B (02 ) 100 MINOR

 003 T067603C (03 ) 010 NOT IN THIS SUBJECT

 004 T067603M (M ) 001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

714

 CONDITIONING VARIABLE ID: TCHR0030

 DESCRIPTION: GRAD MAJOR/MINOR-BILINGUAL EDUCATION/ESL

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067604 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067604A (01 ) 000 MAJOR

 002 T067604B (02 ) 100 MINOR

 003 T067604C (03 ) 010 NOT IN THIS SUBJECT

 004 T067604M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0031

 DESCRIPTION: GRAD MAJOR/MINOR-ADMINSTRATION & SUPERVISION

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067605 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067605A (01 ) 000 MAJOR

 002 T067605B (02 ) 100 MINOR

 003 T067605C (03 ) 010 NOT IN THIS SUBJECT

 004 T067605M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0032

 DESCRIPTION: GRAD MAJOR/MINOR-CURRICULUM AND INSTRUCTION

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067606 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067606A (01 ) 000 MAJOR

 002 T067606B (02 ) 100 MINOR

 003 T067606C (03 ) 010 NOT IN THIS SUBJECT

 004 T067606M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0033

 DESCRIPTION: GRAD MAJOR/MINOR-COUNSELING

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067607 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067607A (01 ) 000 MAJOR

 002 T067607B (02 ) 100 MINOR

 003 T067607C (03 ) 010 NOT IN THIS SUBJECT

 004 T067607M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0034

 DESCRIPTION: GRAD MAJOR/MINOR-ENGLISH

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067608 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067608A (01 ) 000 MAJOR

 002 T067608B (02 ) 100 MINOR

 003 T067608C (03 ) 010 NOT IN THIS SUBJECT

 004 T067608M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0035

 DESCRIPTION: GRAD MAJOR/MINOR-READING AND/OR LANGUAGE ARTS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067609 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067609A (01 ) 000 MAJOR

 002 T067609B (02 ) 100 MINOR

 003 T067609C (03 ) 010 NOT IN THIS SUBJECT

 004 T067609M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0036

 DESCRIPTION: GRAD MAJOR/MINOR-HISTORY

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067610 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067610A (01 ) 000 MAJOR

 002 T067610B (02 ) 100 MINOR

 003 T067610C (03 ) 010 NOT IN THIS SUBJECT

 004 T067610M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0037

 DESCRIPTION: GRAD MAJOR/MINOR-POLITICAL SCIENCE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067611 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067611A (01 ) 000 MAJOR

 002 T067611B (02 ) 100 MINOR

 003 T067611C (03 ) 010 NOT IN THIS SUBJECT

 004 T067611M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0038

 DESCRIPTION: GRAD MAJOR/MINOR-OTHER

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067612 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067612A (01 ) 000 MAJOR

 002 T067612B (02 ) 100 MINOR

 003 T067612C (03 ) 010 NOT IN THIS SUBJECT

 004 T067612M (M ) 001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

715

 CONDITIONING VARIABLE ID: TCHR0039

 DESCRIPTION: LAST 12 MOS, PROF DEV-READING AND WRITING

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067701A (01 ) 00000 NONE

 002 T067701B (02 ) 10000 LESS THAN 6 HOURS

 003 T067701C (03 ) 01000 6 - 15 HOURS

 004 T067701D (04 ) 00100 16 - 35 HOURS

 005 T067701E (05 ) 00010 MORE THAN 35 HOURS

 006 T067701M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0040

 DESCRIPTION: LAST 12 MOS, PROF DEV-SOCIAL STUDIES

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067702 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067702A (01 ) 00000 NONE

 002 T067702B (02 ) 10000 LESS THAN 6 HOURS

 003 T067702C (03 ) 01000 6 - 15 HOURS

 004 T067702D (04 ) 00100 16 - 35 HOURS

 005 T067702E (05 ) 00010 MORE THAN 35 HOURS

 006 T067702M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0041

 DESCRIPTION: PREPARED IN THE USE OF TELECOMMUNICATIONS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067801A (01 ) 000 WELL PREPARED

 002 T067801B (02 ) 100 MODERATELY PREPARED

 003 T067801C (03 ) 010 NOT WELL PREPARED

 004 T067801M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0042

 DESCRIPTION: PREPARED IN THE USE OF COMPUTERS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067802 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067802A (01 ) 000 WELL PREPARED

 002 T067802B (02 ) 100 MODERATELY PREPARED

 003 T067802C (03 ) 010 NOT WELL PREPARED

 004 T067802M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0043

 DESCRIPTION: PREPARED IN COOPERATIVE GROUP INSTRUCTION

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067803 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067803A (01 ) 000 WELL PREPARED

 002 T067803B (02 ) 100 MODERATELY PREPARED

 003 T067803C (03 ) 010 NOT WELL PREPARED

 004 T067803M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0044

 DESCRIPTION: PREPARED IN TEACHING STUDENTS-DIFFERENT CULTURES

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067804 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067804A (01 ) 000 WELL PREPARED

 002 T067804B (02 ) 100 MODERATELY PREPARED

 003 T067804C (03 ) 010 NOT WELL PREPARED

 004 T067804M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0045

 DESCRIPTION: PREPARED IN TEACHING STUDENTS WHO ARE LEP

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067805 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067805A (01 ) 000 WELL PREPARED

 002 T067805B (02 ) 100 MODERATELY PREPARED

 003 T067805C (03 ) 010 NOT WELL PREPARED

 004 T067805M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0046

 DESCRIPTION: PREPARED IN TEACHING STUDENTS WITH DISABILITIES

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067806 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067806A (01 ) 000 WELL PREPARED

 002 T067806B (02 ) 100 MODERATELY PREPARED

 003 T067806C (03 ) 010 NOT WELL PREPARED

 004 T067806M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0047

 DESCRIPTION: PREPARED IN CLASSROOM MANAGEMENT AND ORGANIZATION

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067807 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067807A (01 ) 000 WELL PREPARED

 002 T067807B (02 ) 100 MODERATELY PREPARED

 003 T067807C (03 ) 010 NOT WELL PREPARED

 004 T067807M (M ) 001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

716

 CONDITIONING VARIABLE ID: TCHR0048

 DESCRIPTION: AVAILABILITY OF RESOURCES

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T041201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T041201A (01 ) 0000 GET ALL RESOURCES

 002 T041201B (02 ) 1000 GET MOST RESOURCES

 003 T041201C (03 ) 0100 GET SOME RESOURCES

 004 T041201D (04 ) 0010 DON’T GET RESOURCES

 005 T041201M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0049

 DESCRIPTION: HOW WELL PREPARED TO TEACH READING

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067901A (01 ) 000 WELL PREPARED

 002 T067901B (02 ) 100 MODERATELY PREPARED

 003 T067901C (03 ) 010 NOT WELL PREPARED

 004 T067901M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0050

 DESCRIPTION: HOW WELL PREPARED TO TEACH WRITING

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067902 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067902A (01 ) 000 WELL PREPARED

 002 T067902B (02 ) 100 MODERATELY PREPARED

 003 T067902C (03 ) 010 NOT WELL PREPARED

 004 T067902M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0051

 DESCRIPTION: PREPARED IN LIT-BASED READING INSTRUCTION

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T068001A (01 ) 000 WELL PREPARED

 002 T068001B (02 ) 100 MODERATELY PREPARED

 003 T068001C (03 ) 010 NOT WELL PREPARED

 004 T068001M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0052

 DESCRIPTION: PREPARED IN CONTENT AREA READING

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068002 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T068002A (01 ) 000 WELL PREPARED

 002 T068002B (02 ) 100 MODERATELY PREPARED

 003 T068002C (03 ) 010 NOT WELL PREPARED

 004 T068002M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0053

 DESCRIPTION: PREPARED IN COMBINING RDG AND WRITING

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068003 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T068003A (01 ) 000 WELL PREPARED

 002 T068003B (02 ) 100 MODERATELY PREPARED

 003 T068003C (03 ) 010 NOT WELL PREPARED

 004 T068003M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0054

 DESCRIPTION: PREPARED IN WHOLE LANGUAGE APPROACH TO TEACH RDG

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068004 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T068004A (01 ) 000 WELL PREPARED

 002 T068004B (02 ) 100 MODERATELY PREPARED

 003 T068004C (03 ) 010 NOT WELL PREPARED

 004 T068004M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0055

 DESCRIPTION: PREPARED IN PHONICS IN TEACHING READING

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068005 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T068005A (01 ) 000 WELL PREPARED

 002 T068005B (02 ) 100 MODERATELY PREPARED

 003 T068005C (03 ) 010 NOT WELL PREPARED

 004 T068005M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0056

 DESCRIPTION: PREPARED IN TEACHING MULTICULTURAL LITERATURE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068006 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T068006A (01 ) 000 WELL PREPARED

 002 T068006B (02 ) 100 MODERATELY PREPARED

 003 T068006C (03 ) 010 NOT WELL PREPARED

 004 T068006M (M ) 001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

717

 CONDITIONING VARIABLE ID: TCHR0057

 DESCRIPTION: PREPARED IN COMPUTER SOFTWARE FOR TEACHING RDG

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068007 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T068007A (01 ) 000 WELL PREPARED

 002 T068007B (02 ) 100 MODERATELY PREPARED

 003 T068007C (03 ) 010 NOT WELL PREPARED

 004 T068007M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0058

 DESCRIPTION: PREPARED IN WRITING ACROSS THE CURRICULUM

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068008 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T068008A (01 ) 000 WELL PREPARED

 002 T068008B (02 ) 100 MODERATELY PREPARED

 003 T068008C (03 ) 010 NOT WELL PREPARED

 004 T068008M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0059

 DESCRIPTION: PREPARED IN USING COMPUTER SOFTWARE TO TEACH WRTG

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068009 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T068009A (01 ) 000 WELL PREPARED

 002 T068009B (02 ) 100 MODERATELY PREPARED

 003 T068009C (03 ) 010 NOT WELL PREPARED

 004 T068009M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0060

 DESCRIPTION: PREPARED IN TEACHING SPELLING, GRAMMAR, MECHANICS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068010 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T068010A (01 ) 000 WELL PREPARED

 002 T068010B (02 ) 100 MODERATELY PREPARED

 003 T068010C (03 ) 010 NOT WELL PREPARED

 004 T068010M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0061

 DESCRIPTION: AVERAGE READING CLASS SIZE

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T068101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T068101A (01 ) 00000 1-20 STUDENTS

 002 T068101B (02 ) 10000 21-25 STUDENTS

 003 T068101C (03 ) 01000 26-30 STUDENTS

 004 T068101D (04 ) 00100 31-35 STUDENTS

 005 T068101E (05 ) 00010 36 OR MORE STUDENTS

 006 T068101M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0062

 DESCRIPTION: CLASS ASSIGNMENT BY ABILITY

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T046101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T046101Y (01 ) 00 YES

 002 T046101N (02 ) 10 NO

 003 T046101M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TCHR0063

 DESCRIPTION: ABILITY LEVEL OF STUDENTS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T046201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T046201A (01 ) 0000 MOSTLY HIGH ABILITY

 002 T046201B (02 ) 1000 MOSTLY AVERAGE ABLTY

 003 T046201C (03 ) 0100 MOSTLY LOW ABILITY

 004 T046201D (04 ) 0010 MIXED ABILITY LEVELS

 005 T046201M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0064

 DESCRIPTION: HOW MUCH CLASS TIME PER DAY-READING INSTRUCTION

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T068201A (01 ) 00000 LESS THAN 30 MINUTES

 002 T068201B (02 ) 10000 30-44 MINUTES

 003 T068201C (03 ) 01000 45-59 MINUTES

 004 T068201D (04 ) 00100 60-90 MINUTES

 005 T068201E (05 ) 00010 MORE THAN 90 MINUTES

 006 T068201M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0065

 DESCRIPTION: BASIS FOR CREATING READING INSTRUCTIONAL GROUPS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T068301A (01 ) 00000 ABILITY

 002 T068301B (02 ) 10000 INTEREST

 003 T068301C (03 ) 01000 DIVERSITY

 004 T068301D (04 ) 00100 OTHER

 005 T068301E (05 ) 00010 NOT CREATED

 006 T068301M (M ) 00001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

718

 CONDITIONING VARIABLE ID: TCHR0066

 DESCRIPTION: CLASS DIVIDED INTO HOW MANY INSTRUCTIONAL GROUPS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 T068401A (01 ) 0000000 WHOLE CLASS

 002 T068401B (02 ) 1000000 WHOLE W/FLEX GROUP

 003 T068401C (03 ) 0100000 2 GROUPS

 004 T068401D (04 ) 0010000 3 GROUPS

 005 T068401E (05 ) 0001000 4 GROUPS

 006 T068401F (06 ) 0000100 5 OR MORE GROUPS

 007 T068401G (07 ) 0000010 INDIVIDUALIZED

 008 T068401M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0067

 DESCRIPTION: WRITING ABILITY LEVEL OF CLASS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T068601A (01 ) 0000 PRIMARILY HIGH

 002 T068601B (02 ) 1000 PRIMARILY AVERAGE

 003 T068601C (03 ) 0100 PRIMARILY LOW

 004 T068601D (04 ) 0010 WIDELY MIXED

 005 T068601M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0068

 DESCRIPTION: EACH WEEK, TIME SPENT INSTRUCTING/HELPING-WRITING

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T068701A (01 ) 00000 LESS THAN 30 MINUTES

 002 T068701B (02 ) 10000 30-44 MINUTES

 003 T068701C (03 ) 01000 45-59 MINUTES

 004 T068701D (04 ) 00100 60-90 MINUTES

 005 T068701E (05 ) 00010 MORE THAN 90 MINUTES

 006 T068701M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0069

 DESCRIPTION: HOW OFTEN USE CHILDREN’S NEWSPAPERS/MAGAZINES

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T068801A (01 ) 0000 ALMOST EVERY DAY

 002 T068801B (02 ) 1000 ONCE/TWICE A WEEK

 003 T068801C (03 ) 0100 ONCE/TWICE A MONTH

 004 T068801D (04 ) 0010 NEVER OR HARDLY EVER

 005 T068801M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0070

 DESCRIPTION: HOW OFTEN USE READING KITS TO TEACH READING

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068802 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T068802A (01 ) 0000 ALMOST EVERY DAY

 002 T068802B (02 ) 1000 ONCE/TWICE A WEEK

 003 T068802C (03 ) 0100 ONCE/TWICE A MONTH

 004 T068802D (04 ) 0010 NEVER OR HARDLY EVER

 005 T068802M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0071

 DESCRIPTION: HOW OFTEN USE COMPUTER SOFTWARE FOR READING INSTR

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068803 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T068803A (01 ) 0000 ALMOST EVERY DAY

 002 T068803B (02 ) 1000 ONCE/TWICE A WEEK

 003 T068803C (03 ) 0100 ONCE/TWICE A MONTH

 004 T068803D (04 ) 0010 NEVER OR HARDLY EVER

 005 T068803M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0072

 DESCRIPTION: HOW OFTEN USE BOOKS (NOVELS, POETRY, NONFICTION)

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068804 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T068804A (01 ) 0000 ALMOST EVERY DAY

 002 T068804B (02 ) 1000 ONCE/TWICE A WEEK

 003 T068804C (03 ) 0100 ONCE/TWICE A MONTH

 004 T068804D (04 ) 0010 NEVER OR HARDLY EVER

 005 T068804M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0073

 DESCRIPTION: HOW OFTEN USE MATERIALS FROM OTHER SUBJECTS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068805 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T068805A (01 ) 0000 ALMOST EVERY DAY

 002 T068805B (02 ) 1000 ONCE/TWICE A WEEK

 003 T068805C (03 ) 0100 ONCE/TWICE A MONTH

 004 T068805D (04 ) 0010 NEVER OR HARDLY EVER

 005 T068805M (M ) 0001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

719

 CONDITIONING VARIABLE ID: TCHR0074

 DESCRIPTION: WHAT TYPE OF MATERIALS FORM CORE READING PROGRAM

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T068901A (01 ) 0000 PRIMARILY BASAL

 002 T068901B (02 ) 1000 PRIMARILY TRADE BOOK

 003 T068901C (03 ) 0100 BOTH BASAL AND TRADE

 004 T068901D (04 ) 0010 OTHER

 005 T068901M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0075

 DESCRIPTION: AVAILABILITY OF COMPUTERS FOR USE IN CLASS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T069001A (01 ) 00000 NOT AVAILABLE

 002 T069001B (02 ) 10000 LIMITED ACCESS

 003 T069001C (03 ) 01000 LAB OR LIBRARY

 004 T069001D (04 ) 00100 ONE IN CLASSROOM

 005 T069001E (05 ) 00010 SEVERAL IN CLASSROOM

 006 T069001M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0076

 DESCRIPTION: PROPORTION TIME SPENT ON RDG FOR LIT EXPERIENCE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069101A (01 ) 0000 ALMOST ALL TIME

 002 T069101B (02 ) 1000 TWO-THIRDS OF TIME

 003 T069101C (03 ) 0100 AT LEAST ONE-THIRD

 004 T069101D (04 ) 0010 LITTLE OR NO TIME

 005 T069101M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0077

 DESCRIPTION: PROPORTION TIME SPENT ON RDG TO GAIN INFORMATION

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069102 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069102A (01 ) 0000 ALMOST ALL TIME

 002 T069102B (02 ) 1000 TWO-THIRDS OF TIME

 003 T069102C (03 ) 0100 AT LEAST ONE-THIRD

 004 T069102D (04 ) 0010 LITTLE OR NO TIME

 005 T069102M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0078

 DESCRIPTION: PROPORTION TIME SPENT ON RDG TO PERFORM A TASK

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069103 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069103A (01 ) 0000 ALMOST ALL TIME

 002 T069103B (02 ) 1000 TWO-THIRDS OF TIME

 003 T069103C (03 ) 0100 AT LEAST ONE-THIRD

 004 T069103D (04 ) 0010 LITTLE OR NO TIME

 005 T069103M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0079

 DESCRIPTION: PROPORTION TIME SPENT ON NARRATIVE WRITING

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069201A (01 ) 0000 ALMOST ALL TIME

 002 T069201B (02 ) 1000 TWO-THIRDS OF TIME

 003 T069201C (03 ) 0100 AT LEAST ONE-THIRD

 004 T069201D (04 ) 0010 LITTLE OR NO TIME

 005 T069201M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0080

 DESCRIPTION: PROPORTION TIME SPENT ON INFORMATIVE WRITING

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069202 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069202A (01 ) 0000 ALMOST ALL TIME

 002 T069202B (02 ) 1000 TWO-THIRDS OF TIME

 003 T069202C (03 ) 0100 AT LEAST ONE-THIRD

 004 T069202D (04 ) 0010 LITTLE OR NO TIME

 005 T069202M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0081

 DESCRIPTION: PROPORTION TIME SPENT ON PERSUASIVE WRITING

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069203 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069203A (01 ) 0000 ALMOST ALL TIME

 002 T069203B (02 ) 1000 TWO-THIRDS OF TIME

 003 T069203C (03 ) 0100 AT LEAST ONE-THIRD

 004 T069203D (04 ) 0010 LITTLE OR NO TIME

 005 T069203M (M ) 0001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

720

 CONDITIONING VARIABLE ID: TCHR0082

 DESCRIPTION: DO YOU USE GRAMMAR OR SKILL-BASED INSTRUCTION

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069301A (01 ) 000 YES, CENTRAL PART

 002 T069301B (02 ) 100 YES, SUPPLEMENT PART

 003 T069301N (03 ) 010 NO

 004 T069301M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0083

 DESCRIPTION: DO YOU USE WRITING PROCESS INSTRUCTION

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069302 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069302A (01 ) 000 YES, CENTRAL PART

 002 T069302B (02 ) 100 YES, SUPPLEMENT PART

 003 T069302N (03 ) 010 NO

 004 T069302M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0084

 DESCRIPTION: DO YOU INTEGRATE READING AND WRITING INSTRUCTION

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069303 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069303A (01 ) 000 YES, CENTRAL PART

 002 T069303B (02 ) 100 YES, SUPPLEMENT PART

 003 T069303N (03 ) 010 NO

 004 T069303M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0085

 DESCRIPTION: DO YOU USE WRITING ABOUT LITERATURE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069304 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069304A (01 ) 000 YES, CENTRAL PART

 002 T069304B (02 ) 100 YES, SUPPLEMENT PART

 003 T069304N (03 ) 010 NO

 004 T069304M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0086

 DESCRIPTION: DO YOU USE WRITING ACROSS OTHER SUBJECT AREAS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069305 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069305A (01 ) 000 YES, CENTRAL PART

 002 T069305B (02 ) 100 YES, SUPPLEMENT PART

 003 T069305N (03 ) 010 NO

 004 T069305M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0087

 DESCRIPTION: HOW OFTEN STUDENTS DO SPELLING, PUNCTUATION, GRAMM

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069401A (01 ) 0000 ALMOST EVERY DAY

 002 T069401B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069401C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069401D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069401M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0088

 DESCRIPTION: HOW OFTEN STUDENTS WORK ON WRITING PROCESS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069402 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069402A (01 ) 0000 ALMOST EVERY DAY

 002 T069402B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069402C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069402D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069402M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0089

 DESCRIPTION: HOW OFTEN STUDENTS WRITE IN A LOG/JOURNAL

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069403 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069403A (01 ) 0000 ALMOST EVERY DAY

 002 T069403B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069403C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069403D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069403M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0090

 DESCRIPTION: HOW OFTEN PARENTS SIGN/REVIEW STUDENTS’ HOMEWORK

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069404 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069404A (01 ) 0000 ALMOST EVERY DAY

 002 T069404B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069404C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069404D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069404M (M ) 0001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

721

 CONDITIONING VARIABLE ID: TCHR0091

 DESCRIPTION: HOW OFTEN ASSIGN HOMEWORK TO DO WITH PARENTS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069405 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069405A (01 ) 0000 ALMOST EVERY DAY

 002 T069405B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069405C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069405D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069405M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0092

 DESCRIPTION: EXPECTED TIME SPENT ON WRITING ASSIGNMENTS/WEEK

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T069501A (01 ) 00000 NONE

 002 T069501B (02 ) 10000 LESS THAN 1 HOUR

 003 T069501C (03 ) 01000 1 HOUR

 004 T069501D (04 ) 00100 2 HOURS

 005 T069501E (05 ) 00010 3 HOURS OR MORE

 006 T069501M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0093

 DESCRIPTION: THIS YEAR, PROJECTS TO DO/SHARE WITH PARENTS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069601A (01 ) 0000 NEVER

 002 T069601B (02 ) 1000 ONCE

 003 T069601C (03 ) 0100 TWICE

 004 T069601D (04 ) 0010 THREE OR MORE TIMES

 005 T069601M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0094

 DESCRIPTION: HOW OFTEN ASK STUDENTS TO READ ALOUD

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069701A (01 ) 0000 ALMOST EVERY DAY

 002 T069701B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069701C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069701D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069701M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0095

 DESCRIPTION: HOW OFTEN ASK STUDENTS-DISCUSS WHAT WAS READ

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069702 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069702A (01 ) 0000 ALMOST EVERY DAY

 002 T069702B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069702C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069702D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069702M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0096

 DESCRIPTION: HOW OFTEN ASK STUDENTS- WRITE ABOUT WHAT WAS READ

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069703 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069703A (01 ) 0000 ALMOST EVERY DAY

 002 T069703B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069703C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069703D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069703M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0097

 DESCRIPTION: HOW OFTEN ASK STUDENTS-WRITE IN WORKSHEET/BOOK

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069704 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069704A (01 ) 0000 ALMOST EVERY DAY

 002 T069704B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069704C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069704D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069704M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0098

 DESCRIPTION: HOW OFTEN ASK STUDENTS-READ SILENTLY

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069705 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069705A (01 ) 0000 ALMOST EVERY DAY

 002 T069705B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069705C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069705D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069705M (M ) 0001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

722

 CONDITIONING VARIABLE ID: TCHR0099

 DESCRIPTION: HOW OFTEN GIVE STUDENTS TIME TO READ BOOKS CHOSEN

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069706 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069706A (01 ) 0000 ALMOST EVERY DAY

 002 T069706B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069706C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069706D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069706M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0100

 DESCRIPTION: HOW OFTEN ASK STUDENTS-GROUP ACTIVITY/PROJECT

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069707 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069707A (01 ) 0000 ALMOST EVERY DAY

 002 T069707B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069707C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069707D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069707M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0101

 DESCRIPTION: HOW OFTEN ASK STUDENTS-DISCUSS INTERPRETATIONS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069708 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069708A (01 ) 0000 ALMOST EVERY DAY

 002 T069708B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069708C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069708D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069708M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0102

 DESCRIPTION: HOW OFTEN ASK STUDENTS-EXPLAIN/SUPPORT WHAT READ

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069709 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069709A (01 ) 0000 ALMOST EVERY DAY

 002 T069709B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069709C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069709D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069709M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0103

 DESCRIPTION: HOW OFTEN GIVE READING QUIZZES OR TESTS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069710 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069710A (01 ) 0000 ALMOST EVERY DAY

 002 T069710B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069710C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069710D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069710M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0104

 DESCRIPTION: HOW OFTEN WATCH MOVIES, VIDEOS, FILMSTRIPS, TV, CD

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069711 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069711A (01 ) 0000 ALMOST EVERY DAY

 002 T069711B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069711C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069711D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069711M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0105

 DESCRIPTION: HOW OFTEN HELP STUDENTS UNDERSTAND NEW WORDS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069712 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069712A (01 ) 0000 ALMOST EVERY DAY

 002 T069712B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069712C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069712D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069712M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0106

 DESCRIPTION: HOW OFTEN ASK STUDENTS-ANSWER QUESTIONS IN WRITING

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069713 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069713A (01 ) 0000 ALMOST EVERY DAY

 002 T069713B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069713C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069713D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069713M (M ) 0001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

723

 CONDITIONING VARIABLE ID: TCHR0107

 DESCRIPTION: HOW OFTEN ASK STUDENTS-PREDICT OUTCOME OF READING

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069714 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069714A (01 ) 0000 ALMOST EVERY DAY

 002 T069714B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069714C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069714D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069714M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0108

 DESCRIPTION: HOW OFTEN ASK STUDENTS-MAKE GENERALIZATIONS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069715 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069715A (01 ) 0000 ALMOST EVERY DAY

 002 T069715B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069715C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069715D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069715M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0109

 DESCRIPTION: HOW OFTEN ASK STUDENTS-DESCRIBE STYLE/STRUCTURE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069716 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069716A (01 ) 0000 ALMOST EVERY DAY

 002 T069716B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069716C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069716D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069716M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0110

 DESCRIPTION: HOW OFTEN STUDENTS CHOOSE WRITING TOPIC

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T071801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071801A (01 ) 0000 ALMOST EVERY DAY

 002 T071801B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071801C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071801D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071801M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0111

 DESCRIPTION: HOW OFTEN STUDENTS PLAN THEIR WRITING

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T071802 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071802A (01 ) 0000 ALMOST EVERY DAY

 002 T071802B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071802C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071802D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071802M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0112

 DESCRIPTION: HOW OFTEN STUDENTS DEFINE PURPOSES AND AUDIENCE

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T071803 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071803A (01 ) 0000 ALMOST EVERY DAY

 002 T071803B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071803C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071803D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071803M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0113

 DESCRIPTION: HOW OFTEN STUDENTS MAKE FORMAL OUTLINE

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T071804 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071804A (01 ) 0000 ALMOST EVERY DAY

 002 T071804B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071804C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071804D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071804M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0114

 DESCRIPTION: HOW OFTEN STUDENTS WRITE MORE THAN ONE DRAFT

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T071805 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071805A (01 ) 0000 ALMOST EVERY DAY

 002 T071805B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071805C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071805D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071805M (M ) 0001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

724

 CONDITIONING VARIABLE ID: TCHR0115

 DESCRIPTION: HOW OFTEN STUDENTS USE RESOURCES OTHER THAN TEXT

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T071806 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071806A (01 ) 0000 ALMOST EVERY DAY

 002 T071806B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071806C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071806D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071806M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0116

 DESCRIPTION: HOW OFTEN STUDENTS DISCUSS WRITING WHILE WRITING

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T071807 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071807A (01 ) 0000 ALMOST EVERY DAY

 002 T071807B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071807C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071807D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071807M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0117

 DESCRIPTION: HOW OFTEN STUDENTS DISCUSS OTHERS’ WRITING

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T071808 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071808A (01 ) 0000 ALMOST EVERY DAY

 002 T071808B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071808C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071808D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071808M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0118

 DESCRIPTION: HOW OFTEN STUDENTS CHECK PROPER SPELLING, GRAMMAR

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T071809 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071809A (01 ) 0000 ALMOST EVERY DAY

 002 T071809B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071809C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071809D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071809M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0119

 DESCRIPTION: HOW OFTEN STUDENTS DISCUSS WRITING WITH FAMILY

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T071810 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071810A (01 ) 0000 ALMOST EVERY DAY

 002 T071810B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071810C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071810D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071810M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0120

 DESCRIPTION: HOW OFTEN STUDENTS CONTRIBUTE TO COLLECTION

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T071811 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071811A (01 ) 0000 ALMOST EVERY DAY

 002 T071811B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071811C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071811D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071811M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0121

 DESCRIPTION: HOW OFTEN STUDENTS WORK ON AN ASSIGNED TOPIC

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T071812 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071812A (01 ) 0000 ALMOST EVERY DAY

 002 T071812B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071812C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071812D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071812M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0122

 DESCRIPTION: HOW OFTEN STUDENTS FOLLOW ASSIGNED FORMAT

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T071813 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071813A (01 ) 0000 ALMOST EVERY DAY

 002 T071813B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071813C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071813D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071813M (M ) 0001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

725

 CONDITIONING VARIABLE ID: TCHR0123

 DESCRIPTION: HOW OFTEN WRITING ASSIGNMENTS-LESS THAN ONE PAGE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069901A (01 ) 0000 ALMOST EVERY DAY

 002 T069901B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069901C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069901D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069901M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0124

 DESCRIPTION: HOW OFTEN WRITING ASSIGNMENTS-ONE TO TWO PAGES

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069902 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069902A (01 ) 0000 ALMOST EVERY DAY

 002 T069902B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069902C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069902D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069902M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0125

 DESCRIPTION: HOW OFTEN WRITING ASSIGNMENTS-THREE OR MORE PAGES

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069903 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T069903A (01 ) 0000 ALMOST EVERY DAY

 002 T069903B (02 ) 1000 ONCE/TWICE A WEEK

 003 T069903C (03 ) 0100 ONCE/TWICE A MONTH

 004 T069903D (04 ) 0010 NEVER OR HARDLY EVER

 005 T069903M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0126

 DESCRIPTION: HOW OFTEN STUDENTS USE COMPUTER-SPELL, PUNC, GRAM

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070001A (01 ) 0000 ALMOST EVERY DAY

 002 T070001B (02 ) 1000 ONCE/TWICE A WEEK

 003 T070001C (03 ) 0100 ONCE/TWICE A MONTH

 004 T070001D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070001M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0127

 DESCRIPTION: HOW OFTEN STUDENTS USE COMPUTERS-WRITE DRAFTS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070002 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070002A (01 ) 0000 ALMOST EVERY DAY

 002 T070002B (02 ) 1000 ONCE/TWICE A WEEK

 003 T070002C (03 ) 0100 ONCE/TWICE A MONTH

 004 T070002D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070002M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0128

 DESCRIPTION: HOW OFTEN STUDENTS USE COMPUTERS-READ STORIES

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070003 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070003A (01 ) 0000 ALMOST EVERY DAY

 002 T070003B (02 ) 1000 ONCE/TWICE A WEEK

 003 T070003C (03 ) 0100 ONCE/TWICE A MONTH

 004 T070003D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070003M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0129

 DESCRIPTION: HOW OFTEN READING ASSESSED-MULTIPLE-CHOICE TESTS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070101A (01 ) 0000 ONCE/TWICE A WEEK

 002 T070101B (02 ) 1000 ONCE/TWICE A MONTH

 003 T070101C (03 ) 0100 ONCE/TWICE A YEAR

 004 T070101D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070101M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0130

 DESCRIPTION: HOW OFTEN READING ASSESSED-SHORT-ANSWER TESTS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070102 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070102A (01 ) 0000 ONCE/TWICE A WEEK

 002 T070102B (02 ) 1000 ONCE/TWICE A MONTH

 003 T070102C (03 ) 0100 ONCE/TWICE A YEAR

 004 T070102D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070102M (M ) 0001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

726

 CONDITIONING VARIABLE ID: TCHR0131

 DESCRIPTION: HOW OFTEN READ ASSESSED-PARAGRAPH WRITTEN RESPONSE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070103 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070103A (01 ) 0000 ONCE/TWICE A WEEK

 002 T070103B (02 ) 1000 ONCE/TWICE A MONTH

 003 T070103C (03 ) 0100 ONCE/TWICE A YEAR

 004 T070103D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070103M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0132

 DESCRIPTION: HOW OFTEN STUDENTS ASSESSED-INDIVIDUAL/GROUP PROJ

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070104 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070104A (01 ) 0000 ONCE/TWICE A WEEK

 002 T070104B (02 ) 1000 ONCE/TWICE A MONTH

 003 T070104C (03 ) 0100 ONCE/TWICE A YEAR

 004 T070104D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070104M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0133

 DESCRIPTION: HOW OFTEN STUDENTS ASSESSED-READING PORTFOLIOS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070105 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070105A (01 ) 0000 ONCE/TWICE A WEEK

 002 T070105B (02 ) 1000 ONCE/TWICE A MONTH

 003 T070105C (03 ) 0100 ONCE/TWICE A YEAR

 004 T070105D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070105M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0134

 DESCRIPTION: HOW OFTEN STUDENTS ASSESSED-ESSAYS/PAPERS ASSIGNED

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070106 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070106A (01 ) 0000 ONCE/TWICE A WEEK

 002 T070106B (02 ) 1000 ONCE/TWICE A MONTH

 003 T070106C (03 ) 0100 ONCE/TWICE A YEAR

 004 T070106D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070106M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0135

 DESCRIPTION: HOW OFTEN STUDENTS ASSESSED-ORAL READING

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070107 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070107A (01 ) 0000 ONCE/TWICE A WEEK

 002 T070107B (02 ) 1000 ONCE/TWICE A MONTH

 003 T070107C (03 ) 0100 ONCE/TWICE A YEAR

 004 T070107D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070107M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0136

 DESCRIPTION: HOW OFTEN WRITING ASSESSED-MULTIPLE-CHOICE TESTS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070201A (01 ) 0000 ONCE/TWICE A WEEK

 002 T070201B (02 ) 1000 ONCE/TWICE A MONTH

 003 T070201C (03 ) 0100 ONCE/TWICE A YEAR

 004 T070201D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070201M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0137

 DESCRIPTION: HOW OFTEN WRITING ASSESSED-PARAGRAPH WRITTEN

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070202 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070202A (01 ) 0000 ONCE/TWICE A WEEK

 002 T070202B (02 ) 1000 ONCE/TWICE A MONTH

 003 T070202C (03 ) 0100 ONCE/TWICE A YEAR

 004 T070202D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070202M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0138

 DESCRIPTION: HOW OFTEN WRITING ASSESSED-ESSAYS, REPORTS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070203 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070203A (01 ) 0000 ONCE/TWICE A WEEK

 002 T070203B (02 ) 1000 ONCE/TWICE A MONTH

 003 T070203C (03 ) 0100 ONCE/TWICE A YEAR

 004 T070203D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070203M (M ) 0001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

727

 CONDITIONING VARIABLE ID: TCHR0139

 DESCRIPTION: HOW OFTEN WRITING ASSESSED-WRITING PORTFOLIOS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070204 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070204A (01 ) 0000 ONCE/TWICE A WEEK

 002 T070204B (02 ) 1000 ONCE/TWICE A MONTH

 003 T070204C (03 ) 0100 ONCE/TWICE A YEAR

 004 T070204D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070204M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0140

 DESCRIPTION: HOW IMPORTANT TO GRADE-SPELLING, GRAMMAR, PUNC

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T070301A (01 ) 000 VERY IMPORTANT

 002 T070301B (02 ) 100 MODERATELY IMPORTANT

 003 T070301C (03 ) 010 UNIMPORTANT

 004 T070301M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0141

 DESCRIPTION: HOW IMPORTANT TO GRADE-ORGANIZATION/COHERENCE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070302 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T070302A (01 ) 000 VERY IMPORTANT

 002 T070302B (02 ) 100 MODERATELY IMPORTANT

 003 T070302C (03 ) 010 UNIMPORTANT

 004 T070302M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0142

 DESCRIPTION: HOW IMPORTANT TO GRADE-QUALITY/CREATIVITY OF IDEAS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070303 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T070303A (01 ) 000 VERY IMPORTANT

 002 T070303B (02 ) 100 MODERATELY IMPORTANT

 003 T070303C (03 ) 010 UNIMPORTANT

 004 T070303M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0143

 DESCRIPTION: HOW IMPORTANT TO GRADE-LENGTH OF PAPERS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070304 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T070304A (01 ) 000 VERY IMPORTANT

 002 T070304B (02 ) 100 MODERATELY IMPORTANT

 003 T070304C (03 ) 010 UNIMPORTANT

 004 T070304M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0144

 DESCRIPTION: HOW IMPORTANT TO GRADE-ACCOMPLISH WRITING PURPOSE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T070305 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T070305A (01 ) 000 VERY IMPORTANT

 002 T070305B (02 ) 100 MODERATELY IMPORTANT

 003 T070305C (03 ) 010 UNIMPORTANT

 004 T070305M (M ) 001 MISSING

 004 T070305M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0145

 DESCRIPTION: DO YOU TEACH READING

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T071601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T071601Y (01 ) 0 YES

 002 T071601M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0146

 DESCRIPTION: DO YOU TEACH WRITING

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T071602 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T071602Y (01 ) 0 YES

 002 T071602M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0147

 DESCRIPTION: DO YOU TEACH ENGLISH

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T071603 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T071603Y (01 ) 0 YES

 002 T071603M (M ) 1 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

728

 CONDITIONING VARIABLE ID: TCHR0148

 DESCRIPTION: DO YOU TEACH-OTHER

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T071604 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T071604Y (01 ) 0 YES

 002 T071604M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0149

 DESCRIPTION: YEARS TOTAL TAUGHT ELEMENTARY OR SECONDARY

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T040301A (01 ) 00000 2 YEARS OR LESS

 002 T040301B (02 ) 10000 3-5 YEARS

 003 T040301C (03 ) 01000 6-10 YEARS

 004 T040301D (04 ) 00100 11-24 YEARS

 005 T040301E (05 ) 00010 25 YEARS OR MORE

 006 T040301M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0150

 DESCRIPTION: YEARS TOTAL TAUGHT READING

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T071701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 T071701A (01 ) 000000 NOT TAUGHT

 002 T071701B (02 ) 100000 2 YEARS OR LESS

 003 T071701C (03 ) 010000 3-5 YEARS

 004 T071701D (04 ) 001000 6-10 YEARS

 005 T071701E (05 ) 000100 11-24 YEARS

 006 T071701F (06 ) 000010 25 YEARS OR MORE

 007 T071701M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0151

 DESCRIPTION: YEARS TOTAL TAUGHT WRITING

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T071702 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 T071702A (01 ) 000000 NOT TAUGHT

 002 T071702B (02 ) 100000 2 YEARS OR LESS

 003 T071702C (03 ) 010000 3-5 YEARS

 004 T071702D (04 ) 001000 6-10 YEARS

 005 T071702E (05 ) 000100 11-24 YEARS

 006 T071702F (06 ) 000010 25 YEARS OR MORE

 007 T071702M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0152

 DESCRIPTION: YEARS TOTAL TAUGHT ENGLISH

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T071703 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 T071703A (01 ) 000000 NOT TAUGHT

 002 T071703B (02 ) 100000 2 YEARS OR LESS

 003 T071703C (03 ) 010000 3-5 YEARS

 004 T071703D (04 ) 001000 6-10 YEARS

 005 T071703E (05 ) 000100 11-24 YEARS

 006 T071703F (06 ) 000010 25 YEARS OR MORE

 007 T071703M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0153

 DESCRIPTION: YEARS TOTAL TAUGHT- OTHER

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T071704 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 T071704A (01 ) 000000 NOT TAUGHT

 002 T071704B (02 ) 100000 2 YEARS OR LESS

 003 T071704C (03 ) 010000 3-5 YEARS

 004 T071704D (04 ) 001000 6-10 YEARS

 005 T071704E (05 ) 000100 11-24 YEARS

 006 T071704F (06 ) 000010 25 YEARS OR MORE

 007 T071704M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0154

 DESCRIPTION: LAST 12 MOS, PROF DEV-LITERATURE

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T067703 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067703A (01 ) 00000 NONE

 002 T067703B (02 ) 10000 LESS THAN 6 HOURS

 003 T067703C (03 ) 01000 6 - 15 HOURS

 004 T067703D (04 ) 00100 16 - 35 HOURS

 005 T067703E (05 ) 00010 MORE THAN 35 HOURS

 006 T067703M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0155

 DESCRIPTION: ARE STUDENTS ASSIGNED TO THIS CLASS BY ABILITY

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T068501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T068501Y (01 ) 00 YES

 002 T068501N (02 ) 10 NO

 003 T068501M (M ) 01 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

729

 CONDITIONING VARIABLE ID: TCHR0156

 DESCRIPTION: HOW OFTEN STUDENTS CHOOSE WRITING TOPIC

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069801A (01 ) 000 ALWAYS

 002 T069801B (02 ) 100 SOMETIMES

 003 T069801C (03 ) 010 NEVER

 004 T069801M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0157

 DESCRIPTION: HOW OFTEN STUDENTS PLAN THEIR WRITING

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069802 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069802A (01 ) 000 ALWAYS

 002 T069802B (02 ) 100 SOMETIMES

 003 T069802C (03 ) 010 NEVER

 004 T069802M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0158

 DESCRIPTION: HOW OFTEN STUDENTS DEFINE PURPOSES AND AUDIENCE

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069803 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069803A (01 ) 000 ALWAYS

 002 T069803B (02 ) 100 SOMETIMES

 003 T069803C (03 ) 010 NEVER

 004 T069803M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0159

 DESCRIPTION: HOW OFTEN STUDENTS MAKE FORMAL OUTLINE

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069804 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069804A (01 ) 000 ALWAYS

 002 T069804B (02 ) 100 SOMETIMES

 003 T069804C (03 ) 010 NEVER

 004 T069804M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0160

 DESCRIPTION: HOW OFTEN STUDENTS WRITE MORE THAN ONE DRAFT

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069805 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069805A (01 ) 000 ALWAYS

 002 T069805B (02 ) 100 SOMETIMES

 003 T069805C (03 ) 010 NEVER

 004 T069805M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0161

 DESCRIPTION: HOW OFTEN STUDENTS USE RESOURCES OTHER THAN TEXT

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069806 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069806A (01 ) 000 ALWAYS

 002 T069806B (02 ) 100 SOMETIMES

 003 T069806C (03 ) 010 NEVER

 004 T069806M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0162

 DESCRIPTION: HOW OFTEN STUDENTS DISCUSS WRITING WHILE WRITING

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069807 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069807A (01 ) 000 ALWAYS

 002 T069807B (02 ) 100 SOMETIMES

 003 T069807C (03 ) 010 NEVER

 004 T069807M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0163

 DESCRIPTION: HOW OFTEN STUDENTS DISCUSS OTHERS’ WRITING

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069808 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069808A (01 ) 000 ALWAYS

 002 T069808B (02 ) 100 SOMETIMES

 003 T069808C (03 ) 010 NEVER

 004 T069808M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0164

 DESCRIPTION: HOW OFTEN STUDENTS CHECK PROPER SPELLING, GRAMMAR

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069809 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069809A (01 ) 000 ALWAYS

 002 T069809B (02 ) 100 SOMETIMES

 003 T069809C (03 ) 010 NEVER

 004 T069809M (M ) 001 MISSING
Table F-6 (continued)

1998 Writing Conditioning Variable Specifications

730

 CONDITIONING VARIABLE ID: TCHR0165

 DESCRIPTION: HOW OFTEN STUDENTS DISCUSS WRITING WITH FAMILY

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069810 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069810A (01 ) 000 ALWAYS

 002 T069810B (02 ) 100 SOMETIMES

 003 T069810C (03 ) 010 NEVER

 004 T069810M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0166

 DESCRIPTION: HOW OFTEN STUDENTS CONTRIBUTE TO COLLECTION

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069811 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069811A (01 ) 000 ALWAYS

 002 T069811B (02 ) 100 SOMETIMES

 003 T069811C (03 ) 010 NEVER

 004 T069811M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0167

 DESCRIPTION: HOW OFTEN STUDENTS WORK ON AN ASSIGNED TOPIC

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069812 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069812A (01 ) 000 ALWAYS

 002 T069812B (02 ) 100 SOMETIMES

 003 T069812C (03 ) 010 NEVER

 004 T069812M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0168

 DESCRIPTION: HOW OFTEN STUDENTS FOLLOW ASSIGNED FORMAT

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T069813 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T069813A (01 ) 000 ALWAYS

 002 T069813B (02 ) 100 SOMETIMES

 003 T069813C (03 ) 010 NEVER

 004 T069813M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0169

 DESCRIPTION: WHAT IS THE NUMBER OF STUDENTS IN EACH CLASS? (8TH GRADE)

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL: CLASSIZ8

 NAEP ID: TCSIZE TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 CLASIZ-1 (1 ) 00000 AVERAGE CLASS SIZE: 1-20 STUDENTS

 002 CLASIZ-2 (2 ) 10000 AVERAGE CLASS SIZE: 21-25 STUDENTS

 003 CLASIZ-3 (3 ) 01000 AVERAGE CLASS SIZE: 26-30 STUDENTS

 004 CLASIZ-4 (4 ) 00100 AVERAGE CLASS SIZE: 31-35 STUDENTS

 005 CLASIZ-5 (5 ) 00010 AVERAGE CLASS SIZE: 36 OR MORE STUDENTS

 006 CLASIZ-? (M ) 00001 AVERAGE CLASS SIZE: MISSING, DOES NOT APPLY
Table F-7

1998 Civics Conditioning Variable Specifications

731

 CONDITIONING VARIABLE ID: BACK0001

 DESCRIPTION: GRAND MEAN

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: OVERALL

 NAEP ID: BKSER TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: OTHER NUMBER OF INDEPENDENT CONTRASTS: 1

 001 OVERALL (@ ) 1 GRAND MEAN

 CONDITIONING VARIABLE ID: BACK0002

 DESCRIPTION: DERIVED SEX

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: GENDER

 NAEP ID: DSEX TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 MALE (1,M ) 0 MALE

 002 FEMALE (2 ) 1 FEMALE

 CONDITIONING VARIABLE ID: BACK0003

 DESCRIPTION: DERIVED RACE/ETHNICITY

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: RACE/ETH

 NAEP ID: DRACE TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 WHI/AI/O (1,5,6,M ) 000 RACE/ETHNICITY: WHITE, AMERICAN

INDIAN/ALASKAN NATIVE, OTHER, MISSING, UNCLASSIFIED

 002 BLACK (2 ) 100 RACE/ETHNICITY: BLACK

 003 HISPANIC (3 ) 010 RACE/ETHNICITY: HISPANIC

 004 ASIAN (4 ) 001 RACE/ETHNICITY: ASIAN / PACIFIC ISLANDER

 CONDITIONING VARIABLE ID: BACK0004

 DESCRIPTION: IF HISPANIC, WHAT IS YOUR HISPANIC BACKGROUND?

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: HISPANIC

 NAEP ID: B003101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 NOT HISP (1 ) 0000 HISPANIC: NOT HISPANIC

 002 MEXICAN (2 ) 1000 HISPANIC: MEXICAN, MEXICAN AMERICAN, CHICANO

 003 PUER RIC (3 ) 0100 HISPANIC: PUERTO RICAN

 004 CUBN,OTH (4,5 ) 0010 HISPANIC: CUBAN, OTHER

 005 HISP-? (M ) 0001 HISPANIC: MISSING

 CONDITIONING VARIABLE ID: BACK0005

 DESCRIPTION: TOL 7 - TYPE OF LOCATION

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: TOL7

 NAEP ID: TOL7 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 BIG CTY7 (1 ) 000000 TOL7: LARGE CITY

 002 MID CTY7 (2,M ) 100000 TOL7: MID-SIZE CITY

 003 FR/LCTY7 (3 ) 010000 TOL7: URBAN FRINGE OF LARGE CITY

 004 FR/MCTY7 (4 ) 001000 TOL7: URBAN FRINGE OF MID-SIZE CITY

 005 LAR TWN7 (5 ) 000100 TOL7: LARGE TOWN

 006 SML TWN7 (6 ) 000010 TOL7: SMALL TOWN

 007 OTHER (7 ) 000001 TOL7: OTHER

 CONDITIONING VARIABLE ID: BACK0006

 DESCRIPTION: TYPE OF LOCALE (5 CATEGORIES)

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: TOL5

 NAEP ID: TOL5 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 BIG CTY5 (1 ) 0000 TOL5: LARGE CITY

 002 MID CTY5 (2,M ) 1000 TOL5: MID-SIZE CITY

 003 FR/BTWN5 (3 ) 0100 TOL5: URBAN FRINGE AND LARGE TOWN

 004 SML TWN5 (4 ) 0010 TOL5: SMALL TOWN

 005 RURAL5 (5 ) 0001 TOL5: RURAL (MSA AND NON-MSA)

 CONDITIONING VARIABLE ID: BACK0007

 DESCRIPTION: PARENTS’ HIGHEST LEVEL OF EDUCATION, GRADE 4

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: PARED2

 NAEP ID: PARED2 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 < HS (1 ) 0000 PARED: LESS THAN HIGH SCHOOL

 002 HS GRAD (2 ) 1000 PARED: HIGH SCHOOL GRADUATE

 003 POST HS (3 ) 0100 PARED: POST HIGH SCHOOL

 004 COL GRAD (4 ) 0010 PARED: COLLEGE GRADUATE

 005 PARED-? (5,M ) 0001 PARED: MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0008

 DESCRIPTION: REGION OF THE COUNTRY

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: REGION

 NAEP ID: REGION TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 N EAST (1,M ) 000 REGION: NORTHEAST

 002 S EAST (2 ) 100 REGION: SOUTHEAST

 003 CENTRAL (3 ) 010 REGION: CENTRAL

 004 WEST (4,5 ) 001 REGION: WEST, TERRITORIES (NONE)

 CONDITIONING VARIABLE ID: BACK0009

 DESCRIPTION: SCHOOL TYPE

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: SCHTYPE

 NAEP ID: SCHTYPE TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 PUBLIC (1 ) 00 SCHOOL TYPE: PUBLIC,

 002 PRIVATE (2,4,5,M ) 10 SCHOOL TYPE: PRIVATE, BIA, DEPARTMENT OF

DEFENSE, MISSING

003 CATHOLIC (3 ) 01 SCHOOL TYPE: CATHOLIC
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

732

 CONDITIONING VARIABLE ID: BACK0010

 DESCRIPTION: RACE

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: RACE

 NAEP ID: RACE TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 WHI/AI/O (1,5,6,M ) 000 RACE: WHITE, AMERICAN INDIAN/ALASKAN NATIVE,

OTHER, MISSING, UNCLASSIFIED

 002 BLACK (2 ) 100 RACE: BLACK

 003 HISPANIC (3 ) 010 RACE: HISPANIC

 004 ASIAN (4 ) 001 RACE: ASIAN / PACIFIC ISLANDER

 CONDITIONING VARIABLE ID: BACK0011

 DESCRIPTION: INDIVIDUALIZED EDUCATION PLAN

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: IEP

 NAEP ID: IEP TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 IEP-YES (1 ) 0 IEP: YES

 002 IEP-NO (2,M ) 1 IEP: NO

 CONDITIONING VARIABLE ID: BACK0012

 DESCRIPTION: LIMITED ENGLISH PROFICIENCY

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: LEP

 NAEP ID: LEP TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 LEP-YES (1 ) 0 LEP: YES

 002 LEP-NO (2,M ) 1 LEP: NO

 CONDITIONING VARIABLE ID: BACK0013

 DESCRIPTION: TITLE 1: (BOOK COVER)

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: TITLE 1

 NAEP ID: TITLE1 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 TITLE-Y (1 ) 0 TITLE 1: YES

 002 TITLE-N (2,M ) 1 TITLE 1: NO

 CONDITIONING VARIABLE ID: BACK0014

 DESCRIPTION: DO YOU RECEIVE A FREE OR REDUCED-PRICE LUNCH?

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: LUNCH

 NAEP ID: SLUNCH TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 NOT ELIG (1 ) 00000 LUNCH PROGRAM: NOT ELIGIBLE

 002 RED PRIC (2 ) 10000 LUNCH PROGRAM: REDUCED PRICE

 003 FREE (3 ) 01000 LUNCH PROGRAM: FREE

 004 INFO N/A (4,M ) 00100 LUNCH PROGRAM: INFO NOT AVAILABLE

 005 SCH/REF (5 ) 00010 LUNCH PROGRAM: SCHOOL REFUSAL

 006 SCH/NP (6 ) 00001 LUNCH PROGRAM: SCHOOL NOT PARTIPATE

 CONDITIONING VARIABLE ID: BACK0015

 DESCRIPTION: HOW MUCH TELEVISION/VIDEO GAMES DO YOU USUALLY WATCH EACH DAY? (LINEAR)

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: TVWATCHL

 NAEP ID: B013901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 TVLIN-0 (1 ) 0 TV WATCHING (LINEAR) (0 TO 6+ HOURS PER DAY)

 002 TVLIN-1 (2 ) 1 TV WATCHING (LINEAR)

 003 TVLIN-2 (3 ) 2 TV WATCHING (LINEAR)

 004 TVLIN-3 (4,M ) 3 TV WATCHING (LINEAR)

 005 TVLIN-4 (5 ) 4 TV WATCHING (LINEAR)

 006 TVLIN-5 (6 ) 5 TV WATCHING (LINEAR)

 007 TVLIN-6 (7 ) 6 TV WATCHING (LINEAR)

 CONDITIONING VARIABLE ID: BACK0016

 DESCRIPTION: HOW MUCH TELEVISION/VIDEO GAMES DO YOU USUALLY WATCH EACH DAY? (QUADRATIC)

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: TVWATCHQ

 NAEP ID: B013901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: QUADRATIC NUMBER OF INDEPENDENT CONTRASTS: 1

 001 TV-QUAD (1-7,M=4 ) 1.0 + -2.0*X + 1.0*X**2 TV WATCHING (QUADRATIC)

 CONDITIONING VARIABLE ID: BACK0017

 DESCRIPTION: HOMEWORK ASSIGNED?: BASED ON TIME SPENT ON HOMEWORK EACH DAY.

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: HWASSIGN

 NAEP ID: B006601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 HW-MISS (M ) 00 HOMEWORK ASSIGNED?: MISSING

 002 HW-NO (1 ) 10 HOMEWORK ASSIGNED?: NO

 003 HW-YES (2-5 ) 01 HOMEWORK ASSIGNED?: YES

 CONDITIONING VARIABLE ID: BACK0018

 DESCRIPTION: HOW MUCH TIME DO YOU USUALLY SPEND ON HOMEWORK EACH DAY? (LINEAR)

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: HOMEWRKL

 NAEP ID: B006601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 HWLIN-0 (1,2,M ) 0 HOMEWORK (LINEAR): DON’T HAVE ANY, DON’T DO

ANY, MISSING

 002 HWLIN-1 (3 ) 1 HOMEWORK (LINEAR): 1/2 HOUR OR LESS

 003 HWLIN-2 (4 ) 2 HOMEWORK (LINEAR): 1 HOUR

 004 HWLIN-3 (5 ) 3 HOMEWORK (LINEAR): MORE THAN 1 HOUR
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

733

 CONDITIONING VARIABLE ID: BACK0019

 DESCRIPTION: HOW MUCH TIME DO YOU USUALLY SPEND ON HOMEWORK EACH DAY (QUADRATIC)

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: HOMEWRKQ

 NAEP ID: B006601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: SCALE NUMBER OF INDEPENDENT CONTRASTS: 1

 001 HWQUAD-0 (1,2,M ) 0 HOMEWORK (QUADRATIC): DON’T HAVE ANY, DON’T

DO ANY, MISSING

 002 HWQUAD-1 (3 ) 1 HOMEWORK (QUADRATIC): 1/2 HOUR OR LESS

 003 HWQUAD-2 (4 ) 4 HOMEWORK (QUADRATIC): 1 HOUR

 004 HWQUAD-3 (5 ) 9 HOMEWORK (QUADRATIC): MORE THAN 1 HOUR

 CONDITIONING VARIABLE ID: BACK0020

 DESCRIPTION: NUMBER OF ITEMS IN THE HOME (NEWSPAPER, > 25 BOOKS, ENCYCLOPEDIA, MAGAZINES) (DERIVED)

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: HOMEITMS

 NAEP ID: HOMEEN3 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 HITEM<=2 (1,M ) 00 ITEMS IN HOME: ZERO TO TWO ITEMS, MISSING

 002 HITEM=3 (2 ) 10 ITEMS IN HOME: THREE ITEMS

 003 HITEM=4 (3 ) 01 ITEMS IN HOME: FOUR ITEMS

 CONDITIONING VARIABLE ID: BACK0021

 DESCRIPTION: ABOUT HOW MANY PAGES A DAY DO YOU HAVE TO READ FOR SCHOOL AND HOMEWORK?

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: PGSREAD1

 NAEP ID: B001101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 PGS<6,? (5,M ) 0 PAGES READ: 5 OR FEWER A DAY, MISSING

 002 PGS>5 (1,2,3,4 ) 1 PAGES READ: 6-10, 11-15, 16-20, 20 OR MORE

 CONDITIONING VARIABLE ID: BACK0022

 DESCRIPTION: ABOUT HOW MANY PAGES A DAY DO YOU HAVE TO READ FOR SCHOOL AND HOMEWORK?

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: PGSREAD2

 NAEP ID: B001101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 PGS<11,? (4,5,M ) 0 PAGES READ: 6-10, 5 OR FEWER A DAY, MISSING

 002 PGS>10 (1,2,3 ) 1 PAGES READ: 11-15, 16-20, 20 OR MORE

 CONDITIONING VARIABLE ID: BACK0023

 DESCRIPTION: STUDENTS ACCOMMODATION STATUS

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: ACCOM

 NAEP ID: ACCOM TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 ACCOM (1,2 ) 0 ACCOMMODATED WITH APPROPRIATE BOOK OR WRONG

BOOK

 002 NO ACCOM (3 ) 1 NON ACCOMMODATED

 CONDITIONING VARIABLE ID: BACK0024

 DESCRIPTION: NUMBER OF YEARS TAKING CIVICS COURSES IN HIGH SCHOOL

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: NYRCIV

 NAEP ID: NYRCIV TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 NYRCIV A (1,M ) 0000 NONE

 002 NYRCIV B (2 ) 1000 1 YEAR

 003 NYRCIV C (3 ) 0100 2 YEARS

 004 NYRCIV D (4 ) 0010 3 YEARS

 005 NYRCIV E (5 ) 0001 4 YEARS

 CONDITIONING VARIABLE ID: BACK0025

 DESCRIPTION: CIVICS COURSES TAKING IN 11TH AND 12TH GRADES

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: NYRCIV2

 NAEP ID: NYRCIV2 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 NYRCIV2A (1 ) 00 NEITHER

 002 NYRCIV2B (2 ) 10 EITHER 11TH OR 12TH

 003 NYRCIV2C (3 ) 01 BOTH 11TH AND 12TH

 CONDITIONING VARIABLE ID: BACK0026

 DESCRIPTION: INTERACTION: GENDER BY RACE/ETHNICITY

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: GEND/RAC

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 3

 001 G/R 11 (11 ) 010101 GEND/RAC INTACT: 1. MALE 1. WHI/AI/O

 002 G/R 12 (12 ) -10000 GEND/RAC INTACT: 1. MALE 2. BLACK

 003 G/R 13 (13 ) 00-100 GEND/RAC INTACT: 1. MALE 3. HISPANIC

 004 G/R 14 (14 ) 0000-1 GEND/RAC INTACT: 1. MALE 4. ASIAN

 005 G/R 21 (21 ) -1-1-1 GEND/RAC INTACT: 2. FEMALE 1. WHI/AI/O

 006 G/R 22 (22 ) 010000 GEND/RAC INTACT: 2. FEMALE 2. BLACK

 007 G/R 23 (23 ) 000100 GEND/RAC INTACT: 2. FEMALE 3. HISPANIC

 008 G/R 24 (24 ) 000001 GEND/RAC INTACT: 2. FEMALE 4. ASIAN
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

734

 CONDITIONING VARIABLE ID: BACK0027

 DESCRIPTION: INTERACTION: GENDER BY TYPE OF LOCALE (7 CATEGORIES)

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: GEND/TOL

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 14

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 6

 001 G/T 11 (11 ) 010101010101 GEND/TOL INTACT: 1. MALE 1. BIG CTY7

 002 G/T 12 (12 ) -10000000000 GEND/TOL INTACT: 1. MALE 2. MID CTY7

 003 G/T 13 (13 ) 00-100000000 GEND/TOL INTACT: 1. MALE 3. FR/LCTY7

 004 G/T 14 (14 ) 0000-1000000 GEND/TOL INTACT: 1. MALE 4. FR/MCTY7

 005 G/T 15 (15 ) 000000-10000 GEND/TOL INTACT: 1. MALE 5. LAR TWN7

 006 G/T 16 (16 ) 00000000-100 GEND/TOL INTACT: 1. MALE 6. SML TWN7

 007 G/T 17 (17 ) 0000000000-1 GEND/TOL INTACT: 1. MALE 7. OTHER

 008 G/T 21 (21 ) -1-1-1-1-1-1 GEND/TOL INTACT: 2. FEMALE 1. BIG CTY7

 009 G/T 22 (22 ) 010000000000 GEND/TOL INTACT: 2. FEMALE 2. MID CTY7

 010 G/T 23 (23 ) 000100000000 GEND/TOL INTACT: 2. FEMALE 3. FR/LCTY7

 011 G/T 24 (24 ) 000001000000 GEND/TOL INTACT: 2. FEMALE 4. FR/MCTY7

 012 G/T 25 (25 ) 000000010000 GEND/TOL INTACT: 2. FEMALE 5. LAR TWN7

 013 G/T 26 (26 ) 000000000100 GEND/TOL INTACT: 2. FEMALE 6. SML TWN7

 014 G/T 27 (27 ) 000000000001 GEND/TOL INTACT: 2. FEMALE 7. OTHER

 CONDITIONING VARIABLE ID: BACK0028

 DESCRIPTION: INTERACTION: GENDER BY PARENTS’ EDUCATION ALL GRADES

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: GEND/PAR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 10

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 4

 001 G/P 11 (11 ) 01010101 GEND/PAR INTACT: 1. MALE 1. < HS

 002 G/P 12 (12 ) -1000000 GEND/PAR INTACT: 1. MALE 2. HS GRAD

 003 G/P 13 (13 ) 00-10000 GEND/PAR INTACT: 1. MALE 3. POST HS

 004 G/P 14 (14 ) 0000-100 GEND/PAR INTACT: 1. MALE 4. COL GRAD

 005 G/P 15 (15 ) 000000-1 GEND/PAR INTACT: 1. MALE 5. PARED-?

 006 G/P 21 (21 ) -1-1-1-1 GEND/PAR INTACT: 2. FEMALE 1. < HS

 007 G/P 22 (22 ) 01000000 GEND/PAR INTACT: 2. FEMALE 2. HS GRAD

 008 G/P 23 (23 ) 00010000 GEND/PAR INTACT: 2. FEMALE 3. POST HS

 009 G/P 24 (24 ) 00000100 GEND/PAR INTACT: 2. FEMALE 4. COL GRAD

 010 G/P 25 (25 ) 00000001 GEND/PAR INTACT: 2. FEMALE 5. PARED-?

 CONDITIONING VARIABLE ID: BACK0029

 DESCRIPTION: INTERACTION: GENDER BY SCHOOL TYPE

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: GEND/SCH

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 2

 001 G/S 11 (11 ) 0101 GEND/SCH INTACT: 1. MALE 1. PUBLIC

 002 G/S 12 (12 ) -100 GEND/SCH INTACT: 1. MALE 2. PRIVATE

 003 G/S 13 (13 ) 00-1 GEND/SCH INTACT: 1. MALE 3. CATHOLIC

 004 G/S 21 (21 ) -1-1 GEND/SCH INTACT: 2. FEMALE 1. PUBLIC

 005 G/S 22 (22 ) 0100 GEND/SCH INTACT: 2. FEMALE 2. PRIVATE

 006 G/S 23 (23 ) 0001 GEND/SCH INTACT: 2. FEMALE 3. CATHOLIC

 CONDITIONING VARIABLE ID: BACK0030

 DESCRIPTION: INTERACTION: RACE/ETHNICITY BY TYPE OF LOCALE (7 CATEGORIES)

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: RACE/TOL

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 28

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 18

 001 R/T 11 (11 ) 010101010101010101010101010101010101 RACE/TOL INTACT: 1. WHI/AI/O 1. BIG CTY7

 002 R/T 12 (12 ) -10000000000-10000000000-10000000000 RACE/TOL INTACT: 1. WHI/AI/O 2. MID CTY7

 003 R/T 13 (13 ) 00-10000000000-10000000000-100000000 RACE/TOL INTACT: 1. WHI/AI/O 3. FR/LCTY7

 004 R/T 14 (14 ) 0000-10000000000-10000000000-1000000 RACE/TOL INTACT: 1. WHI/AI/O 4. FR/MCTY7

 005 R/T 15 (15 ) 000000-10000000000-10000000000-10000 RACE/TOL INTACT: 1. WHI/AI/O 5. LAR TWN7

 006 R/T 16 (16 ) 00000000-10000000000-10000000000-100 RACE/TOL INTACT: 1. WHI/AI/O 6. SML TWN7

 007 R/T 17 (17 ) 0000000000-10000000000-10000000000-1 RACE/TOL INTACT: 1. WHI/AI/O 7. OTHER

 008 R/T 21 (21 ) -1-1-1-1-1-1000000000000000000000000 RACE/TOL INTACT: 2. BLACK 1. BIG CTY7

 009 R/T 22 (22 ) 010000000000000000000000000000000000 RACE/TOL INTACT: 2. BLACK 2. MID CTY7

 010 R/T 23 (23 ) 000100000000000000000000000000000000 RACE/TOL INTACT: 2. BLACK 3. FR/LCTY7

 011 R/T 24 (24 ) 000001000000000000000000000000000000 RACE/TOL INTACT: 2. BLACK 4. FR/MCTY7

 012 R/T 25 (25 ) 000000010000000000000000000000000000 RACE/TOL INTACT: 2. BLACK 5. LAR TWN7

 013 R/T 26 (26 ) 000000000100000000000000000000000000 RACE/TOL INTACT: 2. BLACK 6. SML TWN7

 014 R/T 27 (27 ) 000000000001000000000000000000000000 RACE/TOL INTACT: 2. BLACK 7. OTHER

 015 R/T 31 (31 ) 000000000000-1-1-1-1-1-1000000000000 RACE/TOL INTACT: 3. HISPANIC 1. BIG CTY7

 016 R/T 32 (32 ) 000000000000010000000000000000000000 RACE/TOL INTACT: 3. HISPANIC 2. MID CTY7

 017 R/T 33 (33 ) 000000000000000100000000000000000000 RACE/TOL INTACT: 3. HISPANIC 3. FR/LCTY7

 018 R/T 34 (34 ) 000000000000000001000000000000000000 RACE/TOL INTACT: 3. HISPANIC 4. FR/MCTY7

 019 R/T 35 (35 ) 000000000000000000010000000000000000 RACE/TOL INTACT: 3. HISPANIC 5. LAR TWN7

 020 R/T 36 (36 ) 000000000000000000000100000000000000 RACE/TOL INTACT: 3. HISPANIC 6. SML TWN7

 021 R/T 37 (37 ) 000000000000000000000001000000000000 RACE/TOL INTACT: 3. HISPANIC 7. OTHER

 022 R/T 41 (41 ) 000000000000000000000000-1-1-1-1-1-1 RACE/TOL INTACT: 4. ASIAN 1. BIG CTY7

 023 R/T 42 (42 ) 000000000000000000000000010000000000 RACE/TOL INTACT: 4. ASIAN 2. MID CTY7

 024 R/T 43 (43 ) 000000000000000000000000000100000000 RACE/TOL INTACT: 4. ASIAN 3. FR/LCTY7

 025 R/T 44 (44 ) 000000000000000000000000000001000000 RACE/TOL INTACT: 4. ASIAN 4. FR/MCTY7

 026 R/T 45 (45 ) 000000000000000000000000000000010000 RACE/TOL INTACT: 4. ASIAN 5. LAR TWN7

 027 R/T 46 (46 ) 000000000000000000000000000000000100 RACE/TOL INTACT: 4. ASIAN 6. SML TWN7

 028 R/T 47 (47 ) 000000000000000000000000000000000001 RACE/TOL INTACT: 4. ASIAN 7. OTHER
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

735

 CONDITIONING VARIABLE ID: BACK0031

 DESCRIPTION: INTERACTION: RACE/ETHNICITY BY PARENTS’ EDUCATION ALL GRADES

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: RACE/PAR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 20

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 12

 001 R/P 11 (11 ) 010101010101010101010101 RACE/PAR INTACT: 1. WHI/AI/O 1. < HS

 002 R/P 12 (12 ) -1000000-1000000-1000000 RACE/PAR INTACT: 1. WHI/AI/O 2. HS GRAD

 003 R/P 13 (13 ) 00-1000000-1000000-10000 RACE/PAR INTACT: 1. WHI/AI/O 3. POST HS

 004 R/P 14 (14 ) 0000-1000000-1000000-100 RACE/PAR INTACT: 1. WHI/AI/O 4. COL GRAD

 005 R/P 15 (15 ) 000000-1000000-1000000-1 RACE/PAR INTACT: 1. WHI/AI/O 5. PARED-?

 006 R/P 21 (21 ) -1-1-1-10000000000000000 RACE/PAR INTACT: 2. BLACK 1. < HS

 007 R/P 22 (22 ) 010000000000000000000000 RACE/PAR INTACT: 2. BLACK 2. HS GRAD

 008 R/P 23 (23 ) 000100000000000000000000 RACE/PAR INTACT: 2. BLACK 3. POST HS

 009 R/P 24 (24 ) 000001000000000000000000 RACE/PAR INTACT: 2. BLACK 4. COL GRAD

 010 R/P 25 (25 ) 000000010000000000000000 RACE/PAR INTACT: 2. BLACK 5. PARED-?

 011 R/P 31 (31 ) 00000000-1-1-1-100000000 RACE/PAR INTACT: 3. HISPANIC 1. < HS

 012 R/P 32 (32 ) 000000000100000000000000 RACE/PAR INTACT: 3. HISPANIC 2. HS GRAD

 013 R/P 33 (33 ) 000000000001000000000000 RACE/PAR INTACT: 3. HISPANIC 3. POST HS

 014 R/P 34 (34 ) 000000000000010000000000 RACE/PAR INTACT: 3. HISPANIC 4. COL GRAD

 015 R/P 35 (35 ) 000000000000000100000000 RACE/PAR INTACT: 3. HISPANIC 5. PARED-?

 016 R/P 41 (41 ) 0000000000000000-1-1-1-1 RACE/PAR INTACT: 4. ASIAN 1. < HS

 017 R/P 42 (42 ) 000000000000000001000000 RACE/PAR INTACT: 4. ASIAN 2. HS GRAD

 018 R/P 43 (43 ) 000000000000000000010000 RACE/PAR INTACT: 4. ASIAN 3. POST HS

 019 R/P 44 (44 ) 000000000000000000000100 RACE/PAR INTACT: 4. ASIAN 4. COL GRAD

 020 R/P 45 (45 ) 000000000000000000000001 RACE/PAR INTACT: 4. ASIAN 5. PARED-?

 CONDITIONING VARIABLE ID: BACK0032

 DESCRIPTION: INTERACTION: RACE/ETHNICITY BY SCHOOL TYPE

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: RACE/SCH

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 12

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 6

 001 R/S 11 (11 ) 010101010101 RACE/SCH INTACT: 1. WHI/AI/O 1. PUBLIC

 002 R/S 12 (12 ) -100-100-100 RACE/SCH INTACT: 1. WHI/AI/O 2. PRIVATE

 003 R/S 13 (13 ) 00-100-100-1 RACE/SCH INTACT: 1. WHI/AI/O 3. CATHOLIC

 004 R/S 21 (21 ) -1-100000000 RACE/SCH INTACT: 2. BLACK 1. PUBLIC

 005 R/S 22 (22 ) 010000000000 RACE/SCH INTACT: 2. BLACK 2. PRIVATE

 006 R/S 23 (23 ) 000100000000 RACE/SCH INTACT: 2. BLACK 3. CATHOLIC

 007 R/S 31 (31 ) 0000-1-10000 RACE/SCH INTACT: 3. HISPANIC 1. PUBLIC

 008 R/S 32 (32 ) 000001000000 RACE/SCH INTACT: 3. HISPANIC 2. PRIVATE

 009 R/S 33 (33 ) 000000010000 RACE/SCH INTACT: 3. HISPANIC 3. CATHOLIC

 010 R/S 41 (41 ) 00000000-1-1 RACE/SCH INTACT: 4. ASIAN 1. PUBLIC

 011 R/S 42 (42 ) 000000000100 RACE/SCH INTACT: 4. ASIAN 2. PRIVATE

 012 R/S 43 (43 ) 000000000001 RACE/SCH INTACT: 4. ASIAN 3. CATHOLIC

 CONDITIONING VARIABLE ID: BACK0033

 DESCRIPTION: INTERACTION: PARENT’S EDUCATION ALL GRADES BY TYPE OF LOCALE (7 CATEGORIES)

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: PARE/TOL

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 35

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 24

 001 P/T 11 (11 ) 010101010101010101010101010101010101010101010101 PARE/TOL INTACT: 1. < HS 1. BIG CTY7

 002 P/T 12 (12 ) -10000000000-10000000000-10000000000-10000000000 PARE/TOL INTACT: 1. < HS 2. MID CTY7

 003 P/T 13 (13 ) 00-10000000000-10000000000-10000000000-100000000 PARE/TOL INTACT: 1. < HS 3. FR/LCTY7

 004 P/T 14 (14 ) 0000-10000000000-10000000000-10000000000-1000000 PARE/TOL INTACT: 1. < HS 4. FR/MCTY7

 005 P/T 15 (15 ) 000000-10000000000-10000000000-10000000000-10000 PARE/TOL INTACT: 1. < HS 5. LAR TWN7

 006 P/T 16 (16 ) 00000000-10000000000-10000000000-10000000000-100 PARE/TOL INTACT: 1. < HS 6. SML TWN7

 007 P/T 17 (17 ) 0000000000-10000000000-10000000000-10000000000-1 PARE/TOL INTACT: 1. < HS 7. OTHER

 008 P/T 21 (21 ) -1-1-1-1-1-1000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 1. BIG CTY7

 009 P/T 22 (22 ) 010000000000000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 2. MID CTY7

 010 P/T 23 (23 ) 000100000000000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 3. FR/LCTY7

 011 P/T 24 (24 ) 000001000000000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 4. FR/MCTY7

 012 P/T 25 (25 ) 000000010000000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 5. LAR TWN7

 013 P/T 26 (26 ) 000000000100000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 6. SML TWN7

 014 P/T 27 (27 ) 000000000001000000000000000000000000000000000000 PARE/TOL INTACT: 2. HS GRAD 7. OTHER

 015 P/T 31 (31 ) 000000000000-1-1-1-1-1-1000000000000000000000000 PARE/TOL INTACT: 3. POST HS 1. BIG CTY7

 016 P/T 32 (32 ) 000000000000010000000000000000000000000000000000 PARE/TOL INTACT: 3. POST HS 2. MID CTY7

 017 P/T 33 (33 ) 000000000000000100000000000000000000000000000000 PARE/TOL INTACT: 3. POST HS 3. FR/LCTY7

 018 P/T 34 (34 ) 000000000000000001000000000000000000000000000000 PARE/TOL INTACT: 3. POST HS 4. FR/MCTY7

 019 P/T 35 (35 ) 000000000000000000010000000000000000000000000000 PARE/TOL INTACT: 3. POST HS 5. LAR TWN7

 020 P/T 36 (36 ) 000000000000000000000100000000000000000000000000 PARE/TOL INTACT: 3. POST HS 6. SML TWN7

 021 P/T 37 (37 ) 000000000000000000000001000000000000000000000000 PARE/TOL INTACT: 3. POST HS 7. OTHER

 022 P/T 41 (41 ) 000000000000000000000000-1-1-1-1-1-1000000000000 PARE/TOL INTACT: 4. COL GRAD 1. BIG CTY7

 023 P/T 42 (42 ) 000000000000000000000000010000000000000000000000 PARE/TOL INTACT: 4. COL GRAD 2. MID CTY7

 024 P/T 43 (43 ) 000000000000000000000000000100000000000000000000 PARE/TOL INTACT: 4. COL GRAD 3. FR/LCTY7

 025 P/T 44 (44 ) 000000000000000000000000000001000000000000000000 PARE/TOL INTACT: 4. COL GRAD 4. FR/MCTY7

 026 P/T 45 (45 ) 000000000000000000000000000000010000000000000000 PARE/TOL INTACT: 4. COL GRAD 5. LAR TWN7

 027 P/T 46 (46 ) 000000000000000000000000000000000100000000000000 PARE/TOL INTACT: 4. COL GRAD 6. SML TWN7

 028 P/T 47 (47 ) 000000000000000000000000000000000001000000000000 PARE/TOL INTACT: 4. COL GRAD 7. OTHER

 029 P/T 51 (51 ) 000000000000000000000000000000000000-1-1-1-1-1-1 PARE/TOL INTACT: 5. PARED-? 1. BIG CTY7

 030 P/T 52 (52 ) 000000000000000000000000000000000000010000000000 PARE/TOL INTACT: 5. PARED-? 2. MID CTY7

 031 P/T 53 (53 ) 000000000000000000000000000100000000000100000000 PARE/TOL INTACT: 5. PARED-? 3. FR/LCTY7

 032 P/T 54 (54 ) 000000000000000000000000000000000000000001000000 PARE/TOL INTACT: 5. PARED-? 4. FR/MCTY7

 033 P/T 55 (55 ) 000000000000000000000000000000000000000000010000 PARE/TOL INTACT: 5. PARED-? 5. LAR TWN7

 034 P/T 56 (56 ) 000000000000000000000000000000000000000000000100 PARE/TOL INTACT: 5. PARED-? 6. SML TWN7

 035 P/T 57 (57 ) 000000000000000000000000000000000000000000000001 PARE/TOL INTACT: 5. PARED-? 7. OTHER
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

736

 CONDITIONING VARIABLE ID: BACK0034

 DESCRIPTION: INTERACTION: TYPE OF LOCALE (7 CATEGORIES) BY SCHOOL TYPE

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: TOL7/SCH

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 21

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 12

 001 T/S 11 (11 ) 010101010101010101010101 TOL7/SCH INTACT: 1. BIG CTY7 1. PUBLIC

 002 T/S 12 (12 ) -100-100-100-100-100-100 TOL7/SCH INTACT: 1. BIG CTY7 2. PRIVATE

 003 T/S 13 (13 ) 00-100-100-100-100-100-1 TOL7/SCH INTACT: 1. BIG CTY7 3. CATHOLIC

 004 T/S 21 (21 ) -1-100000000000000000000 TOL7/SCH INTACT: 2. MID CTY7 1. PUBLIC

 005 T/S 22 (22 ) 010000000000000000000000 TOL7/SCH INTACT: 2. MID CTY7 2. PRIVATE

 006 T/S 23 (23 ) 000100000000000000000000 TOL7/SCH INTACT: 2. MID CTY7 3. CATHOLIC

 007 T/S 31 (31 ) 0000-1-10000000000000000 TOL7/SCH INTACT: 3. FR/LCTY7 1. PUBLIC

 008 T/S 32 (32 ) 000001000000000000000000 TOL7/SCH INTACT: 3. FR/LCTY7 2. PRIVATE

 009 T/S 33 (33 ) 000000010000000000000000 TOL7/SCH INTACT: 3. FR/LCTY7 3. CATHOLIC

 010 T/S 41 (41 ) 00000000-1-1000000000000 TOL7/SCH INTACT: 4. FR/MCTY7 1. PUBLIC

 011 T/S 42 (42 ) 000000000100000000000000 TOL7/SCH INTACT: 4. FR/MCTY7 2. PRIVATE

 012 T/S 43 (43 ) 000000000001000000000000 TOL7/SCH INTACT: 4. FR/MCTY7 3. CATHOLIC

 013 T/S 51 (51 ) 000000000000-1-100000000 TOL7/SCH INTACT: 5. LAR TWN7 1. PUBLIC

 014 T/S 52 (52 ) 000000000000010000000000 TOL7/SCH INTACT: 5. LAR TWN7 2. PRIVATE

 015 T/S 53 (53 ) 000000000000000100000000 TOL7/SCH INTACT: 5. LAR TWN7 3. CATHOLIC

 016 T/S 61 (61 ) 0000000000000000-1-10000 TOL7/SCH INTACT: 6. SML TWN7 1. PUBLIC

 017 T/S 62 (62 ) 000000000000000001000000 TOL7/SCH INTACT: 6. SML TWN7 2. PRIVATE

 018 T/S 63 (63 ) 000000000000000000010000 TOL7/SCH INTACT: 6. SML TWN7 3. CATHOLIC

 019 T/S 71 (71 ) 00000000000000000000-1-1 TOL7/SCH INTACT: 7. OTHER 1. PUBLIC

 020 T/S 72 (72 ) 000000000000000000000100 TOL7/SCH INTACT: 7. OTHER 2. PRIVATE

 021 T/S 73 (73 ) 000000000000000000000001 TOL7/SCH INTACT: 7. OTHER 3. CATHOLIC

 CONDITIONING VARIABLE ID: BACK0035

 DESCRIPTION: INTERACTION: PARENTS’ EDUCATION ALL GRADES BY SCHOOL TYPE

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: PARE/SCH

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 15

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 8

 001 P/S 11 (11 ) 0101010101010101 PARE/SCH INTACT: 1. < HS 1. PUBLIC

 002 P/S 12 (12 ) -100-100-100-100 PARE/SCH INTACT: 1. < HS 2. PRIVATE

 003 P/S 13 (13 ) 00-100-100-100-1 PARE/SCH INTACT: 1. < HS 3. CATHOLIC

 004 P/S 21 (21 ) -1-1000000000000 PARE/SCH INTACT: 2. HS GRAD 1. PUBLIC

 005 P/S 22 (22 ) 0100000000000000 PARE/SCH INTACT: 2. HS GRAD 2. PRIVATE

 006 P/S 23 (23 ) 0001000000000000 PARE/SCH INTACT: 2. HS GRAD 3. CATHOLIC

 007 P/S 31 (31 ) 0000-1-100000000 PARE/SCH INTACT: 3. POST HS 1. PUBLIC

 008 P/S 32 (32 ) 0000010000000000 PARE/SCH INTACT: 3. POST HS 2. PRIVATE

 009 P/S 33 (33 ) 0000000100000000 PARE/SCH INTACT: 3. POST HS 3. CATHOLIC

 010 P/S 41 (41 ) 00000000-1-10000 PARE/SCH INTACT: 4. COL GRAD 1. PUBLIC

 011 P/S 42 (42 ) 0000000001000000 PARE/SCH INTACT: 4. COL GRAD 2. PRIVATE

 012 P/S 43 (43 ) 0000000000010000 PARE/SCH INTACT: 4. COL GRAD 3. CATHOLIC

 013 P/S 51 (51 ) 000000000000-1-1 PARE/SCH INTACT: 5. PARED-? 1. PUBLIC

 014 P/S 52 (52 ) 0000000000000100 PARE/SCH INTACT: 5. PARED-? 2. PRIVATE

 015 P/S 53 (53 ) 0000000000000001 PARE/SCH INTACT: 5. PARED-? 3. CATHOLIC

 CONDITIONING VARIABLE ID: BACK0036

 DESCRIPTION: INTERACTION: ACCOMMODATED BY GENDER

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: ACCO/GEN

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 1

 001 A/G 11 (11 ) 01 ACCO/GEN INTACT: 1. ACCOM 1. MALE

 002 A/G 12 (12 ) -1 ACCO/GEN INTACT: 1. ACCOM 2. FEMALE

 003 A/G 21 (21 ) -1 ACCO/GEN INTACT: 2. NO ACCOM 1. MALE

 004 A/G 22 (22 ) 01 ACCO/GEN INTACT: 2. NO ACCOM 2. FEMALE

 CONDITIONING VARIABLE ID: BACK0037

 DESCRIPTION: INTERACTION: ACCOMMODATED BY RACE/ETHNICITY

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: ACCO/RAC

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 3

 001 A/R 11 (11 ) 010101 ACCO/RAC INTACT: 1. ACCOM 1. WHI/AI/O

 002 A/R 12 (12 ) -10000 ACCO/RAC INTACT: 1. ACCOM 2. BLACK

 003 A/R 13 (13 ) 00-100 ACCO/RAC INTACT: 1. ACCOM 3. HISPANIC

 004 A/R 14 (14 ) 0000-1 ACCO/RAC INTACT: 1. ACCOM 4. ASIAN

 005 A/R 21 (21 ) -1-1-1 ACCO/RAC INTACT: 2. NO ACCOM 1. WHI/AI/O

 006 A/R 22 (22 ) 010000 ACCO/RAC INTACT: 2. NO ACCOM 2. BLACK

 007 A/R 23 (23 ) 000100 ACCO/RAC INTACT: 2. NO ACCOM 3. HISPANIC

 008 A/R 24 (24 ) 000001 ACCO/RAC INTACT: 2. NO ACCOM 4. ASIAN

 CONDITIONING VARIABLE ID: BACK0038

 DESCRIPTION: INTERACTION: ACCOMMODATED BY TYPE OF LOCALE (7 CATEGORIES)

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: ACCO/TOL

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 14

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 6

 001 A/T 11 (11 ) 010101010101 ACCO/TOL INTACT: 1. ACCOM 1. BIG CTY7

 002 A/T 12 (12 ) -10000000000 ACCO/TOL INTACT: 1. ACCOM 2. MID CTY7

 003 A/T 13 (13 ) 00-100000000 ACCO/TOL INTACT: 1. ACCOM 3. FR/LCTY7

 004 A/T 14 (14 ) 0000-1000000 ACCO/TOL INTACT: 1. ACCOM 4. FR/MCTY7

 005 A/T 15 (15 ) 000000-10000 ACCO/TOL INTACT: 1. ACCOM 5. LAR TWN7

 006 A/T 16 (16 ) 00000000-100 ACCO/TOL INTACT: 1. ACCOM 6. SML TWN7

 007 A/T 17 (17 ) 0000000000-1 ACCO/TOL INTACT: 1. ACCOM 7. OTHER

 008 A/T 21 (21 ) -1-1-1-1-1-1 ACCO/TOL INTACT: 2. NO ACCOM 1. BIG CTY7

 009 A/T 22 (22 ) 010000000000 ACCO/TOL INTACT: 2. NO ACCOM 2. MID CTY7

 010 A/T 23 (23 ) 000100000000 ACCO/TOL INTACT: 2. NO ACCOM 3. FR/LCTY7

 011 A/T 24 (24 ) 000001000000 ACCO/TOL INTACT: 2. NO ACCOM 4. FR/MCTY7

 012 A/T 25 (25 ) 000000010000 ACCO/TOL INTACT: 2. NO ACCOM 5. LAR TWN7

 013 A/T 26 (26 ) 000000000100 ACCO/TOL INTACT: 2. NO ACCOM 6. SML TWN7

 014 A/T 27 (27 ) 000000000001 ACCO/TOL INTACT: 2. NO ACCOM 7. OTHER
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

737

 CONDITIONING VARIABLE ID: BACK0039

 DESCRIPTION: INTERACTION: ACCOMMODATED BY PARENTS’ EDUCATION ALL GRADES

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: ACCO/PAR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 10

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 4

 001 A/P 11 (11 ) 01010101 ACCO/PAR INTACT: 1. ACCOM 1. < HS

 002 A/P 12 (12 ) -1000000 ACCO/PAR INTACT: 1. ACCOM 2. HS GRAD

 003 A/P 13 (13 ) 00-10000 ACCO/PAR INTACT: 1. ACCOM 3. POST HS

 004 A/P 14 (14 ) 0000-100 ACCO/PAR INTACT: 1. ACCOM 4. COL GRAD

 005 A/P 15 (15 ) 000000-1 ACCO/PAR INTACT: 1. ACCOM 5. PARED-?

 006 A/P 21 (21 ) -1-1-1-1 ACCO/PAR INTACT: 2. NO ACCOM 1. < HS

 007 A/P 22 (22 ) 01000000 ACCO/PAR INTACT: 2. NO ACCOM 2. HS GRAD

 008 A/P 23 (23 ) 00010000 ACCO/PAR INTACT: 2. NO ACCOM 3. POST HS

 009 A/P 24 (24 ) 00000100 ACCO/PAR INTACT: 2. NO ACCOM 4. COL GRAD

 010 A/P 25 (25 ) 00000001 ACCO/PAR INTACT: 2. NO ACCOM 5. PARED-?

 CONDITIONING VARIABLE ID: BACK0040

 DESCRIPTION: INTERACTION: ACCOMMODATED BY SCHOOL TYPE

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: ACCO/SCH

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 2

 001 A/S 11 (11 ) 0101 ACCO/SCH INTACT: 1. ACCOM 1. PUBLIC

 002 A/S 12 (12 ) -100 ACCO/SCH INTACT: 1. ACCOM 2. PRIVATE

 003 A/S 13 (13 ) 00-1 ACCO/SCH INTACT: 1. ACCOM 3. CATHOLIC

 004 A/S 21 (21 ) -1-1 ACCO/SCH INTACT: 2. NO ACCOM 1. PUBLIC

 005 A/S 22 (22 ) 0100 ACCO/SCH INTACT: 2. NO ACCOM 2. PRIVATE

 006 A/S 23 (23 ) 0001 ACCO/SCH INTACT: 2. NO ACCOM 3. CATHOLIC

 CONDITIONING VARIABLE ID: BACK0041

 DESCRIPTION: INTERACTION: ACCOMMODATED BY IEP

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: ACCO/IEP

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 1

 001 A/I 11 (11 ) 01 ACCO/IEP INTACT: 1. ACCOM 1. IEP-YES

 002 A/I 12 (12 ) -1 ACCO/IEP INTACT: 1. ACCOM 2. IEP-NO

 003 A/I 21 (21 ) -1 ACCO/IEP INTACT: 2. NO ACCOM 1. IEP-YES

 004 A/I 22 (22 ) 01 ACCO/IEP INTACT: 2. NO ACCOM 2. IEP-NO

 CONDITIONING VARIABLE ID: BACK0042

 DESCRIPTION: INTERACTION: ACCOMMODATED BY LEP

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: ACCO/LEP

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 1

 001 A/L 11 (11 ) 01 ACCO/LEP INTACT: 1. ACCOM 1. LEP-YES

 002 A/L 12 (12 ) -1 ACCO/LEP INTACT: 1. ACCOM 2. LEP-NO

 003 A/L 21 (21 ) -1 ACCO/LEP INTACT: 2. NO ACCOM 1. LEP-YES

 004 A/L 22 (22 ) 01 ACCO/LEP INTACT: 2. NO ACCOM 2. LEP-NO

 CONDITIONING VARIABLE ID: BACK0043

 DESCRIPTION: INTERACTION: GENDER BY YEARS TAKING CIVICS COURSES

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: GEND/NYR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 10

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 4

 001 G/N 11 (11 ) 01010101 GEND/NYR INTACT: 1. MALE 1. NYRCIV A

 002 G/N 12 (12 ) -1000000 GEND/NYR INTACT: 1. MALE 2. NYRCIV B

 003 G/N 13 (13 ) 00-10000 GEND/NYR INTACT: 1. MALE 3. NYRCIV C

 004 G/N 14 (14 ) 0000-100 GEND/NYR INTACT: 1. MALE 4. NYRCIV D

 005 G/N 15 (15 ) 000000-1 GEND/NYR INTACT: 1. MALE 5. NYRCIV E

 006 G/N 21 (21 ) -1-1-1-1 GEND/NYR INTACT: 2. FEMALE 1. NYRCIV A

 007 G/N 22 (22 ) 01000000 GEND/NYR INTACT: 2. FEMALE 2. NYRCIV B

 008 G/N 23 (23 ) 00010000 GEND/NYR INTACT: 2. FEMALE 3. NYRCIV C

 009 G/N 24 (24 ) 00000100 GEND/NYR INTACT: 2. FEMALE 4. NYRCIV D

 010 G/N 25 (25 ) 00000001 GEND/NYR INTACT: 2. FEMALE 5. NYRCIV E

 CONDITIONING VARIABLE ID: BACK0044

 DESCRIPTION: INTERACTION: RACE/ETHNICITY BY YEARS TAKING CIVICS COURSES

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: RACE/NYR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 20

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 12

 001 R/N 11 (11 ) 010101010101010101010101 RACE/NYR INTACT: 1. WHI/AI/O 1. NYRCIV A

 002 R/N 12 (12 ) -1000000-1000000-1000000 RACE/NYR INTACT: 1. WHI/AI/O 2. NYRCIV B

 003 R/N 13 (13 ) 00-1000000-1000000-10000 RACE/NYR INTACT: 1. WHI/AI/O 3. NYRCIV C

 004 R/N 14 (14 ) 0000-1000000-1000000-100 RACE/NYR INTACT: 1. WHI/AI/O 4. NYRCIV D

 005 R/N 15 (15 ) 000000-1000000-1000000-1 RACE/NYR INTACT: 1. WHI/AI/O 5. NYRCIV E

 006 R/N 21 (21 ) -1-1-1-10000000000000000 RACE/NYR INTACT: 2. BLACK 1. NYRCIV A

 007 R/N 22 (22 ) 010000000000000000000000 RACE/NYR INTACT: 2. BLACK 2. NYRCIV B

 008 R/N 23 (23 ) 000100000000000000000000 RACE/NYR INTACT: 2. BLACK 3. NYRCIV C

 009 R/N 24 (24 ) 000001000000000000000000 RACE/NYR INTACT: 2. BLACK 4. NYRCIV D

 010 R/N 25 (25 ) 000000010000000000000000 RACE/NYR INTACT: 2. BLACK 5. NYRCIV E

 011 R/N 31 (31 ) 00000000-1-1-1-100000000 RACE/NYR INTACT: 3. HISPANIC 1. NYRCIV A

 012 R/N 32 (32 ) 000000000100000000000000 RACE/NYR INTACT: 3. HISPANIC 2. NYRCIV B

 013 R/N 33 (33 ) 000000000001000000000000 RACE/NYR INTACT: 3. HISPANIC 3. NYRCIV C

 014 R/N 34 (34 ) 000000000000010000000000 RACE/NYR INTACT: 3. HISPANIC 4. NYRCIV D

 015 R/N 35 (35 ) 000000000000000100000000 RACE/NYR INTACT: 3. HISPANIC 5. NYRCIV E

 016 R/N 41 (41 ) 0000000000000000-1-1-1-1 RACE/NYR INTACT: 4. ASIAN 1. NYRCIV A

 017 R/N 42 (42 ) 000000000000000001000000 RACE/NYR INTACT: 4. ASIAN 2. NYRCIV B

 018 R/N 43 (43 ) 000000000000000000010000 RACE/NYR INTACT: 4. ASIAN 3. NYRCIV C

 019 R/N 44 (44 ) 000000000000000000000100 RACE/NYR INTACT: 4. ASIAN 4. NYRCIV D

 020 R/N 45 (45 ) 000000000000000000000001 RACE/NYR INTACT: 4. ASIAN 5. NYRCIV E
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

738

 CONDITIONING VARIABLE ID: BACK0045

 DESCRIPTION: INTERACTION: YEARS TAKING CIVICS COURSES BY TYPE OF LOCALE (7 CATEGORIES)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: NYRC/TOL

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 35

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 24

 001 N/T 11 (11 ) 010101010101010101010101010101010101010101010101 NYRC/TOL INTACT: 1. NYRCIV A 1. BIG CTY7

 002 N/T 12 (12 ) -10000000000-10000000000-10000000000-10000000000 NYRC/TOL INTACT: 1. NYRCIV A 2. MID CTY7

 003 N/T 13 (13 ) 00-10000000000-10000000000-10000000000-100000000 NYRC/TOL INTACT: 1. NYRCIV A 3. FR/LCTY7

 004 N/T 14 (14 ) 0000-10000000000-10000000000-10000000000-1000000 NYRC/TOL INTACT: 1. NYRCIV A 4. FR/MCTY7

 005 N/T 15 (15 ) 000000-10000000000-10000000000-10000000000-10000 NYRC/TOL INTACT: 1. NYRCIV A 5. LAR TWN7

 006 N/T 16 (16 ) 00000000-10000000000-10000000000-10000000000-100 NYRC/TOL INTACT: 1. NYRCIV A 6. SML TWN7

 007 N/T 17 (17 ) 0000000000-10000000000-10000000000-10000000000-1 NYRC/TOL INTACT: 1. NYRCIV A 7. OTHER

 008 N/T 21 (21 ) -1-1-1-1-1-1000000000000000000000000000000000000 NYRC/TOL INTACT: 2. NYRCIV B 1. BIG CTY7

 009 N/T 22 (22 ) 010000000000000000000000000000000000000000000000 NYRC/TOL INTACT: 2. NYRCIV B 2. MID CTY7

 010 N/T 23 (23 ) 000100000000000000000000000000000000000000000000 NYRC/TOL INTACT: 2. NYRCIV B 3. FR/LCTY7

 011 N/T 24 (24 ) 000001000000000000000000000000000000000000000000 NYRC/TOL INTACT: 2. NYRCIV B 4. FR/MCTY7

 012 N/T 25 (25 ) 000000010000000000000000000000000000000000000000 NYRC/TOL INTACT: 2. NYRCIV B 5. LAR TWN7

 013 N/T 26 (26 ) 000000000100000000000000000000000000000000000000 NYRC/TOL INTACT: 2. NYRCIV B 6. SML TWN7

 014 N/T 27 (27 ) 000000000001000000000000000000000000000000000000 NYRC/TOL INTACT: 2. NYRCIV B 7. OTHER

 015 N/T 31 (31 ) 000000000000-1-1-1-1-1-1000000000000000000000000 NYRC/TOL INTACT: 3. NYRCIV C 1. BIG CTY7

 016 N/T 32 (32 ) 000000000000010000000000000000000000000000000000 NYRC/TOL INTACT: 3. NYRCIV C 2. MID CTY7

 017 N/T 33 (33 ) 000000000000000100000000000000000000000000000000 NYRC/TOL INTACT: 3. NYRCIV C 3. FR/LCTY7

 018 N/T 34 (34 ) 000000000000000001000000000000000000000000000000 NYRC/TOL INTACT: 3. NYRCIV C 4. FR/MCTY7

 019 N/T 35 (35 ) 000000000000000000010000000000000000000000000000 NYRC/TOL INTACT: 3. NYRCIV C 5. LAR TWN7

 020 N/T 36 (36 ) 000000000000000000000100000000000000000000000000 NYRC/TOL INTACT: 3. NYRCIV C 6. SML TWN7

 021 N/T 37 (37 ) 000000000000000000000001000000000000000000000000 NYRC/TOL INTACT: 3. NYRCIV C 7. OTHER

 022 N/T 41 (41 ) 000000000000000000000000-1-1-1-1-1-1000000000000 NYRC/TOL INTACT: 4. NYRCIV D 1. BIG CTY7

 023 N/T 42 (42 ) 000000000000000000000000010000000000000000000000 NYRC/TOL INTACT: 4. NYRCIV D 2. MID CTY7

 024 N/T 43 (43 ) 000000000000000000000000000100000000000000000000 NYRC/TOL INTACT: 4. NYRCIV D 3. FR/LCTY7

 025 N/T 44 (44 ) 000000000000000000000000000001000000000000000000 NYRC/TOL INTACT: 4. NYRCIV D 4. FR/MCTY7

 026 N/T 45 (45 ) 000000000000000000000000000000010000000000000000 NYRC/TOL INTACT: 4. NYRCIV D 5. LAR TWN7

 027 N/T 46 (46 ) 000000000000000000000000000000000100000000000000 NYRC/TOL INTACT: 4. NYRCIV D 6. SML TWN7

 028 N/T 47 (47 ) 000000000000000000000000000000000001000000000000 NYRC/TOL INTACT: 4. NYRCIV D 7. OTHER

 029 N/T 51 (51 ) 000000000000000000000000000000000000-1-1-1-1-1-1 NYRC/TOL INTACT: 5. NYRCIV E 1. BIG CTY7

 030 N/T 52 (52 ) 000000000000000000000000000000000000010000000000 NYRC/TOL INTACT: 5. NYRCIV E 2. MID CTY7

 031 N/T 53 (53 ) 000000000000000000000000000100000000000100000000 NYRC/TOL INTACT: 5. NYRCIV E 3. FR/LCTY7

 032 N/T 54 (54 ) 000000000000000000000000000000000000000001000000 NYRC/TOL INTACT: 5. NYRCIV E 4. FR/MCTY7

 033 N/T 55 (55 ) 000000000000000000000000000000000000000000010000 NYRC/TOL INTACT: 5. NYRCIV E 5. LAR TWN7

 034 N/T 56 (56 ) 000000000000000000000000000000000000000000000100 NYRC/TOL INTACT: 5. NYRCIV E 6. SML TWN7

 035 N/T 57 (57 ) 000000000000000000000000000000000000000000000001 NYRC/TOL INTACT: 5. NYRCIV E 7. OTHER

 CONDITIONING VARIABLE ID: BACK0046

 DESCRIPTION: INTERACTION: PARENT’S EDUCATION BY YEARS TAKING CIVICS COURSES

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: PARE/NYR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 25

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 16

 001 P/N 11 (11 ) 01010101010101010101010101010101 PARE/NYR INTACT: 1. < HS 1. NYRCIV A

 002 P/N 12 (12 ) -1000000-1000000-1000000-1000000 PARE/NYR INTACT: 1. < HS 2. NYRCIV B

 003 P/N 13 (13 ) 00-1000000-1000000-1000000-10000 PARE/NYR INTACT: 1. < HS 3. NYRCIV C

 004 P/N 14 (14 ) 0000-1000000-1000000-1000000-100 PARE/NYR INTACT: 1. < HS 4. NYRCIV D

 005 P/N 15 (15 ) 000000-1000000-1000000-1000000-1 PARE/NYR INTACT: 1. < HS 5. NYRCIV E

 006 P/N 21 (21 ) -1-1-1-1000000000000000000000000 PARE/NYR INTACT: 2. HS GRAD 1. NYRCIV A

 007 P/N 22 (22 ) 01000000000000000000000000000000 PARE/NYR INTACT: 2. HS GRAD 2. NYRCIV B

 008 P/N 23 (23 ) 00010000000000000000000000000000 PARE/NYR INTACT: 2. HS GRAD 3. NYRCIV C

 009 P/N 24 (24 ) 00000100000000000000000000000000 PARE/NYR INTACT: 2. HS GRAD 4. NYRCIV D

 010 P/N 25 (25 ) 00000001000000000000000000000000 PARE/NYR INTACT: 2. HS GRAD 5. NYRCIV E

 011 P/N 31 (31 ) 00000000-1-1-1-10000000000000000 PARE/NYR INTACT: 3. POST HS 1. NYRCIV A

 012 P/N 32 (32 ) 00000000010000000000000000000000 PARE/NYR INTACT: 3. POST HS 2. NYRCIV B

 013 P/N 33 (33 ) 00000000000100000000000000000000 PARE/NYR INTACT: 3. POST HS 3. NYRCIV C

 014 P/N 34 (34 ) 00000000000001000000000000000000 PARE/NYR INTACT: 3. POST HS 4. NYRCIV D

 015 P/N 35 (35 ) 00000000000000010000000000000000 PARE/NYR INTACT: 3. POST HS 5. NYRCIV E

 016 P/N 41 (41 ) 0000000000000000-1-1-1-100000000 PARE/NYR INTACT: 4. COL GRAD 1. NYRCIV A

 017 P/N 42 (42 ) 00000000000000000100000000000000 PARE/NYR INTACT: 4. COL GRAD 2. NYRCIV B

 018 P/N 43 (43 ) 00000000000000000001000000000000 PARE/NYR INTACT: 4. COL GRAD 3. NYRCIV C

 019 P/N 44 (44 ) 00000000000000000000010000000000 PARE/NYR INTACT: 4. COL GRAD 4. NYRCIV D

 020 P/N 45 (45 ) 00000000000000000000000100000000 PARE/NYR INTACT: 4. COL GRAD 5. NYRCIV E

 021 P/N 51 (51 ) 000000000000000000000000-1-1-1-1 PARE/NYR INTACT: 5. PARED-? 1. NYRCIV A

 022 P/N 52 (52 ) 00000000000000000000000001000000 PARE/NYR INTACT: 5. PARED-? 2. NYRCIV B

 023 P/N 53 (53 ) 00000000000000000000000000010000 PARE/NYR INTACT: 5. PARED-? 3. NYRCIV C

 024 P/N 54 (54 ) 00000000000000000000000000000100 PARE/NYR INTACT: 5. PARED-? 4. NYRCIV D

 025 P/N 55 (55 ) 00000000000000000000000000000001 PARE/NYR INTACT: 5. PARED-? 5. NYRCIV E

 CONDITIONING VARIABLE ID: BACK0047

 DESCRIPTION: INTERACTION: YEARS TAKING CIVICS COURSES BY SCHOOL TYPE

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: NYRC/SCH

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 15

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 8

 001 N/S 11 (11 ) 0101010101010101 NYRC/SCH INTACT: 1. NYRCIV A 1. PUBLIC

 002 N/S 12 (12 ) -100-100-100-100 NYRC/SCH INTACT: 1. NYRCIV A 2. PRIVATE

 003 N/S 13 (13 ) 00-100-100-100-1 NYRC/SCH INTACT: 1. NYRCIV A 3. CATHOLIC

 004 N/S 21 (21 ) -1-1000000000000 NYRC/SCH INTACT: 2. NYRCIV B 1. PUBLIC

 005 N/S 22 (22 ) 0100000000000000 NYRC/SCH INTACT: 2. NYRCIV B 2. PRIVATE

 006 N/S 23 (23 ) 0001000000000000 NYRC/SCH INTACT: 2. NYRCIV B 3. CATHOLIC

 007 N/S 31 (31 ) 0000-1-100000000 NYRC/SCH INTACT: 3. NYRCIV C 1. PUBLIC

 008 N/S 32 (32 ) 0000010000000000 NYRC/SCH INTACT: 3. NYRCIV C 2. PRIVATE

 009 N/S 33 (33 ) 0000000100000000 NYRC/SCH INTACT: 3. NYRCIV C 3. CATHOLIC

 010 N/S 41 (41 ) 00000000-1-10000 NYRC/SCH INTACT: 4. NYRCIV D 1. PUBLIC

 011 N/S 42 (42 ) 0000000001000000 NYRC/SCH INTACT: 4. NYRCIV D 2. PRIVATE

 012 N/S 43 (43 ) 0000000000010000 NYRC/SCH INTACT: 4. NYRCIV D 3. CATHOLIC

 013 N/S 51 (51 ) 000000000000-1-1 NYRC/SCH INTACT: 5. NYRCIV E 1. PUBLIC

 014 N/S 52 (52 ) 0000000000000100 NYRC/SCH INTACT: 5. NYRCIV E 2. PRIVATE

 015 N/S 53 (53 ) 0000000000000001 NYRC/SCH INTACT: 5. NYRCIV E 3. CATHOLIC
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

739

 CONDITIONING VARIABLE ID: BACK0048

 DESCRIPTION: INTERACTION: ACCOMMODATED BY YEARS TAKING CIVICS COURSES

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: ACCO/NYR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 10

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 4

 001 A/N 11 (11 ) 01010101 ACCO/NYR INTACT: 1. ACCOM 1. NYRCIV A

 002 A/N 12 (12 ) -1000000 ACCO/NYR INTACT: 1. ACCOM 2. NYRCIV B

 003 A/N 13 (13 ) 00-10000 ACCO/NYR INTACT: 1. ACCOM 3. NYRCIV C

 004 A/N 14 (14 ) 0000-100 ACCO/NYR INTACT: 1. ACCOM 4. NYRCIV D

 005 A/N 15 (15 ) 000000-1 ACCO/NYR INTACT: 1. ACCOM 5. NYRCIV E

 006 A/N 21 (21 ) -1-1-1-1 ACCO/NYR INTACT: 2. NO ACCOM 1. NYRCIV A

 007 A/N 22 (22 ) 01000000 ACCO/NYR INTACT: 2. NO ACCOM 2. NYRCIV B

 008 A/N 23 (23 ) 00010000 ACCO/NYR INTACT: 2. NO ACCOM 3. NYRCIV C

 009 A/N 24 (24 ) 00000100 ACCO/NYR INTACT: 2. NO ACCOM 4. NYRCIV D

 010 A/N 25 (25 ) 00000001 ACCO/NYR INTACT: 2. NO ACCOM 5. NYRCIV E

 CONDITIONING VARIABLE ID: BACK0049

 DESCRIPTION: INTERACTION: GENDER BY CIVICS COURSES TAKING IN 11TH AND 12TH GRADES

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: GEND/NYR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 2

 001 G/N 11 (11 ) 0101 GEND/NYR INTACT: 1. MALE 1. NYRCIV2A

 002 G/N 12 (12 ) -100 GEND/NYR INTACT: 1. MALE 2. NYRCIV2B

 003 G/N 13 (13 ) 00-1 GEND/NYR INTACT: 1. MALE 3. NYRCIV2C

 004 G/N 21 (21 ) -1-1 GEND/NYR INTACT: 2. FEMALE 1. NYRCIV2A

 005 G/N 22 (22 ) 0100 GEND/NYR INTACT: 2. FEMALE 2. NYRCIV2B

 006 G/N 23 (23 ) 0001 GEND/NYR INTACT: 2. FEMALE 3. NYRCIV2C

 CONDITIONING VARIABLE ID: BACK0050

 DESCRIPTION: INTERACTION: RACE/ETHNICITY BY CIVICS COURSES TAKING IN 11TH AND 12TH GRADES

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: RACE/NYR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 12

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 6

 001 R/N 11 (11 ) 010101010101 RACE/NYR INTACT: 1. WHI/AI/O 1. NYRCIV2A

 002 R/N 12 (12 ) -100-100-100 RACE/NYR INTACT: 1. WHI/AI/O 2. NYRCIV2B

 003 R/N 13 (13 ) 00-100-100-1 RACE/NYR INTACT: 1. WHI/AI/O 3. NYRCIV2C

 004 R/N 21 (21 ) -1-100000000 RACE/NYR INTACT: 2. BLACK 1. NYRCIV2A

 005 R/N 22 (22 ) 010000000000 RACE/NYR INTACT: 2. BLACK 2. NYRCIV2B

 006 R/N 23 (23 ) 000100000000 RACE/NYR INTACT: 2. BLACK 3. NYRCIV2C

 007 R/N 31 (31 ) 0000-1-10000 RACE/NYR INTACT: 3. HISPANIC 1. NYRCIV2A

 008 R/N 32 (32 ) 000001000000 RACE/NYR INTACT: 3. HISPANIC 2. NYRCIV2B

 009 R/N 33 (33 ) 000000010000 RACE/NYR INTACT: 3. HISPANIC 3. NYRCIV2C

 010 R/N 41 (41 ) 00000000-1-1 RACE/NYR INTACT: 4. ASIAN 1. NYRCIV2A

 011 R/N 42 (42 ) 000000000100 RACE/NYR INTACT: 4. ASIAN 2. NYRCIV2B

 012 R/N 43 (43 ) 000000000001 RACE/NYR INTACT: 4. ASIAN 3. NYRCIV2C

 CONDITIONING VARIABLE ID: BACK0051

 DESCRIPTION: INTERACTION: TYPE OF LOCALE (7 CATEGORIES) BY CIVICS COURSES TAKING IN 11TH AND 12TH GRADES

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: TOL7/NYR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 21

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 12

 001 T/N 11 (11 ) 010101010101010101010101 TOL7/NYR INTACT: 1. BIG CTY7 1. NYRCIV2A

 002 T/N 12 (12 ) -100-100-100-100-100-100 TOL7/NYR INTACT: 1. BIG CTY7 2. NYRCIV2B

 003 T/N 13 (13 ) 00-100-100-100-100-100-1 TOL7/NYR INTACT: 1. BIG CTY7 3. NYRCIV2C

 004 T/N 21 (21 ) -1-100000000000000000000 TOL7/NYR INTACT: 2. MID CTY7 1. NYRCIV2A

 005 T/N 22 (22 ) 010000000000000000000000 TOL7/NYR INTACT: 2. MID CTY7 2. NYRCIV2B

 006 T/N 23 (23 ) 000100000000000000000000 TOL7/NYR INTACT: 2. MID CTY7 3. NYRCIV2C

 007 T/N 31 (31 ) 0000-1-10000000000000000 TOL7/NYR INTACT: 3. FR/LCTY7 1. NYRCIV2A

 008 T/N 32 (32 ) 000001000000000000000000 TOL7/NYR INTACT: 3. FR/LCTY7 2. NYRCIV2B

 009 T/N 33 (33 ) 000000010000000000000000 TOL7/NYR INTACT: 3. FR/LCTY7 3. NYRCIV2C

 010 T/N 41 (41 ) 00000000-1-1000000000000 TOL7/NYR INTACT: 4. FR/MCTY7 1. NYRCIV2A

 011 T/N 42 (42 ) 000000000100000000000000 TOL7/NYR INTACT: 4. FR/MCTY7 2. NYRCIV2B

 012 T/N 43 (43 ) 000000000001000000000000 TOL7/NYR INTACT: 4. FR/MCTY7 3. NYRCIV2C

 013 T/N 51 (51 ) 000000000000-1-100000000 TOL7/NYR INTACT: 5. LAR TWN7 1. NYRCIV2A

 014 T/N 52 (52 ) 000000000000010000000000 TOL7/NYR INTACT: 5. LAR TWN7 2. NYRCIV2B

 015 T/N 53 (53 ) 000000000000000100000000 TOL7/NYR INTACT: 5. LAR TWN7 3. NYRCIV2C

 016 T/N 61 (61 ) 0000000000000000-1-10000 TOL7/NYR INTACT: 6. SML TWN7 1. NYRCIV2A

 017 T/N 62 (62 ) 000000000000000001000000 TOL7/NYR INTACT: 6. SML TWN7 2. NYRCIV2B

 018 T/N 63 (63 ) 000000000000000000010000 TOL7/NYR INTACT: 6. SML TWN7 3. NYRCIV2C

 019 T/N 71 (71 ) 00000000000000000000-1-1 TOL7/NYR INTACT: 7. OTHER 1. NYRCIV2A

 020 T/N 72 (72 ) 000000000000000000000100 TOL7/NYR INTACT: 7. OTHER 2. NYRCIV2B

 021 T/N 73 (73 ) 000000000000000000000001 TOL7/NYR INTACT: 7. OTHER 3. NYRCIV2C

 CONDITIONING VARIABLE ID: BACK0052

 DESCRIPTION: INTERACTION: PARENTS’ EDUCATION ALL GRADES BY CIVICS COURSES TAKING IN 11TH AND 12TH GRADES

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: PARE/NYR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 15

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 8

 001 P/N 11 (11 ) 0101010101010101 PARE/NYR INTACT: 1. < HS 1. NYRCIV2A

 002 P/N 12 (12 ) -100-100-100-100 PARE/NYR INTACT: 1. < HS 2. NYRCIV2B

 003 P/N 13 (13 ) 00-100-100-100-1 PARE/NYR INTACT: 1. < HS 3. NYRCIV2C

 004 P/N 21 (21 ) -1-1000000000000 PARE/NYR INTACT: 2. HS GRAD 1. NYRCIV2A

 005 P/N 22 (22 ) 0100000000000000 PARE/NYR INTACT: 2. HS GRAD 2. NYRCIV2B

 006 P/N 23 (23 ) 0001000000000000 PARE/NYR INTACT: 2. HS GRAD 3. NYRCIV2C

 007 P/N 31 (31 ) 0000-1-100000000 PARE/NYR INTACT: 3. POST HS 1. NYRCIV2A

 008 P/N 32 (32 ) 0000010000000000 PARE/NYR INTACT: 3. POST HS 2. NYRCIV2B

 009 P/N 33 (33 ) 0000000100000000 PARE/NYR INTACT: 3. POST HS 3. NYRCIV2C

 010 P/N 41 (41 ) 00000000-1-10000 PARE/NYR INTACT: 4. COL GRAD 1. NYRCIV2A

 011 P/N 42 (42 ) 0000000001000000 PARE/NYR INTACT: 4. COL GRAD 2. NYRCIV2B

 012 P/N 43 (43 ) 0000000000010000 PARE/NYR INTACT: 4. COL GRAD 3. NYRCIV2C

 013 P/N 51 (51 ) 000000000000-1-1 PARE/NYR INTACT: 5. PARED-? 1. NYRCIV2A

 014 P/N 52 (52 ) 0000000000000100 PARE/NYR INTACT: 5. PARED-? 2. NYRCIV2B

 015 P/N 53 (53 ) 0000000000000001 PARE/NYR INTACT: 5. PARED-? 3. NYRCIV2C
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

740

 CONDITIONING VARIABLE ID: BACK0053

 DESCRIPTION: INTERACTION: SCHOOL TYPE BY CIVICS COURSES TAKING IN 11TH AND 12TH GRADES

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: SCHT/NYR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 4

 001 S/N 11 (11 ) 01010101 SCHT/NYR INTACT: 1. PUBLIC 1. NYRCIV2A

 002 S/N 12 (12 ) -100-100 SCHT/NYR INTACT: 1. PUBLIC 2. NYRCIV2B

 003 S/N 13 (13 ) 00-100-1 SCHT/NYR INTACT: 1. PUBLIC 3. NYRCIV2C

 004 S/N 21 (21 ) -1-10000 SCHT/NYR INTACT: 2. PRIVATE 1. NYRCIV2A

 005 S/N 22 (22 ) 01000000 SCHT/NYR INTACT: 2. PRIVATE 2. NYRCIV2B

 006 S/N 23 (23 ) 00010000 SCHT/NYR INTACT: 2. PRIVATE 3. NYRCIV2C

 007 S/N 31 (31 ) 0000-1-1 SCHT/NYR INTACT: 3. CATHOLIC 1. NYRCIV2A

 008 S/N 32 (32 ) 00000100 SCHT/NYR INTACT: 3. CATHOLIC 2. NYRCIV2B

 009 S/N 33 (33 ) 00000001 SCHT/NYR INTACT: 3. CATHOLIC 3. NYRCIV2C

 CONDITIONING VARIABLE ID: BACK0054

 DESCRIPTION: INTERACTION: ACCOMMODATED BY CIVICS COURSES TAKING IN 11TH AND 12TH GRADES

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: ACCO/NYR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 2

 001 A/N 11 (11 ) 0101 ACCO/NYR INTACT: 1. ACCOM 1. NYRCIV2A

 002 A/N 12 (12 ) -100 ACCO/NYR INTACT: 1. ACCOM 2. NYRCIV2B

 003 A/N 13 (13 ) 00-1 ACCO/NYR INTACT: 1. ACCOM 3. NYRCIV2C

 004 A/N 21 (21 ) -1-1 ACCO/NYR INTACT: 2. NO ACCOM 1. NYRCIV2A

 005 A/N 22 (22 ) 0100 ACCO/NYR INTACT: 2. NO ACCOM 2. NYRCIV2B

 006 A/N 23 (23 ) 0001 ACCO/NYR INTACT: 2. NO ACCOM 3. NYRCIV2C

 CONDITIONING VARIABLE ID: BACK0055

 DESCRIPTION: WHICH RACE/ETHNICITY BEST DESCRIBES YOU

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B003001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 WHITE (01 ) 000000 WHITE

 002 BLACK (02 ) 100000 BLACK

 003 HISPANIC (03 ) 010000 HISPANIC

 004 ASIAN AM (04 ) 001000 ASIAN/PACIFIC ISLAND

 005 AMER IND (05 ) 000100 AMER IND/ALASKA NATV

 006 OTHER (06 ) 000010 OTHER

 007 B003001M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: BACK0056

 DESCRIPTION: HOW LONG LIVED IN UNITED STATES

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B013001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 B013001A (01 ) 0000 ALL MY LIFE

 002 B013001B (02 ) 1000 MORE THAN 5 YEARS

 003 B013001C (03 ) 0100 3-5 YEARS

 004 B013001D (04 ) 0010 LESS THAN 3 YEARS

 005 B013001M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0057

 DESCRIPTION: HOW OFTEN OTHER THAN ENGLISH SPOKEN AT HOME

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B013101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 B013101A (01 ) 0000 ALL OR MOST OF TIME

 002 B013101B (02 ) 1000 ABOUT HALF OF TIME

 003 B013101C (03 ) 0100 LESS THAN HALF TIME

 004 B013101D (04 ) 0010 NEVER

 005 B013101M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0058

 DESCRIPTION: MOTHER GRADUATED HIGH SCHOOL

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B013201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B013201Y (01 ) 000 YES

 002 B013201N (02 ) 100 NO

 003 B013201M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0059

 DESCRIPTION: MOTHER HAD SOME EDUCATION AFTER HIGH SCHOOL

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B013301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B013301Y (01 ) 000 YES

 002 B013301N (02 ) 100 NO

 003 B013301M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0060

 DESCRIPTION: MOTHER GRADUATED COLLEGE

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B013401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 B013401Y (01 ) 000 YES

 002 B013401N (02 ) 100 NO

 003 B013401M (M, IDK ) 001 MISSING, I DON’T KNOW
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

741

 CONDITIONING VARIABLE ID: BACK0061

 DESCRIPTION: FATHER GRADUATED HIGH SCHOOL

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B013501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B013501Y (01 ) 000 YES

 002 B013501N (02 ) 100 NO

 003 B013501M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0062

 DESCRIPTION: FATHER HAD SOME EDUCATION AFTER HIGH SCHOOL

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B013601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B013601Y (01 ) 000 YES

 002 B013601N (02 ) 100 NO

 003 B013601M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0063

 DESCRIPTION: FATHER GRADUATED COLLEGE

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B013701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B013701Y (01 ) 000 YES

 002 B013701N (02 ) 100 NO

 0034 B013701M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0064

 DESCRIPTION: DOES YOUR FAMILY GET A NEWSPAPER REGULARLY

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B000901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B000901Y (01 ) 000 YES

 002 B000901N (02 ) 100 NO

 003 B000901M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0065

 DESCRIPTION: IS THERE AN ENCYCLOPEDIA IN YOUR HOME

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B000903 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B000903Y (01 ) 000 YES

 002 B000903N (02 ) 100 NO

 003 B000903M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0066

 DESCRIPTION: HOW MANY BOOKS ARE IN YOUR HOME

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B013801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 B013801A (01 ) 0000 0-10 (FEW)

 002 B013801B (02 ) 1000 11-25 (1 SHELF)

 003 B013801C (03 ) 0100 26-100 (1 BOOKCASE)

 004 B013801D (04 ) 0010 >100 (>1 BOOKCASE)

 005 B013801M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0067

 DESCRIPTION: DOES YOUR FAMILY GET MAGAZINES REGULARLY

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B000905 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 B000905Y (01 ) 000 YES

 002 B000905N (02 ) 100 NO

 003 B000905M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0068

 DESCRIPTION: TIME SPENT ON HOMEWORK EACH DAY

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B006601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 B006601N (01 ) 00000 DON’T USUALLY HAVE

 002 B006601B (02 ) 10000 HAVE BUT DON’T DO

 003 B006601C (03 ) 01000 1/2 HOUR OR LESS

 004 B006601D (04 ) 00100 1 HOUR

 005 B006601E (05 ) 00010 MORE THAN 1 HOUR

 006 B006601M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: BACK0069

 DESCRIPTION: DAYS ABSENT FROM SCHOOL LAST MONTH

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B014001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 B014001N (01 ) 00000 NONE

 002 B014001B (02 ) 10000 1 OR 2 DAYS

 003 B014001C (03 ) 01000 3 OR 4 DAYS

 004 B014001D (04 ) 00100 5 TO 9 DAYS

 005 B014001E (05 ) 00010 10 OR MORE DAYS

 006 B014001M (M ) 00001 MISSING
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

742

 CONDITIONING VARIABLE ID: BACK0070

 DESCRIPTION: TIMES CHANGED SCHOOLS IN PAST TWO YEARS

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B007301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 B007301N (01 ) 0000 NONE

 002 B007301B (02 ) 1000 1

 003 B007301C (03 ) 0100 2

 004 B007301D (04 ) 0010 3 OR MORE

 005 B007301M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0071

 DESCRIPTION: HOW OFTEN DISCUSS STUDIES AT HOME

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B007401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 B007401A (01 ) 0000 ALMOST EVERY DAY

 002 B007401B (02 ) 1000 ONCE/TWICE A WEEK

 003 B007401C (03 ) 0100 ONCE/TWICE A MONTH

 004 B007401D (04 ) 0010 NEVER OR HARDLY EVER

 005 B007401M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0072

 DESCRIPTION: HOW OFTEN USE COMPUTER AT HOME FOR SCHOOLWORK

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B014101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 B014101A (01 ) 00000 NO COMPUTER AT HOME

 002 B014101B (02 ) 10000 NEVER OR HARDLY EVER

 003 B014101C (03 ) 01000 ONCE/TWICE A MONTH

 004 B014101D (04 ) 00100 ONCE/TWICE A WEEK

 005 B014101E (05 ) 00010 ALMOST EVERY DAY

 006 B014101M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0001

 DESCRIPTION: HOW HARD TRIED ON THIS SS TEST THAN ON OTHERS

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: P804001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 P804001A (01 ) 0000 TRIED MUCH HARDER

 002 P804001B (02 ) 1000 TRIED HARDER

 003 P804001C (03 ) 0100 TRIED ABOUT AS HARD

 004 P804001N (04 ) 0010 TRIED NOT AS HARD

 005 P804001M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0002

 DESCRIPTION: HOW IMPORTANT TO DO WELL ON THIS SS TEST

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: P804101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 P804101A (01 ) 0000 VERY IMPORTANT

 002 P804101B (02 ) 1000 IMPORTANT

 003 P804101C (03 ) 0100 SOMEWHAT IMPORTANT

 004 P804101N (04 ) 0010 NOT VERY IMPORTANT

 005 P804101M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0003

 DESCRIPTION: HOW OFTEN WRITE LONG ANSWERS ON SS TESTS

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: P804201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 P804201A (01 ) 0000 AT LEAST ONCE A WEEK

 002 P804201B (02 ) 1000 ONCE/TWICE A MONTH

 003 P804201C (03 ) 0100 ONCE/TWICE A YEAR

 004 P804201D (04 ) 0010 NEVER

 005 P804201M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0004

 DESCRIPTION: MY FRIENDS MAKE FUN OF PEOPLE WHO TRY TO DO WELL

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: P804301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 P804301A (01 ) 0000 STRONGLY AGREE

 002 P804301B (02 ) 1000 AGREE

 003 P804301C (03 ) 0100 DISAGREE

 004 P804301D (04 ) 0010 STRONGLY DISAGREE

 005 P804301M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0005

 DESCRIPTION: I HAVE FRIENDS TO TALK TO IF NEED HELP W/SCHOOL

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: P804302 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 P804302A (01 ) 0000 STRONGLY AGREE

 002 P804302B (02 ) 1000 AGREE

 003 P804302C (03 ) 0100 DISAGREE

 004 P804302D (04 ) 0010 STRONGLY DISAGREE

 005 P804302M (M ) 0001 MISSING
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

743

 CONDITIONING VARIABLE ID: SUBJ0006

 DESCRIPTION: HOW OFTEN STUDY SOCIAL STUDIES IN SCHOOL

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: P803501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 P803501A (01 ) 0000 ALMOST EVERY DAY

 002 P803501B (02 ) 1000 ONCE/TWICE A WEEK

 003 P803501C (03 ) 0100 ONCE/TWICE A MONTH

 004 P803501D (04 ) 0010 NEVER OR HARDLY EVER

 005 P803501M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0007

 DESCRIPTION: THIS YEAR-STUDY HOW OUR GOVERNMENT WORKS

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: P803601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P803601Y (01 ) 000 YES

 002 P803601N (02 ) 100 NO

 003 P803601M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0008

 DESCRIPTION: THIS YEAR-STUDY RULES/LAWS OF GOVERNMENT

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: P803602 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P803602Y (01 ) 000 YES

 002 P803602N (02 ) 100 NO

 003 P803602M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0009

 DESCRIPTION: THIS YEAR-STUDY ELECTIONS AND VOTING

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: P803603 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P803603Y (01 ) 000 YES

 002 P803603N (02 ) 100 NO

 003 P803603M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0010

 DESCRIPTION: THIS YEAR-STUDY THE PRESIDENT/LEADERS OF COUNTRY

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: P803604 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P803604Y (01 ) 000 YES

 002 P803604N (02 ) 100 NO

 003 P803604M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0011

 DESCRIPTION: THIS YEAR-STUDY YOUR COMMUNITY

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: P803605 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P803605Y (01 ) 000 YES

 002 P803605N (02 ) 100 NO

 003 P803605M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0012

 DESCRIPTION: THIS YEAR-STUDY RIGHTS/RESPONSIBILITIES-CITIZENS

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: P803606 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P803606Y (01 ) 000 YES

 002 P803606N (02 ) 100 NO

 003 P803606M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0013

 DESCRIPTION: THIS YEAR-STUDY HOW PEOPLE SOLVE DISAGREEMENTS

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: P803607 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P803607Y (01 ) 000 YES

 002 P803607N (02 ) 100 NO

 003 P803607M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0014

 DESCRIPTION: IN SOCIAL STUDIES-READ FROM TEXTBOOK

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: P803701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P803701Y (01 ) 000 YES

 002 P803701N (02 ) 100 NO

 003 P803701M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0015

 DESCRIPTION: IN SOCIAL STUDIES-MEMORIZE READING MATERIAL

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: P803702 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P803702Y (01 ) 000 YES

 002 P803702N (02 ) 100 NO

 003 P803702M (M, IDK ) 001 MISSING, I DON’T KNOW
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

744

 CONDITIONING VARIABLE ID: SUBJ0016

 DESCRIPTION: IN SOCIAL STUDIES-READ EXTRA MATERIAL

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: P803703 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P803703Y (01 ) 000 YES

 002 P803703N (02 ) 100 NO

 003 P803703M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0017

 DESCRIPTION: IN SOCIAL STUDIES-FILL OUT WORKSHEETS

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: P803704 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P803704Y (01 ) 000 YES

 002 P803704N (02 ) 100 NO

 003 P803704M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0018

 DESCRIPTION: IN SOCIAL STUDIES-WRITE REPORTS

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: P803705 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P803705Y (01 ) 000 YES

 002 P803705N (02 ) 100 NO

 003 P803705M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0019

 DESCRIPTION: IN SOCIAL STUDIES-DISCUSS CURRENT EVENTS

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: P803706 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P803706Y (01 ) 000 YES

 002 P803706N (02 ) 100 NO

 003 P803706M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0020

 DESCRIPTION: IN SOCIAL STUDIES-WATCH TV, VIDEOS, FILMSTRIPS

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: P803707 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P803707Y (01 ) 000 YES

 002 P803707N (02 ) 100 NO

 003 P803707M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0021

 DESCRIPTION: IN SOCIAL STUDIES-DISCUSS TV, VIDEOS, FILMSTRIP

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: P803708 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P803708Y (01 ) 000 YES

 002 P803708N (02 ) 100 NO

 003 P803708M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0022

 DESCRIPTION: IN SOCIAL STUDIES-TAKE PART IN DEBATES/PANEL DISC

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: P803709 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P803709Y (01 ) 000 YES

 002 P803709N (02 ) 100 NO

 003 P803709M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0023

 DESCRIPTION: IN SOCIAL STUDIES-ROLE PLAYING, MOCK TRIALS

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: P803710 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P803710Y (01 ) 000 YES

 002 P803710N (02 ) 100 NO

 003 P803710M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0024

 DESCRIPTION: IN SOCIAL STUDIES-WRITE LETTER FOR COMMUNITY

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: P803711 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P803711Y (01 ) 000 YES

 002 P803711N (02 ) 100 NO

 003 P803711M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0025

 DESCRIPTION: IN SOCIAL STUDIES-HAVE VISITORS FROM COMMUNITY

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: P803712 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P803712Y (01 ) 000 YES

 002 P803712N (02 ) 100 NO

 003 P803712M (M, IDK ) 001 MISSING, I DON’T KNOW
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

745

 CONDITIONING VARIABLE ID: SUBJ0026

 DESCRIPTION: HOW OFTEN DO YOU HAVE SOCIAL STUDIES HOMEWORK

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: P803801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 P803801A (01 ) 0000 ALMOST EVERY DAY

 002 P803801B (02 ) 1000 ONCE/TWICE A WEEK

 003 P803801C (03 ) 0100 ONCE/TWICE A MONTH

 004 P803801D (04 ) 0010 NEVER OR HARDLY EVER

 005 P803801M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0027

 DESCRIPTION: DO YOU HAVE A CLASSROOM GOVERNMENT

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: P803901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P803901Y (01 ) 000 YES

 002 P803901N (02 ) 100 NO

 003 P803901M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SCHL0001

 DESCRIPTION: FOURTH GRADERS ASSIGNED TO CLASS BY ABILITY

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C042501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C042501Y (01 ) 00 YES

 002 C042501N (02 ) 10 NO

 003 C042501M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0002

 DESCRIPTION: HOW OFTEN STUDENTS RECEIVE READING INSTRUCTION

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C042601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C042601A (01 ) 00000 EVERY DAY

 002 C042601B (02 ) 10000 3-4 TIMES A WEEK

 003 C042601C (03 ) 01000 ONCE OR TWICE A WEEK

 004 C042601D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C042601N (05 ) 00010 SUBJECT NOT TAUGHT

 006 C042601M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0003

 DESCRIPTION: HOW OFTEN STUDENTS RECEIVE WRITING INSTRUCTION

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C042602 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C042602A (01 ) 00000 EVERY DAY

 002 C042602B (02 ) 10000 3-4 TIMES A WEEK

 003 C042602C (03 ) 01000 ONCE OR TWICE A WEEK

 004 C042602D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C042602N (05 ) 00010 SUBJECT NOT TAUGHT

 006 C042602M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0004

 DESCRIPTION: HOW OFTEN STUDENTS RECEIVE SOC STUDIES INSTRUCT

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C042603 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C042603A (01 ) 00000 EVERY DAY

 002 C042603B (02 ) 10000 3-4 TIMES A WEEK

 003 C042603C (03 ) 01000 ONCE OR TWICE A WEEK

 004 C042603D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C042603N (05 ) 00010 SUBJECT NOT TAUGHT

 006 C042603M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0005

 DESCRIPTION: HOW OFTEN STUDENTS RECEIVE COMPUTER USE INSTRUCT

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C042604 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C042604A (01 ) 00000 EVERY DAY

 002 C042604B (02 ) 10000 3-4 TIMES A WEEK

 003 C042604C (03 ) 01000 ONCE OR TWICE A WEEK

 004 C042604D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C042604N (05 ) 00010 SUBJECT NOT TAUGHT

 006 C042604M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0006

 DESCRIPTION: DOES SCHOOL USE BLOCK SCHEDULING

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C042701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 C042701Y (01 ) 000 YES-ALL SUBJECTS

 002 C042701Y (02 ) 100 YES-SOME SUBJECTS

 003 C042701N (03 ) 010 NO

 004 C042701M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SCHL0007

 DESCRIPTION: ARE COMPUTERS AVAILABLE IN ALL CLASSROOMS

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C042801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C042801Y (01 ) 00 YES

 002 C042801N (02 ) 10 NO

 003 C042801M (M ) 01 MISSING
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

746

 CONDITIONING VARIABLE ID: SCHL0008

 DESCRIPTION: ARE COMPUTERS AVAILABLE IN COMPUTER LAB

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C042802 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C042802Y (01 ) 00 YES

 002 C042802N (02 ) 10 NO

 003 C042802M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0009

 DESCRIPTION: ARE COMPUTERS AVAILABLE TO CLASSROOM WHEN NEEDED

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C042803 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C042803Y (01 ) 00 YES

 002 C042803N (02 ) 10 NO

 003 C042803M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0010

 DESCRIPTION: HOW MANY COMPUTERS AVAILABLE TO STUDENTS

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C042901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 C042901N (01 ) 0000000 NONE

 002 C042901B (02 ) 1000000 1-10

 003 C042901C (03 ) 0100000 11-25

 004 C042901D (04 ) 0010000 26-50

 005 C042901E (05 ) 0001000 51-75

 006 C042901F (06 ) 0000100 76-100

 007 C042901G (07 ) 0000010 MORE THAN 100

 008 C042901M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0011

 DESCRIPTION: PRIMARY WAY LIBRARY IS STAFFED

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C036601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C036601N (01 ) 0000 NO LIBRARY IN SCHOOL

 002 C036601N (02 ) 1000 LIBRARY-NO/VOL STAFF

 003 C036601C (03 ) 0100 PART-TIME STAFF

 004 C036601D (04 ) 0010 FULL-TIME STAFF

 005 C036601M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0012

 DESCRIPTION: PARENTS PARTICIPATE-PARENT-TEACHER ORG

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043001A (01 ) 00000 NOT AVAILABLE

 002 C043001B (02 ) 10000 0-10%

 003 C043001C (03 ) 01000 11-25%

 004 C043001D (04 ) 00100 26-50%

 005 C043001E (05 ) 00010 51-100%

 006 C043001M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0013

 DESCRIPTION: PARENTS PARTICIPATE-OPEN HOUSE

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043002 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043002A (01 ) 00000 NOT AVAILABLE

 002 C043002B (02 ) 10000 0-10%

 003 C043002C (03 ) 01000 11-25%

 004 C043002D (04 ) 00100 26-50%

 005 C043002E (05 ) 00010 51-100%

 006 C043002M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0014

 DESCRIPTION: PARTICIPATE-PARENT-TEACHER CONFERENCE

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043003 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043003A (01 ) 00000 NOT AVAILABLE

 002 C043003B (02 ) 10000 0-10%

 003 C043003C (03 ) 01000 11-25%

 004 C043003D (04 ) 00100 26-50%

 005 C043003E (05 ) 00010 51-100%

 006 C043003M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0015

 DESCRIPTION: PARENTS PARTICIPATE-SCHOOL CURRICULUM DECISIONS

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043004 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043004A (01 ) 00000 NOT AVAILABLE

 002 C043004B (02 ) 10000 0-10%

 003 C043004C (03 ) 01000 11-25%

 004 C043004D (04 ) 00100 26-50%

 005 C043004E (05 ) 00010 51-100%

 006 C043004M (M ) 00001 MISSING
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

747

 CONDITIONING VARIABLE ID: SCHL0016

 DESCRIPTION: PARENTS PARTICIPATE-VOLUNTEER PROGRAMS

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043005 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043005A (01 ) 00000 NOT AVAILABLE

 002 C043005B (02 ) 10000 0-10%

 003 C043005C (03 ) 01000 11-25%

 004 C043005D (04 ) 00100 26-50%

 005 C043005E (05 ) 00010 51-100%

 006 C043005M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0017

 DESCRIPTION: PARENTS PARTICIPATE-PARENTING-SKILLS PROGRAM

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043006 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043006A (01 ) 00000 NOT AVAILABLE

 002 C043006B (02 ) 10000 0-10%

 003 C043006C (03 ) 01000 11-25%

 004 C043006D (04 ) 00100 26-50%

 005 C043006E (05 ) 00010 51-100%

 006 C043006M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0018

 DESCRIPTION: PARENTS PARTICIPATE-SCHOOL ADVISORY COMMITTEES

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043007 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043007A (01 ) 00000 NOT AVAILABLE

 002 C043007B (02 ) 10000 0-10%

 003 C043007C (03 ) 01000 11-25%

 004 C043007D (04 ) 00100 26-50%

 005 C043007E (05 ) 00010 51-100%

 006 C043007M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0019

 DESCRIPTION: PARENTS PARTICIPATE-CLASSROOM ASSISTANTS

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043008 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043008A (01 ) 00000 NOT AVAILABLE

 002 C043008B (02 ) 10000 0-10%

 003 C043008C (03 ) 01000 11-25%

 004 C043008D (04 ) 00100 26-50%

 005 C043008E (05 ) 00010 51-100%

 006 C043008M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0020

 DESCRIPTION: IS STUDENT ABSENTEEISM A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032402 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032402A (01 ) 0000 SERIOUS

 002 C032402B (02 ) 1000 MODERATE

 003 C032402C (03 ) 0100 MINOR

 004 C032402N (04 ) 0010 NOT A PROBLEM

 005 C032402M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0021

 DESCRIPTION: IS STUDENT TARDINESS A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032401A (01 ) 0000 SERIOUS

 002 C032401B (02 ) 1000 MODERATE

 003 C032401C (03 ) 0100 MINOR

 004 C032401N (04 ) 0010 NOT A PROBLEM

 005 C032401M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0022

 DESCRIPTION: ARE PHYSICAL CONFLICTS A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032404 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032404A (01 ) 0000 SERIOUS

 002 C032404B (02 ) 1000 MODERATE

 003 C032404C (03 ) 0100 MINOR

 004 C032404N (04 ) 0010 NOT A PROBLEM

 005 C032404M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0023

 DESCRIPTION: ARE RACIAL/CULT. CONFLICTS A PROBLEM IN SCHOOL

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032407 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032407A (01 ) 0000 SERIOUS

 002 C032407B (02 ) 1000 MODERATE

 003 C032407C (03 ) 0100 MINOR

 004 C032407N (04 ) 0010 NOT A PROBLEM

 005 C032407M (M ) 0001 MISSING
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

748

 CONDITIONING VARIABLE ID: SCHL0024

 DESCRIPTION: IS STUDENT HEALTH A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032408 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032408A (01 ) 0000 SERIOUS

 002 C032408B (02 ) 1000 MODERATE

 003 C032408C (03 ) 0100 MINOR

 004 C032408N (04 ) 0010 NOT A PROBLEM

 005 C032408M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0025

 DESCRIPTION: IS LACK OF PARENT INVLVMNT A PROBLEM IN SCHOOL

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032409 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032409A (01 ) 0000 SERIOUS

 002 C032409B (02 ) 1000 MODERATE

 003 C032409C (03 ) 0100 MINOR

 004 C032409N (04 ) 0010 NOT A PROBLEM

 005 C032409M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0026

 DESCRIPTION: IS STUDENT ALCOHOL USE A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032410 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032410A (01 ) 0000 SERIOUS

 002 C032410B (02 ) 1000 MODERATE

 003 C032410C (03 ) 0100 MINOR

 004 C032410N (04 ) 0010 NOT A PROBLEM

 005 C032410M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0027

 DESCRIPTION: IS STUDENT TOBACCO USE A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032411 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032411A (01 ) 0000 SERIOUS

 002 C032411B (02 ) 1000 MODERATE

 003 C032411C (03 ) 0100 MINOR

 004 C032411N (04 ) 0010 NOT A PROBLEM

 005 C032411M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0028

 DESCRIPTION: IS STUDENT DRUG USE A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032412 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032412A (01 ) 0000 SERIOUS

 002 C032412B (02 ) 1000 MODERATE

 003 C032412C (03 ) 0100 MINOR

 004 C032412N (04 ) 0010 NOT A PROBLEM

 005 C032412M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0029

 DESCRIPTION: ARE GANG ACTIVITIES A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032413 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032413A (01 ) 0000 SERIOUS

 002 C032413B (02 ) 1000 MODERATE

 003 C032413C (03 ) 0100 MINOR

 004 C032413N (04 ) 0010 NOT A PROBLEM

 005 C032413M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0030

 DESCRIPTION: IS STUDENT MISBEHAVIOR A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032414 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032414A (01 ) 0000 SERIOUS

 002 C032414B (02 ) 1000 MODERATE

 003 C032414C (03 ) 0100 MINOR

 004 C032414N (04 ) 0010 NOT A PROBLEM

 005 C032414M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0031

 DESCRIPTION: IS STUDENT CHEATING A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C043101A (01 ) 0000 SERIOUS

 002 C043101B (02 ) 1000 MODERATE

 003 C043101C (03 ) 0100 MINOR

 004 C043101N (04 ) 0010 NOT A PROBLEM

 005 C043101M (M ) 0001 MISSING
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

749

 CONDITIONING VARIABLE ID: SCHL0032

 DESCRIPTION: IS TEACHER ABSENTEEISM A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043102 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C043102A (01 ) 0000 SERIOUS

 002 C043102B (02 ) 1000 MODERATE

 003 C043102C (03 ) 0100 MINOR

 004 C043102N (04 ) 0010 NOT A PROBLEM

 005 C043102M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0033

 DESCRIPTION: ARE PHYSICAL CONFLICTS BETWEEN STUDENTS/TEACHERS

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043103 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C043103A (01 ) 0000 SERIOUS

 002 C043103B (02 ) 1000 MODERATE

 003 C043103C (03 ) 0100 MINOR

 004 C043103N (04 ) 0010 NOT A PROBLEM

 005 C043103M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0034

 DESCRIPTION: IS VANDALISM A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043104 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C043104A (01 ) 0000 SERIOUS

 002 C043104B (02 ) 1000 MODERATE

 003 C043104C (03 ) 0100 MINOR

 004 C043104N (04 ) 0010 NOT A PROBLEM

 005 C043104M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0035

 DESCRIPTION: TEACHER MORALE

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032502 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032502A (01 ) 0000 VERY POSITIVE

 002 C032502B (02 ) 1000 SOMEWHAT POSITIVE

 003 C032502C (03 ) 0100 SOMEWHAT NEGATIVE

 004 C032502D (04 ) 0010 VERY NEGATIVE

 005 C032502M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0036

 DESCRIPTION: STUDENT ATTITUDES TOWARD ACADEMIC ACHIEVEMENT

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032503 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032503A (01 ) 0000 VERY POSITIVE

 002 C032503B (02 ) 1000 SOMEWHAT POSITIVE

 003 C032503C (03 ) 0100 SOMEWHAT NEGATIVE

 004 C032503D (04 ) 0010 VERY NEGATIVE

 005 C032503M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0037

 DESCRIPTION: PARENT SUPPORT FOR STUDENT ACHIEVEMENT

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032505 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032505A (01 ) 0000 VERY POSITIVE

 002 C032505B (02 ) 1000 SOMEWHAT POSITIVE

 003 C032505C (03 ) 0100 SOMEWHAT NEGATIVE

 004 C032505D (04 ) 0010 VERY NEGATIVE

 005 C032505M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0038

 DESCRIPTION: REGARD FOR SCHOOL PROPERTY

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032506 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032506A (01 ) 0000 VERY POSITIVE

 002 C032506B (02 ) 1000 SOMEWHAT POSITIVE

 003 C032506C (03 ) 0100 SOMEWHAT NEGATIVE

 004 C032506D (04 ) 0010 VERY NEGATIVE

 005 C032506M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0039

 DESCRIPTION: TEACHERS’ EXPECTATIONS FOR STUDENT ACHIEVEMENT

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C043201A (01 ) 0000 VERY POSITIVE

 002 C043201B (02 ) 1000 SOMEWHAT POSITIVE

 003 C043201C (03 ) 0100 SOMEWHAT NEGATIVE

 004 C043201D (04 ) 0010 VERY NEGATIVE

 005 C043201M (M ) 0001 MISSING
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

750

 CONDITIONING VARIABLE ID: SCHL0040

 DESCRIPTION: PERCENT STUDENT BODY ABSENT AVERAGE DAY

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 C043301A (01 ) 000000 0-2%

 002 C043301B (02 ) 100000 3-5%

 003 C043301C (03 ) 010000 6-10%

 004 C043301D (04 ) 001000 11-25%

 005 C043301E (05 ) 000100 26-50%

 006 C043301F (06 ) 000010 MORE THAN 50%

 007 C043301M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0041

 DESCRIPTION: PERCENT TEACHING STAFF ABSENT AVERAGE DAY

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 C043401A (01 ) 000000 0-2%

 002 C043401B (02 ) 100000 3-5%

 003 C043401C (03 ) 010000 6-10%

 004 C043401D (04 ) 001000 11-25%

 005 C043401E (05 ) 000100 26-50%

 006 C043401F (06 ) 000010 MORE THAN 50%

 007 C043401M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0042

 DESCRIPTION: ENROLLMENT LAST YEAR COMPARED TO END OF SCHOOL YR

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 C043501A (01 ) 000000 98-100%

 002 C043501B (02 ) 100000 95-97%

 003 C043501C (03 ) 010000 90-94%

 004 C043501D (04 ) 001000 80-89%

 005 C043501E (05 ) 000100 70-79%

 006 C043501F (06 ) 000010 LESS THAN 70%

 007 C043501M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0043

 DESCRIPTION: PERCENT STUDENTS HELD BACK AND REPEATING GRADE

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043601A (01 ) 00000 0%

 002 C043601B (02 ) 10000 1-2%

 003 C043601C (03 ) 01000 3-5%

 004 C043601D (04 ) 00100 6-10%

 005 C043601E (05 ) 00010 MORE THAN 10%

 006 C043601M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0044

 DESCRIPTION: PERCENT TEACHING STAFF LEFT BEFORE END OF YEAR

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C043701A (01 ) 00000 0%

 002 C043701B (02 ) 10000 1-2%

 003 C043701C (03 ) 01000 3-5%

 004 C043701D (04 ) 00100 6-10%

 005 C043701E (05 ) 00010 MORE THAN 10%

 006 C043701M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0045

 DESCRIPTION: IS SCHOOL IN NATIONAL SCHOOL LUNCH PROGRAM

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C038301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C038301Y (01 ) 00 YES

 002 C038301N (02 ) 10 NO

 003 C038301M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0046

 DESCRIPTION: PERCENT ELIGIBLE NATIONAL SCHOOL LUNCH PROGRAM

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 8

 001 C043801A (01 ) 00000000 0%

 002 C043801B (02 ) 10000000 1-5%

 003 C043801C (03 ) 01000000 6-10%

 004 C043801D (04 ) 00100000 11-25%

 005 C043801E (05 ) 00010000 26-50%

 006 C043801F (06 ) 00001000 51-75%

 007 C043801G (07 ) 00000100 76-99%

 008 C043801H (08 ) 00000010 100%

 009 C043801M (M ) 00000001 MISSING
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

751

 CONDITIONING VARIABLE ID: SCHL0047

 DESCRIPTION: DOES SCHOOL RECEIVE CHAPTER 1/TITLE I FUNDING

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C043901Y (01 ) 00 YES

 002 C043901N (02 ) 10 NO

 003 C043901M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0048

 DESCRIPTION: PERCENT STUDENTS RECEIVE CHAPTER1/TITLE I FUNDING

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 8

 001 C044001N (01 ) 00000000 NONE

 002 C044001B (02 ) 10000000 1-5%

 003 C044001C (03 ) 01000000 6-10%

 004 C044001D (04 ) 00100000 11-25%

 005 C044001E (05 ) 00010000 26-50%

 006 C044001F (06 ) 00001000 51-75%

 007 C044001G (07 ) 00000100 76-90%

 008 C044001H (08 ) 00000010 OVER 90%

 009 C044001M (M ) 00000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0049

 DESCRIPTION: PERCENT STUDENTS RECEIVE REMEDIAL READING INSTRUCT

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044002 TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 8

 001 C044002N (01 ) 00000000 NONE

 002 C044002B (02 ) 10000000 1-5%

 003 C044002C (03 ) 01000000 6-10%

 004 C044002D (04 ) 00100000 11-25%

 005 C044002E (05 ) 00010000 26-50%

 006 C044002F (06 ) 00001000 51-75%

 007 C044002G (07 ) 00000100 76-90%

 008 C044002H (08 ) 00000010 OVER 90%

 009 C044002M (M ) 00000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0050

 DESCRIPTION: PERCENT STUDENTS RECEIVE REMEDIAL WRITING INSTRUCT

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044003 TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 8

 001 C044003N (01 ) 00000000 NONE

 002 C044003B (02 ) 10000000 1-5%

 003 C044003C (03 ) 01000000 6-10%

 004 C044003D (04 ) 00100000 11-25%

 005 C044003E (05 ) 00010000 26-50%

 006 C044003F (06 ) 00001000 51-75%

 007 C044003G (07 ) 00000100 76-90%

 008 C044003H (08 ) 00000010 OVER 90%

 009 C044003M (M ) 00000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0051

 DESCRIPTION: PERCENT STUDENTS IN GIFTED AND TALENTED PROGRAM

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044004 TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 8

 001 C044004N (01 ) 00000000 NONE

 002 C044004B (02 ) 10000000 1-5%

 003 C044004C (03 ) 01000000 6-10%

 004 C044004D (04 ) 00100000 11-25%

 005 C044004E (05 ) 00010000 26-50%

 006 C044004F (06 ) 00001000 51-75%

 007 C044004G (07 ) 00000100 76-90%

 008 C044004H (08 ) 00000010 OVER 90%

 009 C044004M (M ) 00000001 MISSING

 CONDITIONING VARIABLE ID: BACK0073

 DESCRIPTION: HOW MUCH EDUCATION DO YOU EXPECT TO RECEIVE

 GRADES/ASSESSMENTS: N08

 CONDITIONING VAR LABEL:

 NAEP ID: B014201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 B014201N (01 ) 000000 WILL NOT FINISH HS

 002 B014201B (02 ) 100000 WILL GRADUATE HS

 003 B014201C (03 ) 010000 SOME ED AFTER HS

 004 B014201D (04 ) 001000 GRADUATE COLLEGE

 005 B014201E (05 ) 000100 GO TO GRAD SCHOOL

 006 B014201M (M, IDK ) 000001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0028

 DESCRIPTION: THIS YEAR-STUDIED U. S. CONSTITUTION

 GRADES/ASSESSMENTS: N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: P804401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P804401Y (01 ) 000 YES

 002 P804401N (02 ) 100 NO

 003 P804401M (M, IDK ) 001 MISSING, I DON’T KNOW
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

752

 CONDITIONING VARIABLE ID: SUBJ0029

 DESCRIPTION: THIS YEAR-STUDIED CONGRESS

 GRADES/ASSESSMENTS: N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: P804402 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P804402Y (01 ) 000 YES

 002 P804402N (02 ) 100 NO

 003 P804402M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0030

 DESCRIPTION: THIS YEAR-STUDIED PRESIDENT AND CABINET

 GRADES/ASSESSMENTS: N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: P804403 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P804403Y (01 ) 000 YES

 002 P804403N (02 ) 100 NO

 003 P804403M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0031

 DESCRIPTION: THIS YEAR-STUDIED HOW LAWS ARE MADE

 GRADES/ASSESSMENTS: N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: P804404 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P804404Y (01 ) 000 YES

 002 P804404N (02 ) 100 NO

 003 P804404M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0032

 DESCRIPTION: THIS YEAR-STUDIED THE COURT SYSTEM

 GRADES/ASSESSMENTS: N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: P804405 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P804405Y (01 ) 000 YES

 002 P804405N (02 ) 100 NO

 003 P804405M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0033

 DESCRIPTION: THIS YEAR-STUDIED POLIT PARTIES, ELECTIONS, VOTE

 GRADES/ASSESSMENTS: N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: P804406 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P804406Y (01 ) 000 YES

 002 P804406N (02 ) 100 NO

 003 P804406M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0034

 DESCRIPTION: THIS YEAR-STUDIED STATE & LOCAL GOVERNMENT

 GRADES/ASSESSMENTS: N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: P804407 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P804407Y (01 ) 000 YES

 002 P804407N (02 ) 100 NO

 003 P804407M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0035

 DESCRIPTION: THIS YEAR-STUDIED OTHER COUNTRIES’ GOVERNMENT

 GRADES/ASSESSMENTS: N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: P804408 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P804408Y (01 ) 000 YES

 002 P804408N (02 ) 100 NO

 003 P804408M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0036

 DESCRIPTION: THIS YEAR-STUDIED INTERNATIONAL ORGANIZATIONS

 GRADES/ASSESSMENTS: N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: P804409 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P804409Y (01 ) 000 YES

 002 P804409N (02 ) 100 NO

 003 P804409M (M, IDK ) 001 MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: SUBJ0037

 DESCRIPTION: HOMEWORK HOURS/WEEK-SOCIAL STUDIES CLASS

 GRADES/ASSESSMENTS: N08

 CONDITIONING VAR LABEL:

 NAEP ID: P804501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 P804501N (01 ) 000000 DON’T USUALLY HAVE

 002 P804501B (02 ) 100000 HAVE BUT DON’T DO

 003 P804501C (03 ) 010000 LESS THAN 1 HOUR

 004 P804501D (04 ) 001000 1-2 HOURS

 005 P804501E (05 ) 000100 3-4 HOURS

 006 P804501F (06 ) 000010 5 HOURS OR MORE

 007 P804501M (M ) 000001 MISSING
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

753

 CONDITIONING VARIABLE ID: SCHL0052

 DESCRIPTION: 8TH GRADE ASSIGNED TO ENGLISH CLASS BY ABILITY

 GRADES/ASSESSMENTS: N08

 CONDITIONING VAR LABEL:

 NAEP ID: C044401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C044401Y (01 ) 00 YES

 002 C044401N (02 ) 10 NO

 003 C044401M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0053

 DESCRIPTION: 8TH GRADE ASSIGNED-HISTORY/SS BY ABILITY

 GRADES/ASSESSMENTS: N08

 CONDITIONING VAR LABEL:

 NAEP ID: C044402 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C044402Y (01 ) 00 YES

 002 C044402N (02 ) 10 NO

 003 C044402M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0054

 DESCRIPTION: IS STUDENT DROPOUT A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043105 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C043105A (01 ) 0000 SERIOUS

 002 C043105B (02 ) 1000 MODERATE

 003 C043105C (03 ) 0100 MINOR

 004 C043105N (04 ) 0010 NOT A PROBLEM

 005 C043105M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0055

 DESCRIPTION: IS TEEN PREGNANCY A PROBLEM IN YOUR SCHOOL

 GRADES/ASSESSMENTS: N08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C043106 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C043106A (01 ) 0000 SERIOUS

 002 C043106B (02 ) 1000 MODERATE

 003 C043106C (03 ) 0100 MINOR

 004 C043106N (04 ) 0010 NOT A PROBLEM

 005 C043106M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0074

 DESCRIPTION: MAIN ACTIVITY YEAR FOLLOWING HIGH SCHOOL

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: B005501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 B005501A (01 ) 000000 WORK FULL-TIME

 002 B005501B (02 ) 100000 VOCA/TECH/BUSINESS

 003 B005501C (03 ) 010000 ATTEND 2 YR COLLEGE

 004 B005501D (04 ) 001000 ATTEND 4 YR COLLEGE

 005 B005501E (05 ) 000100 SERVE IN MILITARY

 006 B005501F (06 ) 000010 OTHER

 007 B005501M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: BACK0075

 DESCRIPTION: VOLUNTEER WORK IN YOUR COMMUNITY THIS YEAR

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: B014301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 B014301Y (01 ) 000 YES, WITH MY SCHOOL

 002 B014301Y (02 ) 100 YES, ON MY OWN

 003 B014301N (03 ) 010 NO

 004 B014301M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: BACK0076

 DESCRIPTION: HOW MANY HOURS/WEEK WORK JOB FOR PAY

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: B014401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 B014401N (01 ) 000000 NONE

 002 B014401B (02 ) 100000 1-5 HOURS

 003 B014401C (03 ) 010000 6-10 HOURS

 004 B014401D (04 ) 001000 11-15 HOURS

 005 B014401E (05 ) 000100 16-20 HOURS

 006 B014401F (06 ) 000010 21 OR MORE HOURS

 007 B014401M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0038

 DESCRIPTION: HOW HARD TRIED ON THIS CIVICS TEST THAN ON OTHERS

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: P802545 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 P802545A (01 ) 0000 TRIED MUCH HARDER

 002 P802545B (02 ) 1000 TRIED HARDER

 003 P802545C (03 ) 0100 TRIED ABOUT AS HARD

 004 P802545N (04 ) 0010 TRIED NOT AS HARD

 005 P802545M (M ) 0001 MISSING
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

754

 CONDITIONING VARIABLE ID: SUBJ0039

 DESCRIPTION: HOW IMPORTANT TO DO WELL ON THIS CIVICS TEST

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: P802546 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 P802546A (01 ) 0000 VERY IMPORTANT

 002 P802546B (02 ) 1000 IMPORTANT

 003 P802546C (03 ) 0100 SOMEWHAT IMPORTANT

 004 P802546N (04 ) 0010 NOT VERY IMPORTANT

 005 P802546M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0040

 DESCRIPTION: HOW OFTEN WRITE LONG ANSWERS ON CIVICS TESTS

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: P802547 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 P802547A (01 ) 0000 AT LEAST ONCE A WEEK

 002 P802547B (02 ) 1000 ONCE/TWICE A MONTH

 003 P802547C (03 ) 0100 ONCE/TWICE A YEAR

 004 P802547D (04 ) 0010 NEVER

 005 P802547M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0041

 DESCRIPTION: GRADE 9 - STUDIED CIVICS OR GOVERNMENT

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: P804601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 P804601Y (01 ) 0 YES

 002 P804601M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SUBJ0042

 DESCRIPTION: GRADE 10 - STUDIED CIVICS OR GOVERNMENT

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: P804602 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 P804602Y (01 ) 0 YES

 002 P804602M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SUBJ0043

 DESCRIPTION: GRADE 11 - STUDIED CIVICS OR GOVERNMENT

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: P804603 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 P804603Y (01 ) 0 YES

 002 P804603M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SUBJ0044

 DESCRIPTION: GRADE 12 - STUDIED CIVICS OR GOVERNMENT

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: P804604 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 P804604Y (01 ) 0 YES

 002 P804604M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SUBJ0045

 DESCRIPTION: HOMEWORK HOURS/WEEK CIVICS-GOVERNMENT CLASS

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: P804701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 P804701N (01 ) 000000 DON’T USUALLY HAVE

 002 P804701B (02 ) 100000 HAVE BUT DON’T DO

 003 P804701C (03 ) 010000 LESS THAN 1 HOUR

 004 P804701D (04 ) 001000 1-2 HOURS

 005 P804701E (05 ) 000100 3-4 HOURS

 006 P804701F (06 ) 000010 5 HOURS OR MORE

 007 P804701M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0046

 DESCRIPTION: DO YOU HAVE A TEXTBOOK TO STUDY CIVICS/GOVERNMENT

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: P804801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P804801Y (01 ) 00 YES

 002 P804801N (02 ) 10 NO

 003 P804801M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SUBJ0047

 DESCRIPTION: ENROLLED IN OR TOOK AP U.S. GOV’T & POLITICS

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: P804901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 P804901Y (01 ) 00 YES

 002 P804901N (02 ) 10 NO

 003 P804901M (M ) 01 MISSING
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

755

 CONDITIONING VARIABLE ID: SCHL0056

 DESCRIPTION: 12TH GRADE ASSIGNED TO ENGLISH CLASS BY ABILITY

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C044301Y (01 ) 00 YES

 002 C044301N (02 ) 10 NO

 003 C044301M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0057

 DESCRIPTION: 12TH GR ASSIGNED- HISTORY/CIVICS/SS CLASS ABILITY

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044302 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C044302Y (01 ) 00 YES

 002 C044302N (02 ) 10 NO

 003 C044302M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0058

 DESCRIPTION: PERCENT LAST YEAR’S TWELFTH-GRADE CLASS GRADUATED

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C044101A (01 ) 00000 99-100%

 002 C044101B (02 ) 10000 95-98%

 003 C044101C (03 ) 01000 90-94%

 004 C044101D (04 ) 00100 75-89%

 005 C044101E (05 ) 00010 LESS THAN 75%

 006 C044101M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0059

 DESCRIPTION: PERCENT GRADUATING CLASS-ATTEND TWO-YEAR COLLEGE

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 8

 001 C044201N (01 ) 00000000 NONE

 002 C044201B (02 ) 10000000 1-5%

 003 C044201C (03 ) 01000000 6-10%

 004 C044201D (04 ) 00100000 11-25%

 005 C044201E (05 ) 00010000 26-50%

 006 C044201F (06 ) 00001000 51-75%

 007 C044201G (07 ) 00000100 76-90%

 008 C044201H (08 ) 00000010 OVER 100%

 009 C044201M (M ) 00000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0060

 DESCRIPTION: PERCENT GRADUATING CLASS-ATTEND FOUR-YEAR COLLEGE

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C044202 TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 8

 001 C044202N (01 ) 00000000 NONE

 002 C044202B (02 ) 10000000 1-5%

 003 C044202C (03 ) 01000000 6-10%

 004 C044202D (04 ) 00100000 11-25%

 005 C044202E (05 ) 00010000 26-50%

 006 C044202F (06 ) 00001000 51-75%

 007 C044202G (07 ) 00000100 76-90%

 008 C044202H (08 ) 00000010 OVER 100%

 009 C044202M (M ) 00000001 MISSING

 009 C044202M (M ) 00000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0001

 DESCRIPTION: DO YOU TEACH READING

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T067001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T067001Y (01 ) 0 YES

 002 T067001M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0002

 DESCRIPTION: DO YOU TEACH WRITING

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T067002 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T067002Y (01 ) 0 YES

 002 T067002M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0003

 DESCRIPTION: DO YOU TEACH LANGUAGE ARTS

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T067003 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T067003Y (01 ) 0 YES

 002 T067003M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0004

 DESCRIPTION: DO YOU TEACH SOCIAL STUDIES

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T067004 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T067004Y (01 ) 0 YES

 002 T067004M (M ) 1 MISSING
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

756

 CONDITIONING VARIABLE ID: TCHR0005

 DESCRIPTION: YEARS TOTAL TAUGHT ELEMENTARY LEVEL

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T067101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067101A (01 ) 00000 2 YEARS OR LESS

 002 T067101B (02 ) 10000 3-5 YEARS

 003 T067101C (03 ) 01000 6-10 YEARS

 004 T067101D (04 ) 00100 11-24 YEARS

 005 T067101E (05 ) 00010 25 YEARS OR MORE

 006 T067101M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0006

 DESCRIPTION: YEARS TOTAL TAUGHT READING

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T067201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067201A (01 ) 00000 2 YEARS OR LESS

 002 T067201B (02 ) 10000 3-5 YEARS

 003 T067201C (03 ) 01000 6-10 YEARS

 004 T067201D (04 ) 00100 11-24 YEARS

 005 T067201E (05 ) 00010 25 YEARS OR MORE

 006 T067201M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0007

 DESCRIPTION: YEARS TOTAL TAUGHT WRITING

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T067202 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067202A (01 ) 00000 2 YEARS OR LESS

 002 T067202B (02 ) 10000 3-5 YEARS

 003 T067202C (03 ) 01000 6-10 YEARS

 004 T067202D (04 ) 00100 11-24 YEARS

 005 T067202E (05 ) 00010 25 YEARS OR MORE

 006 T067202M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0008

 DESCRIPTION: YEARS TOTAL TAUGHT LANGUAGE ARTS

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T067203 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067203A (01 ) 00000 2 YEARS OR LESS

 002 T067203B (02 ) 10000 3-5 YEARS

 003 T067203C (03 ) 01000 6-10 YEARS

 004 T067203D (04 ) 00100 11-24 YEARS

 005 T067203E (05 ) 00010 25 YEARS OR MORE

 006 T067203M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0009

 DESCRIPTION: YEARS TOTAL TAUGHT HISTORY

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T067204 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067204A (01 ) 00000 2 YEARS OR LESS

 002 T067204B (02 ) 10000 3-5 YEARS

 003 T067204C (03 ) 01000 6-10 YEARS

 004 T067204D (04 ) 00100 11-24 YEARS

 005 T067204E (05 ) 00010 25 YEARS OR MORE

 006 T067204M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0010

 DESCRIPTION: YEARS TOTAL TAUGHT SOCIAL STUDIES

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T067205 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067205A (01 ) 00000 2 YEARS OR LESS

 002 T067205B (02 ) 10000 3-5 YEARS

 003 T067205C (03 ) 01000 6-10 YEARS

 004 T067205D (04 ) 00100 11-24 YEARS

 005 T067205E (05 ) 00010 25 YEARS OR MORE

 006 T067205M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0011

 DESCRIPTION: YEARS TOTAL TAUGHT CIVICS

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T067206 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067206A (01 ) 00000 2 YEARS OR LESS

 002 T067206B (02 ) 10000 3-5 YEARS

 003 T067206C (03 ) 01000 6-10 YEARS

 004 T067206D (04 ) 00100 11-24 YEARS

 005 T067206E (05 ) 00010 25 YEARS OR MORE

 006 T067206M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0012

 DESCRIPTION: MAIN ASSIGNMENT FIELD

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T067301A (01 ) 0000 REGULAR CLASSROOM

 002 T067301B (02 ) 1000 SPECIAL CLASSROOM

 003 T067301C (03 ) 0100 ESL/BILINGUAL ED

 004 T067301D (04 ) 0010 OTHER

 005 T067301M (M ) 0001 MISSING
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

757

 CONDITIONING VARIABLE ID: TCHR0013

 DESCRIPTION: TEACHING CERTIF IN THIS STATE IN MAIN FIELD

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T056201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 T056201A (01 ) 000000 ADVANCED PROFESSIONL

 002 T056201B (02 ) 100000 REGULAR/STANDARD ST

 003 T056201C (03 ) 010000 PROBATIONARY STATE

 004 T056201D (04 ) 001000 TEMPORARY/PROVISIONL

 005 T056201E (05 ) 000100 OTHER THAN STATE CRT

 006 T056201F (06 ) 000010 NOT HAVE CERT MAIN

 007 T056201M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0014

 DESCRIPTION: HIGHEST ACADEMIC DEGREE YOU HOLD

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T056301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 T056301A (01 ) 0000000 HIGH SCHOOL DIPLOMA

 002 T056301B (02 ) 1000000 ASSOCIATES/VOCATIONL

 003 T056301C (03 ) 0100000 BACHELOR’S DEGREE

 004 T056301D (04 ) 0010000 MASTER’S DEGREE

 005 T056301E (05 ) 0001000 EDUCATION SPECIALIST

 006 T056301F (06 ) 0000100 DOCTORATE

 007 T056301G (07 ) 0000010 PROFESSIONAL DEGREE

 008 T056301M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0015

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-ELEMENTARY EDUCATION

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067501A (01 ) 000 MAJOR

 002 T067501B (02 ) 100 MINOR

 003 T067501C (03 ) 010 NOT IN THIS SUBJECT

 004 T067501M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0016

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-SECONDARY EDUCATION

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067502 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067502A (01 ) 000 MAJOR

 002 T067502B (02 ) 100 MINOR

 003 T067502C (03 ) 010 NOT IN THIS SUBJECT

 004 T067502M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0017

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-SPECIAL EDUCATION

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067503 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067503A (01 ) 000 MAJOR

 002 T067503B (02 ) 100 MINOR

 003 T067503C (03 ) 010 NOT IN THIS SUBJECT

 004 T067503M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0018

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-BILINGUAL EDUCATION/ESL

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067504 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067504A (01 ) 000 MAJOR

 002 T067504B (02 ) 100 MINOR

 003 T067504C (03 ) 010 NOT IN THIS SUBJECT

 004 T067504M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0019

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-ADMINISTRATION & SUPERVISION

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067505 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067505A (01 ) 000 MAJOR

 002 T067505B (02 ) 100 MINOR

 003 T067505C (03 ) 010 NOT IN THIS SUBJECT

 004 T067505M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0020

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-CURRICULUM & SUPERVISION

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067506 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067506A (01 ) 000 MAJOR

 002 T067506B (02 ) 100 MINOR

 003 T067506C (03 ) 010 NOT IN THIS SUBJECT

 004 T067506M (M ) 001 MISSING
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

758

 CONDITIONING VARIABLE ID: TCHR0021

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-COUNSELING

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067507 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067507A (01 ) 000 MAJOR

 002 T067507B (02 ) 100 MINOR

 003 T067507C (03 ) 010 NOT IN THIS SUBJECT

 004 T067507M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0022

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-ENGLISH

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067508 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067508A (01 ) 000 MAJOR

 002 T067508B (02 ) 100 MINOR

 003 T067508C (03 ) 010 NOT IN THIS SUBJECT

 004 T067508M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0023

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-READING AND/OR LANGUAGE ARTS

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067509 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067509A (01 ) 000 MAJOR

 002 T067509B (02 ) 100 MINOR

 003 T067509C (03 ) 010 NOT IN THIS SUBJECT

 004 T067509M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0024

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-HISTORY

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067510 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067510A (01 ) 000 MAJOR

 002 T067510B (02 ) 100 MINOR

 003 T067510C (03 ) 010 NOT IN THIS SUBJECT

 004 T067510M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0025

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-POLITICAL SCIENCE

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067511 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067511A (01 ) 000 MAJOR

 002 T067511B (02 ) 100 MINOR

 003 T067511C (03 ) 010 NOT IN THIS SUBJECT

 004 T067511M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0026

 DESCRIPTION: UNDERGRAD MAJOR/MINOR-OTHER

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067512 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067512A (01 ) 000 MAJOR

 002 T067512B (02 ) 100 MINOR

 003 T067512C (03 ) 010 NOT IN THIS SUBJECT

 004 T067512M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0027

 DESCRIPTION: GRAD MAJOR/MINOR-ELEMENTARY EDUCATION

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067601A (01 ) 000 MAJOR

 002 T067601B (02 ) 100 MINOR

 003 T067601C (03 ) 010 NOT IN THIS SUBJECT

 004 T067601M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0028

 DESCRIPTION: GRAD MAJOR/MINOR-SECONDARY EDUCATION

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067602 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067602A (01 ) 000 MAJOR

 002 T067602B (02 ) 100 MINOR

 003 T067602C (03 ) 010 NOT IN THIS SUBJECT

 004 T067602M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0029

 DESCRIPTION: GRAD MAJOR/MINOR-SPECIAL EDUCATION

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067603 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067603A (01 ) 000 MAJOR

 002 T067603B (02 ) 100 MINOR

 003 T067603C (03 ) 010 NOT IN THIS SUBJECT

 004 T067603M (M ) 001 MISSING
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

759

 CONDITIONING VARIABLE ID: TCHR0030

 DESCRIPTION: GRAD MAJOR/MINOR-BILINGUAL EDUCATION/ESL

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067604 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067604A (01 ) 000 MAJOR

 002 T067604B (02 ) 100 MINOR

 003 T067604C (03 ) 010 NOT IN THIS SUBJECT

 004 T067604M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0031

 DESCRIPTION: GRAD MAJOR/MINOR-ADMINSTRATION & SUPERVISION

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067605 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067605A (01 ) 000 MAJOR

 002 T067605B (02 ) 100 MINOR

 003 T067605C (03 ) 010 NOT IN THIS SUBJECT

 004 T067605M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0032

 DESCRIPTION: GRAD MAJOR/MINOR-CURRICULUM AND INSTRUCTION

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067606 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067606A (01 ) 000 MAJOR

 002 T067606B (02 ) 100 MINOR

 003 T067606C (03 ) 010 NOT IN THIS SUBJECT

 004 T067606M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0033

 DESCRIPTION: GRAD MAJOR/MINOR-COUNSELING

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067607 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067607A (01 ) 000 MAJOR

 002 T067607B (02 ) 100 MINOR

 003 T067607C (03 ) 010 NOT IN THIS SUBJECT

 004 T067607M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0034

 DESCRIPTION: GRAD MAJOR/MINOR-ENGLISH

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067608 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067608A (01 ) 000 MAJOR

 002 T067608B (02 ) 100 MINOR

 003 T067608C (03 ) 010 NOT IN THIS SUBJECT

 004 T067608M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0035

 DESCRIPTION: GRAD MAJOR/MINOR-READING AND/OR LANGUAGE ARTS

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067609 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067609A (01 ) 000 MAJOR

 002 T067609B (02 ) 100 MINOR

 003 T067609C (03 ) 010 NOT IN THIS SUBJECT

 004 T067609M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0036

 DESCRIPTION: GRAD MAJOR/MINOR-HISTORY

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067610 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067610A (01 ) 000 MAJOR

 002 T067610B (02 ) 100 MINOR

 003 T067610C (03 ) 010 NOT IN THIS SUBJECT

 004 T067610M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0037

 DESCRIPTION: GRAD MAJOR/MINOR-POLITICAL SCIENCE

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067611 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067611A (01 ) 000 MAJOR

 002 T067611B (02 ) 100 MINOR

 003 T067611C (03 ) 010 NOT IN THIS SUBJECT

 004 T067611M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0038

 DESCRIPTION: GRAD MAJOR/MINOR-OTHER

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067612 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067612A (01 ) 000 MAJOR

 002 T067612B (02 ) 100 MINOR

 003 T067612C (03 ) 010 NOT IN THIS SUBJECT

 004 T067612M (M ) 001 MISSING
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

760

 CONDITIONING VARIABLE ID: TCHR0039

 DESCRIPTION: LAST 12 MOS, PROF DEV-READING AND WRITING

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067701A (01 ) 00000 NONE

 002 T067701B (02 ) 10000 LESS THAN 6 HOURS

 003 T067701C (03 ) 01000 6 - 15 HOURS

 004 T067701D (04 ) 00100 16 - 35 HOURS

 005 T067701E (05 ) 00010 MORE THAN 35 HOURS

 006 T067701M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0040

 DESCRIPTION: LAST 12 MOS, PROF DEV-SOCIAL STUDIES

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067702 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T067702A (01 ) 00000 NONE

 002 T067702B (02 ) 10000 LESS THAN 6 HOURS

 003 T067702C (03 ) 01000 6 - 15 HOURS

 004 T067702D (04 ) 00100 16 - 35 HOURS

 005 T067702E (05 ) 00010 MORE THAN 35 HOURS

 006 T067702M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0041

 DESCRIPTION: PREPARED IN THE USE OF TELECOMMUNICATIONS

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067801A (01 ) 000 WELL PREPARED

 002 T067801B (02 ) 100 MODERATELY PREPARED

 003 T067801C (03 ) 010 NOT WELL PREPARED

 004 T067801M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0042

 DESCRIPTION: PREPARED IN THE USE OF COMPUTERS

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067802 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067802A (01 ) 000 WELL PREPARED

 002 T067802B (02 ) 100 MODERATELY PREPARED

 003 T067802C (03 ) 010 NOT WELL PREPARED

 004 T067802M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0043

 DESCRIPTION: PREPARED IN COOPERATIVE GROUP INSTRUCTION

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067803 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067803A (01 ) 000 WELL PREPARED

 002 T067803B (02 ) 100 MODERATELY PREPARED

 003 T067803C (03 ) 010 NOT WELL PREPARED

 004 T067803M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0044

 DESCRIPTION: PREPARED IN TEACHING STUDENTS-DIFFERENT CULTURES

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067804 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067804A (01 ) 000 WELL PREPARED

 002 T067804B (02 ) 100 MODERATELY PREPARED

 003 T067804C (03 ) 010 NOT WELL PREPARED

 004 T067804M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0045

 DESCRIPTION: PREPARED IN TEACHING STUDENTS WHO ARE LEP

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067805 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067805A (01 ) 000 WELL PREPARED

 002 T067805B (02 ) 100 MODERATELY PREPARED

 003 T067805C (03 ) 010 NOT WELL PREPARED

 004 T067805M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0046

 DESCRIPTION: PREPARED IN TEACHING STUDENTS WITH DISABILITIES

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067806 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067806A (01 ) 000 WELL PREPARED

 002 T067806B (02 ) 100 MODERATELY PREPARED

 003 T067806C (03 ) 010 NOT WELL PREPARED

 004 T067806M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0047

 DESCRIPTION: PREPARED IN CLASSROOM MANAGEMENT AND ORGANIZATION

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T067807 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T067807A (01 ) 000 WELL PREPARED

 002 T067807B (02 ) 100 MODERATELY PREPARED

 003 T067807C (03 ) 010 NOT WELL PREPARED

 004 T067807M (M ) 001 MISSING
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

761

 CONDITIONING VARIABLE ID: TCHR0048

 DESCRIPTION: AVAILABILITY OF RESOURCES

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T041201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T041201A (01 ) 0000 GET ALL RESOURCES

 002 T041201B (02 ) 1000 GET MOST RESOURCES

 003 T041201C (03 ) 0100 GET SOME RESOURCES

 004 T041201D (04 ) 0010 DON’T GET RESOURCES

 005 T041201M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0049

 DESCRIPTION: PREPARED IN SOCIAL STUDIES INSTRUCTION

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T070401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T070401A (01 ) 000 WELL PREPARED

 002 T070401B (02 ) 100 MODERATELY PREPARED

 003 T070401C (03 ) 010 NOT WELL PREPARED

 004 T070401M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0050

 DESCRIPTION: PREPARED IN PUBLIC SERVICE OPPORTUNITIES

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T070402 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T070402A (01 ) 000 WELL PREPARED

 002 T070402B (02 ) 100 MODERATELY PREPARED

 003 T070402C (03 ) 010 NOT WELL PREPARED

 004 T070402M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0051

 DESCRIPTION: PREPARED IN INSTRUCTIONAL MATERIALS IN SOC STUDIES

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T070403 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T070403A (01 ) 000 WELL PREPARED

 002 T070403B (02 ) 100 MODERATELY PREPARED

 003 T070403C (03 ) 010 NOT WELL PREPARED

 004 T070403M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0052

 DESCRIPTION: PREPARED IN USE OF COMMUNITY RESOURCES IN INSTRUC

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T070404 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T070404A (01 ) 000 WELL PREPARED

 002 T070404B (02 ) 100 MODERATELY PREPARED

 003 T070404C (03 ) 010 NOT WELL PREPARED

 004 T070404M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0053

 DESCRIPTION: PREPARED IN CLASSROOM CLIMATE AND GOVERNANCE

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T070405 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T070405A (01 ) 000 WELL PREPARED

 002 T070405B (02 ) 100 MODERATELY PREPARED

 003 T070405C (03 ) 010 NOT WELL PREPARED

 004 T070405M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0054

 DESCRIPTION: PREPARED IN USING NATL STANDARDS FOR CIVICS

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T070406 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T070406A (01 ) 000 WELL PREPARED

 002 T070406B (02 ) 100 MODERATELY PREPARED

 003 T070406C (03 ) 010 NOT WELL PREPARED

 004 T070406M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0055

 DESCRIPTION: PREPARED IN USING SOFTWARE FOR SOCIAL STUDIES

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T070407 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T070407A (01 ) 000 WELL PREPARED

 002 T070407B (02 ) 100 MODERATELY PREPARED

 003 T070407C (03 ) 010 NOT WELL PREPARED

 004 T070407M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0056

 DESCRIPTION: WHAT IS YOUR AVERAGE SOCIAL STUDIES CLASS SIZE

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T070501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T070501A (01 ) 00000 1-20 STUDENTS

 002 T070501B (02 ) 10000 21-25 STUDENTS

 003 T070501C (03 ) 01000 26-30 STUDENTS

 004 T070501D (04 ) 00100 31-35 STUDENTS

 005 T070501E (05 ) 00010 36 OR MORE STUDENTS

 006 T070501M (M ) 00001 MISSING
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

762

 CONDITIONING VARIABLE ID: TCHR0057

 DESCRIPTION: ARE STUDENTS ASSIGNED TO THIS CLASS BY ABILITY

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T070601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T070601Y (01 ) 00 YES

 002 T070601N (02 ) 10 NO

 003 T070601M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TCHR0058

 DESCRIPTION: WHAT IS THE ABILITY LEVEL OF THE STUDENTS

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T070701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070701A (01 ) 0000 PRIMARILY HIGH

 002 T070701B (02 ) 1000 PRIMARILY AVERAGE

 003 T070701C (03 ) 0100 PRIMARILY LOW

 004 T070701D (04 ) 0010 WIDELY MIXED

 005 T070701M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0059

 DESCRIPTION: CLASS TIME PER DAY-SOCIAL STUDIES INSTRUCTION

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T070801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070801A (01 ) 0000 LESS THAN 30 MINUTES

 002 T070801B (02 ) 1000 30-44 MINUTES

 003 T070801C (03 ) 0100 45-60 MINUTES

 004 T070801D (04 ) 0010 MORE THAN 60 MINUTES

 005 T070801M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0060

 DESCRIPTION: HOW OFTEN USE SOCIAL STUDIES TEXTBOOK

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T070901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070901A (01 ) 0000 ALMOST EVERY DAY

 002 T070901B (02 ) 1000 ONCE/TWICE A WEEK

 003 T070901C (03 ) 0100 ONCE/TWICE A MONTH

 004 T070901D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070901M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0061

 DESCRIPTION: HOW OFTEN USE BOOKS, NEWSPAPER, MAGAZINES

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T070902 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070902A (01 ) 0000 ALMOST EVERY DAY

 002 T070902B (02 ) 1000 ONCE/TWICE A WEEK

 003 T070902C (03 ) 0100 ONCE/TWICE A MONTH

 004 T070902D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070902M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0062

 DESCRIPTION: HOW OFTEN USE PRIMARY DOCUMENTS

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T070903 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070903A (01 ) 0000 ALMOST EVERY DAY

 002 T070903B (02 ) 1000 ONCE/TWICE A WEEK

 003 T070903C (03 ) 0100 ONCE/TWICE A MONTH

 004 T070903D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070903M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0063

 DESCRIPTION: HOW OFTEN USE QUANTITATIVE DATA-CHARTS, GRAPHS

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T070904 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070904A (01 ) 0000 ALMOST EVERY DAY

 002 T070904B (02 ) 1000 ONCE/TWICE A WEEK

 003 T070904C (03 ) 0100 ONCE/TWICE A MONTH

 004 T070904D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070904M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0064

 DESCRIPTION: HOW OFTEN USE COMPUTER SOFTWARE

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T070905 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070905A (01 ) 0000 ALMOST EVERY DAY

 002 T070905B (02 ) 1000 ONCE/TWICE A WEEK

 003 T070905C (03 ) 0100 ONCE/TWICE A MONTH

 004 T070905D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070905M (M ) 0001 MISSING
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

763

 CONDITIONING VARIABLE ID: TCHR0065

 DESCRIPTION: HOW OFTEN USE FILMS, VIDEOS, FILMSTRIPS

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T070906 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070906A (01 ) 0000 ALMOST EVERY DAY

 002 T070906B (02 ) 1000 ONCE/TWICE A WEEK

 003 T070906C (03 ) 0100 ONCE/TWICE A MONTH

 004 T070906D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070906M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0066

 DESCRIPTION: HOW OFTEN USE MATERIALS FROM OTHER SUBJECT AREAS

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T070907 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T070907A (01 ) 0000 ALMOST EVERY DAY

 002 T070907B (02 ) 1000 ONCE/TWICE A WEEK

 003 T070907C (03 ) 0100 ONCE/TWICE A MONTH

 004 T070907D (04 ) 0010 NEVER OR HARDLY EVER

 005 T070907M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0067

 DESCRIPTION: AVAILABILITY OF COMPUTERS IN SOCIAL STUDIES CLASS

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T071001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T071001A (01 ) 00000 NOT AVAILABLE

 002 T071001B (02 ) 10000 LIMITED ACCESS

 003 T071001C (03 ) 01000 LAB OR LIBRARY

 004 T071001D (04 ) 00100 ONE IN CLASSROOM

 005 T071001E (05 ) 00010 SEVERAL IN CLASSROOM

 006 T071001M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0068

 DESCRIPTION: HOW OFTEN STUDENTS COMPLETE A WORKSHEET

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071101A (01 ) 0000 ALMOST EVERY DAY

 002 T071101B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071101C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071101D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071101M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0069

 DESCRIPTION: HOW OFTEN STUDENTS READ EXTRA MATERIAL

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071102 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071102A (01 ) 0000 ALMOST EVERY DAY

 002 T071102B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071102C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071102D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071102M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0070

 DESCRIPTION: HOW OFTEN GIVE LECTURE ABOUT SOCIAL STUDIES

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071103 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071103A (01 ) 0000 ALMOST EVERY DAY

 002 T071103B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071103C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071103D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071103M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0071

 DESCRIPTION: HOW OFTEN STUDENTS DO GROUP ACTIVITY OR PROJECT

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071104 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071104A (01 ) 0000 ALMOST EVERY DAY

 002 T071104B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071104C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071104D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071104M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0072

 DESCRIPTION: HOW OFTEN STUDENTS WRITE THREE OR MORE PAGE REPORT

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071105 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071105A (01 ) 0000 ALMOST EVERY DAY

 002 T071105B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071105C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071105D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071105M (M ) 0001 MISSING
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

764

 CONDITIONING VARIABLE ID: TCHR0073

 DESCRIPTION: HOW OFTEN STUDENTS WATCH TELEVISION, VIDEOS, FILMS

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071106 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071106A (01 ) 0000 ALMOST EVERY DAY

 002 T071106B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071106C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071106D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071106M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0074

 DESCRIPTION: HOW OFTEN STUDENTS PARTICIPATE-DEBATES

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071107 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071107A (01 ) 0000 ALMOST EVERY DAY

 002 T071107B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071107C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071107D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071107M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0075

 DESCRIPTION: HOW OFTEN STUDENTS PARTICIPATE-MOCK TRIALS

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071108 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071108A (01 ) 0000 ALMOST EVERY DAY

 002 T071108B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071108C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071108D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071108M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0076

 DESCRIPTION: HOW OFTEN STUDENTS WRITE LETTERS

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071109 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071109A (01 ) 0000 ALMOST EVERY DAY

 002 T071109B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071109C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071109D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071109M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0077

 DESCRIPTION: HOW OFTEN VISITORS MEET/DISCUSS IMPORTANT EVENTS

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071110 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071110A (01 ) 0000 ALMOST EVERY DAY

 002 T071110B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071110C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071110D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071110M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0078

 DESCRIPTION: HOW OFTEN STUDENTS VISIT GOVERNMENT/COMMUNITY

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071111 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071111A (01 ) 0000 ALMOST EVERY DAY

 002 T071111B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071111C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071111D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071111M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0079

 DESCRIPTION: HOW OFTEN STUDENTS PARTICIPATE-VOLUNTEER PROJ/SERV

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071112 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071112A (01 ) 0000 ALMOST EVERY DAY

 002 T071112B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071112C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071112D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071112M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0080

 DESCRIPTION: HOW OFTEN STUDENTS ACCESS INTERNET-CLASSROOM

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071113 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071113A (01 ) 0000 ALMOST EVERY DAY

 002 T071113B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071113C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071113D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071113M (M ) 0001 MISSING
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

765

 CONDITIONING VARIABLE ID: TCHR0081

 DESCRIPTION: HOW OFTEN STUDENTS DISCUSS CURRENT EVENTS

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071114 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071114A (01 ) 0000 ALMOST EVERY DAY

 002 T071114B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071114C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071114D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071114M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0082

 DESCRIPTION: HOW OFTEN STUDENTS USE STUDENT GOVERNMENT

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071115 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071115A (01 ) 0000 ALMOST EVERY DAY

 002 T071115B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071115C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071115D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071115M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0083

 DESCRIPTION: HOW OFTEN GIVE STUDENTS SOCIAL STUDIES HOMEWORK

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071116 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071116A (01 ) 0000 ALMOST EVERY DAY

 002 T071116B (02 ) 1000 ONCE/TWICE A WEEK

 003 T071116C (03 ) 0100 ONCE/TWICE A MONTH

 004 T071116D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071116M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0084

 DESCRIPTION: HOW OFTEN USE MULTIPLE-CHOICE,TRUE/FALSE,MATCHING

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071201A (01 ) 0000 ONCE/TWICE A WEEK

 002 T071201B (02 ) 1000 ONCE/TWICE A MONTH

 003 T071201C (03 ) 0100 ONCE/TWICE A YEAR

 004 T071201D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071201M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0085

 DESCRIPTION: HOW OFTEN USE FILL-IN-THE BLANK QUESTIONS

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071202 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071202A (01 ) 0000 ONCE/TWICE A WEEK

 002 T071202B (02 ) 1000 ONCE/TWICE A MONTH

 003 T071202C (03 ) 0100 ONCE/TWICE A YEAR

 004 T071202D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071202M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0086

 DESCRIPTION: HOW OFTEN USE PARAGRAPH WRITTEN RESPONSE

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071203 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071203A (01 ) 0000 ONCE/TWICE A WEEK

 002 T071203B (02 ) 1000 ONCE/TWICE A MONTH

 003 T071203C (03 ) 0100 ONCE/TWICE A YEAR

 004 T071203D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071203M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0087

 DESCRIPTION: HOW OFTEN USE INDIVIDUAL/GROUP PROJECTS

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071204 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071204A (01 ) 0000 ONCE/TWICE A WEEK

 002 T071204B (02 ) 1000 ONCE/TWICE A MONTH

 003 T071204C (03 ) 0100 ONCE/TWICE A YEAR

 004 T071204D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071204M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0088

 DESCRIPTION: HOW OFTEN USE ESSAYS, PAPERS ASSIGNED TOPICS

 GRADES/ASSESSMENTS: N04, N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071205 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071205A (01 ) 0000 ONCE/TWICE A WEEK

 002 T071205B (02 ) 1000 ONCE/TWICE A MONTH

 003 T071205C (03 ) 0100 ONCE/TWICE A YEAR

 004 T071205D (04 ) 0010 NEVER OR HARDLY EVER

 005 T071205M (M ) 0001 MISSING

 005 T071205M (M ) 0001 MISSING
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

766

 CONDITIONING VARIABLE ID: TCHR0089

 DESCRIPTION: AVAILABILITY OF COMPUTERS IN SOCIAL STUDIES CLASS

 GRADES/ASSESSMENTS: N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T071301A (01 ) 0000 NOT AVAILABLE

 002 T071301B (02 ) 1000 LIMITED ACCESS

 003 T071301C (03 ) 0100 READILY AVAILABLE

 004 T071301D (04 ) 0010 AVAILABLE ALL CLASS

 005 T071301M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0090

 DESCRIPTION: DO YOU TEACH HISTORY

 GRADES/ASSESSMENTS: N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T071401Y (01 ) 0 YES

 002 T071401M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0091

 DESCRIPTION: DO YOU TEACH SOCIAL STUDIES

 GRADES/ASSESSMENTS: N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071402 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T071402Y (01 ) 0 YES

 002 T071402M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0092

 DESCRIPTION: DO YOU TEACH GOVERNMENT/CIVICS

 GRADES/ASSESSMENTS: N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071403 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T071403Y (01 ) 0 YES

 002 T071403M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0093

 DESCRIPTION: DO YOU TEACH-OTHER

 GRADES/ASSESSMENTS: N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071404 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T071404Y (01 ) 0 YES

 002 T071404M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0094

 DESCRIPTION: YEARS TOTAL TAUGHT ELEMENTARY OR SECONDARY

 GRADES/ASSESSMENTS: N08

 CONDITIONING VAR LABEL:

 NAEP ID: T040301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T040301A (01 ) 00000 2 YEARS OR LESS

 002 T040301B (02 ) 10000 3-5 YEARS

 003 T040301C (03 ) 01000 6-10 YEARS

 004 T040301D (04 ) 00100 11-24 YEARS

 005 T040301E (05 ) 00010 25 YEARS OR MORE

 006 T040301M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0095

 DESCRIPTION: YEARS TOTAL TAUGHT HISTORY

 GRADES/ASSESSMENTS: N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 T071501A (01 ) 000000 NOT TAUGHT

 002 T071501B (02 ) 100000 2 YEARS OR LESS

 003 T071501C (03 ) 010000 3-5 YEARS

 004 T071501D (04 ) 001000 6-10 YEARS

 005 T071501E (05 ) 000100 11-24 YEARS

 006 T071501F (06 ) 000010 25 YEARS OR MORE

 007 T071501M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0096

 DESCRIPTION: YEARS TOTAL TAUGHT SOCIAL STUDIES

 GRADES/ASSESSMENTS: N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071502 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 T071502A (01 ) 000000 NOT TAUGHT

 002 T071502B (02 ) 100000 2 YEARS OR LESS

 003 T071502C (03 ) 010000 3-5 YEARS

 004 T071502D (04 ) 001000 6-10 YEARS

 005 T071502E (05 ) 000100 11-24 YEARS

 006 T071502F (06 ) 000010 25 YEARS OR MORE

 007 T071502M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0097

 DESCRIPTION: YEARS TOTAL TAUGHT GOVERNMENT/CIVICS

 GRADES/ASSESSMENTS: N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071503 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 T071503A (01 ) 000000 NOT TAUGHT

 002 T071503B (02 ) 100000 2 YEARS OR LESS

 003 T071503C (03 ) 010000 3-5 YEARS

 004 T071503D (04 ) 001000 6-10 YEARS

 005 T071503E (05 ) 000100 11-24 YEARS

 006 T071503F (06 ) 000010 25 YEARS OR MORE

 007 T071503M (M ) 000001 MISSING
Table F-7 (continued)

1998 Civics Conditioning Variable Specifications

767

 CONDITIONING VARIABLE ID: TCHR0098

 DESCRIPTION: YEARS TOTAL TAUGHT-OTHER

 GRADES/ASSESSMENTS: N08

 CONDITIONING VAR LABEL:

 NAEP ID: T071504 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 T071504A (01 ) 000000 NOT TAUGHT

 002 T071504B (02 ) 100000 2 YEARS OR LESS

 003 T071504C (03 ) 010000 3-5 YEARS

 004 T071504D (04 ) 001000 6-10 YEARS

 005 T071504E (05 ) 000100 11-24 YEARS

 006 T071504F (06 ) 000010 25 YEARS OR MORE

 007 T071504M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0099

 DESCRIPTION: WHAT IS THE NUMBER OF STUDENTS IN EACH CLASS? (8TH GRADE)

 GRADES/ASSESSMENTS: N08

 CONDITIONING VAR LABEL: CLASSIZ8

 NAEP ID: TCSIZE TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 CLASIZ-1 (1 ) 00000 AVERAGE CLASS SIZE: 1-20 STUDENTS

 002 CLASIZ-2 (2 ) 10000 AVERAGE CLASS SIZE: 21-25 STUDENTS

 003 CLASIZ-3 (3 ) 01000 AVERAGE CLASS SIZE: 26-30 STUDENTS

 004 CLASIZ-4 (4 ) 00100 AVERAGE CLASS SIZE: 31-35 STUDENTS

 005 CLASIZ-5 (5 ) 00010 AVERAGE CLASS SIZE: 36 OR MORE STUDENTS

 006 CLASIZ-? (M ) 00001 AVERAGE CLASS SIZE: MISSING, DOES NOT APPLY
Table F-8

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Reading Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

768

FEMALE 0.92628

BLACK 0.95307

HISPANIC 0.95189

ASIAN 0.91279

MEXICAN 0.91130

PUER RIC 0.96254

CUBN,OTH 0.96397

HISP-? 0.80692

MID CTY7 0.93456

FR/LCTY7 0.93130

FR/MCTY7 0.93924

LAR TWN7 0.90909

SML TWN7 0.94034

OTHER 0.92528

HS GRAD 0.94413

POST HS 0.94449

COL GRAD 0.94563

PARED-? 0.94188

S EAST 0.87014

CENTRAL 0.86238

WEST 0.88336

PRIVATE 0.90879

CATHOLIC 0.92133

BLACK 0.85377

HISPANIC 0.76016

ASIAN 0.74882

IEP-NO 0.90776

LEP-NO 0.81781

TITLE-N 0.77439

RED PRIC 0.92181

FREE 0.75380

INFO N/A 0.85570

SCH/REF 0.86186

SCH/NP 0.90161

TVLIN-0 0.98175

TV-QUAD 0.98167

HW-NO 0.98390

HW-YES 0.98520

HWLIN-0 0.98273

HWQUAD-0 0.97923

HITEM=3 0.94575

HITEM=4 0.97449

PGS>5 0.82955

PGS>10 0.82962

G/R 22 0.91021

G/R 23 0.90480

G/R 24 0.96274

G/T 22 0.71298

G/T 23 0.72771

G/T 24 0.74975

G/T 25 0.94790

G/T 26 0.72301

G/T 27 0.70027

G/P 22 0.94475

G/P 23 0.93965

G/P 24 0.74374

G/P 25 0.93859

G/S 22 0.93007

G/S 23 0.90185

R/T 24 0.90053

R/T 25 0.91144

R/T 26 0.92731

R/T 27 0.95530

R/T 31 0.91207

R/T 32 0.93387

R/T 33 0.90364

R/T 34 0.89985

R/T 35 0.91026

R/T 36 0.93874

R/T 37 0.92089

R/T 41 0.91700

R/T 42 0.92950

R/T 43 0.91914

R/T 44 0.93379

R/T 45 0.92807

R/T 46 0.94438

R/T 47 0.95159

R/P 24 0.90676

R/P 25 0.90483

R/P 31 0.91488

R/P 32 0.89881

R/P 33 0.90874

R/P 34 0.89696

R/P 35 0.89304

R/P 41 0.89477

R/P 42 0.96782

R/P 43 0.96287

R/P 44 0.95399

R/P 45 0.95120

R/S 31 0.95044

R/S 32 0.96000

R/S 33 0.96247

R/S 41 0.94798

R/S 42 0.95501

R/S 43 0.95510

P/T 25 0.72351

P/T 26 0.71971

P/T 27 0.76992

P/T 31 0.93957

P/T 32 0.72892

P/T 33 0.74133

P/T 34 0.73856

P/T 35 0.73890

P/T 36 0.81032

P/T 37 0.95000

P/T 41 0.75452

P/T 42 0.74692

P/T 43 0.83855

P/T 44 0.87364

P/T 45 0.87675

P/T 46 0.96140

P/T 47 0.84205

P/T 51 0.82294

P/T 52 0.73076

P/T 53 0.75765

P/T 54 0.78923

P/T 55 0.96239

P/T 56 0.77793

P/T 57 0.75745

T/S 41 0.93907

T/S 42 0.93488

T/S 43 0.94698

T/S 51 0.92564

T/S 52 0.94699

T/S 53 0.95233

T/S 61 0.96863

T/S 62 0.95136

T/S 63 0.94575

T/S 71 0.94858

P/S 32 0.95315

P/S 33 0.92456

P/S 41 0.95004

P/S 42 0.92638

P/S 43 0.92637

P/S 51 0.91069

P/S 52 0.94580

P/S 53 0.92271

SAMP S3 0.83925

S/R 22 0.88816

S/R 23 0.89965

S/R 24 0.96330

BLACK 0.92332

HISPANIC 0.82936

ASIAN AM 0.86267

AMER IND 0.97386

OTHER 0.95438

B003001M 0.75316
Table F-8 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Reading Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

769

B014601B 0.95658

B014601C 0.95758

B014601M 0.78688

B003201B 0.84763

B003201C 0.84646

B003201M 0.79767

B013201N 0.74480

B013201M 0.72712

B013301N *0.82246

B013301M 0.78303

B013401N 0.86286

B013401M 0.84829

B013501N 0.77472

B013501M 0.74417

B013601N 0.78895

B013601M 0.78177

B013701N 0.83211

B013701M 0.83416

B000901N 0.88857

B000901M 0.88612

B000903N 0.87263

B000903M 0.90415

B000904N 0.84825

B000904M 0.86494

B000905N 0.87257

B000905M 0.87975

S004001B 0.88141

S004001C 0.89989

S004001D 0.93650

S004001E 0.94786

S004001M 0.87649

B007301B 0.91620

B007301C 0.91502

B007301D 0.90310

B007301M 0.91379

B007401B 0.88168

B007401C 0.91232

B007401D 0.83302

B007401M 0.94347

B014501B 0.94875

B014501C 0.95468

B014501D 0.94315

B014501M 0.91382

R830301B 0.87989

R830301C 0.84610

R830301N 0.91282

R830301M 0.83199

R830401B 0.90460

R830401C 0.92523

R830401N 0.94199

R830401M 0.84463

RM00501B 0.85790

RM00501C 0.87746

RM00501D 0.87721

RM00501M 0.80797

R830501B 0.95123

R830501C 0.93966

R830501D 0.93502

R830501M 0.82531

R830502B 0.84817

R830502C 0.89197

R830502D 0.90167

R830502M 0.83029

R810801B 0.95270

R810801C 0.95963

R810801D 0.94975

R810801M 0.86397

R810201B 0.84563

R810201C 0.85108

R810201D 0.93135

R810201M 0.87667

R810901B 0.88955

R810901C 0.89756

R810901D 0.86104

R810901M 0.72278

R810902B 0.90628

R810902C 0.91598

R810902D 0.89186

R810902M 0.78257

R810903B 0.91407

R810903C 0.92877

R810903D 0.92784

R810903M 0.75943

R810904B 0.90282

R810904C 0.90158

R810904D 0.89567

R810904M 0.74597

R810905B 0.92892

R810905C 0.93860

R810905D 0.91736

R810905M 0.77399

R810906B 0.90056

R810906C 0.90137

R810906D 0.88905

R810906M 0.70272

R811005B 0.93956

R811005C 0.93782

R811005D 0.92966

R811005M 0.77775

R811006B 0.88944

R811006C 0.89682

R811006D 0.89611

R811006M 0.79032

R811007B 0.88462

R811007C 0.91932

R811007D 0.92341

R811007M 0.79994

R811009B 0.89117

R811009C 0.88895

R811009D 0.87111

R811009M 0.74680

R811002B 0.92925

R811002C 0.94188

R811002D 0.92611

R811002M 0.76614

R811004B 0.91914

R811004C 0.91339

R811004D 0.91452

R811004M 0.79021

R818101B 0.91092

R818101C 0.92504

R818101D 0.89325

R818101M 0.83062

R818102B 0.88899

R818102C 0.88819

R818102D 0.88388

R818102M 0.83619

R830001N 0.88434

R830001M 0.87663

R830101N 0.94992

R830101M 0.87425

R811301B 0.93891

R811301C 0.92657

R811301D 0.94903

R811301E 0.92614

R811301M 0.89768

R811302B 0.91965

R811302C 0.92919

R811302D 0.94411

R811302E 0.92825

R811302M 0.93049

R811303B 0.91004

R811303C 0.92281

R811303D 0.95804

R811303E 0.90352

R811303M 0.92395

R811304B 0.89172

R811304C 0.91075

R811304D 0.93738

R811304E 0.87954

R811304M 0.91665

C042501N 0.86706

C042501M 0.91515
Table F-8 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Reading Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

770

C042601B 0.86210

C042601C 0.88207

C042601M 0.95173

C042602B 0.86285

C042602C 0.88190

C042602M 0.96137

C042603B 0.86322

C042603C 0.87847

C042603D 0.87939

C042603M 0.92964

C042604B 0.87102

C042604C 0.91037

C042604D 0.88898

C042604N 0.85131

C042604M 0.91539

C042701Y 0.94252

C042701N 0.94868

C042701M 0.94016

C042801N 0.86703

C042801M 0.91524

C042802N 0.87656

C042802M 0.90476

C042803N 0.87103

C042803M 0.89473

C042901B 0.87919

C042901C 0.88722

C042901D 0.92288

C042901E 0.89066

C042901F 0.89872

C042901G 0.89286

C036601N 0.88440

C036601C 0.91519

C036601D 0.93828

C036601M 0.94393

C043001B 0.89769

C043001C 0.90845

C043001D 0.91907

C043001E 0.90475

C043001M 0.95985

C043002B 0.88816

C043002C 0.87632

C043002D 0.93038

C043002E 0.95446

C043002M 0.98085

C043003B 0.88769

C043003C 0.88139

C043003D 0.92795

C043003E 0.94523

C043004B 0.91028

C043004C 0.90734

C043004D 0.87888

C043004E 0.87202

C043004M 0.98066

C043005B 0.89781

C043005C 0.91582

C043005D 0.89847

C043005E 0.89052

C043005M 0.98558

C043006B 0.89011

C043006C 0.90027

C043006D 0.87666

C043006E 0.87661

C043006M 0.98937

C043007B 0.90713

C043007C 0.89073

C043007D 0.89631

C043007E 0.89424

C043008B 0.89885

C043008C 0.89156

C043008D 0.88621

C043008E 0.88421

C032402B 0.91383

C032402C 0.93303

C032402N 0.92774

C032402M 0.97513

C032401B 0.91209

C032401C 0.93328

C032401N 0.92631

C032404B 0.89928

C032404C 0.93742

C032404N 0.92020

C032404M 0.99525

C032407B 0.88120

C032407C 0.86168

C032408B 0.89640

C032408C 0.94827

C032408N 0.92473

C032408M 0.98120

C032409B 0.88853

C032409C 0.91790

C032409N 0.90994

C032409M 0.96823

C032410C 0.90846

C032411B 0.91149

C032411C 0.92628

C032411N 0.95323

C032412B 0.91569

C032412C 0.90764

C032413B 0.90907

C032413C 0.86486

C032414B 0.90611

C032414C 0.94378

C032414N 0.90190

C032414M 0.98384

C043101B 0.87480

C043101C 0.94895

C043101N 0.94752

C043102B 0.88900

C043102C 0.94003

C043102N 0.94266

C043103C 0.93437

C043103N 0.94832

C043104B 0.87232

C043104C 0.94982

C043104N 0.94924

C032502B 0.87721

C032502C 0.88962

C032503B 0.86411

C032503C 0.89815

C032503D 0.88718

C032505B 0.88158

C032505C 0.87829

C032505D 0.89413

C032505M 0.97286

C032506B 0.87922

C032506C 0.88567

C043201B 0.88754

C043201C 0.88923

C043301B 0.89834

C043301C 0.90737

C043301D 0.88769

C043301M 0.96374

C043401B 0.87043

C043401C 0.88540

C043401D 0.88473

C043501B 0.89433

C043501C 0.88847

C043501D 0.89312

C043501E 0.88921

C043501F 0.88692

C043501M 0.97203

C043601B 0.85716

C043601C 0.86306

C043601D 0.86510

C043601M 0.97090

C043701B 0.86208

C043701C 0.85297

C043701D 0.88800

C043701E 0.86648

C038301N 0.90735
Table F-8 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Reading Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

771

C038301M 0.94521

C043801B 0.88580

C043801C 0.87251

C043801D 0.90383

C043801E 0.90444

C043801F 0.89651

C043801G 0.89022

C043801H 0.87385

C043801M 0.92145

C043901N 0.91029

C043901M 0.95197

C044001B 0.87942

C044001C 0.89393

C044001D 0.90205

C044001E 0.87631

C044001F 0.86634

C044001G 0.87567

C044001H 0.90223

C044001M 0.92299

C044002B 0.87495

C044002C 0.89534

C044002D 0.91167

C044002E 0.89428

C044002F 0.89140

C044002G 0.87944

C044002H 0.91421

C044002M 0.94647

C044003B 0.87137

C044003C 0.87728

C044003D 0.88421

C044003E 0.88389

C044003F 0.89517

C044003G 0.89444

C044003H 0.92104

C044003M 0.95104

C044004B 0.90419

C044004C 0.90170

C044004D 0.89931

C044004E 0.87260

C044004G 0.91410

C044004H 0.88557

C044004M 0.93332

T067001M 0.89347

T067002M 0.88211

T067003M 0.88928

T067004M 0.84686

T067101B 0.89074

T067101C 0.88781

T067101D 0.91626

T067101E 0.93430

T067101M 0.96044

T067201B 0.94406

T067201C 0.94622

T067201D 0.95786

T067201E 0.96152

T067201M 0.96630

T067202B 0.93375

T067202C 0.94325

T067202D 0.94975

T067202E 0.94678

T067202M 0.95821

T067203B 0.94605

T067203C 0.94895

T067203D 0.96387

T067203E 0.96516

T067203M 0.95434

T067204B 0.89331

T067204C 0.88444

T067204D 0.88986

T067204E 0.89253

T067204M 0.88761

T067205B 0.89204

T067205C 0.88559

T067205D 0.91382

T067205E 0.93192

T067205M 0.91813

T067206B 0.85764

T067206C 0.84912

T067206D 0.85802

T067206E 0.86327

T067206M 0.86759

T067301B 0.91344

T067301C 0.85656

T067301D 0.83388

T067301M 0.94066

T056201B 0.88342

T056201C 0.89167

T056201D 0.88017

T056201E 0.87342

T056201F 0.85852

T056201M 0.89638

T056301B 0.90525

T056301C 0.94644

T056301D 0.93300

T056301E 0.88886

T056301F 0.88856

T056301G 0.91012

T056301M 0.95209

T067501B 0.86109

T067501C 0.86894

T067501M 0.87964

T067502B 0.87778

T067502C 0.93992

T067502M 0.97225

T067503B 0.87672

T067503C 0.94230

T067503M 0.96585

T067504B 0.89351

T067504C 0.97773

T067504M 0.98615

T067505B 0.89691

T067505C 0.98614

T067505M 0.98700

T067506B 0.86280

T067506C 0.95095

T067506M 0.99112

T067507B 0.86122

T067507C 0.98128

T067507M 0.99135

T067508B 0.89776

T067508C 0.90943

T067508M 0.95105

T067509B 0.89139

T067509C 0.92792

T067509M 0.94938

T067510B 0.89094

T067510C 0.94201

T067510M 0.96332

T067511B 0.87672

T067511C 0.97303

T067511M 0.98135

T067512B 0.89497

T067512C 0.89354

T067512M 0.87277

T067601B 0.85068

T067601C 0.80874

T067601M 0.89270

T067602B 0.93338

T067602C 0.98032

T067602M 0.97722

T067603B 0.87929

T067603C 0.93263

T067603M 0.97588

T067604B 0.91360

T067604C 0.98360

T067604M 0.98730

T067605B 0.87052

T067605C 0.92621

T067605M 0.97407

T067606B 0.84360

T067606C 0.91290

T067606M 0.96521

T067607B 0.85733
Table F-8 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Reading Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

772

T067607C 0.96492

T067607M 0.97318

T067608B 0.85562

T067608C 0.96731

T067608M 0.98397

T067609B 0.87524

T067609C 0.89762

T067609M 0.95847

T067610B 0.85283

T067610C 0.97910

T067610M 0.98413

T067611B 0.87388

T067611C 0.99489

T067611M 0.99094

T067612B 0.87542

T067612C 0.89424

T067612M 0.89891

T067701B 0.90730

T067701C 0.91856

T067701D 0.90589

T067701E 0.88700

T067701M 0.95399

T067702B 0.85943

T067702C 0.86617

T067702D 0.87432

T067702E 0.87179

T067702M 0.88192

T067801B 0.88423

T067801C 0.89761

T067801M 0.90525

T067802B 0.88293

T067802C 0.87879

T067802M 0.98708

T067803B 0.83030

T067803C 0.84850

T067803M 0.97425

T067804B 0.88049

T067804C 0.87073

T067804M 0.97815

T067805B 0.91266

T067805C 0.93156

T067805M 0.97389

T067806B 0.88682

T067806C 0.88361

T067806M 0.98086

T067807B 0.85015

T067807C 0.88704

T067807M 0.97843

T041201B 0.87723

T041201C 0.89159

T041201D 0.87491

T041201M 0.98493

T067901B 0.84404

T067901C 0.85204

T067901M 0.97199

T067902B 0.84804

T067902C 0.85642

T067902M 0.95933

T068001B 0.84319

T068001C 0.86481

T068001M 0.97374

T068002B 0.82330

T068002C 0.82785

T068002M 0.95189

T068003B 0.85305

T068003C 0.86668

T068003M 0.97532

T068004B 0.85927

T068004C 0.85635

T068004M 0.96887

T068005B 0.85665

T068005C 0.84475

T068005M 0.96962

T068006B 0.85948

T068006C 0.85826

T068006M 0.96043

T068007B 0.87715

T068007C 0.91181

T068007M 0.97896

T068008B 0.84827

T068008C 0.86753

T068008M 0.97839

T068009B 0.88046

T068009C 0.89851

T068009M 0.96812

T068010B 0.85445

T068010C 0.83568

T068010M 0.97561

T068101B 0.88760

T068101C 0.90374

T068101D 0.88266

T068101E 0.83922

T068101M 0.97404

T046101N 0.84667

T046101M 0.97155

T046201B 0.89903

T046201C 0.88667

T046201D 0.91646

T046201M 0.95225

T068201B 0.90581

T068201C 0.93433

T068201D 0.92744

T068201E 0.87694

T068201M 0.96782

T068301B 0.87401

T068301C 0.86053

T068301D 0.86811

T068301E 0.88910

T068301M 0.85064

T068401B 0.87700

T068401C 0.88555

T068401D 0.88698

T068401E 0.89155

T068401F 0.90045

T068401G 0.84200

T068401M 0.87650

T068601B 0.92077

T068601C 0.91078

T068601D 0.91942

T068601M 0.96903

T068701B 0.90376

T068701C 0.90386

T068701D 0.92571

T068701E 0.92016

T068701M 0.97327

T068801B 0.92542

T068801C 0.93716

T068801D 0.91621

T068801M 0.94768

T068802B 0.87743

T068802C 0.89648

T068802D 0.90087

T068802M 0.92013

T068803B 0.86560

T068803C 0.90377

T068803D 0.90812

T068803M 0.97091

T068804B 0.85139

T068804C 0.86643

T068804D 0.85178

T068804M 0.94621

T068805B 0.85524

T068805C 0.84710

T068805D 0.85964

T068805M 0.94281

T068901B 0.88405

T068901C 0.87330

T068901D 0.86959

T068901M 0.90342

T069001B 0.88213
Table F-8 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Reading Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

773

T069001C 0.88984

T069001D 0.90530

T069001E 0.90064

T069001M 0.85770

T069101B 0.85310

T069101C 0.87030

T069101D 0.84848

T069101M 0.97382

T069102B 0.88190

T069102C 0.88592

T069102D 0.86653

T069102M 0.96449

T069103B 0.84638

T069103C 0.90407

T069103D 0.88107

T069103M 0.94796

T069201B 0.89288

T069201C 0.89086

T069201D 0.83997

T069201M 0.97072

T069202B 0.89594

T069202C 0.91413

T069202D 0.85916

T069202M 0.96835

T069203B 0.87634

T069203C 0.93299

T069203D 0.92282

T069203M 0.96225

T069301B 0.83062

T069301N 0.88306

T069301M 0.94049

T069302B 0.83553

T069302N 0.87606

T069302M 0.96346

T069303B 0.84902

T069303N 0.86552

T069303M 0.96549

T069304B 0.84916

T069304N 0.86552

T069304M 0.95498

T069305B 0.87194

T069305N 0.87139

T069305M 0.96855

T069401B 0.85541

T069401C 0.91247

T069401D 0.84662

T069401M 0.98980

T069402B 0.85593

T069402C 0.85946

T069402D 0.87909

T069402M 0.97014

T069403B 0.87923

T069403C 0.85715

T069403D 0.85532

T069403M 0.96111

T069404B 0.86987

T069404C 0.86947

T069404D 0.86018

T069404M 0.96791

T069405B 0.88542

T069405C 0.86991

T069405D 0.88785

T069405M 0.95887

T069501B 0.90424

T069501C 0.93297

T069501D 0.90768

T069501E 0.86159

T069501M 0.97937

T069601B 0.90314

T069601C 0.90246

T069601D 0.91732

T069601M 0.97759

T069701B 0.83875

T069701C 0.85624

T069701D 0.87816

T069701M 0.98434

T069702B 0.86526

T069702C 0.87379

T069702D 0.85797

T069702M 0.98162

T069703B 0.86382

T069703C 0.87545

T069703D 0.84843

T069703M 0.96767

T069704B 0.85608

T069704C 0.87761

T069704D 0.85621

T069704M 0.95287

T069705B 0.86622

T069705C 0.85939

T069705D 0.90017

T069705M 0.97968

T069706B 0.84285

T069706C 0.85369

T069706D 0.84245

T069706M 0.98573

T069707B 0.90221

T069707C 0.92822

T069707D 0.87587

T069707M 0.98554

T069708B 0.87717

T069708C 0.90196

T069708D 0.87361

T069708M 0.96329

T069709B 0.83350

T069709C 0.86183

T069709D 0.85606

T069709M 0.97248

T069710B 0.91787

T069710C 0.92818

T069710D 0.87576

T069710M 0.97970

T069711B 0.91370

T069711C 0.93754

T069711D 0.91302

T069711M 0.94334

T069712B 0.84078

T069712C 0.85975

T069712M 0.96505

T069713B 0.83445

T069713C 0.85810

T069713D 0.86265

T069713M 0.97930

T069714B 0.88003

T069714C 0.85546

T069714D 0.88770

T069714M 0.96413

T069715B 0.86855

T069715C 0.87757

T069715D 0.86807

T069715M 0.98976

T069716B 0.90154

T069716C 0.90864

T069716D 0.85448

T069716M 0.95816

T071801B 0.91534

T071801C 0.92655

T071801D 0.89370

T071801M 0.96023

T071802B 0.87402

T071802C 0.88551

T071802D 0.84789

T071802M 0.96920

T071803B 0.88437

T071803C 0.91129

T071803D 0.88301

T071803M 0.96105

T071804B 0.87797

T071804C 0.92133

T071804D 0.91392

T071804M 0.96659

T071805B 0.90731

T071805C 0.91723
Table F-8 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Reading Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

774

T071805D 0.86105

T071805M 0.97613

T071806B 0.89447

T071806C 0.89563

T071806D 0.86442

T071806M 0.97160

T071807B 0.85958

T071807C 0.88429

T071807D 0.86079

T071807M 0.98035

T071808B 0.89151

T071808C 0.90989

T071808D 0.87678

T071808M 0.97412

T071809B 0.85428

T071809C 0.85776

T071809D 0.87568

T071809M 0.98756

T071810B 0.90564

T071810C 0.93532

T071810D 0.90010

T071810M 0.96100

T071811B 0.89721

T071811C 0.92287

T071811D 0.89700

T071811M 0.95758

T071812B 0.86810

T071812C 0.88415

T071812D 0.89228

T071812M 0.97386

T071813B 0.86450

T071813C 0.88997

T071813D 0.86270

T071813M 0.96782

T069901B 0.86047

T069901C 0.86284

T069901D 0.83807

T069901M 0.94007

T069902B 0.89617

T069902C 0.91788

T069902D 0.88183

T069902M 0.95901

T069903B 0.88731

T069903C 0.93325

T069903D 0.94557

T069903M 0.95889

T070001B 0.86761

T070001C 0.89911

T070001D 0.89884

T070001M 0.98602

T070002B 0.88316

T070002C 0.92605

T070002D 0.91262

T070002M 0.97431

T070003B 0.86705

T070003C 0.87717

T070003D 0.89502

T070003M 0.98401

T070101B 0.88472

T070101C 0.87235

T070101D 0.86267

T070101M 0.96714

T070102B 0.86421

T070102C 0.86842

T070102D 0.84942

T070102M 0.96014

T070103B 0.86328

T070103C 0.87070

T070103D 0.85038

T070103M 0.94464

T070104B 0.90070

T070104C 0.91396

T070104D 0.86893

T070104M 0.96879

T070105B 0.89006

T070105C 0.88878

T070105D 0.89692

T070105M 0.94416

T070106B 0.89429

T070106C 0.91756

T070106D 0.89985

T070106M 0.96151

T070107B 0.86296

T070107C 0.87438

T070107D 0.85925

T070107M 0.97317

T070201B 0.90272

T070201C 0.88546

T070201D 0.89925

T070201M 0.94121

T070202B 0.85865

T070202C 0.85592

T070202D 0.84777

T070202M 0.97055

T070203B 0.90399

T070203C 0.91495

T070203D 0.87945

T070203M 0.96630

T070204B 0.85512

T070204C 0.86661

T070204D 0.88739

T070204M 0.96474

T070301B 0.84249

T070301C 0.86748

T070301M 0.96664

T070302B 0.84155

T070302C 0.90085

T070303B 0.84748

T070303C 0.88062

T070304B 0.94514

T070304C 0.94800

T070304M 0.95792

T070305B 0.84813

T070305C 0.90866
Table F-9

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Reading Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

775

FEMALE 0.91863

BLACK 0.95705

HISPANIC 0.96148

ASIAN 0.93863

MEXICAN 0.93211

PUER RIC 0.95272

CUBN,OTH 0.96588

HISP-? 0.97031

MID CTY7 0.91586

FR/LCTY7 0.93033

FR/MCTY7 0.93499

LAR TWN7 0.90124

SML TWN7 0.92933

OTHER 0.93436

HS GRAD 0.94736

POST HS 0.93850

COL GRAD 0.95215

PARED-? 0.94669

S EAST 0.87562

CENTRAL 0.87726

WEST 0.87890

PRIVATE 0.92919

CATHOLIC 0.92090

BLACK 0.89586

HISPANIC 0.84856

ASIAN 0.80463

IEP-NO 0.82519

LEP-NO 0.82366

TITLE-N 0.77836

RED PRIC 0.93952

FREE 0.73773

INFO N/A 0.86303

SCH/REF 0.88012

SCH/NP 0.88317

TVLIN-0 0.98268

TV-QUAD 0.98224

HW-NO 0.97717

HW-YES 0.97905

HWLIN-0 0.97361

HWQUAD-0 0.96858

HITEM=3 0.89769

HITEM=4 0.98003

PGS>5 0.80949

PGS>10 0.80448

G/R 22 0.89749

G/R 23 0.90602

G/R 24 0.94787

G/T 22 0.67768

G/T 23 0.70935

G/T 24 0.73244

G/T 25 0.96753

G/T 26 0.72234

G/T 27 0.69531

G/P 22 0.87980

G/P 23 0.89785

G/P 24 0.83762

G/P 25 0.79481

G/S 22 0.95234

G/S 23 0.93632

R/T 24 0.91006

R/T 25 0.90745

R/T 26 0.91621

R/T 27 0.97079

R/T 31 0.90578

R/T 32 0.93125

R/T 33 0.90665

R/T 34 0.90406

R/T 35 0.91257

R/T 36 0.95062

R/T 37 0.91252

R/T 41 0.91865

R/T 42 0.91723

R/T 43 0.92475

R/T 44 0.93461

R/T 45 0.92717

R/T 46 0.93824

R/T 47 0.94413

R/P 24 0.89406

R/P 25 0.89686

R/P 31 0.90416

R/P 32 0.84710

R/P 33 0.90059

R/P 34 0.90148

R/P 35 0.90744

R/P 41 0.87693

R/P 42 0.96567

R/P 43 0.95874

R/P 44 0.94964

R/P 45 0.93140

R/S 31 0.96422

R/S 32 0.95771

R/S 33 0.96995

R/S 41 0.94842

R/S 42 0.96487

R/S 43 0.94935

P/T 25 0.81136

P/T 26 0.72670

P/T 27 0.75567

P/T 31 0.97900

P/T 32 0.78921

P/T 33 0.79005

P/T 34 0.80269

P/T 35 0.73589

P/T 36 0.78005

P/T 37 0.97507

P/T 41 0.80984

P/T 42 0.80567

P/T 43 0.79755

P/T 44 0.81877

P/T 45 0.82220

P/T 46 0.97519

P/T 47 0.82315

P/T 51 0.80533

P/T 52 0.85202

P/T 53 0.81801

P/T 54 0.78615

P/T 55 0.98560

P/T 56 0.85477

P/T 57 0.86230

T/S 41 0.93824

T/S 42 0.92253

T/S 43 0.93226

T/S 51 0.91938

T/S 52 0.94786

T/S 53 0.94906

T/S 61 0.97389

T/S 62 0.95477

T/S 63 0.93737

T/S 71 0.93577

T/S 72 0.94114

P/S 32 0.93915

P/S 33 0.90833

P/S 41 0.92178

P/S 42 0.89249

P/S 43 0.92791

P/S 51 0.89801

P/S 52 0.96355

P/S 53 0.93603

SAMP S3 0.83770

S/R 22 0.89296

S/R 23 0.90511

S/R 24 0.95462

BLACK 0.94032

HISPANIC 0.93417

ASIAN AM 0.92572

AMER IND 0.96379

OTHER 0.96543

B003001M 0.96838

B014601B 0.96952

B014601C 0.94112
Table F-9 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Reading Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

776

B014601M 0.86354

B003201B 0.86107

B003201C 0.79038

B003201M 0.88567

B000901N 0.83576

B000901M 0.90876

B000903N 0.89396

B000903M 0.91538

B000904N 0.87719

B000904M 0.91701

B000905N 0.91464

B000905M 0.92531

S004001B 0.85539

S004001C 0.86200

S004001D 0.91163

S004001E 0.93782

S004001M 0.72403

B007301B 0.96557

B007301C 0.96654

B007301D 0.95969

B007301M 0.82873

B007401B 0.91488

B007401C 0.91822

B007401D 0.84838

B007401M 0.89220

B014501B 0.93222

B014501C 0.93729

B014501D 0.92877

B014501M 0.83893

R830301B 0.93938

R830301C 0.93592

R830301N 0.94011

R830301M 0.87582

R830401B 0.91890

R830401C 0.91528

R830401N 0.90695

R830401M 0.86518

RM00501B 0.84859

RM00501C 0.85643

RM00501D 0.91023

RM00501M 0.90814

R830501B 0.96067

R830501C 0.95322

R830501D 0.95171

R830501M 0.85710

R830502B 0.83615

R830502C 0.89617

R830502D 0.88876

R830502M 0.83173

R810801B 0.89777

R810801C 0.91967

R810801D 0.93456

R810801M 0.85336

R810201B 0.89308

R810201C 0.87783

R810201D 0.93005

R810201M 0.73802

R810901B 0.92870

R810901C 0.93287

R810901D 0.91208

R810901M 0.83805

R810902B 0.94470

R810902C 0.95554

R810902D 0.94903

R810902M 0.71952

R810903B 0.96320

R810903C 0.97475

R810903D 0.96695

R810903M 0.73054

R810904B 0.94009

R810904C 0.93875

R810904D 0.93073

R810904M 0.69667

R810905B 0.90483

R810905C 0.91763

R810905D 0.91090

R810905M 0.73206

R810906B 0.88183

R810906C 0.87166

R810906D 0.88824

R810906M 0.72633

R811005B 0.96434

R811005C 0.97036

R811005D 0.96661

R811005M 0.76553

R811006B 0.91043

R811006C 0.90737

R811006D 0.91368

R811006M 0.74494

R811007B 0.88365

R811007C 0.88737

R811007D 0.90187

R811007M 0.91058

R811009B 0.92903

R811009C 0.93521

R811009D 0.91117

R811009M 0.70486

R811002B 0.96090

R811002C 0.96736

R811002D 0.96386

R811002M 0.70313

R811004B 0.92840

R811004C 0.92085

R811004D 0.93110

R811004M 0.68243

R818101B 0.94416

R818101C 0.95015

R818101D 0.93612

R818101M 0.66443

R818102B 0.92122

R818102C 0.90885

R818102D 0.91408

R818102M 0.70298

R830001N 0.92389

R830001M 0.82034

R811301B 0.90102

R811301C 0.94916

R811301D 0.94538

R811301E 0.88166

R811301M 0.90263

R811302B 0.90414

R811302C 0.95623

R811302D 0.95060

R811302E 0.90752

R811302M 0.92864

R811303B 0.92655

R811303C 0.92981

R811303D 0.95300

R811303E 0.92281

R811303M 0.91901

R811304B 0.93010

R811304C 0.92626

R811304D 0.94250

R811304E 0.86650

R811304M 0.89327

C042701Y 0.91267

C042701N 0.90735

C042701M 0.90625

C042801N 0.87526

C042801M 0.88725

C042802N 0.87018

C042802M 0.91528

C042803N 0.87575

C042803M 0.89022

C042901B 0.88318

C042901C 0.90190

C042901D 0.91875

C042901E 0.90476

C042901F 0.91165

C042901G 0.91159

C042901M 0.96124

C036601N 0.88845

C036601C 0.91376
Table F-9 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Reading Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

777

C036601D 0.93745

C036601M 0.97837

C043001B 0.90832

C043001C 0.90321

C043001D 0.89904

C043001E 0.88616

C043001M 0.97449

C043002B 0.89752

C043002C 0.89756

C043002D 0.93120

C043002E 0.93766

C043002M 0.94440

C043003B 0.88581

C043003C 0.89286

C043003D 0.92671

C043003E 0.92552

C043003M 0.95868

C043004B 0.89890

C043004C 0.89905

C043004D 0.86507

C043004E 0.87037

C043004M 0.94055

C043005B 0.93297

C043005C 0.92825

C043005D 0.91753

C043005E 0.88926

C043005M 0.97850

C043006B 0.85907

C043006C 0.88023

C043006D 0.85684

C043006E 0.92890

C043007B 0.89021

C043007C 0.89233

C043007D 0.88127

C043007E 0.86145

C043008B 0.88297

C043008C 0.87043

C043008D 0.91205

C043008E 0.86125

C032402B 0.89241

C032402C 0.92602

C032402N 0.91658

C032402M 0.97966

C032401B 0.90668

C032401C 0.94216

C032401N 0.90636

C032401M 0.97274

C032404B 0.90049

C032404C 0.93799

C032404N 0.89956

C032404M 0.93112

C032407B 0.87310

C032407C 0.94242

C032407N 0.93834

C032407M 0.98607

C032408B 0.87401

C032408C 0.95184

C032408N 0.93834

C032408M 0.96769

C032409B 0.88849

C032409C 0.92175

C032409N 0.90418

C032409M 0.98104

C032410B 0.90897

C032410C 0.93283

C032410N 0.93306

C032410M 0.96299

C032411B 0.92399

C032411C 0.93338

C032411N 0.93532

C032411M 0.96939

C032412B 0.91263

C032412C 0.93198

C032412N 0.93896

C032413B 0.87430

C032413C 0.94145

C032413N 0.94509

C032413M 0.99554

C032414B 0.92051

C032414C 0.94278

C032414N 0.87080

C032414M 0.96624

C043101B 0.88794

C043101C 0.94785

C043101N 0.93543

C043102B 0.89014

C043102C 0.93573

C043102N 0.92517

C043103B 0.88235

C043103C 0.87466

C043104B 0.86881

C043104C 0.95480

C043104N 0.94198

C043104M 0.97017

C032502B 0.87493

C032502C 0.89193

C032502D 0.88420

C032502M 0.96275

C032503B 0.88198

C032503C 0.88120

C032503D 0.86715

C032503M 0.96694

C032505B 0.87885

C032505C 0.89049

C032505D 0.88137

C032505M 0.97895

C032506B 0.86318

C032506C 0.89064

C032506D 0.88601

C032506M 0.95781

C043201B 0.86344

C043201C 0.86471

C043301B 0.88654

C043301C 0.90992

C043301D 0.87510

C043301M 0.94699

C043401B 0.89217

C043401C 0.87829

C043401M 0.95808

C043501B 0.88432

C043501C 0.88798

C043501D 0.88849

C043501E 0.87233

C043501F 0.86884

C043501M 0.94438

C043601B 0.87225

C043601C 0.86533

C043601D 0.88971

C043601E 0.84996

C043601M 0.94016

C043701B 0.85742

C043701C 0.88308

C043701D 0.89284

C043701E 0.88067

C043701M 0.95440

C038301N 0.88624

C038301M 0.94784

C043801B 0.90074

C043801C 0.88508

C043801D 0.91383

C043801E 0.90373

C043801F 0.89845

C043801G 0.89410

C043801H 0.87943

C043801M 0.87178

C043901N 0.92868

C043901M 0.95365

C044001B 0.88896

C044001C 0.88227

C044001D 0.87842

C044001E 0.88004

C044001F 0.88313

C044001G 0.87380
Table F-9 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Reading Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

778

C044001H 0.89032

C044001M 0.93730

C044002B 0.90232

C044002C 0.89913

C044002D 0.89012

C044002E 0.89903

C044002F 0.92336

C044002G 0.86251

C044002H 0.88598

C044002M 0.93534

C044003B 0.89403

C044003C 0.88758

C044003D 0.87957

C044003E 0.90171

C044003F 0.92736

C044003H 0.88160

C044003M 0.93744

C044004B 0.88582

C044004C 0.88889

C044004D 0.90041

C044004E 0.87206

C044004F 0.89495

C044004G 0.88326

C044004H 0.87535

C044004M 0.93394

B003501B 0.86342

B003501C 0.87816

B003501D 0.90019

B003501M 0.82188

B003601B 0.90219

B003601C 0.89128

B003601D 0.89206

B003601M 0.82192

R811010B 0.90374

R811010C 0.90932

R811010D 0.86628

R811010M 0.79875

R811011B 0.91242

R811011C 0.93107

R811011D 0.89797

R811011M 0.82907

R830201N 0.94539

R830201M 0.82114

C044401N 0.90040

C044401M 0.96247

C044402N 0.89351

C044402M 0.92385

C043105B 0.87303

C043105C 0.93898

C043105N 0.94872

C043106B 0.86420

C043106C 0.87430

T067301B 0.83002

T067301C 0.86414

T067301D 0.89140

T067301M 0.91554

T056201B 0.88342

T056201C 0.87072

T056201D 0.88856

T056201E 0.88150

T056201F 0.87515

T056201M 0.87898

T056301B 0.95051

T056301C 0.94794

T056301D 0.94810

T056301E 0.88334

T056301F 0.91644

T056301G 0.94744

T056301M 0.96556

T067501B 0.86504

T067501C 0.89089

T067501M 0.88303

T067502B 0.86582

T067502C 0.86214

T067502M 0.86952

T067503B 0.84525

T067503C 0.96109

T067503M 0.96247

T067504B 0.85746

T067504C 0.97534

T067504M 0.98492

T067505B 0.86946

T067505C 0.97471

T067505M 0.98693

T067506B 0.82962

T067506C 0.95045

T067506M 0.98234

T067507B 0.88695

T067507C 0.96962

T067507M 0.96693

T067508B 0.88640

T067508C 0.85950

T067508M 0.86852

T067509B 0.87765

T067509C 0.87168

T067509M 0.94120

T067510B 0.88570

T067510C 0.89277

T067510M 0.90356

T067511B 0.85688

T067511C 0.94819

T067511M 0.95333

T067512B 0.88722

T067512C 0.87310

T067512M 0.85460

T067601B 0.88495

T067601C 0.91035

T067601M 0.91999

T067602B 0.85188

T067602C 0.86886

T067602M 0.89088

T067603B 0.88561

T067603C 0.95175

T067603M 0.96833

T067604B 0.87053

T067604C 0.97440

T067604M 0.96926

T067605B 0.87513

T067605C 0.89826

T067605M 0.95741

T067606B 0.86640

T067606C 0.91909

T067606M 0.95379

T067607B 0.87522

T067607C 0.96027

T067607M 0.97103

T067608B 0.84801

T067608C 0.87632

T067608M 0.91414

T067609B 0.84488

T067609C 0.88152

T067609M 0.93488

T067610B 0.81574

T067610C 0.96393

T067610M 0.97038

T067611B 0.82538

T067611C 0.98191

T067611M 0.98988

T067612B 0.85234

T067612C 0.87122

T067612M 0.88046

T067701B 0.87608

T067701C 0.89319

T067701D 0.88047

T067701E 0.87131

T067701M 0.93651

T067702B 0.86273

T067702C 0.87397

T067702D 0.87391

T067702E 0.91071

T067702M 0.84583

T067801B 0.89125

T067801C 0.90150
Table F-9 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Reading Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

779

T067801M 0.89699

T067802B 0.89168

T067802C 0.86769

T067802M 0.98843

T067803B 0.84137

T067803C 0.86063

T067803M 0.98661

T067804B 0.86815

T067804C 0.87088

T067804M 0.99456

T067805B 0.89829

T067805C 0.90638

T067805M 0.98077

T067806B 0.89455

T067806C 0.89827

T067806M 0.97319

T067807B 0.85303

T067807C 0.91288

T067807M 0.97919

T041201B 0.88121

T041201C 0.89116

T041201D 0.88708

T041201M 0.98702

T067901B 0.84173

T067901C 0.84517

T067901M 0.96585

T067902B 0.82784

T067902C 0.86096

T067902M 0.96879

T068001B 0.84328

T068001C 0.84198

T068001M 0.97641

T068002B 0.83639

T068002C 0.85688

T068002M 0.97668

T068003B 0.84056

T068003C 0.85292

T068003M 0.98727

T068004B 0.87965

T068004C 0.86226

T068004M 0.97341

T068005B 0.89143

T068005C 0.88305

T068005M 0.96533

T068006B 0.85841

T068006C 0.86404

T068006M 0.98535

T068007B 0.87671

T068007C 0.91028

T068007M 0.96737

T068008B 0.85769

T068008C 0.86591

T068008M 0.98362

T068009B 0.87730

T068009C 0.89651

T068009M 0.98792

T068010B 0.84496

T068010C 0.84611

T068010M 0.99236

T046101N 0.91564

T046101M 0.94114

T046201B 0.86898

T046201C 0.86702

T046201D 0.89382

T046201M 0.95852

T068201B 0.84231

T068201C 0.87574

T068201D 0.85639

T068201E 0.80977

T068201M 0.94873

T068301B 0.87933

T068301C 0.88669

T068301D 0.87810

T068301E 0.90639

T068301M 0.90427

T068401B 0.85730

T068401C 0.88675

T068401D 0.90437

T068401E 0.91011

T068401F 0.89349

T068401G 0.86844

T068401M 0.90481

T068601B 0.86843

T068601C 0.87970

T068601D 0.89223

T068601M 0.95419

T068701B 0.88813

T068701C 0.89934

T068701D 0.89729

T068701E 0.89796

T068701M 0.95857

T068801B 0.88291

T068801C 0.91982

T068801D 0.92210

T068801M 0.94355

T068802B 0.89614

T068802C 0.89825

T068802D 0.93186

T068802M 0.93821

T068803B 0.88705

T068803C 0.88371

T068803D 0.93109

T068803M 0.93770

T068804B 0.85784

T068804C 0.85365

T068804D 0.84838

T068804M 0.95194

T068805B 0.88549

T068805C 0.91479

T068805D 0.89430

T068805M 0.94075

T068901B 0.88545

T068901C 0.90010

T068901D 0.89132

T068901M 0.88949

T069001B 0.90524

T069001C 0.89497

T069001D 0.87514

T069001E 0.88597

T069001M 0.88396

T069101B 0.85247

T069101C 0.86399

T069101D 0.82160

T069101M 0.95206

T069102B 0.86901

T069102C 0.91494

T069102D 0.88672

T069102M 0.94318

T069103B 0.85310

T069103C 0.89956

T069103D 0.90502

T069103M 0.92636

T069201B 0.88895

T069201C 0.91200

T069201D 0.85487

T069201M 0.96578

T069202B 0.88142

T069202C 0.91842

T069202D 0.86672

T069202M 0.96520

T069203B 0.86572

T069203C 0.92425

T069203D 0.90071

T069203M 0.95656

T069301B 0.85281

T069301N 0.86090

T069301M 0.95597

T069302B 0.85273

T069302N 0.86516

T069302M 0.97341

T069303B 0.84798

T069303N 0.85284

T069303M 0.95773
Table F-9 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Reading Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

780

T069304B 0.85491

T069304N 0.85074

T069304M 0.95931

T069305B 0.91037

T069305N 0.91549

T069305M 0.95071

T069401B 0.85503

T069401C 0.85857

T069401D 0.84286

T069401M 0.97799

T069402B 0.85892

T069402C 0.88282

T069402D 0.83477

T069402M 0.98243

T069403B 0.87040

T069403C 0.87269

T069403D 0.87443

T069403M 0.97396

T069404B 0.88140

T069404C 0.92623

T069404D 0.93087

T069404M 0.96773

T069405B 0.88547

T069405C 0.90509

T069405D 0.92905

T069405M 0.97787

T069501B 0.88031

T069501C 0.92756

T069501D 0.91295

T069501E 0.88265

T069501M 0.97747

T069601B 0.89030

T069601C 0.89332

T069601D 0.88509

T069601M 0.97302

T069701B 0.85238

T069701C 0.85192

T069701D 0.85676

T069701M 0.97467

T069702B 0.87784

T069702C 0.90081

T069702D 0.84641

T069702M 0.97512

T069703B 0.88220

T069703C 0.88884

T069703D 0.83626

T069703M 0.97541

T069704B 0.89892

T069704C 0.90709

T069704D 0.88821

T069704M 0.97695

T069705B 0.84927

T069705C 0.87745

T069705D 0.84580

T069705M 0.97948

T069706B 0.87478

T069706C 0.89092

T069706D 0.87236

T069706M 0.96697

T069707B 0.88440

T069707C 0.94875

T069707D 0.89066

T069707M 0.96409

T069708B 0.87726

T069708C 0.90404

T069708D 0.84441

T069708M 0.97534

T069709B 0.87554

T069709C 0.86711

T069709D 0.83957

T069709M 0.97969

T069710B 0.91163

T069710C 0.91522

T069710D 0.85427

T069710M 0.97747

T069711B 0.86668

T069711C 0.94797

T069711D 0.93405

T069711M 0.96690

T069712B 0.84923

T069712C 0.86065

T069712D 0.88757

T069712M 0.98111

T069713B 0.85266

T069713C 0.86438

T069713D 0.85637

T069713M 0.95761

T069714B 0.87542

T069714C 0.87195

T069714D 0.86941

T069714M 0.98084

T069715B 0.87722

T069715C 0.87201

T069715D 0.84682

T069715M 0.98367

T069716B 0.87209

T069716C 0.90771

T069716D 0.86682

T069716M 0.98009

T069901B 0.85174

T069901C 0.87151

T069901D 0.87529

T069901M 0.91948

T069902B 0.91470

T069902C 0.93573

T069902D 0.83852

T069902M 0.95363

T069903B 0.86996

T069903C 0.93031

T069903D 0.92652

T069903M 0.92621

T070001B 0.86588

T070001C 0.89190

T070001D 0.91523

T070001M 0.96582

T070002B 0.88610

T070002C 0.92533

T070002D 0.91529

T070002M 0.96160

T070003B 0.87549

T070003C 0.87631

T070003D 0.90629

T070003M 0.95523

T070101B 0.86856

T070101C 0.89723

T070101D 0.87139

T070101M 0.96167

T070102B 0.86606

T070102C 0.87150

T070102D 0.84820

T070102M 0.96910

T070103B 0.83747

T070103C 0.85906

T070103D 0.85830

T070103M 0.97241

T070104B 0.90954

T070104C 0.92870

T070104D 0.86923

T070104M 0.97513

T070105B 0.87540

T070105C 0.89689

T070105D 0.90075

T070105M 0.96135

T070106B 0.89632

T070106C 0.91948

T070106D 0.86490

T070106M 0.96900

T070107B 0.87044

T070107C 0.87407

T070107D 0.87620

T070107M 0.96444

T070201B 0.89826

T070201C 0.90230
Table F-9 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Reading Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

781

T070201D 0.91690

T070201M 0.95467

T070202B 0.84303

T070202C 0.85569

T070202D 0.84843

T070202M 0.96665

T070203B 0.89680

T070203C 0.89991

T070203D 0.86306

T070203M 0.96877

T070204B 0.87789

T070204C 0.87409

T070204D 0.87810

T070204M 0.96464

T070301B 0.84069

T070301C 0.84191

T070301M 0.96198

T070302B 0.83558

T070302C 0.85024

T070302M 0.96958

T070303B 0.85563

T070303C 0.83576

T070303M 0.96322

T070304B 0.92611

T070304C 0.92677

T070304M 0.96145

T070305B 0.83924

T070305C 0.84039

T070305M 0.96200

T071601M 0.87257

T071602M 0.87182

T071603M 0.86012

T071604M 0.87211

T040301B 0.86358

T040301C 0.88428

T040301D 0.90299

T040301E 0.92153

T040301M 0.97388

T071701B 0.85481

T071701C 0.85462

T071701D 0.88252

T071701E 0.88374

T071701F 0.88370

T071701M 0.88327

T071702B 0.89244

T071702C 0.89268

T071702D 0.90677

T071702E 0.91266

T071702F 0.91712

T071702M 0.90123

T071703B 0.89839

T071703C 0.91148

T071703D 0.90516

T071703E 0.91676

T071703F 0.92695

T071703M 0.87981

T071704B 0.87064

T071704C 0.87638

T071704D 0.87588

T071704E 0.88656

T071704F 0.85706

T071704M 0.92345

T067703B 0.85549

T067703C 0.87321

T067703D 0.86680

T067703E 0.87364

T067703M 0.87077

T068501N 0.90908

T068501M 0.94726

T069801B 0.88969

T069801C 0.85555

T069801M 0.97853

T069802B 0.81889

T069802C 0.80335

T069802M 0.98046

T069803B 0.82804

T069803C 0.84551

T069803M 0.97289

T069804B 0.86311

T069804C 0.86561

T069804M 0.97645

T069805B 0.82502

T069805C 0.83843

T069805M 0.97913

T069806B 0.82269

T069806C 0.82037

T069806M 0.97759

T069807B 0.83808

T069807C 0.85058

T069807M 0.98176

T069808B 0.83549

T069808C 0.83281

T069808M 0.97631

T069809B 0.83543

T069809C 0.82326

T069809M 0.98677

T069810B 0.88730

T069810C 0.89291

T069810M 0.98140

T069811B 0.87914

T069811C 0.89690

T069811M 0.97520

T069812B 0.85616

T069812C 0.87411

T069812M 0.97451

T069813B 0.85141

T069813C 0.86230

T069813M 0.97485

CLASIZ-2 0.89334

CLASIZ-3 0.88514

CLASIZ-4 0.91240

CLASIZ-5 0.92105

CLASIZ-? 0.82479
Table F-10

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Reading Conditioning Variables, Grade 12

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

782

FEMALE 0.92556

BLACK 0.96266

HISPANIC 0.96052

ASIAN 0.94800

MEXICAN 0.95006

PUER RIC 0.97750

CUBN,OTH 0.97190

HISP-? 0.97661

MID CTY7 0.91920

FR/LCTY7 0.92127

FR/MCTY7 0.93821

LAR TWN7 0.91496

SML TWN7 0.92617

OTHER 0.93123

HS GRAD 0.93917

POST HS 0.93724

COL GRAD 0.95058

PARED-? 0.92974

S EAST 0.84277

CENTRAL 0.86971

WEST 0.88509

PRIVATE 0.89567

CATHOLIC 0.92089

BLACK 0.89781

HISPANIC 0.84808

ASIAN 0.84659

IEP-NO 0.96435

LEP-NO 0.88193

TITLE-N 0.81111

RED PRIC 0.95448

FREE 0.77943

INFO N/A 0.86564

SCH/REF 0.91498

SCH/REF 0.82294

TVLIN-0 0.98053

TV-QUAD 0.97956

HW-NO 0.98894

HW-YES 0.99005

HWLIN-0 0.97335

HWQUAD-0 0.96716

HITEM=3 0.93489

HITEM=4 0.98138

PGS>5 0.81759

PGS>10 0.82434

G/R 22 0.90119

G/R 23 0.90941

G/R 24 0.94910

G/T 22 0.67817

G/T 23 0.73920

G/T 24 0.69002

G/T 25 0.94884

G/T 26 0.64242

G/T 27 0.67020

G/P 22 0.91946

G/P 23 0.95609

G/P 24 0.89890

G/P 25 0.89159

G/S 22 0.95188

G/S 23 0.93042

R/T 24 0.90409

R/T 25 0.90054

R/T 26 0.91009

R/T 27 0.96559

R/T 31 0.90532

R/T 32 0.91274

R/T 33 0.90715

R/T 34 0.89826

R/T 35 0.89648

R/T 36 0.93198

R/T 37 0.91416

R/T 41 0.92774

R/T 42 0.92199

R/T 43 0.91686

R/T 44 0.92078

R/T 45 0.92911

R/T 46 0.94090

R/T 47 0.95255

R/P 24 0.88235

R/P 25 0.88800

R/P 31 0.89566

R/P 32 0.80255

R/P 33 0.88214

R/P 34 0.88508

R/P 35 0.89583

R/P 41 0.84197

R/P 42 0.93544

R/P 43 0.94672

R/P 44 0.93287

R/P 45 0.85388

R/S 31 0.96648

R/S 32 0.94676

R/S 33 0.95396

R/S 41 0.94626

R/S 42 0.96773

R/S 43 0.95163

P/T 25 0.80537

P/T 26 0.79446

P/T 27 0.82859

P/T 31 0.95359

P/T 32 0.79972

P/T 33 0.80781

P/T 34 0.77873

P/T 35 0.76154

P/T 36 0.81262

P/T 37 0.95056

P/T 41 0.78351

P/T 42 0.78719

P/T 43 0.79359

P/T 44 0.80266

P/T 45 0.83330

P/T 46 0.95621

P/T 47 0.79274

P/T 51 0.80628

P/T 52 0.88842

P/T 53 0.92641

P/T 54 0.89651

P/T 55 0.96385

P/T 56 0.92049

P/T 57 0.94191

T/S 41 0.92516

T/S 42 0.91245

T/S 43 0.94292

T/S 51 0.92583

T/S 52 0.95720

T/S 53 0.96226

T/S 61 0.95341

T/S 62 0.94916

T/S 63 0.92347

T/S 72 0.92561

P/S 32 0.93276

P/S 33 0.90572

P/S 41 0.91998

P/S 42 0.89010

P/S 43 0.92442

P/S 51 0.90200

P/S 52 0.96608

P/S 53 0.95421

SAMP S3 0.79554

S/R 22 0.88010

S/R 23 0.89515

S/R 24 0.95525

BLACK 0.93907

HISPANIC 0.94083

ASIAN AM 0.92057

AMER IND 0.99021

OTHER 0.98045

B003001M 0.96914

B014601B 0.92147

B014601C 0.90406

B014601M 0.74598
Table F-10 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Reading Conditioning Variables, Grade 12

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

783

B003201B 0.86397

B003201C 0.75995

B003201M 0.77033

B000901N 0.87363

B000901M 0.75400

B000903N 0.92996

B000903M 0.71515

B000904N 0.91798

B000904M 0.90066

B000905N 0.96085

B000905M 0.74232

S004001B 0.85833

S004001C 0.87728

S004001D 0.92195

S004001E 0.94919

S004001M 0.76851

B007301B 0.98596

B007301C 0.98506

B007301D 0.97074

B007301M 0.85229

B007401B 0.91516

B007401C 0.91994

B007401D 0.87221

B007401M 0.86043

B014501B 0.91292

B014501C 0.91183

B014501D 0.91925

B014501M 0.82808

R830301B 0.97727

R830301C 0.97085

R830301N 0.96885

R830301M 0.93656

R830401B 0.95510

R830401C 0.95922

R830401N 0.95309

R830401M 0.94177

RM00501B 0.85455

RM00501C 0.85107

RM00501D 0.91709

RM00501M 0.95813

R830501B 0.97442

R830501C 0.96723

R830501D 0.96150

R830501M 0.93054

R830502B 0.81317

R830502C 0.91556

R830502D 0.91915

R830502M 0.93508

R810801B 0.86061

R810801C 0.90599

R810801D 0.94047

R810801M 0.89822

R810201B 0.91262

R810201C 0.88625

R810201D 0.95355

R810201M 0.78711

R810901B 0.94167

R810901C 0.92924

R810901D 0.92250

R810901M 0.91129

R810902B 0.93355

R810902C 0.94014

R810902D 0.94393

R810902M 0.81851

R810903B 0.96956

R810903C 0.97729

R810903D 0.97017

R810903M 0.90392

R810904B 0.94347

R810904C 0.94476

R810904D 0.94823

R810904M 0.84205

R810905B 0.89092

R810905C 0.89391

R810905D 0.89973

R810905M 0.73369

R810906B 0.88088

R810906C 0.87370

R810906D 0.94355

R810906M 0.68444

R811005B 0.97079

R811005C 0.96812

R811005D 0.96649

R811005M 0.86718

R811006B 0.93676

R811006C 0.93532

R811006D 0.94289

R811006M 0.79333

R811007B 0.86994

R811007C 0.86626

R811007D 0.92156

R811007M 0.63211

R811009B 0.96657

R811009C 0.96539

R811009D 0.95445

R811009M 0.83030

R811002B 0.95733

R811002C 0.95586

R811002D 0.94606

R811002M 0.86185

R811004B 0.92321

R811004C 0.91601

R811004D 0.94231

R811004M 0.81117

R818101B 0.95591

R818101C 0.96143

R818101D 0.94459

R818101M 0.78738

R818102B 0.93294

R818102C 0.92232

R818102D 0.92049

R818102M 0.72662

R830001N 0.95687

R830001M 0.85759

R811301B 0.89848

R811301C 0.95806

R811301D 0.95055

R811301E 0.87751

R811301M 0.88578

R811302B 0.89518

R811302C 0.95902

R811302D 0.96056

R811302E 0.89975

R811302M 0.90287

R811303B 0.93668

R811303C 0.93201

R811303D 0.94847

R811303E 0.92862

R811303M 0.89102

R811304B 0.93722

R811304C 0.93732

R811304D 0.94002

R811304E 0.88211

R811304M 0.88068

C042701Y 0.88575

C042701N 0.86347

C042701M 0.96282

C042801N 0.84234

C042801M 0.87521

C042802N 0.88996

C042802M 0.88454

C042803N 0.82181

C042803M 0.86245

C042901B 0.86530

C042901C 0.87832

C042901D 0.89470

C042901E 0.90018

C042901F 0.92584

C042901G 0.93966

C036601N 0.87034

C036601C 0.92233

C036601D 0.95962

C036601M 0.98315
Table F-10 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Reading Conditioning Variables, Grade 12

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

784

C043001B 0.88304

C043001C 0.90297

C043001D 0.90494

C043001E 0.82678

C043001M 0.96936

C043002B 0.87970

C043002C 0.92306

C043002D 0.94163

C043002E 0.90319

C043002M 0.99537

C043003B 0.91687

C043003C 0.92339

C043003D 0.94049

C043003E 0.89342

C043004B 0.88908

C043004C 0.89671

C043004D 0.89673

C043004E 0.85294

C043004M 0.97950

C043005B 0.91257

C043005C 0.94358

C043005D 0.91665

C043005E 0.88043

C043006B 0.87537

C043006C 0.86316

C043006D 0.83690

C043006E 0.89414

C043007B 0.87091

C043007C 0.86215

C043007D 0.85550

C043007E 0.92831

C043007M 0.99123

C043008B 0.86222

C043008C 0.85184

C043008D 0.94500

C043008M 0.96252

C032402B 0.90360

C032402C 0.93402

C032402N 0.89889

C032401B 0.91878

C032401C 0.93697

C032401N 0.88352

C032404B 0.88765

C032404C 0.95659

C032404N 0.90087

C032407B 0.86856

C032407C 0.94962

C032407N 0.94210

C032408B 0.87010

C032408C 0.95151

C032408N 0.93056

C032408M 0.98245

C032409B 0.91441

C032409C 0.92440

C032409N 0.89459

C032409M 0.96261

C032410B 0.89295

C032410C 0.90683

C032410N 0.85585

C032410M 0.98637

C032411B 0.91240

C032411C 0.91745

C032411N 0.85115

C032412B 0.89553

C032412C 0.92107

C032412N 0.87415

C032413B 0.86196

C032413C 0.94414

C032413N 0.94105

C032413M 0.98045

C032414B 0.90745

C032414C 0.95656

C032414N 0.88228

C032414M 0.96390

C043101B 0.90808

C043101C 0.96053

C043101N 0.89772

C043101M 0.96016

C043102B 0.90855

C043102C 0.93552

C043102N 0.90900

C043103B 0.83897

C043103C 0.80742

C043104B 0.88360

C043104C 0.93648

C043104N 0.91318

C032502B 0.86667

C032502C 0.89727

C032502M 0.95010

C032503B 0.86274

C032503C 0.88418

C032503D 0.89217

C032505B 0.83288

C032505C 0.87298

C032505D 0.86999

C032506B 0.88075

C032506C 0.88345

C032506D 0.87020

C032506M 0.97520

C043201B 0.85358

C043201C 0.89831

C043201M 0.97415

C043301B 0.91436

C043301C 0.90894

C043301D 0.86567

C043301E 0.86671

C043301M 0.94129

C043401B 0.86600

C043401C 0.86882

C043401D 0.87768

C043401M 0.98161

C043501B 0.90410

C043501C 0.88360

C043501D 0.89743

C043501E 0.91971

C043501F 0.87506

C043501M 0.95386

C043601B 0.86602

C043601C 0.88368

C043601D 0.85693

C043601E 0.89070

C043601M 0.95363

C043701B 0.85514

C043701C 0.88042

C043701D 0.86392

C043701E 0.87586

C038301N 0.84367

C038301M 0.95649

C043801B 0.87833

C043801C 0.90458

C043801D 0.94938

C043801E 0.92564

C043801F 0.87878

C043801G 0.87225

C043801H 0.84276

C043801M 0.86248

C043901N 0.92376

C043901M 0.90846

C044001B 0.87698

C044001C 0.90992

C044001D 0.87828

C044001E 0.89033

C044001F 0.86449

C044001G 0.83804

C044001H 0.87468

C044001M 0.81822

C044002B 0.89583

C044002C 0.88789

C044002D 0.89806

C044002E 0.89605

C044002F 0.95014

C044002G 0.85461

C044002H 0.88234
Table F-10 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Reading Conditioning Variables, Grade 12

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

785

C044002M 0.91193

C044003B 0.86468

C044003C 0.89377

C044003D 0.88056

C044003E 0.88336

C044003F 0.94047

C044003G 0.88655

C044003M 0.91818

C044004B 0.88744

C044004C 0.89993

C044004D 0.88832

C044004E 0.89427

C044004F 0.89061

C044004G 0.85647

C044004H 0.91655

C044004M 0.86840

B003501B 0.86042

B003501C 0.89419

B003501D 0.90425

B003501M 0.83066

B003601B 0.89048

B003601C 0.86826

B003601D 0.88233

B003601M 0.76207

R811010B 0.85752

R811010C 0.92199

R811010D 0.78365

R811010M 0.84796

R811011B 0.86748

R811011C 0.93164

R811011D 0.82845

R811011M 0.89340

R830201N 0.98108

R830201M 0.83522

C043105B 0.90993

C043105C 0.90936

C043105N 0.86405

C043105M 0.95678

C043106B 0.88466

C043106C 0.91913

C043106N 0.85432

C043106M 0.97291

B005501B 0.97512

B005501C 0.96147

B005501D 0.93842

B005501E 0.98014

B005501F 0.98201

B005501M 0.67580

R820201N 0.95754

R820201M 0.80416

C044301N 0.85229

C044301M 0.96908

C044302N 0.88233

C044302M 0.89047

C044101B 0.87288

C044101C 0.87827

C044101D 0.87620

C044101E 0.87625

C044101M 0.96446

C044201B 0.87631

C044201C 0.88432

C044201D 0.94953

C044201E 0.93262

C044201F 0.90545

C044201G 0.88449

C044201H 0.87252

C044201M 0.92210

C044202B 0.89284

C044202C 0.87966

C044202D 0.90416

C044202E 0.93099

C044202F 0.92740

C044202G 0.86936

C044202H 0.87673

C044202M 0.93443
Table F-11

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Writing Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

786

FEMALE 0.94244

BLACK 0.95815

HISPANIC 0.95803

ASIAN 0.91322

MEXICAN 0.92150

PUER RIC 0.97705

CUBN,OTH 0.96755

HISP-? 0.96639

MID CTY7 0.94612

FR/LCTY7 0.94949

FR/MCTY7 0.94295

LAR TWN7 0.93987

SML TWN7 0.94622

OTHER 0.95187

HS GRAD 0.95494

POST HS 0.95538

COL GRAD 0.95482

PARED-? 0.95313

S EAST 0.86378

CENTRAL 0.85133

WEST 0.86190

PRIVATE 0.92664

CATHOLIC 0.94864

BLACK 0.83360

HISPANIC 0.72536

ASIAN 0.70319

IEP-NO 0.96283

LEP-NO 0.93339

TITLE-N 0.76751

RED PRIC 0.94086

FREE 0.75317

INFO N/A 0.87859

SCH/REF 0.88491

SCH/NP 0.86105

TVLIN-0 0.96218

TV-QUAD 0.98458

HW-NO 0.97691

HW-YES 0.97851

HWLIN-0 0.98919

HWQUAD-0 0.97029

HITEM=3 0.98152

HITEM=4 0.97329

PGS>5 0.98067

PGS>10 0.81814

NO ACCOM 0.93966

G/R 22 0.90939

G/R 23 0.90465

G/R 24 0.96613

G/T 22 0.67149

G/T 23 0.78597

G/T 24 0.74567

G/T 25 0.95922

G/T 26 0.67630

G/T 27 0.69983

G/P 22 0.97629

G/P 23 0.96222

G/P 24 0.74045

G/P 25 0.98022

G/S 22 0.91847

G/S 23 0.87845

R/T 24 0.91777

R/T 25 0.92052

R/T 26 0.92075

R/T 27 0.96934

R/T 31 0.90534

R/T 32 0.91216

R/T 33 0.90594

R/T 34 0.89971

R/T 35 0.89886

R/T 36 0.95164

R/T 37 0.89337

R/T 41 0.88759

R/T 42 0.93809

R/T 43 0.92924

R/T 44 0.92669

R/T 45 0.93200

R/T 46 0.94033

R/T 47 0.94162

R/P 24 0.91869

R/P 25 0.90656

R/P 31 0.91135

R/P 32 0.89912

R/P 33 0.91960

R/P 34 0.89946

R/P 35 0.89281

R/P 41 0.90527

R/P 42 0.97639

R/P 43 0.96885

R/P 44 0.95538

R/P 45 0.96062

R/S 31 0.95822

R/S 32 0.95099

R/S 33 0.95921

R/S 41 0.93595

R/S 42 0.95302

R/S 43 0.95988

P/T 25 0.76432

P/T 26 0.77105

P/T 27 0.73335

P/T 31 0.95862

P/T 32 0.70994

P/T 33 0.78830

P/T 34 0.71999

P/T 35 0.70776

P/T 36 0.76352

P/T 37 0.95982

P/T 41 0.70822

P/T 42 0.73031

P/T 43 0.82918

P/T 44 0.86080

P/T 45 0.86735

P/T 46 0.97450

P/T 47 0.82690

P/T 51 0.81442

P/T 52 0.74812

P/T 53 0.73660

P/T 54 0.74457

P/T 55 0.96202

P/T 56 0.71607

P/T 57 0.75964

T/S 41 0.92640

T/S 42 0.91169

T/S 43 0.92180

T/S 51 0.91204

T/S 52 0.93756

T/S 53 0.92410

T/S 61 0.95855

T/S 62 0.94810

T/S 63 0.92658

T/S 71 0.92613

T/S 72 0.94786

T/S 73 0.95072

P/S 32 0.93964

P/S 33 0.90329

P/S 41 0.93253

P/S 42 0.90370

P/S 43 0.90882

P/S 51 0.89333

P/S 52 0.93168

P/S 53 0.92567

A/G 22 0.86063

A/R 22 0.90816

A/R 23 0.93344

A/R 24 0.95447

A/T 22 0.91748

A/T 23 0.91370

A/T 24 0.92386

A/T 25 0.97287

A/T 26 0.93132

A/T 27 0.92380
Table F-11 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Writing Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

787

A/P 22 0.84593

A/P 23 0.85837

A/P 24 0.85840

A/P 25 0.82029

A/S 22 0.95301

A/S 23 0.95460

A/I 22 0.97843

A/L 22 0.92559

BLACK 0.91647

HISPANIC 0.81554

ASIAN AM 0.86411

AMER IND 0.98658

OTHER 0.98844

B003001M 0.70073

CUBAN 0.97870

B013001B 0.98138

B013001C 0.98148

B013001D 0.97868

B013001M 0.76220

B013101B 0.95932

B013101C 0.93226

B013101D 0.90591

B013101M 0.74885

B013201N 0.81343

B013201M 0.74996

B013301N 0.80509

B013301M 0.75254

B013401N 0.85375

B013401M 0.83776

B013501N 0.79732

B013501M 0.75752

B013601N 0.79972

B013601M 0.77886

B013701N 0.81626

B013701M 0.83254

B000901N 0.96054

B000901M 0.95439

B000903N 0.96628

B000903M 0.96934

B013801B 0.97488

B013801C 0.96737

B013801D 0.96019

B013801M 0.69085

B000905N 0.95289

B000905M 0.94115

B013901B 0.92253

B013901C 0.95175

B013901D 0.96945

B013901E 0.96899

B013901F 0.96033

B006601B 0.93413

B001101B 0.94211

B001101C 0.93882

B001101E 0.97926

B014001B 0.91304

B014001C 0.92447

B014001D 0.95465

B014001E 0.95472

B014001M 0.81203

B007301B 0.95938

B007301C 0.95846

B007301D 0.93867

B007301M 0.85067

B007401B 0.86972

B007401C 0.91739

B007401D 0.81643

B007401M 0.84988

B014101B 0.90941

B014101C 0.91103

B014101D 0.93678

B014101E 0.93842

B014101M 0.85440

W803001B 0.88490

W803001C 0.84525

W803001N 0.92233

W803001M 0.82121

W803101B 0.92144

W803101C 0.93485

W803101N 0.95596

W803101M 0.83087

W803201B 0.92532

W803201C 0.91376

W803201D 0.90619

W803201M 0.76549

W803301B 0.96527

W803301C 0.94166

W803301D 0.93665

W803301M 0.80196

W803302B 0.82769

W803302C 0.88960

W803302D 0.92585

W803302M 0.81930

W801901B 0.90453

W801901C 0.92150

W801901D 0.94299

W801901E 0.95103

W801901M 0.83169

W801902B 0.89950

W801902C 0.92525

W801902D 0.94531

W801902E 0.96143

W801902M 0.92955

W802001B 0.81229

W802001C 0.80665

W802001M 0.75585

W802101B 0.84013

W802101C 0.84225

W802101M 0.79406

W802201B 0.92133

W802201C 0.92171

W802201M 0.75102

W802301B 0.86779

W802301C 0.91373

W802301D 0.87894

W802301M 0.77447

W802302B 0.95070

W802302C 0.94156

W802302D 0.95155

W802302M 0.75796

W802303B 0.94057

W802303C 0.93988

W802303D 0.93166

W802303M 0.74540

W802304B 0.88399

W802304C 0.92969

W802304D 0.87136

W802304M 0.72189

W802401N 0.95002

W802401M 0.95064

W802501B 0.93785

W802501N 0.95243

W802501M 0.77006

W802502B 0.86685

W802502N 0.87695

W802502M 0.85521

W802503B 0.93334

W802503N 0.93479

W802503M 0.81991

W802504B 0.87671

W802504N 0.87122

W802504M 0.84328

W802601B 0.91988

W802601C 0.94395

W802601D 0.88329

W802601M 0.90272

W802602B 0.90198

W802602C 0.94486

W802602D 0.86750

W802602M 0.92774

W802603B 0.94774

W802603C 0.93983

W802603D 0.93501

W802603M 0.91162
Table F-11 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Writing Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

788

C042501N 0.85419

C042501M 0.91454

C042601B 0.84940

C042601C 0.90979

C042601D 0.89760

C042601M 0.93660

C042602B 0.85014

C042602C 0.85661

C042602D 0.90151

C042602N 0.93131

C042602M 0.91356

C042603B 0.85057

C042603C 0.87354

C042603D 0.87299

C042603M 0.89395

C042604B 0.87213

C042604C 0.90265

C042604D 0.87510

C042604N 0.87611

C042604M 0.88801

C042701Y 0.94557

C042701N 0.94441

C042701M 0.93273

C042801N 0.85290

C042801M 0.87638

C042802N 0.87245

C042802M 0.86269

C042803N 0.85694

C042803M 0.87465

C042901B 0.87239

C042901C 0.89511

C042901D 0.92653

C042901E 0.91543

C042901F 0.90296

C042901G 0.89219

C042901M 0.87159

C036601N 0.88533

C036601C 0.91640

C036601D 0.93275

C036601M 0.93186

C043001B 0.88219

C043001C 0.91329

C043001D 0.90726

C043001E 0.88886

C043001M 0.96145

C043002B 0.88318

C043002C 0.86602

C043002D 0.92674

C043002E 0.95135

C043002M 0.98604

C043003B 0.86216

C043003C 0.87580

C043003D 0.93885

C043003E 0.95284

C043004B 0.89558

C043004C 0.90144

C043004D 0.85878

C043004E 0.85191

C043004M 0.97949

C043005B 0.88990

C043005C 0.92251

C043005D 0.90962

C043005E 0.87975

C043005M 0.98590

C043006B 0.86621

C043006C 0.87832

C043006D 0.85182

C043006E 0.86408

C043006M 0.99008

C043007B 0.90094

C043007C 0.89890

C043007D 0.89965

C043007E 0.87421

C043008B 0.88132

C043008C 0.88986

C043008D 0.87546

C043008E 0.88219

C043008M 0.97720

C032402B 0.88324

C032402C 0.92795

C032402N 0.92523

C032402M 0.95915

C032401B 0.89346

C032401C 0.93020

C032401N 0.91101

C032401M 0.98369

C032404B 0.88103

C032404C 0.94600

C032404N 0.92637

C032404M 0.98933

C032407B 0.88689

C032407C 0.94483

C032407N 0.94336

C032408B 0.87251

C032408C 0.94901

C032408N 0.92006

C032408M 0.97057

C032409B 0.87659

C032409C 0.92048

C032409N 0.89023

C032409M 0.96607

C032410B 0.90705

C032410C 0.88366

C032411B 0.91026

C032411C 0.91705

C032411N 0.94852

C032412B 0.90953

C032412C 0.89550

C032413B 0.88222

C032413C 0.82215

C032414B 0.89653

C032414C 0.95116

C032414N 0.90868

C032414M 0.97860

C043101B 0.88391

C043101C 0.94846

C043101N 0.94907

C043102B 0.88352

C043102C 0.93600

C043102N 0.93755

C043103B 0.89660

C043103C 0.94203

C043103N 0.95888

C043104B 0.86618

C043104C 0.94993

C043104N 0.95103

C032502B 0.86011

C032502C 0.86111

C032502D 0.85292

C032502M 0.99409

C032503B 0.85025

C032503C 0.87897

C032503D 0.90774

C032503M 0.97593

C032505B 0.84961

C032505C 0.85544

C032505D 0.89384

C032505M 0.98818

C032506B 0.86175

C032506C 0.85719

C043201B 0.85017

C043201C 0.86582

C043301B 0.86355

C043301C 0.89772

C043301D 0.88457

C043301M 0.96301

C043401B 0.86440

C043401C 0.88493

C043401D 0.86389

C043501B 0.89666

C043501C 0.89885

C043501D 0.90027

C043501E 0.87925
Table F-11 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Writing Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

789

C043501F 0.88218

C043501M 0.96963

C043601B 0.84710

C043601C 0.89175

C043601D 0.85680

C043601M 0.97293

C043701B 0.86478

C043701C 0.85151

C043701D 0.87824

C043701E 0.87909

C043701M 0.97867

C038301N 0.87055

C038301M 0.94878

C043801B 0.87580

C043801C 0.87721

C043801D 0.88617

C043801E 0.91215

C043801F 0.89889

C043801G 0.87676

C043801H 0.87074

C043801M 0.89572

C043901N 0.90309

C043901M 0.95754

C044001B 0.87571

C044001C 0.88049

C044001D 0.87917

C044001E 0.87044

C044001F 0.85421

C044001G 0.86779

C044001H 0.89664

C044001M 0.92154

C044002B 0.88520

C044002C 0.89124

C044002D 0.90349

C044002E 0.88854

C044002F 0.89277

C044002G 0.88976

C044002H 0.90361

C044002M 0.93629

C044003B 0.86084

C044003C 0.86560

C044003D 0.87409

C044003E 0.86411

C044003F 0.88359

C044003G 0.88504

C044003H 0.90089

C044003M 0.93954

C044004B 0.89851

C044004C 0.89178

C044004D 0.89323

C044004E 0.85603

C044004F 0.87226

C044004G 0.96501

C044004H 0.87281

C044004M 0.92930

T067001M 0.87694

T067002M 0.86110

T067003M 0.87259

T067004M 0.81662

T067101B 0.86615

T067101C 0.87379

T067101D 0.90126

T067101E 0.91032

T067101M 0.95428

T067201B 0.93433

T067201C 0.93481

T067201D 0.95298

T067201E 0.95857

T067201M 0.96135

T067202B 0.92406

T067202C 0.92752

T067202D 0.93950

T067202E 0.94298

T067202M 0.95033

T067203B 0.93295

T067203C 0.93782

T067203D 0.95785

T067203E 0.95941

T067203M 0.95446

T067204B 0.86924

T067204C 0.86916

T067204D 0.88645

T067204E 0.86979

T067204M 0.86770

T067205B 0.86476

T067205C 0.86373

T067205D 0.89518

T067205E 0.92163

T067205M 0.89337

T067206B 0.84599

T067206C 0.83590

T067206D 0.85027

T067206E 0.84120

T067206M 0.86030

T067301B 0.88793

T067301C 0.85143

T067301D 0.85650

T067301M 0.93470

T056201B 0.86632

T056201C 0.88924

T056201D 0.90058

T056201E 0.86828

T056201F 0.86111

T056201M 0.88140

T056301B 0.92888

T056301C 0.94686

T056301D 0.94831

T056301E 0.89143

T056301F 0.87472

T056301G 0.87768

T056301M 0.94691

T067501B 0.89186

T067501C 0.85757

T067501M 0.85861

T067502B 0.89090

T067502C 0.93552

T067502M 0.96862

T067503B 0.90155

T067503C 0.92760

T067503M 0.95926

T067504B 0.86693

T067504C 0.97668

T067504M 0.98247

T067505B 0.91315

T067505C 0.98971

T067505M 0.98892

T067506B 0.86083

T067506C 0.93945

T067506M 0.98667

T067507B 0.89296

T067507C 0.98439

T067507M 0.99059

T067508B 0.91196

T067508C 0.90653

T067508M 0.93440

T067509B 0.89792

T067509C 0.92610

T067509M 0.93885

T067510B 0.91750

T067510C 0.93354

T067510M 0.94961

T067511B 0.85369

T067511C 0.96311

T067511M 0.97407

T067512B 0.91300

T067512C 0.88208

T067512M 0.86105

T067601B 0.87462

T067601C 0.78618

T067601M 0.86576

T067602B 0.91552

T067602C 0.97302

T067602M 0.97538
Table F-11 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Writing Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

790

T067603B 0.87045

T067603C 0.93059

T067603M 0.97790

T067604B 0.85615

T067604C 0.97623

T067604M 0.98554

T067605B 0.84784

T067605C 0.91119

T067605M 0.96731

T067606B 0.83923

T067606C 0.88671

T067606M 0.96062

T067607B 0.87449

T067607C 0.95163

T067607M 0.97458

T067608B 0.89015

T067608C 0.96318

T067608M 0.97463

T067609B 0.88167

T067609C 0.89892

T067609M 0.95076

T067610B 0.86420

T067610C 0.97506

T067610M 0.98006

T067611B 0.85651

T067611C 0.99297

T067612B 0.87892

T067612C 0.87329

T067612M 0.87743

T067701B 0.90942

T067701C 0.92907

T067701D 0.92294

T067701E 0.89624

T067701M 0.94048

T067702B 0.85499

T067702C 0.88949

T067702D 0.87855

T067702E 0.87155

T067702M 0.86186

T067801B 0.89637

T067801C 0.91116

T067801M 0.90322

T067802B 0.88713

T067802C 0.87009

T067802M 0.98637

T067803B 0.84317

T067803C 0.86972

T067803M 0.97454

T067804B 0.86456

T067804C 0.86457

T067804M 0.97111

T067805B 0.90342

T067805C 0.92049

T067805M 0.96882

T067806B 0.88163

T067806C 0.87781

T067806M 0.97776

T067807B 0.83360

T067807C 0.86186

T067807M 0.97745

T041201B 0.89339

T041201C 0.90118

T041201D 0.88912

T041201M 0.98065

T067901B 0.82376

T067901C 0.85259

T067901M 0.96869

T067902B 0.82726

T067902C 0.86355

T067902M 0.95563

T068001B 0.83763

T068001C 0.85544

T068001M 0.96505

T068002B 0.80274

T068002C 0.84561

T068002M 0.95336

T068003B 0.83646

T068003C 0.85141

T068003M 0.97091

T068004B 0.85952

T068004C 0.86524

T068004M 0.96112

T068005B 0.85942

T068005C 0.83874

T068005M 0.95849

T068006B 0.85011

T068006C 0.84971

T068006M 0.96633

T068007B 0.87135

T068007C 0.89072

T068007M 0.97687

T068008B 0.83918

T068008C 0.84347

T068008M 0.97038

T068009B 0.87002

T068009C 0.89532

T068009M 0.96881

T068010B 0.83944

T068010C 0.84251

T068010M 0.98176

T068101B 0.88126

T068101C 0.90202

T068101D 0.87040

T068101E 0.87320

T068101M 0.95552

T046101N 0.81590

T046101M 0.94893

T046201B 0.87864

T046201C 0.86826

T046201D 0.91185

T046201M 0.93840

T068201B 0.91659

T068201C 0.94581

T068201D 0.93917

T068201E 0.90746

T068201M 0.95673

T068301B 0.89675

T068301C 0.88106

T068301D 0.87602

T068301E 0.88150

T068301M 0.81871

T068401B 0.88636

T068401C 0.86742

T068401D 0.89396

T068401E 0.89722

T068401F 0.89517

T068401G 0.86322

T068401M 0.85075

T068601B 0.90724

T068601C 0.89272

T068601D 0.91058

T068601M 0.95780

T068701B 0.91485

T068701C 0.92735

T068701D 0.93632

T068701E 0.92594

T068701M 0.96052

T068801B 0.94182

T068801C 0.94346

T068801D 0.93390

T068801M 0.93967

T068802B 0.88665

T068802C 0.88174

T068802D 0.88792

T068802M 0.90053

T068803B 0.88457

T068803C 0.89768

T068803D 0.91295

T068803M 0.94539

T068804B 0.86918

T068804C 0.87002

T068804D 0.84838

T068804M 0.94207
Table F-11 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Writing Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

791

T068805B 0.85811

T068805C 0.85848

T068805D 0.85192

T068805M 0.93758

T068901B 0.88472

T068901C 0.86594

T068901D 0.87604

T068901M 0.88940

T069001B 0.88164

T069001C 0.90496

T069001D 0.92194

T069001E 0.90294

T069001M 0.82750

T069101B 0.86697

T069101C 0.86942

T069101D 0.89327

T069101M 0.96819

T069102B 0.87824

T069102C 0.88728

T069102D 0.87428

T069102M 0.96243

T069103B 0.85461

T069103C 0.90190

T069103D 0.89215

T069103M 0.93193

T069201B 0.90004

T069201C 0.89637

T069201D 0.84135

T069201M 0.97099

T069202B 0.89286

T069202C 0.92617

T069202D 0.86317

T069202M 0.95665

T069203B 0.86042

T069203C 0.93868

T069203D 0.93149

T069203M 0.95789

T069301B 0.83612

T069301N 0.86309

T069301M 0.93524

T069302B 0.83455

T069302N 0.82860

T069302M 0.96376

T069303B 0.83123

T069303N 0.84808

T069303M 0.95870

T069304B 0.84150

T069304N 0.83710

T069304M 0.94913

T069305B 0.86577

T069305N 0.86645

T069305M 0.96611

T069401B 0.85749

T069401C 0.88641

T069401D 0.85588

T069401M 0.98276

T069402B 0.84664

T069402C 0.86750

T069402D 0.86680

T069402M 0.96939

T069403B 0.88400

T069403C 0.87911

T069403D 0.86670

T069403M 0.95569

T069404B 0.87198

T069404C 0.87720

T069404D 0.85927

T069404M 0.96024

T069405B 0.88151

T069405C 0.88641

T069405D 0.87953

T069405M 0.94995

T069501B 0.92444

T069501C 0.94369

T069501D 0.92058

T069501E 0.91236

T069501M 0.97210

T069601B 0.91200

T069601C 0.92208

T069601D 0.91847

T069601M 0.97545

T069701B 0.83457

T069701C 0.90406

T069701D 0.85529

T069701M 0.98195

T069702B 0.86943

T069702C 0.86740

T069702D 0.84351

T069702M 0.97892

T069703B 0.86222

T069703C 0.87878

T069703D 0.85522

T069703M 0.96861

T069704B 0.86485

T069704C 0.87358

T069704D 0.85963

T069704M 0.95860

T069705B 0.86964

T069705C 0.87066

T069705D 0.89535

T069705M 0.97685

T069706B 0.85941

T069706C 0.87492

T069706D 0.86492

T069706M 0.98366

T069707B 0.92659

T069707C 0.93421

T069707D 0.88344

T069707M 0.98208

T069708B 0.88270

T069708C 0.90700

T069708D 0.86970

T069708M 0.95587

T069709B 0.85068

T069709C 0.85902

T069709D 0.84510

T069709M 0.96059

T069710B 0.92799

T069710C 0.93689

T069710D 0.87260

T069710M 0.97289

T069711B 0.92262

T069711C 0.95441

T069711D 0.91382

T069711M 0.95331

T069712B 0.83198

T069712C 0.85442

T069712M 0.96675

T069713B 0.82208

T069713C 0.85395

T069713D 0.86333

T069713M 0.98084

T069714B 0.86699

T069714C 0.85480

T069714D 0.83243

T069714M 0.96263

T069715B 0.84536

T069715C 0.85736

T069715D 0.83461

T069715M 0.98831

T069716B 0.91405

T069716C 0.92772

T069716D 0.87145

T069716M 0.95593

T071801B 0.91846

T071801C 0.94173

T071801D 0.90655

T071801M 0.95558

T071802B 0.86560

T071802C 0.86835

T071802D 0.85177

T071802M 0.97358

T071803B 0.87959
Table F-11 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Writing Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

792

T071803C 0.91035

T071803D 0.87927

T071803M 0.94612

T071804B 0.91300

T071804C 0.93578

T071804D 0.93163

T071804M 0.95971

T071805B 0.90936

T071805C 0.92335

T071805D 0.87012

T071805M 0.97505

T071806B 0.90425

T071806C 0.90489

T071806D 0.87286

T071806M 0.96453

T071807B 0.87765

T071807C 0.88568

T071807D 0.85874

T071807M 0.97541

T071808B 0.89236

T071808C 0.92096

T071808D 0.87651

T071808M 0.97093

T071809B 0.85807

T071809C 0.84758

T071809D 0.86393

T071809M 0.98460

T071810B 0.91355

T071810C 0.94927

T071810D 0.91355

T071810M 0.95974

T071811B 0.91486

T071811C 0.94454

T071811D 0.91739

T071811M 0.94816

T071812B 0.87729

T071812C 0.87556

T071812D 0.84713

T071812M 0.97343

T071813B 0.86686

T071813C 0.88898

T071813D 0.85829

T071813M 0.94210

T069901B 0.86211

T069901C 0.87066

T069901D 0.85177

T069901M 0.92895

T069902B 0.89542

T069902C 0.94316

T069902D 0.89158

T069902M 0.94479

T069903B 0.90316

T069903C 0.93796

T069903D 0.95014

T069903M 0.92439

T070001B 0.86982

T070001C 0.91497

T070001D 0.90415

T070001M 0.96957

T070002B 0.88925

T070002C 0.93392

T070002D 0.91617

T070002M 0.95633

T070003B 0.84744

T070003C 0.89197

T070003D 0.87982

T070003M 0.96527

T070101B 0.87421

T070101C 0.87423

T070101D 0.86259

T070101M 0.94915

T070102B 0.85757

T070102C 0.88006

T070102D 0.86323

T070102M 0.94312

T070103B 0.86074

T070103C 0.87860

T070103D 0.85832

T070103M 0.95284

T070104B 0.91559

T070104C 0.93021

T070104D 0.87792

T070104M 0.96588

T070105B 0.88423

T070105C 0.88449

T070105D 0.90262

T070105M 0.92494

T070106B 0.89059

T070106C 0.92470

T070106D 0.89082

T070106M 0.95379

T070107B 0.88123

T070107C 0.86465

T070107D 0.86279

T070107M 0.95436

T070201B 0.90519

T070201C 0.88554

T070201D 0.90667

T070201M 0.93095

T070202B 0.84959

T070202C 0.88142

T070202D 0.84801

T070202M 0.96967

T070203B 0.90883

T070203C 0.92285

T070203D 0.86944

T070203M 0.96261

T070204B 0.85678

T070204C 0.87504

T070204D 0.87103

T070204M 0.96188

T070301B 0.85697

T070301C 0.86627

T070301M 0.96018

T070302B 0.83259

T070302C 0.85804

T070302M 0.96114

T070303B 0.87447

T070303C 0.88603

T070304B 0.95233

T070304C 0.95338

T070304M 0.95960

T070305B 0.85933

T070305C 0.88833

T070305M 0.95583
Table F-12

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Writing Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

793

FEMALE 0.94661

BLACK 0.96558

HISPANIC 0.96553

ASIAN 0.95215

MEXICAN 0.91960

PUER RIC 0.96607

CUBN,OTH 0.87450

HISP-? 0.97747

MID CTY7 0.94264

FR/LCTY7 0.94421

FR/MCTY7 0.94556

LAR TWN7 0.92854

SML TWN7 0.93971

OTHER 0.95599

HS GRAD 0.96267

POST HS 0.96328

COL GRAD 0.96144

PARED-? 0.93476

S EAST 0.87198

CENTRAL 0.87166

WEST 0.87573

PRIVATE 0.93274

CATHOLIC 0.93455

BLACK 0.88134

HISPANIC 0.81521

ASIAN 0.79436

IEP-NO 0.97348

LEP-NO 0.92877

TITLE-N 0.77833

RED PRIC 0.93996

FREE 0.74009

INFO N/A 0.84808

SCH/REF 0.87891

SCH/NP 0.84629

TVLIN-0 0.94961

TV-QUAD 0.91447

HW-NO 0.92476

HW-YES 0.94085

HWLIN-0 0.97759

HWQUAD-0 0.88119

HITEM=3 0.97007

HITEM=4 0.97414

PGS>5 0.98220

PGS>10 0.88350

NO ACCOM 0.94327

G/R 22 0.90286

G/R 23 0.90353

G/R 24 0.95344

G/T 22 0.65101

G/T 23 0.67302

G/T 24 0.70261

G/T 25 0.94584

G/T 26 0.68709

G/T 27 0.69093

G/P 22 0.93422

G/P 23 0.93744

G/P 24 0.87824

G/P 25 0.88667

G/S 22 0.92261

G/S 23 0.89935

R/T 24 0.90536

R/T 25 0.89595

R/T 26 0.90876

R/T 27 0.95978

R/T 31 0.88617

R/T 32 0.91207

R/T 33 0.90618

R/T 34 0.90245

R/T 35 0.89902

R/T 36 0.93983

R/T 37 0.90192

R/T 41 0.90608

R/T 42 0.92051

R/T 43 0.90943

R/T 44 0.92182

R/T 45 0.90696

R/T 46 0.93349

R/T 47 0.93938

R/P 24 0.90633

R/P 25 0.89485

R/P 31 0.90079

R/P 32 0.83569

R/P 33 0.89756

R/P 34 0.87818

R/P 35 0.88115

R/P 41 0.85856

R/P 42 0.95606

R/P 43 0.94963

R/P 44 0.93628

R/P 45 0.91340

R/S 31 0.95498

R/S 32 0.94850

R/S 33 0.95766

R/S 41 0.94022

R/S 42 0.95639

R/S 43 0.94966

P/T 25 0.79536

P/T 26 0.77917

P/T 27 0.75310

P/T 31 0.96132

P/T 32 0.79327

P/T 33 0.77080

P/T 34 0.78194

P/T 35 0.75112

P/T 36 0.77901

P/T 37 0.96352

P/T 41 0.76985

P/T 42 0.76416

P/T 43 0.80393

P/T 44 0.79532

P/T 45 0.82287

P/T 46 0.96243

P/T 47 0.81089

P/T 51 0.80010

P/T 52 0.84349

P/T 53 0.81064

P/T 54 0.79976

P/T 55 0.96938

P/T 56 0.86311

P/T 57 0.84892

T/S 41 0.91801

T/S 42 0.91589

T/S 43 0.92212

T/S 51 0.91800

T/S 52 0.93588

T/S 53 0.94130

T/S 61 0.95095

T/S 62 0.94994

T/S 63 0.92751

T/S 71 0.93973

T/S 72 0.94823

T/S 73 0.95446

P/S 32 0.94226

P/S 33 0.91772

P/S 41 0.94035

P/S 42 0.91872

P/S 43 0.91943

P/S 51 0.90359

P/S 52 0.95161

P/S 53 0.95076

A/G 22 0.89026

A/R 22 0.92539

A/R 23 0.93278

A/R 24 0.95291

A/T 22 0.93779

A/T 23 0.92620

A/T 24 0.94563

A/T 25 0.96900

A/T 26 0.94574

A/T 27 0.93552
Table F-12 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Writing Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

794

A/P 22 0.88254

A/P 23 0.88767

A/P 24 0.89579

A/P 25 0.86127

A/S 22 0.96036

A/S 23 0.95533

A/L 22 0.93462

BLACK 0.93846

HISPANIC 0.92631

ASIAN AM 0.92732

AMER IND 0.98256

OTHER 0.98029

B003001M 0.78363

CUBAN 0.81279

B013001B 0.96577

B013001C 0.97007

B013001D 0.94980

B013001M 0.76508

B013101B 0.96528

B013101C 0.93964

B013101D 0.90162

B013101M 0.90891

B013201N 0.76658

B013201M 0.75574

B013301N 0.85926

B013301M 0.84967

B013401N 0.87705

B013401M 0.85205

B013501N 0.73347

B013501M 0.82160

B013601N 0.82035

B013601M 0.80263

B013701N 0.86618

B013701M 0.81955

B000901N 0.92345

B000901M 0.79416

B000903N 0.95912

B000903M 0.86335

B013801B 0.97054

B013801C 0.96675

B013801D 0.95719

B013801M 0.70136

B000905N 0.95000

B000905M 0.76176

B013901B 0.94143

B013901C 0.97109

B013901D 0.96319

B013901E 0.89520

B013901F 0.81022

B006601B 0.76974

B001101B 0.94798

B001101C 0.94220

B001101E 0.98280

B014001B 0.86375

B014001C 0.87116

B014001D 0.92395

B014001E 0.92569

B014001M 0.85210

B007301B 0.97846

B007301C 0.97483

B007301D 0.96774

B007301M 0.86715

B007401B 0.88755

B007401C 0.90978

B007401D 0.83162

B007401M 0.87453

B014101B 0.95714

B014101C 0.93103

B014101D 0.93204

B014101E 0.93498

B014101M 0.85435

W803001B 0.92952

W803001C 0.92383

W803001N 0.94511

W803001M 0.92639

W803101B 0.91984

W803101C 0.90501

W803101N 0.91101

W803101M 0.93378

W803201B 0.93943

W803201C 0.93556

W803201D 0.95094

W803201M 0.93149

W803301B 0.96249

W803301C 0.94772

W803301D 0.94960

W803301M 0.88146

W803302B 0.82412

W803302C 0.88534

W803302D 0.87476

W803302M 0.89607

W801901B 0.90776

W801901C 0.93338

W801901D 0.95806

W801901E 0.83696

W801901M 0.73473

W801902B 0.91588

W801902C 0.93931

W801902D 0.96286

W801902E 0.82782

W801902M 0.91800

W802001B 0.88700

W802001C 0.86189

W802001M 0.84764

W802101B 0.87201

W802101C 0.82708

W802101M 0.85268

W802201B 0.91565

W802201C 0.90668

W802201M 0.80176

W802301B 0.86644

W802301C 0.88457

W802301D 0.88585

W802301M 0.84850

W802302B 0.94826

W802302C 0.95658

W802302D 0.95654

W802302M 0.82947

W802303B 0.94838

W802303C 0.95111

W802303D 0.94664

W802303M 0.83561

W802304B 0.90187

W802304C 0.92516

W802304D 0.86249

W802304M 0.84491

W802401N 0.93143

W802401M 0.67206

W802501B 0.91219

W802501N 0.81814

W802501M 0.90306

W802502B 0.89036

W802502N 0.81601

W802502M 0.92333

W802503B 0.90806

W802503N 0.82979

W802503M 0.91775

W802504B 0.83948

W802504N 0.84315

W802504M 0.91388

W802601B 0.94454

W802601C 0.95295

W802601D 0.91351

W802601M 0.95295

W802602B 0.94668

W802602C 0.95759

W802602D 0.90626

W802602M 0.96109

W802603B 0.95597

W802603C 0.94960

W802603D 0.94618

W802603M 0.95046

C042701Y 0.89732
Table F-12 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Writing Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

795

C042701N 0.89067

C042701M 0.90787

C042801N 0.84823

C042801M 0.88553

C042802N 0.85380

C042802M 0.90074

C042803N 0.85012

C042803M 0.87914

C042901B 0.84718

C042901C 0.89157

C042901D 0.90275

C042901E 0.90891

C042901F 0.90742

C042901G 0.91647

C042901M 0.93899

C036601N 0.89563

C036601C 0.91860

C036601D 0.93658

C036601M 0.97577

C043001B 0.89613

C043001C 0.89276

C043001D 0.89252

C043001E 0.86841

C043001M 0.98100

C043002B 0.87730

C043002C 0.89394

C043002D 0.92748

C043002E 0.93065

C043002M 0.94991

C043003B 0.87642

C043003C 0.88996

C043003D 0.92978

C043003E 0.92170

C043003M 0.95769

C043004B 0.88362

C043004C 0.89224

C043004D 0.87216

C043004E 0.87858

C043004M 0.93718

C043005B 0.91853

C043005C 0.91301

C043005D 0.89974

C043005E 0.85741

C043005M 0.97338

C043006B 0.86642

C043006C 0.85143

C043006D 0.83848

C043006E 0.93592

C043006M 0.98184

C043007B 0.87506

C043007C 0.88642

C043007D 0.85538

C043007E 0.89918

C043007M 0.97825

C043008B 0.87230

C043008C 0.86978

C043008D 0.89959

C043008E 0.85921

C032402B 0.87890

C032402C 0.92404

C032402N 0.90186

C032402M 0.97889

C032401B 0.89958

C032401C 0.93714

C032401N 0.89560

C032401M 0.96949

C032404B 0.90128

C032404C 0.93633

C032404N 0.87873

C032404M 0.91689

C032407B 0.85349

C032407C 0.93390

C032407N 0.93360

C032407M 0.98299

C032408B 0.86917

C032408C 0.95583

C032408N 0.93414

C032408M 0.96908

C032409B 0.88682

C032409C 0.92443

C032409N 0.87582

C032409M 0.98238

C032410B 0.90137

C032410C 0.92180

C032410N 0.92916

C032410M 0.96667

C032411B 0.90358

C032411C 0.91706

C032411N 0.91739

C032411M 0.97183

C032412B 0.89369

C032412C 0.92611

C032412N 0.92988

C032412M 0.96969

C032413B 0.86877

C032413C 0.94101

C032413N 0.94437

C032413M 0.99620

C032414B 0.91972

C032414C 0.94444

C032414N 0.84435

C032414M 0.97385

C043101B 0.87979

C043101C 0.95042

C043101N 0.92271

C043102B 0.90042

C043102C 0.94278

C043102N 0.92442

C043103B 0.87988

C043103C 0.83820

C043104B 0.87059

C043104C 0.95597

C043104N 0.93616

C043104M 0.97196

C032502B 0.86905

C032502C 0.87972

C032502D 0.86048

C032502M 0.97302

C032503B 0.86855

C032503C 0.88579

C032503D 0.85768

C032503M 0.95608

C032505B 0.87498

C032505C 0.86911

C032505D 0.88326

C032505M 0.98148

C032506B 0.85928

C032506C 0.87143

C032506D 0.87099

C032506M 0.96447

C043201B 0.85313

C043201C 0.86118

C043201M 0.97937

C043301B 0.86449

C043301C 0.90593

C043301D 0.87467

C043301M 0.94035

C043401B 0.88293

C043401C 0.87598

C043401M 0.95163

C043501B 0.86924

C043501C 0.88925

C043501D 0.87246

C043501E 0.87177

C043501F 0.85545

C043501M 0.93702

C043601B 0.87087

C043601C 0.87334

C043601D 0.88511

C043601E 0.86323

C043601M 0.93473

C043701B 0.85126

C043701C 0.86978
Table F-12 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Writing Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

796

C043701D 0.88359

C043701E 0.86855

C043701M 0.95105

C038301N 0.85229

C038301M 0.93370

C043801B 0.87989

C043801C 0.88766

C043801D 0.89818

C043801E 0.90364

C043801F 0.89864

C043801G 0.89697

C043801H 0.86359

C043801M 0.87100

C043901N 0.92886

C043901M 0.94739

C044001B 0.87877

C044001C 0.87273

C044001D 0.88778

C044001E 0.87528

C044001F 0.85709

C044001G 0.85445

C044001H 0.90004

C044001M 0.93396

C044002B 0.89004

C044002C 0.89629

C044002D 0.89123

C044002E 0.89697

C044002F 0.91439

C044002G 0.87621

C044002H 0.90692

C044002M 0.93327

C044003B 0.87699

C044003C 0.86400

C044003D 0.87821

C044003E 0.90093

C044003F 0.90964

C044003H 0.88841

C044003M 0.93569

C044004B 0.87530

C044004C 0.89628

C044004D 0.88137

C044004E 0.86359

C044004F 0.91819

C044004G 0.91939

C044004H 0.86732

C044004M 0.93692

B014201B 0.98409

B014201C 0.98763

B014201D 0.98058

B014201E 0.97283

B014201M 0.91004

W802701B 0.89888

W802701C 0.89547

W802701D 0.92704

W802701M 0.74106

W802702B 0.93040

W802702C 0.94475

W802702D 0.92825

W802702M 0.71688

W802703B 0.96893

W802703C 0.96751

W802703D 0.95889

W802703M 0.84590

W802801B 0.95561

W802801C 0.95051

W802801D 0.95622

W802801M 0.83618

W802802B 0.96817

W802802C 0.96612

W802802D 0.96690

W802802M 0.86555

W802803B 0.96754

W802803C 0.97064

W802803D 0.96695

W802803M 0.86795

W802804B 0.96484

W802804C 0.95473

W802804D 0.95664

W802804M 0.88140

W802901B 0.92806

W802901C 0.91508

W802901D 0.87903

W802901M 0.85621

W802902B 0.92304

W802902C 0.93706

W802902D 0.91134

W802902M 0.88452

W802903B 0.93665

W802903C 0.94540

W802903D 0.93725

W802903M 0.86053

W802904B 0.93143

W802904C 0.91749

W802904D 0.92251

W802904M 0.86137

C044401N 0.87684

C044401M 0.95448

C044402N 0.87273

C044402M 0.91770

C043105B 0.86622

C043105C 0.93297

C043105N 0.94465

C043106B 0.85406

C043106C 0.93359

C043106N 0.93196

T067301B 0.83957

T067301C 0.84938

T067301D 0.91768

T067301M 0.95107

T056201B 0.87839

T056201C 0.88036

T056201D 0.89958

T056201E 0.87976

T056201F 0.89172

T056201M 0.91283

T056301B 0.92037

T056301C 0.94088

T056301D 0.95143

T056301E 0.87433

T056301F 0.92160

T056301G 0.91174

T056301M 0.97625

T067501B 0.85261

T067501C 0.88442

T067501M 0.88964

T067502B 0.86645

T067502C 0.84515

T067502M 0.88668

T067503B 0.83799

T067503C 0.95583

T067503M 0.97058

T067504B 0.88605

T067504C 0.98004

T067504M 0.98659

T067505B 0.86750

T067505C 0.97839

T067505M 0.98940

T067506B 0.84356

T067506C 0.96503

T067506M 0.98433

T067507B 0.83834

T067507C 0.97067

T067507M 0.97048

T067508B 0.87951

T067508C 0.85544

T067508M 0.86549

T067509B 0.87797

T067509C 0.87634

T067509M 0.94153

T067510B 0.87919

T067510C 0.89396

T067510M 0.91549

T067511B 0.86062
Table F-12 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Writing Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

797

T067511C 0.95614

T067511M 0.96183

T067512B 0.85895

T067512C 0.86721

T067512M 0.85366

T067601B 0.87026

T067601C 0.90157

T067601M 0.92397

T067602B 0.84654

T067602C 0.86685

T067602M 0.89927

T067603B 0.85275

T067603C 0.94980

T067603M 0.97145

T067604B 0.86700

T067604C 0.97658

T067604M 0.97469

T067605B 0.85947

T067605C 0.91921

T067605M 0.96487

T067606B 0.86459

T067606C 0.92165

T067606M 0.96015

T067607B 0.89239

T067607C 0.96034

T067607M 0.96743

T067608B 0.85734

T067608C 0.87551

T067608M 0.91322

T067609B 0.84194

T067609C 0.88535

T067609M 0.94409

T067610B 0.85923

T067610C 0.97160

T067610M 0.97362

T067611B 0.88367

T067611C 0.98773

T067611M 0.99138

T067612B 0.89447

T067612C 0.87396

T067612M 0.88270

T067701B 0.86855

T067701C 0.91022

T067701D 0.88319

T067701E 0.86477

T067701M 0.95154

T067702B 0.83376

T067702C 0.87290

T067702D 0.88936

T067702E 0.92572

T067702M 0.84114

T067801B 0.88071

T067801C 0.90396

T067801M 0.93671

T067802B 0.85972

T067802C 0.87116

T067802M 0.98913

T067803B 0.82403

T067803C 0.84618

T067803M 0.98537

T067804B 0.87206

T067804C 0.86377

T067804M 0.98928

T067805B 0.89025

T067805C 0.90628

T067805M 0.98697

T067806B 0.89071

T067806C 0.88991

T067806M 0.99115

T067807B 0.81887

T067807C 0.88629

T067807M 0.99189

T041201B 0.87850

T041201C 0.89308

T041201D 0.91231

T041201M 0.97595

T067901B 0.85343

T067901C 0.84191

T067901M 0.98325

T067902B 0.83396

T067902C 0.84546

T067902M 0.98566

T068001B 0.82011

T068001C 0.82408

T068001M 0.98688

T068002B 0.82030

T068002C 0.85148

T068002M 0.98478

T068003B 0.82848

T068003C 0.84027

T068003M 0.98791

T068004B 0.84674

T068004C 0.84392

T068004M 0.98457

T068005B 0.87827

T068005C 0.87712

T068005M 0.98428

T068006B 0.85722

T068006C 0.86806

T068006M 0.99086

T068007B 0.88934

T068007C 0.91059

T068007M 0.98750

T068008B 0.84947

T068008C 0.84404

T068008M 0.98765

T068009B 0.86387

T068009C 0.90066

T068009M 0.99622

T068010B 0.83191

T068010C 0.84792

T068010M 0.99599

T046101N 0.89270

T046101M 0.95382

T046201B 0.85771

T046201C 0.85866

T046201D 0.89745

T046201M 0.96838

T068201B 0.84552

T068201C 0.87075

T068201D 0.88386

T068201E 0.92090

T068201M 0.95251

T068301B 0.86404

T068301C 0.88676

T068301D 0.87705

T068301E 0.91016

T068301M 0.91549

T068401B 0.84871

T068401C 0.90742

T068401D 0.89942

T068401E 0.89865

T068401F 0.86436

T068401G 0.84571

T068401M 0.90825

T068601B 0.87331

T068601C 0.88078

T068601D 0.89409

T068601M 0.97031

T068701B 0.89543

T068701C 0.90153

T068701D 0.91409

T068701E 0.88036

T068701M 0.96945

T068801B 0.89312

T068801C 0.92174

T068801D 0.94023

T068801M 0.95602

T068802B 0.87659

T068802C 0.89730

T068802D 0.93538

T068802M 0.95274

T068803B 0.88648
Table F-12 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Writing Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

798

T068803C 0.87416

T068803D 0.93232

T068803M 0.94621

T068804B 0.82483

T068804C 0.84462

T068804D 0.83317

T068804M 0.95518

T068805B 0.88829

T068805C 0.89687

T068805D 0.89585

T068805M 0.95002

T068901B 0.89339

T068901C 0.89247

T068901D 0.90538

T068901M 0.91183

T069001B 0.86972

T069001C 0.87273

T069001D 0.86983

T069001E 0.85755

T069001M 0.89899

T069101B 0.86136

T069101C 0.85369

T069101D 0.81501

T069101M 0.95723

T069102B 0.87330

T069102C 0.90801

T069102D 0.87506

T069102M 0.94290

T069103B 0.87942

T069103C 0.90651

T069103D 0.89708

T069103M 0.92891

T069201B 0.88045

T069201C 0.90744

T069201D 0.85381

T069201M 0.96654

T069202B 0.89412

T069202C 0.92740

T069202D 0.87159

T069202M 0.96866

T069203B 0.86070

T069203C 0.91553

T069203D 0.87698

T069203M 0.96854

T069301B 0.83096

T069301N 0.85082

T069301M 0.96560

T069302B 0.84615

T069302N 0.82458

T069302M 0.97483

T069303B 0.83905

T069303N 0.81904

T069303M 0.96046

T069304B 0.83574

T069304N 0.84706

T069304M 0.95989

T069305B 0.89219

T069305N 0.90396

T069305M 0.94628

T069401B 0.83343

T069401C 0.86227

T069401D 0.82789

T069401M 0.97887

T069402B 0.83341

T069402C 0.87815

T069402D 0.82362

T069402M 0.98311

T069403B 0.85822

T069403C 0.88693

T069403D 0.85280

T069403M 0.97714

T069404B 0.87867

T069404C 0.92396

T069404D 0.93008

T069404M 0.98101

T069405B 0.87326

T069405C 0.91111

T069405D 0.92317

T069405M 0.98198

T069501B 0.87475

T069501C 0.93045

T069501D 0.91542

T069501E 0.88622

T069501M 0.98356

T069601B 0.88519

T069601C 0.87986

T069601D 0.88327

T069601M 0.98136

T069701B 0.83827

T069701C 0.84762

T069701D 0.85643

T069701M 0.97579

T069702B 0.86453

T069702C 0.88530

T069702D 0.84067

T069702M 0.98235

T069703B 0.86537

T069703C 0.88337

T069703D 0.83176

T069703M 0.98173

T069704B 0.87854

T069704C 0.91263

T069704D 0.89868

T069704M 0.98014

T069705B 0.84219

T069705C 0.86474

T069705D 0.83502

T069705M 0.98056

T069706B 0.86319

T069706C 0.88263

T069706D 0.87067

T069706M 0.97545

T069707B 0.87434

T069707C 0.94113

T069707D 0.87336

T069707M 0.96837

T069708B 0.87006

T069708C 0.89205

T069708D 0.84870

T069708M 0.97585

T069709B 0.85614

T069709C 0.85361

T069709D 0.85051

T069709M 0.98092

T069710B 0.90041

T069710C 0.91698

T069710D 0.84552

T069710M 0.98168

T069711B 0.87361

T069711C 0.93865

T069711D 0.92797

T069711M 0.96846

T069712B 0.81506

T069712C 0.83490

T069712D 0.84536

T069712M 0.98163

T069713B 0.84673

T069713C 0.85435

T069713D 0.83985

T069713M 0.97682

T069714B 0.86334

T069714C 0.87423

T069714D 0.86297

T069714M 0.98114

T069715B 0.86764

T069715C 0.87707

T069715D 0.84827

T069715M 0.98670

T069716B 0.88361

T069716C 0.91563

T069716D 0.86383

T069716M 0.98800

T069901B 0.86189
Table F-12 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Writing Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

799

T069901C 0.86555

T069901D 0.86308

T069901M 0.93321

T069902B 0.90240

T069902C 0.92937

T069902D 0.85062

T069902M 0.96394

T069903B 0.85544

T069903C 0.91671

T069903D 0.92503

T069903M 0.93127

T070001B 0.85674

T070001C 0.88172

T070001D 0.91374

T070001M 0.97513

T070002B 0.86443

T070002C 0.92673

T070002D 0.90360

T070002M 0.97497

T070003B 0.87021

T070003C 0.88776

T070003D 0.90116

T070003M 0.96567

T070101B 0.86169

T070101C 0.87982

T070101D 0.85749

T070101M 0.96634

T070102B 0.85137

T070102C 0.86988

T070102D 0.84615

T070102M 0.97370

T070103B 0.83782

T070103C 0.85804

T070103D 0.84296

T070103M 0.97729

T070104B 0.89124

T070104C 0.92323

T070104D 0.84465

T070104M 0.97945

T070105B 0.88359

T070105C 0.89372

T070105D 0.90471

T070105M 0.97147

T070106B 0.89098

T070106C 0.91028

T070106D 0.86329

T070106M 0.97551

T070107B 0.87282

T070107C 0.87397

T070107D 0.88049

T070107M 0.97457

T070201B 0.89810

T070201C 0.89895

T070201D 0.90938

T070201M 0.96203

T070202B 0.83722

T070202C 0.87135

T070202D 0.82983

T070202M 0.97549

T070203B 0.88974

T070203C 0.90493

T070203D 0.84204

T070203M 0.97840

T070204B 0.88004

T070204C 0.89354

T070204D 0.88058

T070204M 0.97151

T070301B 0.81244

T070301C 0.84857

T070301M 0.96689

T070302B 0.81779

T070302C 0.81995

T070302M 0.97257

T070303B 0.80471

T070303C 0.83331

T070303M 0.96282

T070304B 0.92411

T070304C 0.92763

T070304M 0.96255

T070305B 0.82927

T070305C 0.83583

T070305M 0.97180

T071601M 0.87751

T071602M 0.87675

T071603M 0.85916

T071604M 0.84677

T040301B 0.86109

T040301C 0.88232

T040301D 0.90198

T040301E 0.92247

T040301M 0.98638

T071701B 0.86168

T071701C 0.84052

T071701D 0.86143

T071701E 0.87232

T071701F 0.86893

T071701M 0.87858

T071702B 0.88509

T071702C 0.89586

T071702D 0.89948

T071702E 0.91200

T071702F 0.90780

T071702M 0.89586

T071703B 0.88759

T071703C 0.90713

T071703D 0.90089

T071703E 0.91646

T071703F 0.93441

T071703M 0.92296

T071704B 0.88512

T071704C 0.87127

T071704D 0.88691

T071704E 0.86799

T071704F 0.83767

T071704M 0.92361

T067703B 0.83305

T067703C 0.87394

T067703D 0.87331

T067703E 0.86008

T067703M 0.88685

T068501N 0.90370

T068501M 0.94391

T069801B 0.89045

T069801C 0.86076

T069801M 0.98477

T069802B 0.81329

T069802C 0.75937

T069802M 0.98496

T069803B 0.84238

T069803C 0.82304

T069803M 0.96887

T069804B 0.87458

T069804C 0.87791

T069804M 0.98188

T069805B 0.83246

T069805C 0.81049

T069805M 0.98343

T069806B 0.82430

T069806C 0.84908

T069806M 0.97607

T069807B 0.82876

T069807C 0.83968

T069807M 0.98089

T069808B 0.81602

T069808C 0.81800
Table F-12 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Writing Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

800

T069808M 0.98044

T069809C 0.79717

T069809M 0.98744

T069810B 0.89154

T069810C 0.89129

T069810M 0.97984

T069811B 0.88805

T069811C 0.89076

T069812B 0.83209

T069812C 0.82252

T069812M 0.97738

T069813B 0.82007

T069813C 0.80599

T069813M 0.97496

CLASIZ-2 0.87538

CLASIZ-3 0.89403

CLASIZ-4 0.89068

CLASIZ-5 0.91377

CLASIZ-? 0.82560
Table F-13

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Writing Conditioning Variables, Grade 12

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

801

FEMALE 0.95191

BLACK 0.97176

HISPANIC 0.96968

ASIAN 0.95122

MEXICAN 0.91458

PUER RIC 0.98254

CUBN,OTH 0.87055

HISP-? 0.98494

MID CTY7 0.94952

FR/LCTY7 0.94425

FR/MCTY7 0.95569

LAR TWN7 0.93424

SML TWN7 0.94516

OTHER 0.94788

HS GRAD 0.96649

POST HS 0.96295

COL GRAD 0.96388

PARED-? 0.91645

S EAST 0.83206

CENTRAL 0.86784

WEST 0.88792

PRIVATE 0.94171

CATHOLIC 0.95350

BLACK 0.88449

HISPANIC 0.83305

ASIAN 0.82226

IEP-NO 0.97310

LEP-NO 0.92591

TITLE-N 0.76425

RED PRIC 0.96441

FREE 0.80303

INFO N/A 0.86893

SCH/REF 0.90665

SCH/NP 0.79840

TVLIN-0 0.93768

TV-QUAD 0.89662

HW-NO 0.95069

HW-YES 0.95814

HWLIN-0 0.97578

HWQUAD-0 0.86446

HITEM=3 0.97549

HITEM=4 0.97572

PGS>5 0.98466

PGS>10 0.85578

NO ACCOM 0.93137

G/R 22 0.90774

G/R 23 0.91202

G/R 24 0.95197

G/T 22 0.66599

G/T 23 0.69281

G/T 24 0.68631

G/T 25 0.95448

G/T 26 0.63907

G/T 27 0.67636

G/P 22 0.90655

G/P 23 0.95319

G/P 24 0.88805

G/P 25 0.89935

G/S 22 0.93711

G/S 23 0.89789

R/T 24 0.89480

R/T 25 0.89764

R/T 26 0.90073

R/T 27 0.96902

R/T 31 0.88755

R/T 32 0.91133

R/T 33 0.90780

R/T 34 0.90602

R/T 35 0.90900

R/T 36 0.94774

R/T 37 0.91487

R/T 41 0.91470

R/T 42 0.91708

R/T 43 0.90972

R/T 44 0.91447

R/T 45 0.92336

R/T 46 0.93910

R/T 47 0.94418

R/P 24 0.89572

R/P 25 0.89043

R/P 31 0.90062

R/P 32 0.80518

R/P 33 0.88953

R/P 34 0.88968

R/P 35 0.89324

R/P 41 0.87959

R/P 42 0.93830

R/P 43 0.94026

R/P 44 0.93243

R/P 45 0.88694

R/S 31 0.95490

R/S 32 0.93665

R/S 33 0.95220

R/S 41 0.93590

R/S 42 0.96289

R/S 43 0.95176

P/T 25 0.82763

P/T 26 0.81483

P/T 27 0.82958

P/T 31 0.95840

P/T 32 0.82433

P/T 33 0.79431

P/T 34 0.77342

P/T 35 0.75955

P/T 36 0.79693

P/T 37 0.95866

P/T 41 0.77232

P/T 42 0.75929

P/T 43 0.79580

P/T 44 0.77919

P/T 45 0.82047

P/T 46 0.95540

P/T 47 0.79580

P/T 51 0.78604

P/T 52 0.89744

P/T 53 0.88735

P/T 54 0.88052

P/T 55 0.94611

P/T 56 0.89421

P/T 57 0.88316

T/S 41 0.92944

T/S 42 0.90570

T/S 43 0.93045

T/S 51 0.91351

T/S 52 0.95495

T/S 53 0.96623

T/S 61 0.96194

T/S 62 0.94465

T/S 63 0.91433

T/S 72 0.93438

P/S 32 0.95402

P/S 33 0.92006

P/S 41 0.94693

P/S 42 0.90868

P/S 43 0.92761

P/S 51 0.89767

P/S 52 0.94960

P/S 53 0.93069

A/G 22 0.92198

A/R 22 0.95962

A/R 23 0.95572

A/R 24 0.96444

A/T 22 0.93870

A/T 23 0.92890

A/T 24 0.94144

A/T 25 0.94950

A/T 26 0.93557

A/T 27 0.93203

A/P 22 0.92628

A/P 23 0.92823
Table F-13 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Writing Conditioning Variables, Grade 12

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

802

A/P 24 0.92783

A/P 25 0.89837

A/S 22 0.96452

A/S 23 0.96358

A/I 22 0.97985

A/L 22 0.93412

BLACK 0.93702

HISPANIC 0.93948

ASIAN AM 0.91725

AMER IND 0.98647

OTHER 0.98425

B003001M 0.68217

CUBAN 0.86313

B013001B 0.89864

B013001C 0.87917

B013001D 0.88350

B013001M 0.81963

B013101B 0.95567

B013101C 0.91583

B013101D 0.85166

B013101M 0.79184

B013201N 0.64460

B013201M 0.77834

B013301N 0.80361

B013301M 0.81536

B013401N 0.84191

B013401M 0.81605

B013501N 0.67696

B013501M 0.79235

B013601N 0.81178

B013601M 0.80117

B013701N 0.83749

B013701M 0.80360

B000901N 0.96433

B000901M 0.73071

B000903N 0.97988

B000903M 0.73698

B013801B 0.97331

B013801C 0.96741

B013801D 0.96095

B013801M 0.84009

B000905N 0.97595

B000905M 0.73051

B013901B 0.91819

B013901C 0.96699

B013901D 0.95802

B013901E 0.89002

B013901F 0.84370

B006601B 0.83069

B001101B 0.95031

B001101C 0.95099

B001101E 0.98571

B014001B 0.86017

B014001C 0.88125

B014001D 0.92208

B014001E 0.94246

B014001M 0.83626

B007301B 0.98061

B007301C 0.98657

B007301D 0.96817

B007301M 0.87175

B007401B 0.88867

B007401C 0.91040

B007401D 0.85538

B007401M 0.87001

B014101B 0.96053

B014101C 0.92954

B014101D 0.92684

B014101E 0.93252

B014101M 0.88114

W803001B 0.98270

W803001C 0.97037

W803001N 0.97029

W803001M 0.94667

W803101B 0.95798

W803101C 0.95520

W803101N 0.94996

W803101M 0.96753

W803201B 0.93958

W803201C 0.93643

W803201D 0.95257

W803201M 0.96624

W803301B 0.97630

W803301C 0.96456

W803301D 0.95987

W803301M 0.94777

W803302B 0.80828

W803302C 0.89343

W803302D 0.89770

W803302M 0.95418

W801901B 0.88601

W801901C 0.93350

W801901D 0.95001

W801901E 0.80213

W801901M 0.86018

W801902B 0.91002

W801902C 0.93969

W801902D 0.96184

W801902E 0.81435

W801902M 0.65089

W802001B 0.87361

W802001C 0.81842

W802001M 0.89624

W802101B 0.86760

W802101C 0.80383

W802101M 0.91373

W802201B 0.93042

W802201C 0.92368

W802201M 0.85049

W802301B 0.93064

W802301C 0.94163

W802301D 0.91024

W802301M 0.89747

W802302B 0.96402

W802302C 0.96862

W802302D 0.96780

W802302M 0.88117

W802303B 0.94433

W802303C 0.94517

W802303D 0.94166

W802303M 0.87178

W802304B 0.90348

W802304C 0.92289

W802304D 0.86787

W802304M 0.86904

W802401N 0.94667

W802401M 0.64851

W802501B 0.91833

W802501N 0.85621

W802501M 0.90539

W802502B 0.92895

W802502N 0.79915

W802502M 0.90534

W802503B 0.95808

W802503N 0.81966

W802503M 0.90363

W802504B 0.83457

W802504N 0.82091

W802504M 0.89326

W802601B 0.95631

W802601C 0.95092

W802601D 0.91965

W802601M 0.94306

W802602B 0.95695

W802602C 0.95836

W802602D 0.91680

W802602M 0.95276

W802603B 0.95833

W802603C 0.94324

W802603D 0.94744

W802603M 0.94351

C042701Y 0.87813

C042701N 0.86569
Table F-13 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Writing Conditioning Variables, Grade 12

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

803

C042701M 0.96720

C042801N 0.81872

C042801M 0.85451

C042802N 0.87761

C042802M 0.86338

C042803N 0.84402

C042803M 0.84935

C042901B 0.86136

C042901C 0.89016

C042901D 0.88673

C042901E 0.90692

C042901F 0.93052

C042901G 0.94360

C036601N 0.89002

C036601C 0.93340

C036601D 0.95768

C036601M 0.95728

C043001B 0.87014

C043001C 0.89843

C043001D 0.90806

C043001E 0.84881

C043001M 0.97614

C043002B 0.88585

C043002C 0.91434

C043002D 0.94461

C043002E 0.89961

C043002M 0.99570

C043003B 0.90854

C043003C 0.91455

C043003D 0.92692

C043003E 0.89097

C043003M 0.98973

C043004B 0.88883

C043004C 0.90655

C043004D 0.86373

C043004E 0.84631

C043004M 0.98080

C043005B 0.91094

C043005C 0.93580

C043005D 0.90155

C043005E 0.85800

C043005M 0.99388

C043006B 0.86458

C043006C 0.85563

C043006D 0.87894

C043006E 0.96595

C043007B 0.87712

C043007C 0.87248

C043007D 0.87101

C043007E 0.89861

C043007M 0.98611

C043008B 0.85753

C043008C 0.82481

C043008D 0.97642

C043008E 0.94378

C043008M 0.95598

C032402B 0.88789

C032402C 0.93685

C032402N 0.89208

C032401B 0.92219

C032401C 0.92963

C032401N 0.86076

C032404B 0.87548

C032404C 0.95252

C032404N 0.88986

C032407B 0.85807

C032407C 0.95357

C032407N 0.94890

C032408B 0.83815

C032408C 0.94707

C032408N 0.92798

C032408M 0.95697

C032409B 0.90491

C032409C 0.92364

C032409N 0.88757

C032409M 0.96381

C032410B 0.87629

C032410C 0.89200

C032410N 0.85160

C032410M 0.98536

C032411B 0.90092

C032411C 0.91579

C032411N 0.86160

C032412B 0.89146

C032412C 0.91086

C032412N 0.87010

C032413B 0.85753

C032413C 0.94046

C032413N 0.92405

C032413M 0.97191

C032414B 0.90509

C032414C 0.96253

C032414N 0.87869

C032414M 0.96145

C043101B 0.89825

C043101C 0.96027

C043101N 0.88984

C043101M 0.95906

C043102B 0.89548

C043102C 0.93504

C043102N 0.91840

C043103B 0.83413

C043103C 0.78842

C043104B 0.87388

C043104C 0.94250

C043104N 0.91392

C032502B 0.84781

C032502C 0.88585

C032502M 0.94373

C032503B 0.85869

C032503C 0.87014

C032503D 0.87569

C032505B 0.83084

C032505C 0.86525

C032505D 0.85202

C032506B 0.86257

C032506C 0.87019

C032506D 0.86451

C032506M 0.97741

C043201B 0.84737

C043201C 0.87660

C043201M 0.97183

C043301B 0.91850

C043301C 0.91474

C043301D 0.87307

C043301E 0.87186

C043301M 0.93772

C043401B 0.87590

C043401C 0.86120

C043401D 0.86822

C043401M 0.97698

C043501B 0.90100

C043501C 0.88759

C043501D 0.88932

C043501E 0.89145

C043501F 0.88885

C043501M 0.93992

C043601B 0.85691

C043601C 0.89233

C043601D 0.86657

C043601E 0.87158

C043601M 0.94222

C043701B 0.85650

C043701C 0.86238

C043701D 0.85697

C043701E 0.84955

C038301N 0.81749

C038301M 0.95604

C043801B 0.86754

C043801C 0.89972

C043801D 0.93389

C043801E 0.93060

C043801F 0.88368
Table F-13 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Writing Conditioning Variables, Grade 12

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

804

C043801G 0.87464

C043801H 0.83844

C043801M 0.85464

C043901N 0.92420

C043901M 0.90697

C044001B 0.88371

C044001C 0.91163

C044001D 0.87181

C044001E 0.84221

C044001F 0.87104

C044001G 0.83233

C044001H 0.85688

C044001M 0.81136

C044002B 0.88206

C044002C 0.88100

C044002D 0.89204

C044002E 0.87685

C044002F 0.91448

C044002G 0.87670

C044002H 0.92619

C044002M 0.90458

C044003B 0.85647

C044003C 0.87253

C044003D 0.88227

C044003E 0.89597

C044003F 0.91786

C044003G 0.89528

C044003H 0.89558

C044003M 0.91147

C044004B 0.87446

C044004C 0.89771

C044004D 0.89038

C044004E 0.90561

C044004F 0.90098

C044004G 0.90143

C044004H 0.89120

C044004M 0.85854

W802701B 0.91595

W802701C 0.91604

W802701D 0.92479

W802702B 0.94485

W802702C 0.95692

W802702D 0.95185

W802702M 0.74565

W802703B 0.97828

W802703C 0.97835

W802703D 0.96718

W802703M 0.81986

W802801B 0.94419

W802801C 0.94155

W802801D 0.95321

W802801M 0.88440

W802802B 0.95012

W802802C 0.94973

W802802D 0.95809

W802802M 0.88891

W802803B 0.97998

W802803C 0.98055

W802803D 0.97590

W802803M 0.87974

W802804B 0.97096

W802804C 0.96462

W802804D 0.95922

W802804M 0.87862

W802901B 0.92886

W802901C 0.93543

W802901D 0.86545

W802901M 0.88042

W802902B 0.82731

W802902C 0.95189

W802902D 0.91204

W802902M 0.88611

W802903B 0.91812

W802903C 0.95341

W802903D 0.93278

W802903M 0.85832

W802904B 0.95147

W802904C 0.92835

W802904D 0.93152

W802904M 0.87692

C043105B 0.89449

C043105C 0.91272

C043105N 0.85971

C043105M 0.95607

C043106B 0.88822

C043106C 0.91695

C043106N 0.85095

C043106M 0.97074

B005501B 0.97399

B005501C 0.95883

B005501E 0.98563

B005501F 0.98376

B005501M 0.78954

B014301Y 0.87214

B014301N 0.81898

B014301M 0.82186

B014401B 0.95539

B014401C 0.94214

B014401D 0.93044

B014401E 0.90919

B014401F 0.89598

B014401M 0.84688

C044301N 0.85451

C044301M 0.96673

C044302N 0.86717

C044302M 0.87427

C044101B 0.86328

C044101C 0.85223

C044101D 0.86840

C044101E 0.86170

C044101M 0.96084

C044201B 0.84719

C044201C 0.88552

C044201D 0.95126

C044201E 0.92083

C044201F 0.90288

C044201G 0.86707

C044201H 0.84178

C044201M 0.90978

C044202B 0.86966

C044202C 0.86046

C044202D 0.90422

C044202E 0.93946

C044202F 0.93307

C044202G 0.87028

C044202H 0.87040

C044202M 0.93691
Table F-14

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Civics Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

805

FEMALE 0.94713

BLACK 0.96088

HISPANIC 0.95935

ASIAN 0.90748

MEXICAN 0.90975

PUER RIC 0.95216

CUBN,OTH 0.96079

HISP-? 0.89772

MID CTY7 0.94436

FR/LCTY7 0.94835

FR/MCTY7 0.94908

LAR TWN7 0.94576

SML TWN7 0.94899

OTHER 0.95791

HS GRAD 0.96032

POST HS 0.95648

COL GRAD 0.95772

PARED-? 0.95512

S EAST 0.86469

CENTRAL 0.86157

WEST 0.85987

PRIVATE 0.94768

CATHOLIC 0.95176

BLACK 0.84289

HISPANIC 0.74401

ASIAN 0.74957

IEP-NO 0.96591

LEP-NO 0.90986

TITLE-N 0.77394

RED PRIC 0.93504

FREE 0.77910

INFO N/A 0.88398

SCH/REF 0.88457

SCH/NP 0.85016

TVLIN-0 0.98046

TV-QUAD 0.98029

HW-NO 0.97303

HW-YES 0.97575

HWLIN-0 0.98636

HWQUAD-0 0.96491

HITEM=3 0.96860

HITEM=4 0.96988

PGS>5 0.83742

PGS>10 0.84326

NO ACCOM 0.94013

G/R 22 0.90826

G/R 23 0.90095

G/R 24 0.96422

G/T 22 0.72229

G/T 23 0.77473

G/T 24 0.76794

G/T 25 0.96351

G/T 26 0.72013

G/T 27 0.74267

G/P 22 0.91743

G/P 23 0.93189

G/P 24 0.77884

G/P 25 0.93914

G/S 22 0.91477

G/S 23 0.86972

R/T 24 0.91636

R/T 25 0.91980

R/T 26 0.93077

R/T 27 0.96912

R/T 31 0.91828

R/T 32 0.91827

R/T 33 0.90122

R/T 34 0.89684

R/T 35 0.89922

R/T 36 0.94836

R/T 37 0.90449

R/T 41 0.89402

R/T 42 0.93523

R/T 43 0.92777

R/T 44 0.93263

R/T 45 0.93153

R/T 46 0.94197

R/T 47 0.94124

R/P 24 0.92119

R/P 25 0.91233

R/P 31 0.90923

R/P 32 0.90198

R/P 33 0.89725

R/P 34 0.89484

R/P 35 0.88130

R/P 41 0.90628

R/P 42 0.97218

R/P 43 0.97057

R/P 44 0.94622

R/P 45 0.95353

R/S 31 0.95496

R/S 32 0.95141

R/S 33 0.95826

R/S 41 0.93514

R/S 42 0.95311

R/S 43 0.95911

P/T 25 0.79537

P/T 26 0.78361

P/T 27 0.76262

P/T 31 0.96411

P/T 32 0.73256

P/T 33 0.74005

P/T 34 0.75028

P/T 35 0.72884

P/T 36 0.79400

P/T 37 0.97406

P/T 41 0.73385

P/T 42 0.74621

P/T 43 0.83987

P/T 44 0.88500

P/T 45 0.89341

P/T 46 0.97601

P/T 47 0.85350

P/T 51 0.83025

P/T 52 0.76737

P/T 53 0.76172

P/T 54 0.83939

P/T 55 0.95889

P/T 56 0.74170

T/S 41 0.91994

T/S 42 0.90282

T/S 43 0.91969

T/S 51 0.90334

T/S 52 0.93151

T/S 53 0.91947

T/S 61 0.95322

T/S 62 0.94493

T/S 63 0.92526

T/S 71 0.92384

T/S 72 0.95025

T/S 73 0.95198

P/S 32 0.95037

P/S 33 0.91175

P/S 41 0.93641

P/S 42 0.90460

P/S 43 0.93109

P/S 51 0.91165

P/S 52 0.94396

P/S 53 0.91978

A/G 22 0.88118

A/R 22 0.93288

A/R 23 0.92886

A/R 24 0.95617

A/T 22 0.92170

A/T 23 0.91766

A/T 24 0.94266

A/T 25 0.97252

A/T 26 0.93573

A/T 27 0.93337

A/P 22 0.88935
Table F-14 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Civics Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

806

A/P 23 0.86753

A/P 24 0.87932

A/P 25 0.83208

A/S 22 0.96461

A/S 23 0.95663

A/I 22 0.97439

A/L 22 0.93452

BLACK 0.92379

HISPANIC 0.81294

ASIAN AM 0.87182

AMER IND 0.96403

OTHER 0.96717

B003001M 0.79543

B013001B 0.95788

B013001C 0.96270

B013001D 0.94573

B013001M 0.81328

B013101B 0.95138

B013101C 0.92798

B013101D 0.90344

B013101M 0.82389

B013201N 0.79120

B013201M 0.74890

B013301N 0.81449

B013301M 0.78832

B013401N 0.86962

B013401M 0.85059

B013501N 0.73944

B013501M 0.74838

B013601N 0.80299

B013601M 0.76938

B013701N 0.83936

B013701M 0.82708

B000901N 0.91580

B000901M 0.87141

B000903N 0.91466

B000903M 0.90053

B013801B 0.95692

B013801C 0.96542

B013801D 0.95533

B013801M 0.69124

B000905N 0.90721

B000905M 0.87407

B006601B 0.92690

B014001B 0.88541

B014001C 0.89416

B014001D 0.93144

B014001E 0.92104

B014001M 0.79246

B007301B 0.92179

B007301C 0.91866

B007301D 0.90383

B007301M 0.79449

B007401B 0.87854

B007401C 0.92715

B007401D 0.85988

B007401M 0.85865

B014101B 0.90182

B014101C 0.90820

B014101D 0.93268

B014101E 0.93393

B014101M 0.85480

P804001B 0.87582

P804001C 0.85619

P804001N 0.90949

P804001M 0.82424

P804101B 0.91980

P804101C 0.92007

P804101N 0.93915

P804101M 0.87020

P804201B 0.87327

P804201C 0.87867

P804201D 0.87365

P804201M 0.84864

P804301B 0.95783

P804301C 0.93819

P804301D 0.93750

P804301M 0.81763

P804302B 0.85815

P804302C 0.88285

P804302D 0.91064

P804302M 0.78982

P803501B 0.88493

P803501C 0.91114

P803501D 0.88408

P803501M 0.93039

P803601N 0.82900

P803601M 0.85179

P803602N 0.81848

P803602M 0.86787

P803603N 0.83349

P803603M 0.82361

P803604N 0.84165

P803604M 0.81089

P803605N 0.88568

P803605M 0.79264

P803606N 0.83939

P803606M 0.80830

P803607N 0.82962

P803607M 0.79453

P803701N 0.91004

P803701M 0.84919

P803702N 0.90253

P803702M 0.81668

P803703N 0.86673

P803703M 0.82883

P803704N 0.93512

P803704M 0.78633

P803705N 0.90226

P803705M 0.81688

P803706N 0.86565

P803706M 0.77149

P803707N 0.89538

P803707M 0.83896

P803708N 0.86305

P803708M 0.82131

P803709N 0.86223

P803709M 0.86262

P803710N 0.86652

P803710M 0.82866

P803711N 0.85031

P803711M 0.82239

P803712N 0.86042

P803712M 0.80076

P803801B 0.91475

P803801C 0.93033

P803801D 0.91375

P803801M 0.86139

P803901N 0.91559

P803901M 0.91119

C042501N 0.85754

C042501M 0.91185

C042601B 0.84390

C042601C 0.92654

C042601M 0.92512

C042602B 0.85876

C042602C 0.85026

C042602D 0.91013

C042602N 0.95865

C042602M 0.90499

C042603B 0.83472

C042603C 0.85098

C042603D 0.84494

C042603M 0.89041

C042604B 0.88062

C042604C 0.90590

C042604D 0.87960

C042604N 0.85431

C042604M 0.89058

C042701Y 0.94541

C042701N 0.95054

C042701M 0.92604

C042801N 0.84199
Table F-14 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Civics Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

807

C042801M 0.86318

C042802N 0.85951

C042802M 0.85633

C042803N 0.83706

C042803M 0.86435

C042901B 0.86353

C042901C 0.89195

C042901D 0.92817

C042901E 0.90477

C042901F 0.91273

C042901G 0.89018

C042901M 0.86015

C036601N 0.89368

C036601C 0.91448

C036601D 0.93684

C036601M 0.92353

C043001B 0.88197

C043001C 0.92188

C043001D 0.89112

C043001E 0.87855

C043001M 0.96021

C043002B 0.87123

C043002C 0.85295

C043002D 0.93656

C043002E 0.95346

C043002M 0.98320

C043003B 0.89702

C043003C 0.87718

C043003D 0.93759

C043003E 0.95471

C043004B 0.89644

C043004C 0.89738

C043004D 0.86272

C043004E 0.84451

C043004M 0.97512

C043005B 0.90112

C043005C 0.93325

C043005D 0.89635

C043005E 0.85620

C043005M 0.98150

C043006B 0.87106

C043006C 0.86510

C043006D 0.84852

C043006E 0.86817

C043006M 0.98859

C043007B 0.87523

C043007C 0.89131

C043007D 0.87409

C043007E 0.87804

C043007M 0.99225

C043008B 0.87845

C043008C 0.89465

C043008D 0.86813

C043008E 0.87183

C043008M 0.97826

C032402B 0.89034

C032402C 0.92448

C032402N 0.92063

C032402M 0.95057

C032401B 0.88449

C032401C 0.93005

C032401N 0.92059

C032401M 0.98018

C032404B 0.86992

C032404C 0.94593

C032404N 0.92281

C032404M 0.98711

C032407B 0.88608

C032407C 0.93970

C032407N 0.94233

C032408B 0.87136

C032408C 0.95669

C032408N 0.92686

C032408M 0.97458

C032409B 0.88054

C032409C 0.92588

C032409N 0.89220

C032409M 0.96170

C032410B 0.90061

C032410C 0.87989

C032411B 0.91331

C032411C 0.91489

C032411N 0.94869

C032412B 0.90059

C032412C 0.89238

C032413B 0.86976

C032413C 0.80866

C032414B 0.88366

C032414C 0.95137

C032414N 0.90235

C032414M 0.97788

C043101B 0.88796

C043101C 0.95009

C043101N 0.95310

C043102B 0.86914

C043102C 0.94179

C043102N 0.93900

C043103B 0.91529

C043103C 0.94306

C043103N 0.96018

C043104B 0.85915

C043104C 0.95278

C043104N 0.95170

C032502B 0.86022

C032502C 0.85325

C032502D 0.85202

C032503B 0.83797

C032503C 0.87500

C032503D 0.89886

C032503M 0.97488

C032505B 0.83081

C032505C 0.86201

C032505D 0.88665

C032505M 0.98741

C032506B 0.86560

C032506C 0.84790

C043201B 0.84913

C043201C 0.87329

C043301B 0.86793

C043301C 0.88753

C043301D 0.91095

C043301M 0.96298

C043401B 0.85756

C043401C 0.88257

C043401D 0.90197

C043501B 0.90076

C043501C 0.88883

C043501D 0.89045

C043501E 0.88436

C043501F 0.87581

C043501M 0.96483

C043601B 0.83983

C043601C 0.89171

C043601D 0.87114

C043601M 0.97035

C043701B 0.86681

C043701C 0.87892

C043701D 0.87694

C043701E 0.87495

C043701M 0.97977

C038301N 0.86275

C038301M 0.94396

C043801B 0.85834

C043801C 0.89250

C043801D 0.87572

C043801E 0.89770

C043801F 0.90550

C043801G 0.87925

C043801H 0.85450

C043801M 0.89009

C043901N 0.90094

C043901M 0.95468

C044001B 0.86047
Table F-14 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Civics Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

808

C044001C 0.88307

C044001D 0.89274

C044001E 0.85582

C044001F 0.84707

C044001G 0.87337

C044001H 0.89391

C044001M 0.92065

C044002B 0.86478

C044002C 0.88498

C044002D 0.90084

C044002E 0.88270

C044002F 0.89053

C044002G 0.89521

C044002H 0.90560

C044002M 0.93198

C044003B 0.83957

C044003C 0.86690

C044003D 0.85756

C044003E 0.87091

C044003F 0.88001

C044003G 0.88584

C044003H 0.90412

C044003M 0.93561

C044004B 0.89848

C044004C 0.89955

C044004D 0.89110

C044004E 0.84256

C044004F 0.86455

C044004G 0.96482

C044004H 0.89948

C044004M 0.92625

T067001M 0.87758

T067002M 0.88512

T067003M 0.89885

T067004M 0.79328

T067101B 0.84513

T067101C 0.85212

T067101D 0.89232

T067101E 0.89543

T067101M 0.95997

T067201B 0.92320

T067201C 0.92915

T067201D 0.95315

T067201E 0.96004

T067201M 0.95688

T067202B 0.92139

T067202C 0.91938

T067202D 0.93972

T067202E 0.94459

T067202M 0.93706

T067203B 0.91676

T067203C 0.93616

T067203D 0.95304

T067203E 0.95650

T067203M 0.93802

T067204B 0.85918

T067204C 0.87486

T067204D 0.88297

T067204E 0.87083

T067204M 0.87287

T067205B 0.86695

T067205C 0.88042

T067205D 0.89883

T067205E 0.91956

T067205M 0.93570

T067206B 0.84076

T067206C 0.83295

T067206D 0.85507

T067206E 0.85548

T067206M 0.86901

T067301B 0.91973

T067301C 0.85821

T067301D 0.89673

T067301M 0.94444

T056201B 0.86119

T056201C 0.89811

T056201D 0.88340

T056201E 0.88502

T056201F 0.82269

T056201M 0.89155

T056301B 0.89101

T056301C 0.94491

T056301D 0.94351

T056301E 0.87121

T056301F 0.91314

T056301G 0.86230

T056301M 0.95236

T067501B 0.89029

T067501C 0.85187

T067501M 0.86432

T067502B 0.90958

T067502C 0.93722

T067502M 0.97128

T067503B 0.90121

T067503C 0.92971

T067503M 0.95593

T067504B 0.89421

T067504C 0.97735

T067504M 0.98581

T067505B 0.91269

T067505C 0.99080

T067505M 0.99150

T067506B 0.85823

T067506C 0.93445

T067506M 0.98724

T067507B 0.89665

T067507C 0.98275

T067507M 0.99004

T067508B 0.89116

T067508C 0.91018

T067508M 0.95105

T067509B 0.89737

T067509C 0.92459

T067509M 0.94381

T067510B 0.91273

T067510C 0.93607

T067510M 0.95265

T067511B 0.87061

T067511C 0.96058

T067511M 0.97036

T067512B 0.90402

T067512C 0.88991

T067512M 0.84116

T067601B 0.88426

T067601C 0.77620

T067601M 0.86957

T067602B 0.91028

T067602C 0.97387

T067602M 0.98211

T067603B 0.86304

T067603C 0.93111

T067603M 0.98417

T067604B 0.89015

T067604C 0.97501

T067604M 0.98490

T067605B 0.87356

T067605C 0.91394

T067605M 0.97279

T067606B 0.86785

T067606C 0.88483

T067606M 0.95970

T067607B 0.86569

T067607C 0.95595

T067607M 0.97976

T067608B 0.89944

T067608C 0.96402

T067608M 0.97485

T067609B 0.88470

T067609C 0.90035

T067609M 0.95327

T067610B 0.85712

T067610C 0.97414

T067610M 0.97848
Table F-14 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Civics Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

809

T067611B 0.85130

T067611C 0.99253

T067612B 0.87829

T067612C 0.86335

T067612M 0.87399

T067701B 0.91304

T067701C 0.93147

T067701D 0.92980

T067701E 0.90737

T067701M 0.93144

T067702B 0.85333

T067702C 0.86549

T067702D 0.87293

T067702E 0.88840

T067702M 0.88688

T067801B 0.88392

T067801C 0.91217

T067801M 0.89992

T067802B 0.87655

T067802C 0.87011

T067802M 0.98711

T067803B 0.82970

T067803C 0.86497

T067803M 0.96560

T067804B 0.85759

T067804C 0.85377

T067804M 0.97450

T067805B 0.90929

T067805C 0.92213

T067805M 0.98156

T067806B 0.86858

T067806C 0.87493

T067806M 0.98257

T067807B 0.81642

T067807C 0.88869

T067807M 0.98034

T041201B 0.88693

T041201C 0.90183

T041201D 0.89030

T041201M 0.98227

T070401B 0.85154

T070401C 0.85591

T070401M 0.97951

T070402B 0.88532

T070402C 0.89190

T070402M 0.96003

T070403B 0.85888

T070403C 0.84267

T070403M 0.96939

T070404B 0.88170

T070404C 0.88355

T070404M 0.97601

T070405B 0.85765

T070405C 0.85069

T070405M 0.96370

T070406B 0.91978

T070406C 0.93198

T070406M 0.93944

T070407B 0.91137

T070407C 0.92434

T070407M 0.97051

T070501B 0.88923

T070501C 0.90082

T070501D 0.88093

T070501E 0.90999

T070501M 0.97027

T070601N 0.83570

T070601M 0.96441

T070701B 0.94790

T070701C 0.91212

T070701D 0.95349

T070701M 0.96059

T070801B 0.89617

T070801C 0.91600

T070801D 0.82747

T070801M 0.96929

T070901B 0.87467

T070901C 0.88192

T070901D 0.83821

T070901M 0.97106

T070902B 0.91137

T070902C 0.92186

T070902D 0.85099

T070902M 0.95523

T070903B 0.90423

T070903C 0.94167

T070903D 0.92631

T070903M 0.86007

T070904B 0.86201

T070904C 0.86885

T070904D 0.90314

T070904M 0.97264

T070905B 0.90061

T070905C 0.93205

T070905D 0.92261

T070905M 0.95278

T070906B 0.88783

T070906C 0.92884

T070906D 0.88865

T070906M 0.96919

T070907B 0.90424

T070907C 0.92639

T070907D 0.86936

T070907M 0.96095

T071001B 0.87884

T071001C 0.89274

T071001D 0.90952

T071001E 0.87547

T071001M 0.82144

T071101B 0.91317

T071101C 0.92147

T071101D 0.88579

T071101M 0.97353

T071102B 0.90754

T071102C 0.92876

T071102D 0.88164

T071102M 0.99058

T071103B 0.89707

T071103C 0.90660

T071103D 0.89819

T071103M 0.95560

T071104B 0.92217

T071104C 0.93341

T071104D 0.87991

T071104M 0.97470

T071105B 0.87776

T071105C 0.95508

T071105D 0.95897

T071105M 0.95635

T071106B 0.87377

T071106C 0.93772

T071106D 0.91782

T071106M 0.97304

T071107B 0.89836

T071107C 0.95848

T071107D 0.96346

T071107M 0.98131

T071108B 0.85458

T071108C 0.96070

T071108D 0.96150

T071108M 0.98194

T071109B 0.86477

T071109C 0.96262

T071109D 0.96374

T071110B 0.85927

T071110C 0.95997

T071110D 0.96581

T071111B 0.86818

T071111C 0.94576

T071111D 0.96403

T071112B 0.90834

T071112C 0.95875

T071112D 0.97052
Table F-14 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Civics Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

810

T071112M 0.97706

T071113B 0.87978

T071113D 0.95181

T071113M 0.97579

T071114B 0.87575

T071114C 0.88997

T071114D 0.85311

T071114M 0.98028

T071115B 0.90064

T071115C 0.93988

T071115D 0.93087

T071115M 0.96204

T071116B 0.89302

T071116C 0.92509

T071116D 0.87187

T071116M 0.98305

T071201C 0.86854

T071201D 0.85645

T071201M 0.97687

T071202B 0.87791

T071202C 0.88543

T071202D 0.86543

T071202M 0.99072

T071203B 0.90384

T071203C 0.91674

T071203D 0.87769

T071203M 0.98767

T071204B 0.91945

T071204C 0.94033

T071204D 0.86365

T071204M 0.98455

T071205B 0.90197

T071205C 0.95981

T071205D 0.92655
Table F-15

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Civics Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

811

FEMALE 0.94845

BLACK 0.96445

HISPANIC 0.96227

ASIAN 0.95150

MEXICAN 0.93909

PUER RIC 0.97128

CUBN,OTH 0.96681

HISP-? 0.96614

MID CTY7 0.94681

FR/LCTY7 0.94761

FR/MCTY7 0.94708

LAR TWN7 0.94094

SML TWN7 0.94224

OTHER 0.95779

HS GRAD 0.96420

POST HS 0.96528

COL GRAD 0.96307

PARED-? 0.93804

S EAST 0.87212

CENTRAL 0.87043

WEST 0.88469

PRIVATE 0.93863

CATHOLIC 0.94450

BLACK 0.86519

HISPANIC 0.80950

ASIAN 0.80354

IEP-NO 0.98167

LEP-NO 0.93754

TITLE-N 0.79509

RED PRIC 0.94002

FREE 0.76730

INFO N/A 0.85391

SCH/REF 0.87977

SCH/NP 0.83309

TVLIN-0 0.98149

TV-QUAD 0.98117

HW-NO 0.93428

HW-YES 0.94326

HWLIN-0 0.97115

HWQUAD-0 0.90346

HITEM=3 0.95612

HITEM=4 0.97440

PGS>5 0.80778

PGS>10 0.82723

NO ACCOM 0.95387

G/R 22 0.90306

G/R 23 0.90389

G/R 24 0.95333

G/T 22 0.69518

G/T 23 0.74378

G/T 24 0.72053

G/T 25 0.94773

G/T 26 0.70647

G/T 27 0.71562

G/P 22 0.92411

G/P 23 0.92073

G/P 24 0.87327

G/P 25 0.88492

G/S 22 0.92302

G/S 23 0.89994

R/T 24 0.91088

R/T 25 0.89919

R/T 26 0.90708

R/T 27 0.95337

R/T 31 0.89057

R/T 32 0.91408

R/T 33 0.91890

R/T 34 0.90389

R/T 35 0.91354

R/T 36 0.94232

R/T 37 0.90436

R/T 41 0.91571

R/T 42 0.92585

R/T 43 0.91479

R/T 44 0.92140

R/T 45 0.92371

R/T 46 0.94052

R/T 47 0.93658

R/P 24 0.90594

R/P 25 0.89051

R/P 31 0.90200

R/P 32 0.82568

R/P 33 0.89785

R/P 34 0.88308

R/P 35 0.88870

R/P 41 0.86253

R/P 42 0.96373

R/P 43 0.95047

R/P 44 0.94513

R/P 45 0.91509

R/S 31 0.95807

R/S 32 0.95455

R/S 33 0.96089

R/S 41 0.93630

R/S 42 0.95352

R/S 43 0.95800

P/T 25 0.80532

P/T 26 0.76549

P/T 27 0.77380

P/T 31 0.95384

P/T 32 0.78682

P/T 33 0.79571

P/T 34 0.78541

P/T 35 0.74816

P/T 36 0.79374

P/T 37 0.96022

P/T 41 0.77612

P/T 42 0.78918

P/T 43 0.81244

P/T 44 0.82055

P/T 45 0.82643

P/T 46 0.96067

P/T 47 0.80898

P/T 51 0.80733

P/T 52 0.84014

P/T 53 0.80809

P/T 54 0.81844

P/T 55 0.97051

P/T 56 0.84263

P/T 57 0.85899

T/S 41 0.91766

T/S 42 0.91266

T/S 43 0.91789

T/S 51 0.91272

T/S 52 0.93455

T/S 53 0.94675

T/S 61 0.95434

T/S 62 0.94719

T/S 63 0.92860

T/S 71 0.94036

T/S 72 0.95147

T/S 73 0.95245

P/S 32 0.94722

P/S 33 0.91769

P/S 41 0.94828

P/S 42 0.92203

P/S 43 0.92881

P/S 51 0.90805

P/S 52 0.95932

P/S 53 0.94760

A/G 22 0.88884

A/R 22 0.93490

A/R 23 0.93255

A/R 24 0.96413

A/T 22 0.93952

A/T 23 0.92685

A/T 24 0.94826

A/T 25 0.97291

A/T 26 0.94201
Table F-15 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Civics Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

812

A/T 27 0.93058

A/P 22 0.89833

A/P 23 0.89056

A/P 24 0.89973

A/P 25 0.85537

A/S 22 0.96328

A/S 23 0.95543

A/I 22 0.98156

A/L 22 0.94985

BLACK 0.93531

HISPANIC 0.91095

ASIAN AM 0.92120

AMER IND 0.97321

OTHER 0.96502

B003001M 0.83598

B013001B 0.93532

B013001C 0.95491

B013001D 0.92962

B013001M 0.76664

B013101B 0.95590

B013101C 0.93315

B013101D 0.89763

B013101M 0.89512

B013201N 0.76537

B013201M 0.77139

B013301N 0.86189

B013301M 0.86623

B013401N 0.89107

B013401M 0.86293

B013501N 0.76175

B013501M 0.79743

B013601N 0.83057

B013601M 0.80874

B013701N 0.86946

B013701M 0.83769

B000901N 0.91810

B000901M 0.83204

B000903N 0.93927

B000903M 0.85102

B013801B 0.96261

B013801C 0.96620

B013801D 0.95679

B013801M 0.74256

B000905N 0.92277

B000905M 0.84091

B006601B 0.79841

B014001B 0.86211

B014001C 0.86742

B014001D 0.92196

B014001E 0.93064

B014001M 0.82109

B007301B 0.96219

B007301C 0.96582

B007301D 0.94809

B007301M 0.85039

B007401B 0.89362

B007401C 0.91028

B007401D 0.85047

B007401M 0.89906

B014101B 0.95070

B014101C 0.93300

B014101D 0.92688

B014101E 0.93384

B014101M 0.87377

P804001B 0.93955

P804001C 0.92631

P804001N 0.93748

P804001M 0.88245

P804101B 0.90955

P804101C 0.89875

P804101N 0.89729

P804101M 0.87599

P804201B 0.85714

P804201C 0.86261

P804201D 0.88520

P804201M 0.86588

P804301B 0.95925

P804301C 0.94562

P804301D 0.94818

P804301M 0.83719

P804302B 0.85168

P804302C 0.90330

P804302D 0.91491

P804302M 0.83831

P803501B 0.94823

P803501C 0.97086

P803501D 0.92959

P803501M 0.88980

P803701N 0.92829

P803701M 0.72254

P803702N 0.93287

P803702M 0.88247

P803703N 0.86717

P803703M 0.79660

P803704N 0.95248

P803704M 0.73926

P803705N 0.85813

P803705M 0.79196

P803706N 0.89929

P803706M 0.83332

P803707N 0.81733

P803707M 0.72963

P803708N 0.81677

P803708M 0.83522

P803709N 0.84013

P803709M 0.82489

P803710N 0.83527

P803710M 0.82363

P803711N 0.82721

P803711M 0.80988

P803712N 0.85010

P803712M 0.81879

P803901N 0.88924

P803901M 0.88322

C042701Y 0.91063

C042701N 0.90514

C042701M 0.91487

C042801N 0.86687

C042801M 0.88709

C042802N 0.84756

C042802M 0.90801

C042803N 0.85052

C042803M 0.87572

C042901B 0.83263

C042901C 0.89033

C042901D 0.91543

C042901E 0.91603

C042901F 0.91752

C042901G 0.91393

C042901M 0.93483

C036601N 0.91743

C036601C 0.93655

C036601D 0.95001

C036601M 0.97420

C043001B 0.89882

C043001C 0.90102

C043001D 0.90190

C043001E 0.87087

C043001M 0.98443

C043002B 0.87823

C043002C 0.88995

C043002D 0.93285

C043002E 0.94270

C043002M 0.95677

C043003B 0.87481

C043003C 0.89867

C043003D 0.93750

C043003E 0.92104

C043003M 0.95332

C043004B 0.87353

C043004C 0.89226

C043004D 0.88453

C043004E 0.86804
Table F-15 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Civics Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

813

C043004M 0.94581

C043005B 0.92423

C043005C 0.92432

C043005D 0.90959

C043005E 0.86518

C043005M 0.97449

C043006B 0.86042

C043006C 0.85756

C043006D 0.83307

C043006E 0.91734

C043006M 0.98281

C043007B 0.87709

C043007C 0.89310

C043007D 0.85444

C043007E 0.89006

C043007M 0.97971

C043008B 0.86537

C043008C 0.86935

C043008D 0.91115

C043008E 0.83616

C032402B 0.87814

C032402C 0.92609

C032402N 0.90076

C032402M 0.97666

C032401B 0.90394

C032401C 0.93585

C032401N 0.87973

C032401M 0.96770

C032404B 0.88936

C032404C 0.93746

C032404N 0.87097

C032404M 0.92359

C032407B 0.85600

C032407C 0.93991

C032407N 0.93241

C032407M 0.97909

C032408B 0.86451

C032408C 0.95381

C032408N 0.92979

C032408M 0.97437

C032409B 0.90118

C032409C 0.92506

C032409N 0.90065

C032409M 0.98418

C032410B 0.90342

C032410C 0.93319

C032410N 0.93925

C032410M 0.96490

C032411B 0.90878

C032411C 0.92868

C032411N 0.92227

C032411M 0.97335

C032412B 0.89248

C032412C 0.92155

C032412N 0.92838

C032412M 0.96713

C032413B 0.90063

C032413C 0.94132

C032413N 0.94478

C032413M 0.99632

C032414B 0.92179

C032414C 0.94296

C032414N 0.85191

C032414M 0.97112

C043101B 0.87694

C043101C 0.95591

C043101N 0.93579

C043102B 0.90021

C043102C 0.94819

C043102N 0.92700

C043103B 0.88591

C043103C 0.83470

C043104B 0.85614

C043104C 0.95158

C043104N 0.93380

C043104M 0.97075

C032502B 0.87271

C032502C 0.87820

C032502D 0.85012

C032502M 0.97079

C032503B 0.87997

C032503C 0.88136

C032503D 0.86240

C032503M 0.95923

C032505B 0.87650

C032505C 0.87031

C032505D 0.87776

C032505M 0.98365

C032506B 0.84981

C032506C 0.87123

C032506D 0.86207

C032506M 0.97014

C043201B 0.84825

C043201C 0.86886

C043201M 0.97876

C043301B 0.88691

C043301C 0.91072

C043301D 0.86000

C043301M 0.94380

C043401B 0.87144

C043401C 0.87060

C043401M 0.95078

C043501B 0.88474

C043501C 0.88973

C043501D 0.86841

C043501E 0.88983

C043501F 0.87241

C043501M 0.93190

C043601B 0.86748

C043601C 0.87376

C043601D 0.88352

C043601E 0.85955

C043601M 0.93963

C043701B 0.85627

C043701C 0.87304

C043701D 0.90017

C043701E 0.90061

C043701M 0.94824

C038301N 0.84644

C038301M 0.93321

C043801B 0.87795

C043801C 0.89195

C043801D 0.89471

C043801E 0.89904

C043801F 0.88709

C043801G 0.89625

C043801H 0.86095

C043801M 0.87473

C043901N 0.92072

C043901M 0.94631

C044001B 0.88648

C044001C 0.87577

C044001D 0.88224

C044001E 0.87898

C044001F 0.87205

C044001G 0.87426

C044001H 0.89529

C044001M 0.93271

C044002B 0.89071

C044002C 0.90245

C044002D 0.87740

C044002E 0.88550

C044002F 0.90784

C044002G 0.87259

C044002H 0.89743

C044002M 0.93358

C044003B 0.86866

C044003C 0.87141

C044003D 0.86229

C044003E 0.89241

C044003F 0.90089

C044003H 0.86757

C044003M 0.93576
Table F-15 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Civics Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

814

C044004B 0.88350

C044004C 0.90158

C044004D 0.88337

C044004E 0.85085

C044004F 0.87565

C044004G 0.93310

C044004H 0.91149

C044004M 0.93655

B014201B 0.97002

B014201C 0.98000

B014201D 0.97976

B014201E 0.96559

B014201M 0.93073

P804401N 0.77604

P804401M 0.81127

P804402N 0.76741

P804402M 0.80149

P804403N 0.77448

P804403M 0.87516

P804404N 0.74843

P804404M 0.86739

P804405N 0.77890

P804405M 0.83680

P804406N 0.77480

P804406M 0.86683

P804407N 0.79771

P804407M 0.86906

P804408N 0.84693

P804408M 0.83763

P804409N 0.83482

P804409M 0.84151

P804501B 0.85348

P804501C 0.92684

P804501D 0.92713

P804501E 0.94917

P804501F 0.96074

P804501M 0.72352

C044401N 0.88527

C044401M 0.95808

C044402N 0.87436

C044402M 0.92614

C043105B 0.88136

C043105C 0.93912

C043105N 0.95228

C043106B 0.86723

C043106C 0.93351

C043106N 0.93435

T067301B 0.88489

T067301C 0.85086

T067301D 0.93649

T067301M 0.96687

T056201B 0.86696

T056201C 0.87118

T056201D 0.88622

T056201E 0.90545

T056201F 0.82839

T056201M 0.92599

T056301B 0.86632

T056301C 0.94880

T056301D 0.94881

T056301E 0.86926

T056301F 0.88692

T056301G 0.89375

T056301M 0.96785

T067501B 0.87182

T067501C 0.87087

T067501M 0.88121

T067502B 0.85470

T067502C 0.83419

T067502M 0.85140

T067503B 0.87397

T067503C 0.97148

T067503M 0.98003

T067504B 0.91230

T067504C 0.98484

T067504M 0.98988

T067505B 0.84747

T067505C 0.98785

T067505M 0.99012

T067506B 0.87183

T067506C 0.97466

T067506M 0.98914

T067507B 0.85962

T067507C 0.98093

T067507M 0.98249

T067508B 0.88220

T067508C 0.91130

T067508M 0.92147

T067509B 0.86033

T067509C 0.94934

T067509M 0.98050

T067510B 0.86800

T067510C 0.84393

T067510M 0.83242

T067511B 0.88094

T067511C 0.88798

T067511M 0.88289

T067512B 0.89003

T067512C 0.85800

T067512M 0.84951

T067601B 0.86824

T067601C 0.88964

T067601M 0.92557

T067602B 0.85524

T067602C 0.85590

T067602M 0.88857

T067603B 0.84736

T067603C 0.96949

T067603M 0.97890

T067604B 0.88215

T067604C 0.97761

T067604M 0.98097

T067605B 0.85332

T067605C 0.88480

T067605M 0.92374

T067606B 0.88152

T067606C 0.92954

T067606M 0.96353

T067607B 0.86456

T067607C 0.94910

T067607M 0.94952

T067608B 0.86220

T067608C 0.97467

T067608M 0.97281

T067609B 0.87655

T067609C 0.94633

T067609M 0.96373

T067610B 0.86769

T067610C 0.88803

T067610M 0.91493

T067611B 0.87295

T067611C 0.95620

T067611M 0.97030

T067612B 0.88966

T067612C 0.85810

T067612M 0.86529

T067701B 0.88371

T067701C 0.89417

T067701D 0.88230

T067701E 0.88126

T067701M 0.86589

T067702B 0.89336

T067702C 0.89321

T067702D 0.87839

T067702E 0.90113

T067702M 0.90566

T067801B 0.88534

T067801C 0.87959

T067801M 0.92906

T067802B 0.87928

T067802C 0.87910

T067802M 0.98974

T067803B 0.82885
Table F-15 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Civics Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

815

T067803C 0.85113

T067803M 0.97991

T067804B 0.86707

T067804C 0.85668

T067804M 0.98992

T067805B 0.91873

T067805C 0.91981

T067805M 0.98315

T067806B 0.89922

T067806C 0.90604

T067806M 0.97136

T067807B 0.85906

T067807C 0.85180

T067807M 0.96099

T041201B 0.89104

T041201C 0.90699

T041201D 0.86863

T041201M 0.98823

T070401B 0.84665

T070401C 0.86838

T070401M 0.99014

T070402B 0.88292

T070402C 0.88464

T070402M 0.98015

T070403B 0.84924

T070403C 0.87046

T070403M 0.98573

T070404B 0.88611

T070404C 0.88907

T070404M 0.98093

T070405B 0.84937

T070405C 0.85612

T070405M 0.98448

T070406B 0.90574

T070406C 0.91483

T070406M 0.95758

T070407B 0.89413

T070407C 0.91205

T070407M 0.98695

T070601N 0.81174

T070601M 0.95548

T070701B 0.90837

T070701C 0.90239

T070701D 0.91686

T070701M 0.96260

T070801B 0.93660

T070801C 0.93690

T070801D 0.87300

T070801M 0.96242

T070901B 0.83552

T070901C 0.86949

T070901D 0.85344

T070901M 0.97280

T070902B 0.87302

T070902C 0.90199

T070902D 0.86629

T070902M 0.96570

T070903B 0.92256

T070903C 0.94777

T070903D 0.89671

T070903M 0.95356

T070904B 0.87195

T070904C 0.88714

T070904D 0.85017

T070904M 0.97511

T070905B 0.88035

T070905C 0.92167

T070905D 0.93452

T070905M 0.97034

T070906B 0.89576

T070906C 0.92695

T070906D 0.86967

T070906M 0.97424

T070907B 0.91943

T070907C 0.94125

T070907D 0.90157

T070907M 0.95445

T071101B 0.89144

T071101C 0.91066

T071101D 0.90608

T071101M 0.98168

T071102B 0.87797

T071102C 0.89345

T071102D 0.87337

T071102M 0.95878

T071103B 0.85519

T071103C 0.88011

T071103D 0.86620

T071103M 0.98696

T071104B 0.91479

T071104C 0.94124

T071104D 0.86781

T071104M 0.98744

T071105B 0.89341

T071105C 0.95132

T071105D 0.94602

T071105M 0.98645

T071106B 0.89821

T071106C 0.92396

T071106D 0.89817

T071106M 0.98348

T071107B 0.89320

T071107C 0.94239

T071107D 0.92647

T071107M 0.98957

T071108B 0.87870

T071108C 0.94072

T071108D 0.92847

T071108M 0.99281

T071109B 0.87971

T071109C 0.95144

T071109D 0.95989

T071109M 0.98982

T071110B 0.90068

T071110C 0.91767

T071110D 0.95547

T071110M 0.98746

T071111B 0.83903

T071111C 0.90271

T071111D 0.96574

T071111M 0.98826

T071112B 0.86829

T071112C 0.93109

T071112D 0.96226

T071112M 0.98671

T071113B 0.87020

T071113C 0.95196

T071113D 0.94767

T071113M 0.99071

T071114B 0.86804

T071114C 0.87833

T071114D 0.87071

T071114M 0.99199

T071115B 0.87842

T071115C 0.93099

T071115D 0.93701

T071115M 0.97029

T071116B 0.84865

T071116C 0.84439

T071116D 0.84389

T071116M 0.99256

T071201B 0.85847

T071201C 0.87032

T071201D 0.85249

T071201M 0.97232

T071202B 0.88807

T071202C 0.88662

T071202D 0.85975

T071202M 0.97271

T071203B 0.86067

T071203C 0.87297

T071203D 0.84453

T071203M 0.98073
Table F-15 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Civics Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

816

T071204B 0.90278

T071204C 0.92661

T071204D 0.84431

T071204M 0.98471

T071205B 0.90089

T071205C 0.94359

T071205D 0.87910

T071205M 0.97442

T071301B 0.90069

T071301C 0.90977

T071301D 0.87784

T071301M 0.92135

T071401M 0.84352

T071402M 0.85345

T071403M 0.88033

T071404M 0.84831

T040301B 0.86044

T040301C 0.85678

T040301D 0.89932

T040301E 0.88761

T040301M 0.95633

T071501B 0.85737

T071501C 0.88150

T071501D 0.85323

T071501E 0.86411

T071501F 0.85716

T071501M 0.87859

T071502B 0.85122

T071502C 0.86136

T071502D 0.86861

T071502E 0.87343

T071502F 0.85434

T071502M 0.87731

T071503B 0.82484

T071503C 0.88346

T071503D 0.87727

T071503E 0.88783

T071503F 0.85516

T071503M 0.87297

T071504B 0.88817

T071504C 0.89637

T071504D 0.87513

T071504E 0.88133

T071504F 0.86190

T071504M 0.91370

CLASIZ-2 0.89568

CLASIZ-3 0.90022

CLASIZ-4 0.90105

CLASIZ-5 0.93049

CLASIZ-? 0.82343
Table F-16

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Civics Conditioning Variables, Grade 12

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

817

FEMALE 0.94934

BLACK 0.97088

HISPANIC 0.97106

ASIAN 0.95455

MEXICAN 0.94580

PUER RIC 0.97328

CUBN,OTH 0.97283

HISP-? 0.97285

MID CTY7 0.94779

FR/LCTY7 0.94975

FR/MCTY7 0.95934

LAR TWN7 0.94780

SML TWN7 0.95153

OTHER 0.93465

HS GRAD 0.96351

POST HS 0.96398

COL GRAD 0.96660

PARED-? 0.91531

S EAST 0.85715

CENTRAL 0.88010

WEST 0.89486

PRIVATE 0.95110

CATHOLIC 0.95562

BLACK 0.88230

HISPANIC 0.82595

ASIAN 0.81428

IEP-NO 0.97551

LEP-NO 0.94751

TITLE-N 0.80768

RED PRIC 0.96619

FREE 0.84003

INFO N/A 0.89676

SCH/REF 0.92981

SCH/NP 0.84211

TVLIN-0 0.97831

TV-QUAD 0.97808

HW-NO 0.95942

HW-YES 0.96668

HWLIN-0 0.97864

HWQUAD-0 0.89491

HITEM=3 0.97788

HITEM=4 0.97566

PGS>5 0.83230

PGS>10 0.83822

NO ACCOM 0.94609

NYRCIV B 0.96669

NYRCIV C 0.95642

NYRCIV D 0.95675

NYRCIV E 0.96404

NYRCIV2B 0.96605

NYRCIV2C 0.96765

G/R 22 0.91024

G/R 23 0.91351

G/R 24 0.95034

G/T 22 0.78957

G/T 23 0.84176

G/T 24 0.72174

G/T 25 0.95984

G/T 26 0.71068

G/T 27 0.74878

G/P 22 0.93278

G/P 23 0.96109

G/P 24 0.90911

G/P 25 0.92806

G/S 22 0.93496

G/S 23 0.90516

R/T 24 0.91627

R/T 25 0.90567

R/T 26 0.91173

R/T 27 0.97079

R/T 31 0.89430

R/T 32 0.90983

R/T 33 0.91774

R/T 34 0.91559

R/T 35 0.90895

R/T 36 0.94931

R/T 37 0.91087

R/T 41 0.92434

R/T 42 0.92940

R/T 43 0.91749

R/T 44 0.92552

R/T 45 0.92702

R/T 46 0.93249

R/T 47 0.94070

R/P 24 0.89620

R/P 25 0.89568

R/P 31 0.91151

R/P 32 0.84085

R/P 33 0.89928

R/P 34 0.89769

R/P 35 0.90419

R/P 41 0.88435

R/P 42 0.93613

R/P 43 0.94288

R/P 44 0.93717

R/P 45 0.90765

R/S 31 0.94727

R/S 32 0.94331

R/S 33 0.94810

R/S 41 0.94076

R/S 42 0.95657

R/S 43 0.95203

P/T 25 0.84584

P/T 26 0.82375

P/T 27 0.85162

P/T 31 0.95567

P/T 32 0.81737

P/T 33 0.80104

P/T 34 0.80039

P/T 35 0.79572

P/T 36 0.81721

P/T 37 0.96593

P/T 41 0.77688

P/T 42 0.77277

P/T 43 0.81782

P/T 44 0.81842

P/T 45 0.83551

P/T 46 0.96239

P/T 47 0.81893

P/T 51 0.81371

P/T 52 0.92148

P/T 53 0.92009

P/T 54 0.93432

P/T 55 0.95709

P/T 56 0.88045

P/T 57 0.91748

T/S 41 0.92885

T/S 42 0.90670

T/S 43 0.93258

T/S 51 0.91792

T/S 52 0.95956

T/S 53 0.96904

T/S 61 0.96396

T/S 62 0.94860

T/S 63 0.91904

T/S 72 0.93354

P/S 32 0.94384

P/S 33 0.90852

P/S 41 0.93891

P/S 42 0.89834

P/S 43 0.92766

P/S 51 0.89571

P/S 52 0.94041

P/S 53 0.92633

A/G 22 0.92245

A/R 22 0.95035

A/R 23 0.95505

A/R 24 0.96601

A/T 22 0.94194

A/T 23 0.93513
Table F-16 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Civics Conditioning Variables, Grade 12

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

818

A/T 24 0.94559

A/T 25 0.95285

A/T 26 0.94244

A/P 22 0.93072

A/P 23 0.93001

A/P 24 0.93550

A/P 25 0.90807

A/S 22 0.96215

A/S 23 0.96102

A/I 22 0.97816

A/L 22 0.94576

G/N 22 0.87342

G/N 23 0.93783

G/N 24 0.91273

G/N 25 0.92314

R/N 24 0.90131

R/N 25 0.90903

R/N 31 0.90932

R/N 32 0.90778

R/N 33 0.91069

R/N 34 0.90975

R/N 35 0.89878

R/N 41 0.90818

R/N 42 0.94364

R/N 43 0.94848

R/N 44 0.95085

R/N 45 0.94462

N/T 25 0.85303

N/T 26 0.84298

N/T 27 0.87920

N/T 31 0.96749

N/T 32 0.87207

N/T 33 0.86213

N/T 34 0.73287

N/T 35 0.79780

N/T 36 0.77108

N/T 37 0.97199

N/T 41 0.74908

N/T 42 0.77735

N/T 43 0.80269

N/T 44 0.89990

N/T 45 0.79900

N/T 46 0.94752

N/T 47 0.78972

N/T 51 0.79346

N/T 52 0.84293

N/T 53 0.86933

N/T 54 0.84904

N/T 55 0.97629

N/T 56 0.87206

N/T 57 0.85220

P/N 25 0.84596

P/N 31 0.90297

P/N 32 0.90278

P/N 33 0.90486

P/N 34 0.88141

P/N 35 0.94026

P/N 41 0.91959

P/N 42 0.91415

P/N 43 0.87743

P/N 44 0.91419

P/N 45 0.90549

P/N 51 0.88846

P/N 52 0.90834

P/N 53 0.94505

P/N 54 0.92052

P/N 55 0.92086

N/S 32 0.92796

N/S 33 0.89664

N/S 41 0.93662

N/S 42 0.89840

N/S 43 0.93936

N/S 51 0.89214

N/S 52 0.94053

N/S 53 0.90242

A/N 22 0.92329

A/N 23 0.91557

A/N 24 0.92196

A/N 25 0.92354

G/N 22 0.88415

G/N 23 0.88554

R/N 31 0.89922

R/N 32 0.89966

R/N 33 0.91471

R/N 41 0.90359

R/N 42 0.94429

R/N 43 0.93880

T/N 41 0.81675

T/N 42 0.92146

T/N 43 0.78898

T/N 51 0.91045

T/N 52 0.79521

T/N 53 0.91459

T/N 61 0.97517

T/N 62 0.95992

T/N 63 0.80627

T/N 71 0.92480

T/N 72 0.80572

T/N 73 0.91860

P/N 32 0.85609

P/N 33 0.90426

P/N 41 0.89596

P/N 42 0.90622

P/N 43 0.86903

P/N 51 0.91289

P/N 52 0.91991

P/N 53 0.91374

S/N 23 0.92954

S/N 31 0.93608

S/N 32 0.91557

S/N 33 0.92148

A/N 22 0.92424

A/N 23 0.92659

BLACK 0.93587

HISPANIC 0.93141

ASIAN AM 0.91401

AMER IND 0.98884

OTHER 0.98201

B003001M 0.74364

B013001B 0.93459

B013001C 0.93199

B013001D 0.90967

B013001M 0.81342

B013101B 0.95737

B013101C 0.92129

B013101D 0.85693

B013101M 0.80173

B013201N 0.74601

B013201M 0.79006

B013301N 0.82487

B013301M 0.85558

B013401N 0.85191

B013401M 0.85052

B013501N 0.80090

B013501M 0.82410

B013601N 0.82916

B013601M 0.83672

B013701N 0.84905

B013701M 0.83855

B000901N 0.96658

B000901M 0.77723

B000903N 0.97076

B000903M 0.76656

B013801B 0.97154

B013801C 0.96608

B013801D 0.96007

B013801M 0.85026

B000905N 0.97483

B000905M 0.74518

B006601B 0.81797

B014001B 0.87302

B014001C 0.89114

B014001D 0.93218
Table F-16 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Civics Conditioning Variables, Grade 12

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

819

B014001E 0.93731

B014001M 0.88206

B007301B 0.97545

B007301C 0.98471

B007301D 0.96625

B007301M 0.89159

B007401B 0.89418

B007401C 0.91545

B007401D 0.86082

B007401M 0.88191

B014101B 0.95779

B014101C 0.93125

B014101D 0.92505

B014101E 0.93535

B014101M 0.87184

P804301B 0.98186

P804301C 0.97081

P804301D 0.96849

P804301M 0.94391

P804302B 0.95054

P804302C 0.96699

P804302D 0.97522

P804302M 0.94468

P803701N 0.90151

P803701M 0.79921

P803702N 0.94977

P803702M 0.69049

P803703N 0.96534

P803703M 0.69650

P803704N 0.94443

P803704M 0.78991

P803705N 0.95172

P803705M 0.72074

P803706N 0.96560

P803706M 0.76154

P803707N 0.83737

P803707M 0.80366

P803708N 0.81362

P803708M 0.74296

P803709N 0.91112

P803709M 0.72446

P803710N 0.89027

P803710M 0.72648

P803711N 0.87730

P803711M 0.73896

P803712N 0.93109

P803712M 0.72761

P803901N 0.84393

P803901M 0.82221

C042701Y 0.89661

C042701N 0.87374

C042701M 0.96944

C042801N 0.84863

C042801M 0.87489

C042802N 0.90084

C042802M 0.87711

C042803N 0.86792

C042803M 0.86333

C042901B 0.88267

C042901C 0.91057

C042901D 0.92162

C042901E 0.92155

C042901F 0.93999

C042901G 0.95327

C036601N 0.90253

C036601C 0.93843

C036601D 0.96001

C036601M 0.96014

C043001B 0.89178

C043001C 0.90548

C043001D 0.91920

C043001E 0.86320

C043001M 0.97664

C043002B 0.89319

C043002C 0.92090

C043002D 0.94812

C043002E 0.91745

C043002M 0.99608

C043003B 0.92692

C043003C 0.92696

C043003D 0.94974

C043003E 0.90465

C043003M 0.99155

C043004B 0.89752

C043004C 0.91490

C043004D 0.90124

C043004E 0.85554

C043004M 0.98313

C043005B 0.92098

C043005C 0.94583

C043005D 0.92609

C043005E 0.87717

C043006B 0.89483

C043006C 0.87960

C043006D 0.88105

C043006E 0.96252

C043007B 0.88682

C043007C 0.88751

C043007D 0.89832

C043007E 0.88890

C043007M 0.98854

C043008B 0.87191

C043008C 0.86212

C043008D 0.98401

C043008E 0.93730

C043008M 0.96181

C032402B 0.90073

C032402C 0.94410

C032402N 0.90108

C032401B 0.93023

C032401C 0.93390

C032401N 0.88756

C032404B 0.88480

C032404C 0.95833

C032404N 0.90438

C032407B 0.91266

C032407C 0.96181

C032407N 0.95908

C032408B 0.86681

C032408C 0.95730

C032408N 0.93663

C032408M 0.96303

C032409B 0.92122

C032409C 0.94177

C032409N 0.90336

C032409M 0.96528

C032410B 0.89052

C032410C 0.90429

C032410N 0.85974

C032410M 0.98695

C032411B 0.91505

C032411C 0.92794

C032411N 0.87517

C032412B 0.90053

C032412C 0.91830

C032412N 0.88006

C032413B 0.87071

C032413C 0.94574

C032413N 0.93412

C032413M 0.97392

C032414B 0.91259

C032414C 0.96318

C032414N 0.90069

C032414M 0.96316

C043101B 0.91624

C043101C 0.96823

C043101N 0.90812

C043101M 0.96239

C043102B 0.90489

C043102C 0.94445

C043102N 0.92652

C043103B 0.85757

C043103C 0.82104
Table F-16 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Civics Conditioning Variables, Grade 12

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

820

C043104B 0.90071

C043104C 0.95871

C043104N 0.93074

C032502B 0.86474

C032502C 0.89367

C032502M 0.95251

C032503B 0.86686

C032503C 0.87694

C032503D 0.89709

C032505B 0.85189

C032505C 0.87696

C032505D 0.86781

C032506B 0.88635

C032506C 0.88894

C032506D 0.87528

C032506M 0.97941

C043201B 0.85637

C043201C 0.89877

C043201M 0.97340

C043301B 0.92888

C043301C 0.92997

C043301D 0.89427

C043301E 0.87594

C043301M 0.93766

C043401B 0.89189

C043401C 0.88493

C043401D 0.88518

C043401M 0.97966

C043501B 0.90620

C043501C 0.89906

C043501D 0.90199

C043501E 0.91928

C043501F 0.91003

C043501M 0.94386

C043601B 0.87133

C043601C 0.89844

C043601D 0.87814

C043601E 0.89165

C043601M 0.94488

C043701B 0.87917

C043701C 0.87439

C043701D 0.88386

C043701E 0.90469

C038301N 0.83286

C038301M 0.96224

C043801B 0.88098

C043801C 0.90638

C043801D 0.94547

C043801E 0.94036

C043801F 0.89351

C043801G 0.88739

C043801H 0.88106

C043801M 0.86475

C043901N 0.92834

C043901M 0.91454

C044001B 0.90296

C044001C 0.92159

C044001D 0.89312

C044001E 0.86322

C044001F 0.89676

C044001G 0.85605

C044001H 0.87454

C044001M 0.83375

C044002B 0.89405

C044002C 0.89704

C044002D 0.89928

C044002E 0.90414

C044002F 0.92560

C044002G 0.89739

C044002H 0.93263

C044002M 0.91576

C044003B 0.86846

C044003C 0.88765

C044003D 0.89744

C044003E 0.90203

C044003F 0.92897

C044003G 0.91085

C044003H 0.91536

C044003M 0.92084

C044004B 0.88034

C044004C 0.91166

C044004D 0.89127

C044004E 0.91831

C044004F 0.91651

C044004G 0.90356

C044004H 0.91340

C044004M 0.87315

P804401N 0.79972

P804401M 0.74462

P804402N 0.82734

P804402M 0.74825

P804403N 0.76843

P804403M 0.68234

P804404N 0.78919

P804404M 0.72012

P804405N 0.78563

P804405M 0.71796

P804406N 0.77293

P804406M 0.71090

P804407N 0.73331

P804407M 0.69103

P804408N 0.84031

P804408M 0.78321

P804409N 0.82184

P804409M 0.82357

C043105B 0.90385

C043105C 0.92373

C043105N 0.87108

C043105M 0.96741

C043106B 0.90681

C043106C 0.92625

C043106N 0.87956

C043106M 0.97736

B005501B 0.97129

B005501C 0.96065

B005501D 0.93778

B005501E 0.98095

B005501F 0.97418

B005501M 0.79557

B014301Y 0.87564

B014301N 0.82781

B014301M 0.84991

B014401B 0.95334

B014401C 0.94183

B014401D 0.93226

B014401E 0.91122

B014401F 0.89327

B014401M 0.86441

P802545B 0.97784

P802545C 0.96551

P802545N 0.96597

P802545M 0.87927

P802546B 0.96034

P802546C 0.95642

P802546N 0.95655

P802546M 0.95284

P802547B 0.91066

P802547C 0.91017

P802547D 0.87600

P802547M 0.95030

P804601M 0.85123

P804603M 0.80948

P804701B 0.92148

P804701C 0.92254

P804701D 0.92640

P804701E 0.95469

P804701F 0.97043

P804701M 0.87641

P804801N 0.91200

P804801M 0.92153

P804901N 0.94894

P804901M 0.89461

C044301N 0.87259
Table F-16 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

National Civics Conditioning Variables, Grade 12

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

821

C044301M 0.96960

C044302N 0.89023

C044302M 0.88920

C044101B 0.87863

C044101C 0.87043

C044101D 0.89007

C044101E 0.86630

C044101M 0.96328

C044201B 0.87486

C044201C 0.89977

C044201D 0.95516

C044201E 0.93389

C044201F 0.91593

C044201G 0.87949

C044201H 0.92831

C044201M 0.91415

C044202B 0.92408

C044202C 0.89347

C044202D 0.92639

C044202E 0.94405

C044202F 0.93901

C044202G 0.89545

C044202H 0.88406

C044202M 0.93634

823

Appendix G

REPORTING SUBGROUPS AND SPECIAL VARIABLES

FOR THE 1998 NAEP ASSESSMENT

G.1 MAJOR REPORTING SUBGROUPS

Results for the 1998 assessment were reported for student subgroups defined by gender,

race/ethnicity, type of location, parents’ level of education, eligibility for the National School Lunch

Program, enrollment in Title I funding, school type, and geographical region. The following explains

how each of these subgroups was derived.

Gender (DSEX)

The variable SEX is the gender of the student being assessed, as taken from school records. For a

few students, data for this variable was missing and was imputed by ETS after the assessment. The

resulting variable DSEX contains a value for every student and is used for gender comparisons among

students.

Race/Ethnicity (DRACE)

The variable DRACE is an imputed definition of race/ethnicity, derived from up to three sources

of information. This variable is used for race/ethnicity subgroup comparisons in the 1998 national and

state assessments (reading, writing, and civics). Two items from the student demographics questionnaire

were used in determining derived race/ethnicity:

Demographic Item Number 2:

2. If you are Hispanic, what is your Hispanic background?

 I am not Hispanic.

 Mexican, Mexican American, or Chicano

 Puerto Rican

 Cuban

 Other Spanish or Hispanic background
824

Students who responded to Item Number 2 by filling in the second, third, fourth, or fifth oval

were considered Hispanic. For students who filled in the first oval, did not respond to the item, or

provided information that was illegible or could not be classified, responses to item number 1 were

examined in an effort to determine race/ethnicity. Item Number 1 read as follows:

Demographic Item Number 1:

1. Which best describes you?

 White (not Hispanic)

 Black (not Hispanic)

 Hispanic (“Hispanic” means someone who is Mexican, Mexican

American, Chicano, Puerto Rican, Cuban, or from some other

Spanish or Hispanic background.)

 Asian or Pacific Islander (“Asian or Pacific Islander” means

someone who is Chinese, Japanese, Korean, Filipino, Vietnamese,

or from some other Asian or Pacific Island background.)

 American Indian or Alaskan Native (“American Indian or Alaskan

Native” means someone who is from one of the American Indian

tribes, or one of the original people of Alaska.)

 Other (What?) _________

Students’ race/ethnicity was then assigned to correspond with their selection. For students who

filled in the sixth oval (Other), provided illegible information or information that could not be classified,

or did not respond at all, race/ethnicity as provided from school records was used. Derived race/ethnicity

could not be determined for students who did not respond to background items 1 or 2 and for whom

race/ethnicity was not provided by the school.

Type of Location (TOL3)

The variable TOL3 is used in the 1998 national and state assessments to provide information

about school location types:

1 Central City (Large Central City and Midsize Central City) This category

includes central cities of all MSAs. Central City is a geographic

term and is not synonymous with “inner city.”

2 Urban Fringe/Large Town (Urban Fringe of Large City, Urban Fringe of Midsize City, and

Large Town) An Urban Fringe includes all densely settled

places and areas within MSAs that are classified as urban by the

Bureau of Census. A Large Town is defined as a place outside

MSAs with a population greater than or equal to 25,000.
825

3 Rural/Small Town (Small Town, Rural MSA, and Rural Non–MSA) Rural includes

all places and areas with a population of less than 2,500 that are

classified as rural by the Bureau of Census. A Small Town is

defined as a place outside MSAs with a population of less than

25,000 but greater than or equal to 2,500.

Parents’ Education Level (PARED2, PARED)

Parents’ education was reported at four levels—did not finish high school, graduated high

school, had some education after high school, or graduated college—gathered from student responses to

questions about the extent of schooling experienced by each of their parents. In the 1998 assessment, this

information was gathered in two different ways.

Students at grades 4, 8, and 12 in the writing and civics assessments and at grade 4 in the reading

assessment were asked to respond to six questions (three for each parent) requiring a yes/no response.

The response indicating the highest level of education was selected for reporting (PARED2).

At grades 8 and 12 in the reading assessment, a different procedure (one that had been used in

previous reading assessments) was used to gather parental education data. Students were asked to select

the appropriate level of education from one overall question for each parent. Again, the response

indicating the highest level of education was selected for reporting (PARED).

Region of the Country (REGION)

Jurisdictions were grouped into four geographical regionsNortheast, Southeast, Central, and

Westas shown in Table G-1. All 50 states and the District of Columbia are listed. The part of Virginia

that is included in the Washington, D.C., metropolitan statistical area is included in the Northeast region;

the remainder of the state is included in the Southeast region.

Table G-1

NAEP Geographic Regions

NORTHEAST SOUTHEAST CENTRAL WEST

Connecticut

Delaware

District of Columbia

Maine

Maryland

Massachusetts

New Hampshire

New Jersey

New York

Pennsylvania

Rhode Island

Vermont

Virginia

Alabama

Arkansas

Florida

Georgia

Kentucky

Louisiana

Mississippi

North Carolina

South Carolina

Tennessee

Virginia

West Virginia

Illinois

Indiana

Iowa

Kansas

Michigan

Minnesota

Missouri

Nebraska

North Dakota

Ohio

South Dakota

Wisconsin

Alaska

Arizona

California

Colorado

Hawaii

Idaho

Montana

Nevada

New Mexico

Oklahoma

Oregon

Texas

Utah

Washington

Wyoming
826

Title I Participation (TITLE1 )

Based on available school records, students were classified as either currently participating in a

Title I program or receiving Title I services, or as not receiving such services. The classification applies

only to the school year when the assessment was administered and is not based on participation in

previous years. If the school did not offer any Title I programs or services, all students in that school

were classified as not participating.

Eligibility for the Free/Reduced-Price School Lunch Program (SLUNCH1)

Based on available school records, students were classified as either currently eligible or not

currently eligible for the free/reduced-price lunch component of the Department of Agriculture’s

National School Lunch Program. The classification refers only to the school year when the assessment

was administered and is not based on eligibility in previous years. If school records were not available,

the student was classified as "Information not available." If the school did not participate in the program,

all students in that school were classified as "Information not available."

Type of School (SCHTY98, SCHTYPE)

School type information was initially provided by Westat and was used to determine the type of

school that a student attended. The values for the variable SCHTY98 were identified as:

1 Public

2 Religious

3 Other

4 Catholic

5 Bureau of Indian Affairs

6 Department of Defense

7 State Department of Education (Charter)

The SCHTY98 values were collapsed into a five-level variable called SCHTYPE:

1 Public (SCHTY98 categories 1 and 7)

2 Private (SCHTY98 categories 2 and 3)

3 Catholic

4 Bureau of Indian Affairs

5 Department of Defense
827

G.2 WRITING DERIVED VARIABLES

Times Prewriting was Observed (WRIPRE)

For each cognitive item taken by each student, the corresponding rater 1 prewriting field was

checked. Since students were given at most 2 essays, this variable ranged from 0-2. It was coded on the

database as follows:

1 = no evidence of prewriting

2 = one essay showed evidence of prewriting

3 = both essays showed evidence of prewriting missing if

both prewriting variables were missing.

This variable was used at all grades sampled for the national assessment (grade 4, grade 8, and grade 12).

It was also used at grade 8 in the state assessment.

Types of Writing Assignments Reported (WRIASGN) 

 The following variables can be checked for any indication that these types of writing were

assigned:

 W802801 Reports

 W802802 Essays--themes

 W802803 Essays-persuasive

 W802804 Story—narrative

Possible values for this variable were:

1 = none were assigned

2 = one was assigned

3 = two were assigned

4 = three were assigned

5 = all four were assigned

miss = if two or more of the above variables were missing,

a missing value code was assigned.

This variable was used at grades 8 and 12 for the national assessment and was used at grade 8 in the state

assessment.

Writing Steps Used in Planning (WRISTEP) 

The following variables can be checked for any indication that these types of writing were

assigned:

 W802901 Asked to plan writing

 W802903 Define Purpose of Writing

 W802904 Use other sources besides textbook
828

Values 1-4 correspond to 0-3 steps used. If two or more were missing, the variable was coded as missing.

This variable was used at grades 8 and 12 for the national assessment and was used at grade 8 in the state

assessment.

 

Number of Types of Writing Feedback Received (WRIFDBK)

The following variables can be checked for any indication that these types of writing were

assigned:

 W802001 Teacher Talks about what you are writing

 W802101 Teacher Asks to Write more than one Draft

 W802201 Teacher Asks to contribute Writing to a collection.

Values 1-4 correspond to 0-3 types used. If two or more were missing, the variable was coded as missing.

This variable was used at all grades sampled for the national assessment (grade 4, grade 8, and

grade 12). It was also used at grade 8 in the state assessment.

G.3 CIVICS DERIVED VARIABLES

Number of Years of Civics Classes Taken in High School (NYRCIV)

1 = none

2 = 1 year

3 = 2 years

4 = 3 years

5 = 4 years

(Value determined by number of “yes” responses to items P804601, P804602, P804603, and P804604)

Number of Years of Civics Classes Taken in High School - Grades 11 & 12 (NYRCIV2)

1 = none

2 = 1 year

3 = 2 years

(Value determined by number of “yes” responses to items P804603 and P804604)

P804601 = grade 9 - studied civics or government

P804602 = grade 10 - studied civics or government

P804603 = grade 11 - studied civics or government

P804604 = grade 12 - studied civics or government

A response of 1 = yes.
829

G.4 VARIABLES RELATED TO SCALING

Scale Score Variables

Student responses to the assessment questions were analyzed to determine the percentage of

students responding correctly to each multiple-choice question and the percentage of students achieving

each of the score categories for constructed-response questions. Item response theory (IRT) methods

were used to produce scales that summarized results for each of the domains in the subject area. The

scales for the state assessment were defined identically to, but separately from, those used for the scaling

of the national data. Although the questions comprising each scale were identical to those used in the

national assessment, the item parameters for the state assessment scales were estimated from combined

public-school data from the jurisdictions participating in the state assessment program.

In 1992, a reading scale ranging from 0 to 500 was created to report performance for each reading

purpose—Reading for Literary Experience, Reading to Gain Information, and Reading to Perform a Task

(grades 8 and 12 only). The scales summarize student performance across all three types of questions in

the assessment (multiple-choice, short constructed-response, and extended constructed-response). Results

from subsequent reading assessments (1994 and 1998) are reported on these scales.

Each reading scale was initially based on the distribution of student performance across all three

grades in the 1992 national assessment (grades 4, 8, and 12). In that year, the scales had an average of

250 and a standard deviation of 50. In addition, a composite scale was created as an overall measure of

students’ reading performance. This composite scale is a weighted average of the three separate scales

for the three reading purposes.

The 1998 writing assessment results are reported on an overall scale for each of the grades—4, 8,

and 12. For each grade, the range of the scale was 0 to 300, with an average of 150 and a standard

deviation of 35. While the scale-score ranges are identical across grades, the scale was derived

independently for each grade. The scales summarize performance across all three purposes for writing

(narrative, informative, and persuasive) in the assessment. Note that the 50-minute prompts were not

included in the scales.

The 1998 civics assessment results are reported on an overall scale for each of the grades—4, 8,

and 12. For each grade, the range of the scale was 0 to 300, with an average of 150 and a standard

deviation of 35. While the scale-score ranges are identical across grades, the scale was derived

independently for each grade.

The scale score variable names for each subject area are shown in Table G-2.
830

Table G-2

Scaling Variables for the 1998 National and State Assessment Samples

Sample Scale Data Variables

Reading Main Reading for Literary Experience RRPS11 to RRPS15

Reading to Gain Information RRPS21 to RRPS25

Reading to Perform a Task RRPS31 to RRPS35

Composite RRPCM1 to RRPCM5

Writing Main — WRIRP1 to WRIRP5

Civics Main — CIVRP1 to CIVRP5

Reading State Reading for Literary Experience RRPS11 to RRPS15

Reading to Gain Information RRPS21 to RRPS25

Reading to Perform a Task RRPS31 to RRPS35

Composite RRPCM1 to RRPCM5

Writing State — WRIRP1 to WRIRP5

G.5 QUALITY EDUCATION DATA (QED) VARIABLES

The data files contain several variables obtained from information supplied by Quality Education

Data, Inc. (QED). QED maintains and annually updates lists of schools showing grade span, total

enrollment, instructional dollars per pupil, and other information for each school. These data variables

are retained on both the school and student files and are identified in the data layouts by “(QED)” in the

SHORT LABEL field.

Most of the QED variables are defined sufficiently in the data codebooks. Explanations of others

are provided below.

ORSHPT is the Orshansky Percentile, an indicator of relative wealth that specifies the

percentage of school-age children in a district who fall below the poverty line.

IDP represents, at the school district level, dollars per student spent for textbooks and

supplemental materials. The range code for instructional dollars spent per pupil excluding teacher

salaries are:

0 = Unclassified

1 = Under $10

2 = $10–49

3 = $50–99

4 = $100–149

5 = $150–299

6 = $300–399

7 = $400–499

8 = $500–999

9 = $1,000 +

ADULTED indicates whether or not adult education courses are offered at the school site.

URBAN defines the school’s urbanization: urban (central city); suburban (area surrounding

central city, but still located within the counties constituting the metropolitan statistical area); or rural

(area outside any metropolitan statistical area).
831

Appendix H

ESTIMATION ERROR VARIANCE OF THE MEAN

BY GENDER AND RACE/ETHNICITY

Table H-1

Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Reading Grade 4 Literary Scale

Proportion of Variance Due to . . .

Total Variance

Student

Sampling

Latency

of 

Total 0.72 0.84 0.16

Male 1.34 0.84 0.16

Female 0.69 0.75 0.25

White 0.98 0.82 0.18

Black 2.89 0.73 0.27

Hispanic 3.93 0.74 0.26

Asian American 10.65 0.60 0.40

Native American 15.65 0.67 0.33

Other Race/Ethnicity 278.37 0.75 0.25

Public Schools 0.79 0.85 0.15

Private Schools 19.60 0.88 0.12

Catholic Schools 7.83 0.84 0.16

Table H-2

Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Reading Grade 4 Information Scale

Proportion of Variance Due to . . .

Total Variance

Student

Sampling

Latency

of 

Total 0.88 0.85 0.15

Male 1.67 0.85 0.15

Female 0.86 0.71 0.29

White 0.99 0.81 0.19

Black 4.09 0.77 0.23

Hispanic 3.55 0.78 0.22

Asian American 10.63 0.68 0.32

Native American 12.94 0.57 0.43

Other Race/Ethnicity 272.48 0.74 0.26

Public Schools 1.02 0.86 0.14

Private Schools 28.32 0.92 0.08

Catholic Schools 7.64 0.82 0.18
832

Table H-3

Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Reading Grade 4 Composite Scale

Proportion of Variance Due to . . .

Total Variance

Student

Sampling

Latency

of 

Total 0.64 0.89 0.11

Male 1.27 0.91 0.09

Female 0.58 0.78 0.22

White 0.76 0.87 0.13

Black 2.75 0.83 0.17

Hispanic 3.17 0.82 0.18

Asian American 8.02 0.77 0.23

Native American 11.26 0.73 0.27

Other Race/Ethnicity 256.98 0.78 0.22

Public Schools 0.72 0.90 0.10

Private Schools 20.83 0.94 0.06

Catholic Schools 6.17 0.90 0.10

Table H-4

Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Reading Grade 8 Literary Scale

Proportion of Variance Due to . . .

Total Variance

Student

Sampling

Latency

of 

Total 0.75 0.85 0.15

Male 1.14 0.85 0.15

Female 0.76 0.75 0.25

White 0.99 0.81 0.19

Black 2.15 0.70 0.30

Hispanic 4.83 0.86 0.14

Asian American 9.76 0.74 0.26

Native American 32.90 0.63 0.37

Other Race/Ethnicity 56.85 0.72 0.28

Public Schools 0.79 0.85 0.15

Private Schools 22.69 0.88 0.12

Catholic Schools 3.56 0.74 0.26
833

Table H-5

Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Reading Grade 8 Information Scale

Proportion of Variance Due to . . .

Total Variance

Student

Sampling

Latency

of 

Total 0.77 0.91 0.09

Male 1.05 0.85 0.15

Female 0.94 0.87 0.13

White 1.01 0.88 0.12

Black 2.36 0.78 0.22

Hispanic 5.48 0.90 0.10

Asian American 10.70 0.86 0.14

Native American 31.86 0.77 0.23

Other Race/Ethnicity 107.14 0.82 0.18

Public Schools 0.80 0.90 0.10

Private Schools 14.86 0.88 0.12

Catholic Schools 4.25 0.77 0.23

Table H-6

Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Reading Grade 8 Perform a Task Scale

Proportion of Variance Due to . . .

Total Variance

Student

Sampling

Latency

of 

Total 0.89 0.87 0.13

Male 1.21 0.84 0.16

Female 1.23 0.76 0.24

White 1.07 0.87 0.13

Black 3.65 0.73 0.27

Hispanic 6.24 0.83 0.17

Asian American 47.06 0.92 0.08

Native American 32.81 0.57 0.43

Other Race/Ethnicity 69.90 0.73 0.27

Public Schools 1.10 0.87 0.13

Private Schools 11.17 0.72 0.28

Catholic Schools 7.68 0.78 0.22
834

Table H-7

Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Reading Grade 8 Composite Scale

Proportion of Variance Due to . . .

Total Variance

Student

Sampling

Latency

of 

Total 0.62 0.93 0.07

Male 0.89 0.92 0.08

Female 0.69 0.89 0.11

White 0.78 0.91 0.09

Black 1.83 0.87 0.13

Hispanic 4.45 0.94 0.06

Asian American 13.44 0.94 0.06

Native American 24.42 0.82 0.18

Other Race/Ethnicity 59.77 0.81 0.19

Public Schools 0.67 0.93 0.07

Private Schools 13.75 0.93 0.07

Catholic Schools 2.89 0.88 0.12

Table H-8

Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Reading Grade 12 Literary Scale

Proportion of Variance Due to . . .

Total Variance

Student

Sampling

Latency

of 

Total 1.07 0.79 0.21

Male 2.14 0.78 0.22

Female 1.16 0.59 0.41

White 1.06 0.70 0.30

Black 5.37 0.69 0.31

Hispanic 5.30 0.71 0.29

Asian American 35.61 0.88 0.12

Native American 69.31 0.45 0.55

Other Race/Ethnicity 343.58 0.82 0.18

Public Schools 1.37 0.81 0.19

Private Schools 32.39 0.85 0.15

Catholic Schools 7.99 0.72 0.28
835

Table H-9

Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Reading Grade 12 Information Scale

Proportion of Variance Due to . . .

Total Variance

Student

Sampling

Latency

of 

Total 0.44 0.80 0.20

Male 0.84 0.82 0.18

Female 0.51 0.71 0.29

White 0.59 0.78 0.22

Black 2.27 0.83 0.17

Hispanic 2.24 0.79 0.21

Asian American 6.81 0.82 0.18

Native American 29.52 0.69 0.31

Other Race/Ethnicity 125.18 0.80 0.20

Public Schools 0.49 0.81 0.19

Private Schools 19.48 0.92 0.08

Catholic Schools 2.98 0.76 0.24

Table H-10

Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Reading Grade 12 Perform a Task Scale

Proportion of Variance Due to . . .

Total Variance

Student

Sampling

Latency

of 

Total 0.62 0.75 0.25

Male 1.24 0.76 0.24

Female 0.84 0.56 0.44

White 0.76 0.75 0.25

Black 3.35 0.61 0.39

Hispanic 3.44 0.68 0.32

Asian American 6.60 0.49 0.51

Native American 60.54 0.61 0.39

Other Race/Ethnicity 352.21 0.91 0.09

Public Schools 0.73 0.77 0.23

Private Schools 24.47 0.84 0.16

Catholic Schools 6.07 0.70 0.30
836

Table H-11

Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Reading Grade 12 Composite Scale

Proportion of Variance Due to . . .

Total Variance

Student

Sampling

Latency

of 

Total 0.51 0.88 0.12

Male 1.04 0.90 0.10

Female 0.53 0.75 0.25

White 0.57 0.85 0.15

Black 2.54 0.85 0.15

Hispanic 2.56 0.84 0.16

Asian American 11.26 0.91 0.09

Native American 32.66 0.74 0.26

Other Race/Ethnicity 212.15 0.89 0.11

Public Schools 0.62 0.88 0.12

Private Schools 21.23 0.95 0.05

Catholic Schools 3.62 0.87 0.13

Table H-12

Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Writing Grade 4

Proportion of Variance Due to . . .

Total Variance

Student

Sampling

Latency

of 

Total 0.37 0.90 0.10

Male 0.48 0.83 0.17

Female 0.46 0.82 0.18

White 0.48 0.85 0.15

Black 0.76 0.79 0.21

Hispanic 1.54 0.92 0.08

Asian American 6.05 0.82 0.18

Native American 4.31 0.69 0.31

Other Race/Ethnicity 26.04 0.82 0.18

Public Schools 0.48 0.91 0.09

Private Schools 4.26 0.81 0.19

Catholic Schools 1.86 0.83 0.17
837

Table H-13

Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Writing Grade 8

Proportion of Variance Due to . . .

Total Variance

Student

Sampling

Latency

of 

Total 0.41 0.94 0.06

Male 0.63 0.92 0.08

Female 0.42 0.83 0.17

White 0.56 0.92 0.08

Black 0.95 0.74 0.26

Hispanic 1.70 0.91 0.09

Asian American 12.38 0.95 0.05

Native American 7.50 0.72 0.28

Other Race/Ethnicity 16.15 0.61 0.39

Public Schools 0.44 0.94 0.06

Private Schools 5.04 0.87 0.13

Catholic Schools 1.90 0.79 0.21

Table H-14

Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Writing Grade 12

Proportion of Variance Due to . . .

Total Variance

Student

Sampling

Latency

of 

Total 0.44 0.93 0.07

Male 0.58 0.90 0.10

Female 0.50 0.85 0.15

White 0.54 0.89 0.11

Black 1.70 0.87 0.13

Hispanic 1.29 0.80 0.20

Asian American 9.31 0.92 0.08

Native American 14.32 0.71 0.29

Other Race/Ethnicity 68.28 0.88 0.12

Public Schools 0.56 0.94 0.06

Private Schools 8.90 0.89 0.11

Catholic Schools 3.48 0.90 0.10
838

Table H-15

Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Civics Grade 4

Proportion of Variance Due to . . .

Total Variance

Student

Sampling

Latency

of 

Total 0.54 0.90 0.10

Male 0.88 0.89 0.11

Female 0.70 0.84 0.16

White 0.76 0.89 0.11

Black 1.43 0.66 0.34

Hispanic 3.08 0.89 0.11

Asian American 7.47 0.76 0.24

Native American 13.47 0.81 0.19

Other Race/Ethnicity 62.68 0.81 0.19

Public Schools 0.60 0.90 0.10

Private Schools 18.89 0.93 0.07

Catholic Schools 3.20 0.82 0.18

Table H-16

Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Civics Grade 8

Proportion of Variance Due to . . .

Total Variance

Student

Sampling

Latency

of 

Total 0.52 0.91 0.09

Male 0.95 0.92 0.08

Female 0.58 0.85 0.15

White 0.73 0.93 0.07

Black 1.34 0.77 0.23

Hispanic 1.40 0.80 0.20

Asian American 32.61 0.97 0.03

Native American 12.70 0.79 0.21

Other Race/Ethnicity 86.74 0.81 0.19

Public Schools 0.56 0.90 0.10

Private Schools 35.17 0.98 0.02

Catholic Schools 2.78 0.93 0.07
839

Table H-17

Estimation Error Variance of the Mean for the 1998 NAEP Assessment

National Main Civics Grade 12

Proportion of Variance Due to . . .

Total Variance

Student

Sampling

Latency

of 

Total 0.62 0.95 0.05

Male 1.22 0.93 0.07

Female 0.65 0.92 0.08

White 0.80 0.95 0.05

Black 2.79 0.91 0.09

Hispanic 1.86 0.80 0.20

Asian American 18.08 0.96 0.04

Native American 36.61 0.87 0.13

Other Race/Ethnicity 79.30 0.75 0.25

Public Schools 0.78 0.95 0.05

Private Schools 9.19 0.93 0.07

Catholic Schools 2.37 0.93 0.07

841

Appendix I

SETTING THE ACHIEVEMENT LEVELS

FOR THE 1998 NAEP READING ASSESSMENT

Mary Lyn Bourque

National Assessment Governing Board

I.1 INTRODUCTION

The 1998 National Assessment of Educational Progress (NAEP) reading assessment used the

same achievement levels that were developed for the 1994 assessment. This appendix describes the

process originally used in 1994.

Since 1984, NAEP has reported the performance of students in the nation and for specific

subpopulations on a 0-to-500 score scale. The history and development of the scale and the anchoring

procedure used to interpret specific points on that scale are described in Appendix G of The NAEP 1992

Technical Report (Johnson & Carlson, 1994).

The 1988 NAEP legislation (Hawkins-Stafford Education Improvement Act Amendments of

1988) created an independent board, the National Assessment Governing Board (NAGB), responsible for

setting policy for the NAEP program. The 1994 NAEP reauthorization (Improving America’s Schools

Act of 1994) continued many of the board's statutory responsibilities, including developing appropriate

student performance standards for each age and grade in each subject area to be tested under the national

assessment. Consistent with this directive, and striving to achieve one of the primary mandates of the

statute to improve the form and use of NAEP results, the board has been developing student performance

standards (called achievement levels by NAGB) on the national assessment since 1990.

The 1990 standard-setting effort, initiated in December 1989 with the dissemination of a draft

policy statement (NAGB, 1989) and culminating 22 months later in the publication of the NAGB report,

The Levels of Mathematics Achievement (Bourque & Garrison, 1991), consisted of two phases: the main

study and a replication-validation study. Although there were slight differences between the two phases,

there were many common elements. Both phases used a modified (iterative/empirical) Angoff (1971)

procedure for arriving at the levels; both focused on estimating performance levels based on a review of

the 1990 NAEP mathematics item pool; and both phases employed policy definitions for basic,

proficient, and advanced levels (NAGB, 1990) as the criteria for rating items. The 1990 process was

evaluated by a number of different groups (for a discussion, see Hambleton & Bourque, 1991) who

identified technical flaws in the 1990 process. These evaluations influenced the board's decision to set

the levels again in 1992, and not to use the 1990 levels as benchmarks for progress toward the national

goals during the coming decade. It is interesting to note, however, that the 1990 and 1992 processes

produced remarkably similar results.

In September 1991, the board contracted with American College Testing (ACT) to convene the

panels of judges that would recommend the levels on the 1992 NAEP assessments in reading, writing,

and mathematics. While the 1992 level-setting activities were not unlike those undertaken by the board in

1990, there were significant improvements made in the process for 1992. There was a concerted effort to

bring greater technical expertise to the process: The contractor selected by the board has a national

reputation for setting standards in a large number of certification and licensure exams; an internal and

external advisory team monitored all the technical decisions made by the contractor throughout the
842

process; and state assessment directors periodically provided their expertise and technical assistance at

key stages in the project.

Setting achievement levels is a method for setting standards on the NAEP assessment that

identify what students should know and be able to do at various points along the score scale. The initial

policy definitions of the achievement levels were presented to panelists along with an illustrative

framework for more in-depth development and operationalization of the levels. Panelists were asked to

determine descriptions or definitions of the three levels from the specific framework developed for the

NAEP assessment with respect to the content and skills to be assessed. The operationalized definitions

were refined throughout the level-setting process, as well as validated with a supplementary group of

judges subsequent to the level-setting meetings. Panelists were also asked to develop a list of illustrative

tasks associated with each of the levels, after which sample items from the NAEP item pool were

identified to exemplify the full range of performance of the intervals between levels. The emphasis in

operationalizing the definitions and in identifying and selecting exemplar items and papers was to

represent the full range of performance from the lower level to the next higher level. The details of the

implementation procedures are outlined in the remainder of this appendix.

I.2 1992 PREPARATION FOR THE READING LEVEL SETTING MEETING

It is important for the planning of any standard-setting effort to know how various process

elements interact with each other. For example, panelists interact with premeeting materials, meeting

materials (i.e., the assessment items, rating forms, rater feedback, and so forth), each other, and the

project staff. All of these elements combine to promote or degrade what has been called intrajudge

consistency and interjudge consensus (Friedman & Ho, 1990).

Previous research has conceptualized the effects of two major kinds of interaction: (1) people

interacting with text (Smith & Smith, 1988), and (2) people interacting with each other (Curry, 1987;

Fitzpatrick, 1989). In order to assess the effects of textual and social interaction and adjust the standardsetting procedures accordingly, a pilot study was conducted as the first phase of the 1992 initiative.

Reading was chosen as the single content area to be pilot tested, since it combined all of the

various features found in the other NAEP assessments, including multiple-choice and both short and

extended constructed-response items. The pilot study provided the opportunity to implement and evaluate

all aspects of the operational plan—background materials, meeting materials, study design, meeting

logistics, staff function, and participant function.

The overall pilot was quite successful. The level-setting process worked well, and the pilot

allowed the contractor to make improvements in the design before implementation activities began. For

example, schedule changes were made that allowed the panelists more time to operationalize the policy

definitions before beginning the item-rating task. Also, the feedback mechanisms used to inform panelists

about interjudge and intrajudge consistency data were improved for clarity and utility to the entire

process.

I.3 1992 READING LEVEL SETTING PANEL

Sixty-four panelists representing 32 jurisdictions (31 states and Virgin Islands) were selected

from the 366 nominees and invited to participate in the level-setting process. They represented

reading/language arts teachers at grades 4, 8, and 12, nonteacher educators, and members of the

noneducator (general public) community. The group was balanced by gender, race/ethnicity, NAEP

regions of the country, community type (low SES, not low SES), district size, and school type
843

(public/nonpublic). Two panelists were unable to attend for personal reasons, resulting in 62 participants,

22 at grade 4, 20 at grade 8, and 20 at grade 12.

I.4 1992 PROCESS FOR DEVELOPING THE ACHIEVEMENT LEVELS

The four-and-one-half-day session began with a brief overview of NAEP and NAGB, a

presentation on the policy definitions of the achievement levels, a review of the NAEP reading

assessment framework, and a discussion of factors that influence item difficulty. The purpose of the

presentation was to focus panelists’ attention on the reading framework and to emphasize the fact that

panelists’ work was directly related to the NAEP assessment, not to the whole domain of reading. 

All panelists completed and self-scored an appropriate grade-level form of the NAEP assessment.

The purpose of this exercise was to familiarize panelists with the test content and scoring protocols—as

well as time constraints—before beginning to develop the preliminary operationalized descriptions of the

three levels.

Working in small groups of five or six, then eventually in grade-level groups, panelists expanded

and operationalized the policy definitions of basic, proficient, and advanced in terms of specific reading

skills, knowledge, and behaviors that were judged to be appropriate expectations for students in each

grade, and to be in accordance with the current reading assessment framework.

The policy definitions1 are as follows:

Basic This level, below proficient, denotes partial mastery of the knowledge and

skills that are fundamental for proficient work at each grade—4, 8, and 12.

Proficient This central level represents solid academic performance for each grade

tested—4, 8, and 12. Students reaching this level have demonstrated

competency over challenging subject matter and are well prepared for the

next level of schooling.

Advanced This higher level signifies superior performance beyond proficient gradelevel mastery at grades 4, 8, and 12.

The small groups were allowed to brainstorm about what student performance should be, using

the framework and their experience in completing the NAEP assessment as guides.2 In addition, a

practice task caused panelists to examine items in the half of the item pool that they would not be rating

later. A comprehensive listing of grade-level descriptors was developed, and panelists were asked to

identify the five or six that best described what students should be able to do at each of the levels. Those

descriptors appearing with the greatest frequency were compiled into a discussion list for the grade-level

groups. Additions, deletions, and modifications were made as a result of discussions, and the groups

reached general agreement that the final list of descriptors represented what students should be able to do

at each achievement level.

 

1 NAGB revised its policy definitions on achievement levels in late 1993. The Proficient level now reads: This level represents

solid academic performance for each grade assessed. Students reaching this level have demonstrated competency over

challenging subject matter, including subject-matter knowledge, application of such knowledge to real-world situations, and

analytical skills appropriate to the subject matter. Basic and Advanced remain virtually unchanged. 2 The panelists also reviewed about half the item pool (the half they would not be rating later) so that the descriptors could be

further modified if that was deemed appropriate.
844

Panelists next received training in the Angoff method, which was customized to reflect the

unique item formats of the particular subject-area assessment. Once a conceptual consensus was reached

about the characteristics of marginally acceptable performance at each of the three levels, practice items

from the released pool were rated by the panelists according to the process defined in the contractor’s

plan. For multiple-choice and short constructed-response items (both of which were scored right or

wrong), panelists were asked to rate each item for the expected probability of a correct response for a

group of marginally acceptable examinees at the basic, proficient, and advanced levels. For extended

constructed-response items (which were scored on a four-point rating scale using a partial-credit model),

panelists were asked to review a set of student response papers and select three papers, one for each

achievement level, that typified marginally acceptable examinee performance for that level.

Following training in the Angoff method, the judges began the rating and paper selection

process, inspecting and rating each dichotomously scored item in the pool for the expected probabilities

of answering the item correctly at each level. For polytomously scored items, panelists reviewed a

representative set of 24 to 28 student response papers for each item and selected the paper that best

represented marginally acceptable student performance at each level. Panelists completed three rounds of

item ratings and paper selections. For Round 1, panelists first answered the items related to a reading

passage, then reviewed their answers using scoring keys and protocols. This process helped ensure that

panelists would be thoroughly familiar with each item, including the foils and scoring rubrics, before

rating the item. Panelists provided item ratings and paper selections for all three achievement levels, one

item at a time, for all the items related to a reading passage. They then proceeded to the next reading

passage and set of items, for which the process was repeated. Panelists rated items for half the items in

their grade-level assessment; one block of exercises was common to both halves of the grade-level

groups. During Round 1, panelists used their lists of descriptors and other training materials for guidance

in the rating process.

Following Round 1, item response theory (IRT) was used to convert the rating results3

 for each

rater to a latent ability scale, represented by the Greek letter theta (θ ). This θ scale was the same scale

to which the NAEP items evaluated by each panelist were calibrated. In order to provide meaningful

feedback about item ratings, a special relative scale was constructed, which was a linear transformation

of the theta scale having a mean of 75 and standard deviation of 15. Before Round 2 of the rating process,

panelists were given interjudge consistency information using this relative scale. This information

allowed panelists to see where their individual mean item ratings were on the scale, relative to the mean

for the group and to the means for other panelists. Reasons for extreme mean ratings, including the

possibility that some panelists misinterpreted the item rating task, were discussed.

Before Round 2, panelists were also given item difficulty data. This information was presented as

the overall percentage of students who answered each item correctly during the actual NAEP

administration, for items scored “correct” or “incorrect” (i.e., multiple-choice and short constructedresponse items), and as the mean score for student responses (on a scale of 1 to 4) for the extended

constructed-response items. Panelists were told that this item difficulty information should be used as a

reality check. For items on which item ratings differed substantially from the item difficulty value,

panelists were asked to reexamine the item to determine if they had misinterpreted the item or misjudged

its difficulty. Results of the data analysis, and panelists’ own evaluations, indicated that the item

difficulty information was perceived as very useful but had little impact on panelists’ ratings.

For Round 2, panelists reviewed the same set of items they rated in Round 1 and, using the

interjudge consistency information, the item difficulty information, and the information provided prior to

Round 1, they either confirmed their initial item ratings and paper selections or adjusted their ratings to

 

3 Because the IRT item parameters were not available for the polytomously scored (extended constructed-response) items, these

items were not included in the following discussion of results.
845

reflect the additional information. About one-half of Round 1 item ratings and paper selections were

adjusted during Round 2.

Prior to Round 3, panelists’ ratings were reanalyzed and additional information was presented to

panelists concerning intrajudge variability. For each panelist, the intrajudge variability information

consisted of those items that they had rated differently than items having similar difficulty, taking into

consideration the panelist’s aggregated item ratings. That is, the panelists’ aggregated item ratings were

converted to the theta (θ ) scale. All items rated by the panelists were then analyzed in terms of the

panelist’s achievement level (θ ) in comparison to actual student performance on the items. The observed

item rating from each panelist was contrasted to an expected item rating. Those items with the largest

differences between observed and expected ratings were identified. Panelists were given this information

and asked to review each of these items and decide if their Round 2 ratings still accurately reflected their

best judgments of the items. The intrajudge consistency data was to be used to flag items for

reconsideration in the final round of rating.

For Round 3, panelists reviewed the same set of items they rated in Rounds 1 and 2 using both

the new intrajudge variability information and the information made available during Rounds 1 and 2. In

addition, panelists could discuss, within their small groups, ratings and paper selections for specific items

about which they were unsure. About one-third of the item ratings were adjusted during Round 3.

I.5 1992 PROCESS FOR SELECTING EXEMPLAR ITEMS

On the final day of the achievement level-setting process, panelists reviewed items from the 1992

item pool scheduled for release to the public. The released item pool was the set from which the panelists

could select items illustrative of the achievement levels for their grade. Exercises are organized in blocks,

consisting of a reading passage, followed by several items, usually employing each of the three item

formats, (i.e., multiple-choice, short constructed-response, and extended constructed-response). A total of

10 blocks from the 1992 exercise pool were scheduled for release: 2 blocks from the fourth-grade pool,

totaling 19 items; 4 blocks from the eighth-grade pool, totaling 52 items; and 4 blocks from the twelfthgrade pool, totaling 46 items.

Panelists who had rated specific blocks of released items were asked to review those same items

again to select particular ones as exemplary of each achievement level. The items were preassigned to

each achievement level based on the final round of the judges’ rating data, and using the following

statistical criteria. For any given level (basic, proficient, or advanced),

1. items having an expected p-value4

 >.501 and <.750, at that level, were assigned to

that level;

2. items meeting the criteria at more than one level were assigned to one level taking

both the expected p-value and the appropriateness of the item for one of the levels

into account; and

3. because the content of items was given equal consideration in the selection process,

items with expected p-values <.501 were assigned to levels where a specific passage

had few or no items at that level.

For example, the raters' expected p-value for one of the released items might have been .366 at

the basic level, .701 at the proficient level, and .932 at the advanced level. This item would have been

 

4 Expected p-values were based on the average predicted performance at the cut point for each achievement level.
846

identified for review as a potential exemplar item for the proficient level. The expected p-value at the

basic level was too low for consideration as a basic-level exemplar (that is, the item was judged to be too

difficult), and the expected p-value at the advanced level was too high for consideration at the advanced

level (that is, the item was judged to be too easy). Table I-1 shows the results of this process for each

grade and level.

Table I-1

Results of First Review for Achievement-Level Exemplars

Level/Status Grade 4 Grade 8 Grade 12 All Grades

Total Released 19 52 46 117

Basic

Reviewed

Recommended

4

3

12

5

18

14

34

22

Proficient

Reviewed

Recommended

5

4

14

12

20

9

39

25

Advanced

Reviewed

Recommended

5

5

6

6

7

8

18

19

Panelists were asked to review the items as classified, and form an individual judgment regarding

the suitability of each item to illustrate and further communicate the meaning of the levels. Each item’s

classification could be accepted, rejected, or reassigned, although the procedure was primarily designed

to eliminate items that did not meet panelists’ expectations for any reason. Items were reclassified if a

strong consensus was found to hold for that change.

During the validation process, described in the next section, items were again reviewed. Those

that had been selected by the original standard-setting panel were grouped into sets of preselected items.

All remaining items in the released blocks that met the statistical criteria, but were not recommended by

the original panel, were grouped into a set identified as additional items for review. Exercises that had

been recommended for reclassification into another achievement-level category were presented in their

original classification for purposes of this review. As Table I-2 shows, 21 items were recommended as

exemplars for the basic level, 17 for the proficient level, and 9 for the advanced.
847

Table I-2

Results of Review of Additional Items for Achievement-Level Exemplars

Level/Status Grade 4 Grade 8 Grade 12 All Grades

Total Items Recommended 13 13 21 47

Basic

Reviewed

Recommended

3

6

12

7

12

8

27

21

Proficient

Reviewed

Recommended

4

6

13

3

11

8

28

17

Advanced

Reviewed

Recommended

5

1

8

3

9

5

22

9

I.6 1992 PROCESS FOR VALIDATING THE LEVELS

Nineteen reading educators participated in the item-selection and content-validation process. Ten

of the panelists were reading teachers who had participated in the original achievement level-setting

process and who had been identified as outstanding panelists by grade group facilitators during this

meeting, who were extensively involved with professional organizations (e.g., the International Reading

Association, the National Reading Conference, or the National Council for Teachers of English), and

who had outstanding service credentials. The other nine panelists represented state-level reading

curriculum supervisors or assessment directors, as well as university faculty teaching in disciplines

related to this subject area. To the extent possible, the group was balanced by race/ethnicity and gender.

The two-and-one-half-day meeting began by briefing panelists on the purpose of the meeting and

by giving them an overview of the level-setting process and results. Panelists first reviewed the

operationalized descriptions of the achievement levels for qualities such as (1) within- and across-grade

consistency, (2) grade-level appropriateness, and (3) utility for increasing the public’s understanding of

the NAEP reading results. Next, panelists reviewed the operationalized descriptions of the achievement

levels for consistency with the NAGB policy definitions of basic, proficient, and advanced with the

NAEP reading objectives. Working in grade-level (4, 8, and 12) groups of six to seven panelists each,

then as a whole group, panelists reviewed the operationalized descriptions to provide within- and acrossgrade consistency, and to align the language and concepts of the descriptions more closely with the

language of the NAEP reading objectives. (Both the original descriptions and the revised descriptions are

included later in this appendix.) Finally, panelists suggested revisions they thought would improve the

operational descriptions based on their earlier reviews.

On the final day, panelists worked in grade-level groups to review the possible exemplar items.

The task was to select a set of items, for each achievement level for their grade, that would best

communicate to the public the levels of reading ability and the types of skills needed to perform in

reading at that level.

After selecting sets of items for their grades, the three grade-level groups met as a whole group to

review item selection. During this process, cross-grade items that had been selected as exemplars for two

grades (two such items were selected for grades 8 and 12) were assigned to one grade by whole-group

consensus. In addition, items were evaluated by the whole group for overall quality. This process yielded

13 items as recommended exemplars for grade 4, 13 items as recommended exemplars for grade 8, and

21 items as recommended exemplars for grade 12.
848

I.7 EVALUATION OF THE 1992 LEVELS

The 1992 achievement levels in both mathematics and reading were evaluated under a

Congressional mandate by the National Academy of Education (NAE). A series of research studies were

mounted by the NAE (1993a; 1993b) to look at various aspects of the validity of the level-setting process

and the levels finally adopted by NAGB. Three of the studies focused specifically on the reading

achievement levels, and were conducted for the NAE panel by staff at the Center for the Study of

Reading at the University of Illinois at Urbana–Champaign. The first study examined the process for

setting the levels in reading; the second study provided an analysis of the reading achievement levels

descriptions; and the third focused on a comparison of the reading cut scores with those set by alternative

means. Based on these studies the NAE’s policy report concluded that the achievement levels were

flawed and should be discontinued as a means of reporting NAEP data.

While NAGB did not agree with the conclusions reached in the NAE studies, and while the

board’s technical advisors and contractor did not believe the weight of the evidence supported the

conclusions reached by the NAE (American College Testing, 1993; Cizek, 1993; Kane, 1993), the board

agreed to support further investigation into the validity of the reading achievement levels through

additional studies prior to the release of the 1994 NAEP reading data, since the board planned on using

the levels to report the 1994 NAEP data.

I.8 1994 PROCESS FOR VALIDATING THE LEVELS

The methodology developed by ACT to examine the reading achievement levels descriptions

required the use of reading professionals (teachers and nonteacher educators) to review the descriptions

in relation to the 1992 reading item pool. Fifty-eight panelists (about 20 at each grade level) were

assigned to two different task groups, A and B. Group A employed the item difficulty categorization

(IDC) procedure, while Group B used a judgmental item categorization (JIC) procedure. The goal of both

task groups was to identify any lack of congruence between the item pool and the achievement-level

descriptions.

The IDC procedure examined the level of support for the descriptions as evidenced by

performance on the NAEP items. Items were preselected for each achievement level using a response

probability (rp) criterion of 0.50 at the lower borderline (can do items). Those items not meeting the

same rp criterion at the upper borderline of the level were categorized as “can’t do” items, while those

items meeting the rp criterion anywhere in the range (from lower borderline to upper borderline) of the

achievement level were labeled “challenging” items. Panelists were trained to examine the items in each

of the three categories and determine whether or not the cognitive demand of the item matched the skills

and knowledge identified in the descriptions. Mismatches were identified and later resolved or accounted

for through a grade-level procedure involving the JIC group.

The JIC procedure asked panelists to assign items to levels based on their judgment of where it

belonged, given the achievement-levels descriptions. Items were assigned to the lowest level of

performance required to respond correctly to the item. All items were assigned to levels independently by

judges in the first round. Then, working in small groups and finally in the total group, assignments were

confirmed or moderated through a consensus process.

The final grade-level procedure brought both groups A and B together to jointly evaluate the

descriptions vis a vis performance on the item pool. The goal of the grade-level procedure was to reach

general agreement on the extent of (or lack of) agreement between the descriptions and the item pool,

employing somewhat different approaches to the question.
849

On the basis of the validation process only one recommendation was made by the panelists to

improve the descriptions and bring them more in line with the performance data they had examined

during the process. The general conclusion was that reference to an ability to make inferences should be

included in the description of Basic-level achievement at each grade level. An adjustment has been made

in the 1994 descriptions to reflect that recommendation.

I.9 1994 EXEMPLARS

The purpose of providing exemplar exercises is to provide readers with a sample of the kind of

skills and knowledge that students reaching the achievement levels are likely to be able to respond to

successfully. They are meant also to represent the kind of knowledge and skills embodied in the reading

framework.

The selection of exemplar items for the 1994 reading assessment augment the 1992 exemplars by

providing three additional passages (one for each grade level) and 13 additional exercises associated with

the passages. The choice was made on the basis of criteria similar to those used in 1992, with one

additional selection criterion, namely, item format. Since the percent of constructed-response items

increased by approximately 10 percent over the 1992 assessment, the choice of 1994 exemplars reflects

this focus.

It should be noted that although some exemplars are associated with performance data from the

1992 and 1994 assessments (overall and conditional p-values), others have only 1992 performance

estimates, since they were released items in 1992 and not readministered in 1994. However, they are all

reflective of the assessment framework.

I.10 MAPPING THE LEVELS ONTO THE NAEP SCALE

The process of mapping panelists’ ratings to the NAEP scales used item response theory (IRT).

IRT provided statistically sophisticated methods for determining the expected performance of examinees

on particular test items in terms of an appropriate measurement scale. The same measurement scale

simultaneously described the characteristics of the test items and the performance of the examinees. Once

the item characteristics were set, it was possible to determine precisely how examinees were likely to

perform on the test items at different points of the measurement scale.

The panelists’ ratings of the NAEP test items were likewise linked, by definition, to the expected

performance of examinees at the theoretical achievement-level cut points. It was therefore feasible to use

the IRT item characteristics to calculate the values on the measurement scale corresponding to each

achievement level. This was done by averaging the item ratings over panelists for each achievement level

and then simply using the item characteristics to find the corresponding achievement-level cut points on

the IRT measurement scale. This process was repeated for each of the NAEP reading scales within each

grade (4, 8, and 12).

For the multiple-choice and short constructed-response items that were dichotomously scored,

the judges each rated half of the items in the NAEP pool in terms of the expected probability that a

student at a borderline achievement level would answer the item correctly, based on the judges’

operationalization of the policy definitions and the factors that influence item difficulty. To assist the

judges in generating consistently scaled ratings, the rating process was repeated twice, with feedback.

Information on consistency among different judges and on the difficulty of each item5

 was fed back into

 

5 Item difficulty estimates were based on a preliminary, partial set of responses to the national assessment.
850

the first repetition (Round 2), while information on consistency within each judge’s set of ratings was fed

back into the second repetition (Round 3). The third round of ratings permitted the judges to discuss their

ratings among themselves to resolve problematic ratings. The judges’ mean final rating aggregated across

multiple-choice and short constructed-response items, yielded the threshold values for these items in the

percent correct metric. These cut scores were then mapped onto the NAEP scale (which is defined and

scored using item response theory, rather than percent correct).

For extended constructed-response items, judges were asked to select student papers that

exemplified performance at the cut point of each achievement level. Then for each achievement level, the

mean of the scores assigned to the selected papers was mapped onto the NAEP scale in a manner similar

to that used for the items scored dichotomously.

The final cut score for each achievement level was a weighted average of the cut score for the

multiple-choice and short constructed-response items and the cut score for the extended constructedresponse items, with the weights being proportional to the information supplied by the two classes of

items. The judges’ ratings, in both metrics, are shown for grade 4 in Table I-3.

Table I-3

Cut Points for Achievement Levels – Grade 4

Mean Percent Correct,

Multiple-Choice and

Short ConstructedResponse (Round 3)

Mean Paper Rating,

Extended

Constructed-Response

(Round 3)

Scale

Score*

Standard

Error of

Scale

Score**

Basic

Proficient

Advanced

38

62

80

2.72

3.14

3.48

208

238

268

(3.6)

(1.4)

(6.1)

* Scale score is derived from a weighted average of the mean percents correct for multiple-choice and short

constructed-response items and the mean paper ratings for extended constructed-response items after both were

mapped onto the NAEP scale.

** The standard error of the scale is estimated from the difference in mean scale scores for the two equivalent

subgroups of judges.

In the final stage of the mapping process, the achievement-level cut points on the IRT

measurement scale were combined over content areas and rescaled to the NAEP score scale. Weighted

averages of the achievement-level cut points were computed. The weighting constants accounted for the

measurement precision of the test items evaluated by the panelists, the proportion of items belonging to

each NAEP content area, and the linear NAEP scale transformations. These weighted averages produced

the final cut points for the basic, proficient, and advanced achievement levels within each grade.
851

Figure I-1

Final Descriptions of 1992 Reading Achievement Levels

PREAMBLE

Reading for meaning involves a dynamic, complex interaction between and among the reader, the text, and the

context. Readers, for example, bring to the process their prior knowledge about the topic, their reasons for reading it,

their individual reading skills and strategies, and their understanding of differences in text structures.

The texts used in the reading assessment are representative of common real world reading demands. Students at

grade 4 are asked to respond to literary and informational texts which differ in structure, organization, and features.

Literary texts include short stories, poems, and plays that engage the reader in a variety of ways, not the least of

which is reading for fun. Informational texts include selections from textbooks, magazines, encyclopedias, and other

written sources whose purpose is to increase the reader’s knowledge.

In addition to literary and informational texts, students at grades 8 and 12 are asked to respond to practical texts

(e.g., bus schedules or directions for building a model airplane) that describe how to perform a task.

The context of the reading situation includes the purposes for reading that the reader might use in building a meaning

of the text. For example, in reading for literary experience, students may want to see how the author explores or

uncovers experiences, or they may be looking for vicarious experience through the story’s characters. On the other

hand, the student’s purpose in reading informational texts may be to learn about a topic (such as the Civil War or the

oceans) or to accomplish a task (such as getting somewhere, completing a form, or building something).

The assessment asks students at all three grades to build, extend, and examine text meaning from four stances or

orientations:

Initial Understanding–Students are asked to provide the overall or general meaning of the

selection. This includes summaries, main points, or themes.

Developing Interpretation–Students are asked to extend the ideas in the text by making

inferences and connections. This includes making connections between cause and effect, analyzing

the motives of characters, and drawing conclusions.

Personal Response–Students are asked to make explicit connections between the ideas in the text

and their own background knowledge and experiences. This includes comparing story characters

with themselves or people they know, for example, or indicating whether they found a passage

useful or interesting.

Critical Stance–Students are asked to consider how the author crafted a text. This includes

identifying stylistic devices such as mood and tone.

These stances are not considered hierarchical or completely independent of each other. Rather, they provide a frame

for generating questions and considering student performance at all levels. All students at all levels should be able to

respond to reading selections from all of these orientations. What varies with students’ developmental and

achievement levels is the amount of prompting or support needed for response, the complexity of the texts to which

they can respond, and the sophistication of their answers.

(continued)
Figure I-1 (continued)

Final Descriptions of 1992 Reading Achievement Levels

852

INTRODUCTION

The following achievement-level descriptions focus on the interaction of the reader, the text, and the context. They

provide some specific examples of reading behaviors that should be familiar to most readers of this document. The

specific examples are not inclusive; their purpose is to help clarify and differentiate what readers performing at each

achievement level should be able to do. While a number of other reading achievement indicators exist at every level,

space and efficiency preclude an exhaustive listing.

It should also be noted that the achievement levels are cumulative from basic to proficient to advanced. One level

builds on the previous levels such that knowledge at the proficient level presumes mastery of the basic level, and

knowledge at the advanced level presumes mastery at both the basic and proficient.

Grade 4–Basic

Fourth-grade students performing at the basic level should demonstrate an understanding of the overall meaning of

what they read. When reading texts appropriate for fourth graders, they should be able to make relatively obvious

connections between the text and their own experiences6

.

For example, when reading literary text, they should be able to tell what the story is generally about–providing

details to support their understanding–and be able to connect aspects of the stories to their own experiences.

When reading informational text, basic-level fourth graders should be able to tell what the selection is generally

about or identify the purpose for reading it; provide details to support their understanding; and connect ideas from

the text to their background knowledge and experiences.

Grade 4–Proficient

Fourth grade students performing at the proficient level should be able to demonstrate an overall understanding of

the text, providing inferential as well as literal information. When reading text appropriate to fourth grade, they

should be able to extend the ideas in the text by making inferences, drawing conclusions, and making connections to

their own experiences. The connection between the text and what the student infers should be clear.

For example, when reading literary text, proficient-level fourth graders should be able to summarize the story, draw

conclusions about the characters or plot, and recognize relationships such as cause and effect.

When reading informational text, proficient-level students should be able to summarize the information and identify

the author’s intent or purpose. They should be able to draw reasonable conclusions from the text, recognize

relationships such as cause and effect or similarities and differences, and identify the meaning of the selection’s key

concepts.

(continued)

 

6 Based on the recommendations of the 1994 reading revisit study, the phrase and extend the ideas in the text by making simple

inferenceshas been added here to the description of Basic.
Figure I-1 (continued)

Final Descriptions of 1992 Reading Achievement Levels

853

Grade 4–Advanced

Fourth-grade students performing at the advanced level should be able to generalize about topics in the reading

selection and demonstrate an awareness of how authors compose and use literary devices. When reading text

appropriate to fourth grade, they should be able to judge texts critically and, in general, give thorough answers that

indicate careful thought.

For example, when reading literary text, advanced-level students should be able to make generalizations about the

point of the story and extend its meaning by integrating personal experiences and other readings with the ideas

suggested by the text. They should be able to identify literary devices such as figurative language.

When reading informational text, advanced-level fourth graders should be able to explain the author’s intent by

using supporting material from the text. They should be able to make critical judgments of the form and content of

the text and explain their judgments clearly.

Grade 8–Basic

Eighth-grade students performing at the basic level should demonstrate a literal understanding of what they read

and be able to make some interpretations. When reading text appropriate to eighth grade, they should be able to

identify specific aspects of the text that reflect the overall meaning,7

 recognize and relate interpretations and

connections among ideas in the text to personal experience, and draw conclusions based on the text.

For example, when reading literary text, basic-level eighth graders should be able to identify themes and make

inferences and logical predictions about aspects such as plot and characters.

When reading informative text, they should be able to identify the main idea and the author’s purpose. They should

make inferences and draw conclusions supported by information in the text. They should recognize the relationships

among the facts, ideas, events, and concepts of the text (e.g., cause and effect and chronological order).

When reading practical text, they should be able to identify the main purpose and make predictions about the

relatively obvious outcomes of procedures in the text.

(continued)

 

7

 Based on the recommendations of the 1994 reading revisit study, the phrase extend the ideas in the text by making simple

inferences,has been added here to the description of Basic.
Figure I-1 (continued)

Final Descriptions of 1992 Reading Achievement Levels

854

Grade 8–Proficient

Eighth-grade students performing at the proficient level should be able to show an overall understanding of the text,

including inferential as well as literal information. When reading text appropriate to eighth grade, they should

extend the ideas in the text by making clear inferences from it, by drawing conclusions, and by making connections

to their own experiences–including other reading experiences. Proficient eighth graders should be able to identify

some of the devices authors use in composing text.

For example, when reading literary text, students at the proficient level should be able to give details and examples

to support themes that they identify. They should be able to use implied as well as explicit information in articulating

themes; to interpret the actions, behaviors, and motives of characters; and to identify the use of literary devices such

as personification and foreshadowing.

When reading informative text, they should be able to summarize the text using explicit and implied information

and support conclusions with inferences based on the text.

When reading practical text, proficient-level students should be able to describe its purpose and support their views

with examples and details. They should be able to judge the importance of certain steps and procedures.

Grade 8–Advanced

Eighth-grade students performing at the advanced level should be able to describe the more abstract themes and

ideas of the overall text. When reading text appropriate to eighth grade, they should be able to analyze both meaning

and form and support their analyses explicitly with examples from the text; they should be able to extend text

information by relating it to their experiences and to world events. At this level, student responses should be

thorough, thoughtful, and extensive.

For example, when reading literary text, advanced-level eighth graders should be able to make complex, abstract

summaries and theme statements. They should be able to describe the interactions of various literary elements (i.e.,

setting, plot, characters, and theme); to explain how the use of literary devices affects both the meaning of the text

and their response to the author’s style. They should be able critically to analyze and evaluate the composition of the

text.

When reading informative text, they should be able to analyze the author’s purpose and point of view. They should

be able to use cultural and historical background information to develop perspectives on the text and be able to apply

text information to broad issues and world situations.

When reading practical text, advanced-level students should be able to synthesize information that will guide their

performance, apply text information to new situations, and critique the usefulness of the form and content.

(continued)
Figure I-1 (continued)

Final Descriptions of 1992 Reading Achievement Levels

855

Grade 12–Basic

Twelfth-grade students performing at the basic level should be able to demonstrate an overall understanding and

make some interpretations of the text. When reading text appropriate to twelfth grade, they should be able to identify

and relate aspects of the text to its overall meaning,8

 recognize interpretations, make connections among and relate

ideas in the text to their personal experiences, and draw conclusions. They should be able to identify elements of an

author’s style.

For example, when reading literary text, twelfth-grade students should be able to explain the theme, support their

conclusions with information from the text, and make connections between aspects of the text and their own

experiences.

When reading informational text, basic-level twelfth graders should be able to explain the main idea or purpose of a

selection and use text information to support a conclusion or make a point. They should be able to make logical

connections between the ideas in the text and their own background knowledge.

When reading practical text, they should be able to explain its purpose and the significance of specific details or

steps.

Grade 12–Proficient

Twelfth-grade students performing at the proficient level should be able to show an overall understanding of the

text, which includes inferential as well as literal information. When reading text appropriate to twelfth grade, they

should be able to extend the ideas of the text by making inferences, drawing conclusions, and making connections to

their own personal experiences and other readings. Connections between inferences and the text should be clear,

even when implicit. These students should be able to analyze the author’s use of literary devices.

When reading literary text, proficient-level twelfth graders should be able to integrate their personal experiences

with ideas in the text to draw and support conclusions. They should be able to explain the author’s use of literary

devices such as irony or symbolism.

When reading informative text, they should be able to apply text information appropriately to specific situations and

integrate their background information with ideas in the text to draw and support conclusions.

When reading practical texts, they should be able to apply information or directions appropriately. They should be

able to use personal experiences to evaluate the usefulness of text information.

Grade 12–Advanced

Twelfth-grade students performing at the advanced level should be able to describe more abstract themes and ideas

in the overall text. When reading text appropriate to twelfth grade, they should be able to analyze both the meaning

and the form of the text and explicitly support their analyses with specific examples from the text. They should be

able to extend the information from the text by relating it to their experiences and to the world. Their responses

should be thorough, thoughtful, and extensive.

(continued)

 

8

 Based on the recommendations of the 1994 reading revisit study, the phrase extend the ideas in the text by making simple

inferences,has been added here to the description of Basic.
Figure I-1 (continued)

Final Descriptions of 1992 Reading Achievement Levels

856

For example, when reading literary text, advanced-level twelfth graders should be able to produce complex, abstract

summaries and theme statements. They should be able to use cultural, historical, and personal information to develop

and explain text perspectives and conclusions. They should be able to evaluate the text, applying knowledge gained

from other texts.

When reading informational text, they should be able to analyze, synthesize, and evaluate points of view. They

should be able to identify the relationship between the author’s stance and elements of the text. They should be able

to apply text information to new situations and to the process of forming new responses to problems or issues.

When reading practical texts, advanced-level twelfth graders should be able to make a critical evaluation of the

usefulness of the text and apply directions from the text to new situations.
857

Figure I-2

Draft Descriptions of the Achievement Levels

Prepared by the Original Level-Setting Panel

Fourth-Grade Draft Descriptions

BASIC performance in reading should include:

• Determining what a text is about

• Identifying characterizations, settings, conflicts, or plots in a story

• Supporting one’s understanding of a text with appropriate details

• Explaining why one likes or dislikes a text

• Connecting material in a text to personal experiences

• Making predictions about situations beyond the confines of a text

• Demonstrating an ability to maintain a focus over the entirety of a longer text

PROFICIENT performance in reading should include:

• Summarizing a text

• Recognizing an author’s intent or purpose

• Making simple inferences based on information provided in a text

• Using information from a text to draw a basic conclusion

• Determining the meaning of key concepts in the text and connecting them to the main idea

• Recognizing the progression of ideas and the cause-and-effect relationships in a text

• Using the surrounding text to assign meaning to a word or phrase

ADVANCED performance in reading should include:

• Explaining an authors intent, using supporting material from the text

• Describing the similarities and differences in characters

• Demonstrating an awareness of the use of literary devices and figurative language

• Applying inferences drawn from a text to personal experiences

• Extending the meaning of a text by integrating experiences and information outside of the text

• Making and explaining a critical judgment of a text

• Demonstrating an ability to adapt reading purpose to genre and/or writing style

(continued)
Figure I-2 (continued)

Draft Descriptions of the Achievement Levels

Prepared by the Original Level-Setting Panel

858

Eighth-Grade Draft Descriptions

BASIC performance in reading should include:

• Identifying the main idea or purpose of a text using information both stated and implied

• Expressing an author’s purpose, viewpoint, and/or theme

• Using information from a text to draw and support conclusions

• Making inferences appropriate to the information provided in a text

• Recognizing the cause-and-effect relationships in a text

• Making logical connections from the material in a text to personal knowledge and experience

PROFICIENT performance in reading should include:

• Restating the main idea using supportive details and examples from a text

• Summarizing a text using information both stated and implied

• Making inferences from a text in order to draw valid conclusions

• Interpreting the actions, behaviors, and motives of characters

• Integrating personal knowledge and experience to enhance one’s understanding of a text

• Identifying an author’s use of literary devices

ADVANCED performance in reading should include:

• Describing how specific literary elements interact with each other

• Synthesizing the information in a text to obtain abstract meaning or to perform a task

• Finding new applications for information derived from a text

• Making personal and critical evaluations of a text

• Analyzing an author’s purpose, viewpoint, and/or theme

• Explaining an author’s use of literary devices

(continued)
Figure I-2 (continued)

Draft Descriptions of the Achievement Levels

Prepared by the Original Level-Setting Panel

859

Twelfth-Grade Draft Descriptions

BASIC performance in reading should include:

• Explaining the main idea of a text

• Describing the main purpose in reading a selection

• Recognizing the significance of details from a reading in order to support a conclusion or perform a task

• Applying the information gathered from reading to meet an objective or support a conclusion

• Explaining the basic elements of an author’s literary devices

PROFICIENT performance in reading should include:

• Drawing conclusions from and making inferences about information from different texts and writing styles

• Integrating background information with newly acquired information to support conclusions

• Applying information from a text in an appropriate manner

• Bringing personal experience and accumulated knowledge into the process of critically evaluating a text

• Explaining an author’s purpose in using complex literary devices

ADVANCED performance in reading should include:

• Providing innovative elaborations from textual information

• Analyzing and evaluating different points of view by means of comparison and contrast

• Identifying the relationships between an author’s or narrator’s stance and the various elements of the text

• Critically evaluating a text within a specific frame of reference

• Bringing the knowledge of other texts to the process of critical evaluation

• Using cultural or historical information provided in a text to develop perspectives on other situations

• Using cultural or historical information to develop perspectives on a text
860

Figure I-3

Revised Draft Descriptions of the Achievement Levels

Recommended by the Follow-Up Validation Panel

Revised Fourth-Grade Draft Descriptions

BASIC performance in reading should include:

• Determining what a story/informational text is about (i.e., topic, main idea)

• Determining the main purpose for reading a selection

• Identifying character(s), setting(s), conflict(s), or plot(s) in a story

• Supporting one’s understanding of a story/informational text with appropriate details

• Explaining why one likes or dislikes what they have read [a reading]

• Connecting material from a story/informational text to personal experiences

• Making predictions about situations beyond the confines of the printed material

• Maintaining a focus over the entirety of a story/informational text

PROFICIENT performance in reading should include:

• Summarizing a story/informational text

• Recognizing an author’s intent or purpose

• Making simple inferences based on information provided in a story/informational text

• Drawing a valid conclusion from a story/informational text

• Determining the meaning of key concepts in the story/informational text and connecting them to the main

idea

• Recognizing relationships in a story/informational text (i.e., time order, cause/effect, compare/contrast)

ADVANCED performance in reading should include:

• Explaining an author’s intent, using supporting material from the story/informational text

• Describing the similarities and difference in characters, settings, and plots

• Demonstrating an awareness of the use of literary devices, such as figurative language

• Applying inferences drawn from a story/informational text to personal experiences

• Extending the meaning of a story/informational text by integrating experiences and information outside of

the text

• Making and explaining a critical judgment of a story/informational text

• Demonstrating an ability to adapt reading purpose to a variety of printed material and/or writing style

(continued)
Figure I-3 (continued)

Revised Draft Descriptions of the Achievement Levels

Recommended by the Follow-Up Validation Panel

861

Revised Eighth-Grade Draft Descriptions

BASIC performance in reading should include:

• Identifying the main idea, theme, or purpose of a text

• Describing the main purpose for reading a selection

• Expressing an author’s purpose and viewpoint

• Making inferences, predictions, and drawing conclusions that are supported by information in a text

• Recognizing the relationships among facts, ideas, events, and concepts within a text (i.e., cause and effect,

chronological order, and characterization)

• Making logical connections between the text and personal knowledge

• Maintaining a focus over the entirety of a story/informational text

PROFICIENT performance in reading should include:

• Restating the main idea, theme, or purpose of a text using supporting details and examples

• Summarizing a text using both stated and implied information

• Interpreting the actions, behaviors, and motives of characters

• Using personal knowledge and experience to enhance one’s understanding of a text

• Identifying an author’s use of literary devices (i.e., personification, foreshadowing, and so forth)

• Using inferences from a text in order to draw valid conclusions

ADVANCED performance in reading should include:

• Describing how specific literary elements (i.e., setting, plot, characters, and theme) interact with each other

• Synthesizing the information in a text to obtain implied meaning or to perform a task

• Applying information derived from a text to new situations.

• Explaining an author’s use of literary devices (i.e., irony, personification, and foreshadowing)

• Responding personally and critically to a text

• Analyzing an author’s purpose and viewpoint

• Using cultural or historical information to develop perspectives on a text

• Using cultural or historical information provided in a text to develop perspectives on other situations

(continued)
Figure I-3 (continued)

Revised Draft Descriptions of the Achievement Levels

Recommended by the Follow-Up Validation Panel

862

Revised Twelfth-Grade Draft Descriptions

BASIC performance in reading should include:

• Explaining the main idea, theme, or purpose of a text

• Describing the main purpose for reading a selection

• Recognizing the significance of details from a reading in order to support a conclusion or perform a task

• Applying the information gathered from reading to meet an objective or support a conclusion

• Identifying and explaining the basic elements of an author’s literary devices

• Making logical connections between a text and personal knowledge and experience

• Maintaining a focus over the entirety of a story/informational text

PROFICIENT performance in reading should include:

• Drawing conclusions and making inferences from different texts and writing styles

• Integrating background information with newly acquired information to support conclusions

• Applying information from a text in an appropriate manner

• Applying personal experience and accumulated knowledge to the process of critically evaluating a text

• Explaining an author’s purpose in using complex literary devices (i.e., irony, symbolism)

ADVANCED performance in reading should include:

• All basic and proficient reading behaviors listed previously

• Prompted by information from a text, innovating in new situations and creating new answers to old

situations

• Analyzing, synthesizing, and evaluating different points of view by means of comparison and contrast

• Identifying the relationships between an author’s or narrator’s stance and the various elements of the text

• Critically evaluating a text within a frame of reference

• Applying the knowledge of other texts to the process of critical evaluation

• Using cultural or historical information to develop perspectives on a text

• Using cultural or historical information provided in a text to develop perspectives on other situations
863

Figure I-4

Meeting Participants, NAEP Reading Achievement Level Setting

Original Meeting, St. Louis, Missouri, August 21–25, 1992

Paula Abrams

City Hall

Bedford, KY

Freda Andrews

Durham Public Schools

Durham, NC

David Awbrey

Wichita Eagle

Wichita, KS

Tim Barnes

Ashdown Public Schools

Ashdown, AR

Larry Barretto

Maplewood Elementary School

Coral Springs, FL

Linda Borsum

Lakeview School District

Battlecreek, MI

Dorothy Botham

Milwaukee Public Library

Milwaukee, WI

Constance Boyd

Owen J. Roberts SD

King of Prussia, PA

P. Richard Brackett

Brackett & Assoc. Motivational

Marketing Company

Brentwood, TN

Anna Caballero

Attorney

Salinas, CA

Rhonda Cantrell Dunn

Nashville Urban League

Nashville, TN

Kathy Casseday

WFSP Radio Station

Kingwood, WV

Wilma Centers

Wolfe County Middle School

Campton, KY

Eunice Coakley

Greenville School

Greenville, SC

Eugenia Constantinou

Prince Georges County Schools

Silver Spring, MD

Walt Cottingham

Henderson City Schools

Zirconia, NC

Cora Cummins

Conway Public Schools

Conway, AR

Gloria Darling

Conway Public Schools

Conway, AR

Deborah Davidson

Westhampton Beach UFSD

Patchogue, NY

Julia Dominique

Department of Education USVI

Sunnyisle, VI

Dee Ellis

Trimble Banner Newspaper

Milton, KY

Kathryn Flannery

Indiana University

Bloomington, IN

Stanley Fraundorf

Cuba City Public Schools

Cuba City, WI

Nina Frederick

Marion County School System

Hackleburg, AL

Karen Fugita

Oak Grove SD

San Jose, CA

Harlon Gaskill (CPA)

Gaskill, Pharis & Pharis

Dalhart, TX

Patricia Gerdes

Waelder ISD

Schulenburg, TX

Mary Gonzalez

Mesa Public Schools

Mesa, AZ

Anne Gregory

Durham Public Schools

Durham, NC

Kathleen Harkey

Corporate Presentations

Nashville, TN

Catherine Hatala

School District of Philadelphia

Philadelphia, PA

Georgia Howard

Volusia County Schools

Holly Hill, FL

Joseph Howard

Josiah Quincy School

West Roxbury, MA

Roberta Johnson

Cleveland Public Schools

Cleveland, OH

Marcia Jolicoeur

Lisbon Falls School

Lewiston, ME

Anne Kraut

Elementary Supervisor

Princeton, WV
Figure I-4 (continued)

Meeting Participants, NAEP Reading Achievement-Level Setting

Original Meeting, St. Louis, Missouri, August 21 - 25, 1992

864

Roger Larsen

Campbell County SD

Gillette, WY

Mary Ann Ledbetter

East Baton Rouge Parish School

Board

Baton Rouge, LA

Leslie Leech

Elkton School

Elkton, SD

Belva Leffel

Whittier Christian Jr. High

Norwalk, CA

Elizabeth Litchfield

Westwood School District

Emerson, NJ

Judith Lusk

Norfield School District

Rockbury, VT

Harriett McAllaster

Volusia County Schools

DeLand, FL

Jean McManis

Local/State Education Volunteer

State College, PA

Donnie McQuinn

Wolfe County Board of

Education

Pine Ridge, KY

Raymond Morgan

Old Dominion University

Virginia Beach, VA

Patricia Oliverez

Salinas Public Library

Salinas, CA

Mary Orear

Camden–Rockport HS & MS

Rockport, ME

Meredith Powers

Swansea School

Providence, RI

Beth Schieber

Kingfisher Schools

Okarche, OK

James Schindler

Jordan SD

Salt Lake City, UT

Christine Sentz

North Milwaukee Branch

Library

Milwaukee, WI

Nona Smith

NAACP

New York, NY

Lillaine Speese

Oakdale Elementary School

Oroville, CA

Carolyn Sullivan

Planters & Merchants Bank

Gillett, AR

Clifton Whetten

Retired Construction Sprvsr.

Elfrida, AZ

Robert Williams

Macomb Intermediate SD

Clinton Township, MI

Carolyn Sue Wilson

Greenville, SC

Berton Wiser

Columbus Public School

Columbus, OH

Jean Young

Houston ISD

Houston, TX

Sue Zak

Cleveland Board of Education

Garfield Heights, OH

Judith Zinsser

Houston ISD

Houston, TX
865

Figure I-5

Meeting Participants, NAEP Reading Achievement Level Setting

Follow-Up Validation Meeting, San Diego, California, October 9–11, 1992

Larry Barretto

Maplewood Elementary School

Coral Springs, FL

Martha Carter

Milwaukee Public Schools

Milwaukee, WI

Eunice Coakley

Greenville School

Greenville, SC

Clyde Colwell

Norfolk Public School

Norfolk, VA

Mark Conley

Michigan State University

Holt, MI

Eugenia Constantinou

Prince George’s County School

Silver Spring, MD

Debra Davidson

Westhampton Beach UFSD

Patchogue, NY

Peggy Dutcher

Michigan Education Assessment

Program

Lansing, MI

Anne Gregory

Durham Public Schools

Durham, NC

Gene Jongsma

IRA Subcommittee Member

San Antonio, TX

Roger Larsen

Campbell County SD

Gillett, WY

Elizabeth Litchfield

Westwood School District

Emmerson, NJ

Nancy Livingston

Brigham Young University

Salt Lake City, UT

Susan McIntyre

University Wisconsin-Eau Claire

Eau Claire, WI

Mary Orear

Camden–Rockport HS & MS

Rockport, ME

Shelia Potter

Michigan Department of Education

Lansing, MI

Meredith Powers

Swansea School

Providence, RI

Jo Prather

Mississippi Department of Education

Jackson, MS

Beth Schieber

Kingfisher Schools

Okarche, OK
866

Figure I-6

Meeting Participants, NAEP Reading Revisit

Validation Meeting, St. Louis, Missouri, October 14–16, 1994

Jody Alexander

Madison No. 1

Phoenix, AZ

Evelyn Alford

East Baton Rouge Public

Schools

Baton Rouge, LA

Winfrey Bates

Mannsville Elementary School

Mannsville, KY

Joyce Boone

John Strange Elementary School

Indianapolis, IN

Linda Brooks

Alcorn County Public Schools

Corinth, MS

Katie Burnham

Pa Wau Lu Middle School

Gardnerville, NV

Martha Carter

Milwaukee Public Schools

Milwaukee, WI

Carol Case

Mirabean B. Lamar High School

Houston, TX

Molly Chun

Applegate Elementary School

Portland, OR

Roseine Church

Cheyenne, WY

Connie Clayton

Franklin High School

Franklin, WV

David Colburn

Flathead High School

Kalispell, MT

Brenda Creel

Jessup Elementary School

Cheyenne, WY

Pam Diamond

Hellgate Middle School

Missoula, MT

Caroline Downs

Worland Middle School

Worland, WY

Esther Dunnington

Grandview High School

Grandview, MO

Sandra Forsythe

Green Valley High School

Henderson, NV

David Fredette

Westborough High School

Westborough, MA

Cynthia Freeman

Maryville High School

Maryville, TN

Rita Gallagher

Roswell, NM

Lorraine Gerhart

Elmbrook Middle School

Elm Grove, WI

Bill Hammond

GA Department of Education

Atlanta, GA

Sally Hellman

Las Vegas, NV

Grace Herr

West Linn High School

West Linn, OR

Sarah Herz

Coleytown Middle School

Westport, CT

Susan Hodgin

Moscow Public Schools

Moscow, ID

Beverly Hoffmaster

Berkeley Heights

Elem School

Martinsburg, WV

Roberta Horton

Custer County High School

Miles City, MT

Lory Johnson

Iowa Department of Education

Des Moines, IA

Ruth Johnson

Holmes High School

Covington, KY

Theresa Lowe

Rancho Viejo School

Yuma, AZ

Ruby Mayes

S.P. Waltrip High School

Houston, TX

Robert McKean

Havre Public Schools

Havre, MT

Pamela McNair

Lemon G. Hine Jr. High School

Washington, DC

Daniel McQuagge

Delta State University

Cleveland, MS

Cheryl Miller

Buchanan Elementary School

Baton Rouge, LA
Figure I-6 (continued)

Meeting Participants, NAEP Reading Revisit

Validation Meeting, St. Louis, Missouri, October 14 - 16, 1994

867

Donna Miller

Chinook High School

Chinook, MT

Lynn Minderman

Honeoye Falls–Lima Public

Schools

Honeoye Falls, NY

John Morrissey

Huntley Project

Elem School

Worden, MT

Pamela Perryman

Selah Middle School

Selah, WA

Kathleen Sanders

Los Angeles Unified S.D.

Wilmington, CA

Helen Schotanus

NH Department of Education

Concord, NH

Terrence Smith

Verona School

Battle Creek, MI

Faith Stevens

Haslett Public Schools

Haslett, MI

Richard Telfer

Univ. of Wisconsin–Whitewater

Whitewater, WI

Cara Terry

Lakewood High School

St. Petersburg, FL

James Thompson

Simpson–Waverly School

Hartford, CT

Patsy Turner

Great River Co-operative

West Helena, AR

Maria Valeri-Gold

Georgia State University

Atlanta, GA

Florence Wakuya

Hawaii Department of Education

Honolulu, HI

Barbara Watson

Agricola Elementary School

Lucedale, MS

Janet Williams

Bluewell Elementary School

Bluefield, WV

Sarah Williams

Maryville Middle School

Maryville, TN

Philip Yeaton

Concord, NH

Katie Young

Louisiana Department of Ed

Baton Rouge, L

869

Appendix J

SETTING THE ACHIEVEMENT LEVELS

FOR THE 1998 NAEP CIVICS AND WRITING ASSESSMENTS

Mary Lyn Bourque

National Assessment Governing Board

J.1 INTRODUCTION

The 1988 NAEP legislation (Hawkins-Stafford Education Improvement Act Amendments of

1988) created an independent board, the National Assessment Governing Board (NAGB), responsible for

setting policy for the NAEP program. The 1994 NAEP reauthorization (Improving America’s Schools

Act of 1994) continued many of the board’s statutory responsibilities, including “developing appropriate

student performance standards for each age and grade in each subject area to be tested under the National

Assessment.” Consistent with this directive, and striving to achieve one of the primary mandates of the

statute “to improve the form and use of NAEP results,” the board has been developing student

performance standards (called achievement levels by NAGB) on the national assessment since 1990.

From 1984 to 1996, NAEP reported the performance of students in the nation and for specific

subpopulations on a 0-to-500 score scale. This scale was a cross-grade scale, that is, a single performance

scale was developed for grades 4, 8, and 12, so that comparisons could be made between and among the

three grade cohorts. In 1996, NAGB policy required that a unique scale be developed for each grade

level. The new metric chosen ranged from 0-to-300 to minimize confusion between the earlier crossgrade scale and the new within-grade scale. The history and development of the scales in civics and

writing are described in Chapter 12 of this report.

Setting achievement levels is a method for setting standards on the NAEP assessment that

identify what students should know and be able to do at various points along the score scale. The policy

definitions and the final content descriptions of the achievement levels were presented to panelists along

with the assessment framework and the full NAEP item pool in order to estimate the cut scores for the

levels. Panelists were asked to internalize the achievement-level descriptions and to become familiar with

the NAEP item pool for the particular NAEP assessment with respect to the content and skills assessed.

In addition to recommending cut scores, panelists were also asked to select illustrative exercises

associated with each level, selecting from the released exercises in the NAEP item pool those sample

items and student responses (in the case of constructed-response exercises) that best exemplified the full

range of performance of the intervals between levels. The emphasis in operationalizing the definitions

and in identifying and selecting exemplar items and papers was to represent the full range of performance

from the lower level to the next higher level. The details of the implementation procedures are outlined in

the remainder of this appendix.

J.2 PREPARING THE FINAL DESCRIPTIONS

The 1998 levels setting process was different in some significant ways from earlier level-setting

processes that had been used in other NAEP subject areas. The first of these differences occurred at the

very beginning of the process. In the past, panelists were given the preliminary descriptions of the levels

developed initially by the framework consensus groups and asked to craft recommended descriptions

during the process. The descriptions continued to be refined throughout the level-setting process, and

usually were validated by a supplementary group of judges subsequent to the level-setting meetings.
870

In 1998, the finalized achievement-level descriptions (ALDs) were givens in the process, much

like the framework and the item pool are givens. Prior to the level-setting meetings, focus groups in each

NAEP region were conducted to evaluate the preliminary ALDs for each subject (civics and writing).

The focus group recommendations were reviewed by expert content panels and appropriate modifications

were made. The revised ALDs were then reviewed and evaluated by the framework consensus panels and

all focus group participants. These proposed final ALDs were then reviewed and modified by the NAGB

Achievement Levels Committee and approved for use—without change—in the remainder of the process.

J.3 1998 FIELD TRIALS IN CIVICS AND WRITING

A second difference between the 1998 process and earlier processes was the field trials. In the

past, the pilot studies combined both the pilot work (to test out the operational procedures) and the initial

research work (to try out various methods). The 1998 process separated these two tasks by having two

stages: first stage, field trials; and second stage, pilot studies.

In 1998, two field trials in each subject were conducted to identify rating methods and

procedures. Prior to the field trials, a number of computer simulations were completed to determine the

feasibility of the proposed new methods. Field trial 1 was designed to test a new method (item score

string estimation, or ISSE) in comparison to the “current” method. For civics, the current method was a

combination of modified-Angoff method for multiple-choice items and the mean estimation method for

constructed-response items. For writing, the current method was mean estimation, since the NAEP

writing assessment is a direct writing assessment and consists entirely of constructed response.

Field trial 2 was originally designed to compare an item-mapping procedure with the new method

from field trial 1 (ISSE) and to test the provision of consequences data to panelists at various points in

the process. However, analysis of the field trial 1 data led to the conclusion the ISSE method was biased,

and further exploration with it was not recommended by the Technical Advisory Committee on Standard

Setting (TACSS), the external group of advisors to ACT. Therefore, field trial 2 in civics compared an

alternative, the Reckase method, with the mean estimation method combined with item maps in civics,

and the Reckase method with the booklet classification method in writing. Full details of the field trials

in each subject can be found in the ACT reports (ACT, 1998; 1999c). The recommendations from the

two field trials resulted in using the modified Angoff and mean estimation methods with Reckase charts

in civics, and the mean estimation method with Reckase charts in writing.

J.4 PREPARATION FOR CIVICS AND WRITING LEVEL SETTING MEETINGS

It is important for the planning of any standard-setting effort to know how various process

elements interact with each other. For example, panelists interact with premeeting materials, meeting

materials (i.e., the assessment items, rating forms, rater feedback, and so forth), each other, and the

project staff. All of these elements combine to promote or degrade what has been called intrajudge

consistency and interjudge consensus (Friedman & Ho, 1990).

Previous research has conceptualized the effects of two major kinds of interaction: (1) people

interacting with text (Smith & Smith, 1988), and (2) people interacting with each other (Curry, 1987;

Fitzpatrick, 1989). To assess the effects of textual and social interaction and adjust the standard-setting

procedures accordingly, a pilot study in each content area was conducted in preparation for the 1998

level setting.
871

J.5 1998 PILOT STUDIES IN CIVICS AND WRITING

As a result of the earlier field trials, the pilot studies were more focused, concentrating on the

methodologies that would be used in the operational level setting, and adding those elements that were

thought to be positive enhancements to the process. In civics, the pilot studies implemented an item-byitem rating method (the modified Angoff) for multiple-choice items, and the mean estimation method for

the constructed-response items. In writing, the mean estimation method was used exclusively. However,

there were two enhancements not used previously that were incorporated into the feedback loop for both

civics and writing.

The first of these, Reckase charts, were provided to the panelists after Rounds 1 and 2. Figure J–

1 displays an enlargement of a portion of a Reckase chart. The chart displays a matrix of information

about the items on the assessment where the horizontal rows represent the probability of a correct

response (for multiple-choice items) or the expected mean score (for constructed-response items) at a

specific point on the score scale, for all the items in a particular block (or, in the case of writing, for the

prompts in the exercise pool); and the vertical columns represent the same information (probability of a

correct response or expected mean score) across the score scale range for a single item or exercise. The

Reckase charts are an aggregated and tabularized version of the item characteristics curves (ICCs) for a

block of items or a portion of the exercise pool.

The Reckase charts were provided to panelists after the first round so that they could “plot” their

grade-level and individual cut scores on the chart to compare their individual data with the group’s data.

If panelists plot their own item ratings on the chart and they are very consistent in their ratings, they

should see very few peaks and valleys in their plot. A flat line on the chart indicates that panelists were

able to judge the items consistently1

 for their item difficulty and discrimination. An erratic line with

many peaks and valleys would indicate that panelists were unable to judge item difficulty and

discrimination consistently from item to item in the block, or across the pool of exercises. Further,

panelists could look at the distance between their individual line on the chart and that of the grade group.

The wider the gap, the more deviant the individual is from the mean of the group. Panelists were given

updated charts again after Round 2 for additional feedback (according to the new cut scores set in Round

2). Figure J–2 displays a completed portion of a Reckase chart for one of the civics blocks. This “ideal”

panelist is somewhat consistent at the Proficient level, but much less so at the Basic level. The charts

also allowed panelists to “see” their extreme ratings for any particular item. For example, those items that

were rated particularly low (e.g., at or below the guessing parameter) were “off the chart”; while those at

the high end (e.g., at or above a selected theta value) were also “off the chart.” This gave panelists their

first indication that they needed to reconsider the item to understand what was causing them to have such

extreme ratings.

 

1 Consistency in this case refers to the panelists’ judgment about the difficulty and discrimination of the item with respect to the

achievement-level descriptions and its consistency with the model-based estimates of item difficulty and discrimination. It is

important to note that model-based estimates take into account other information that is not generally known to the panelists.
872

The second enhancement to the process was the introduction of consequences data during the

rating process. The field trial data supported the idea of providing panelists with consequences data (that

is, the percentage of students at or above the levels) after Round 3 to estimate final cut points for the final

recommendation to the NAGB. This change was introduced in the pilots partly in response to the

National Academy of Sciences’ evaluation and partly due to the recent effort in other standard-setting

venues to provide such information to judges (National Academy of Sciences, 1998). This change

resulted in four estimates of cut scores for the levels, three using item-by-item approaches, and the final

round using a more holistic approach and consequences data. In the final estimate, panelists were asked

to judge the reasonableness of their standards, taking into account the percentage of students at or above

the levels, and to decide whether or not some final adjustment was necessary.

Figure J-1

Sample Reckase Chart Portion

ACT

NAEPCivics Items for Block Y1X1

Like Score 1 2 3 4 5 6 7 8 9 10 11

273 99 99 99 3.0 3.0 100 3.0 99 99 4.0 99

185 98 96 98 2.9 2.7 99 2.5 94 69 3.0 99

183 98 95 98 2.9 2.7 99 2.4 93 66 2.9 99

181 97 95 97 2.8 2.6 99 2.4 91 63 2.8 99

179 97 94 96 2.8 2.6 99 2.3 89 61 2.7 98

177 96 93 95 2.8 2.5 99 2.2 87 58 2.6 98

175 96 92 93 2.8 2.5 89 2.2 84 55 2.5 98

173 95 91 91 2.7 2.4 97 2.1 81 52 2.4 98

171 94 89 89 2.7 2.4 94 2.1 78 49 2.3 97

169 92 88 85 2.7 2.3 90 2.0 74 47 2.2 97

167 91 86 81 2.5 2.3 83 1.9 70 44 2.1 97

165 89 84 76 2.5 2.2 73 1.9 65 42 2.0 96

163 87 82 70 2.5 2.2 61 1.8 61 40 1.9 95

161 85 80 64 2.5 2.1 50 1.7 56 38 1.8 95

159 82 77 58 2.4 2.0 40 1.7 52 36 1.7 94

157 79 75 52 2.4 2.0 33 1.6 48 34 1.6 93

155 76 72 46 2.3 1.9 29 1.6 45 33 1.6 92

153 72 69 41 2.3 1.8 27 1.5 42 31 1.5 90

151 68 66 37 2.2 1.8 26 1.5 39 30 1.5 89

149 65 63 34 2.1 1.7 25 1.4 37 29 1.4 87

147 61 60 31 2.1 1.7 25 1.4 35 28 1.4 85

39 27 26 23 1.0 1.0 24 1.0 26 20 1.0 34
873

Figure J-2

Sample Reckase Chart – Complete

ACT NAEP- Civics Items for Block Y1X1

Like Score 1 2 3 4 5 6 7 8 9 10 11

273 99 99 99 3.0 3.0 100 3.0 99 99 4.0 99

225 99 99 99 3.0 3.0 99 2.9 99 97 3.9 99

223 99 99 99 3.0 3.0 99 2.9 99 96 3.8 99

221 99 99 99 3.0 3.0 99 2.9 99 96 3.8 99

219 99 99 99 3.0 2.9 99 2.9 99 95 {3.8} 99

217 99 99 99 3.0 2.9 99 2.9 99 95 3.8 99

215 99 99 99 3.0 2.9 99 2.9 99 94 3.8 99

213 99 99 99 3.0 2.9 99 2.9 99 93 3.7 99

211 99 99 99 3.0 {2.9} 99 2.9 99 93 3.7 99

209 99 99 99 3.0 {2.9} 99 2.9 99 92 3.7 99

207 99 99 99 3.0 2.9 99 2.8 99 {91} 3.6 99

205 99 99 99 3.0 2.9 99 {2.8} 99 {89} 3.6 99

203 99 99 99 3.0 2.9 99 {2.8} 99 88 3.5 99

201 99 99 99 3.0 2.9 99 2.8 99 86 3.5 99

199 99 98 99 2.9 2.8 99 2.7 98 85 3.4 99

197 99 98 99 2.9 2.8 99 2.7 98 83 3.4 99

195 99 98 99 2.9 2.8 99 2.7 98 81 3.3 99

193 99 98 99 2.9 2.8 99 2.6 97 79 3.3 99

191 99 97 99 2.9 2.8 99 2.6 97 77 3.2 99

189 99 97 99 2.9 2.7 99 2.6 96 74 [3.1] 99

187 98 96 99 2.9 2.7 99 [2.5] 95 72 3.0 99

185 98 96 98 2.9 2.7 99 [2.5] 94 69 3.0 99

183 98 95 98 2.9 2.7 99 2.4 93 66 2.9 99

181 97 95 97 2.8 [2.6] 99 2.4 91 63 2.8 99

179 97 94 96 {2.8} [2.6] 99 2.3 (89) 61 2.7 98

177 96 93 95 {2.8} 2.5 99 2.2 87 58 2.6 98

175 96 92 93 2.8 2.5 {89} 2.2 84 [55] 2.5 98

173 95 91 {91} 2.7 2.4 97 [2.1] 81 [52] 2.4 98

171 94 89 89 2.7 2.4 94 [2.1] 78 49 2.3 97

169 92 88 85 2.7 2.3 [90] 2.0 74 47 2.2 97

167 91 86 81 [2.5] 2.3 [83] 1.9 70 44 (2.1) 97

165 89 84 76 [2.5] (2.2) 73 1.9 [65] 42 2.0 {96}

163 87 {82} 70 2.5 (2.2) 61 1.8 [61] 40 1.9 95

161 {85} 80 64 2.5 2.1 50 1.7 56 38 1.8 95

159 82 77 [58] 2.4 2.0 (40) 1.7 52 36 1.7 94

157 79 75 [52] 2.4 2.0 33 1.6 (48) 34 1.6 93

155 76 72 46 2.3 1.9 29 1.6 45 33 1.6 [92]

153 72 69 41 2.3 1.8 27 1.5 42 31 1.5 [90]

151 68 [66] 37 2.2 1.8 26 1.5 39 30 1.5 89

149 65 63 34 2.1 1.7 25 1.4 37 (29) 1.4 87

147 [61] 60 (31) 2.1 1.7 25 1.4 35 28 1.4 85

145 57 57 (29) (2.0) 1.6 25 1.4 33 27 1.3 83

143 53 54 27 1.9 1.6 24 1.3 32 26 1.3 81

141 50 51 26 1.9 1.5 24 1.3 31 25 1.2 78

139 47 48 25 1.8 1.5 24 1.3 30 25 1.2 75

137 44 46 25 1.7 1.4 24 1.2 29 24 1.2 73

135 41 44 24 1.7 1.4 24 1.2 28 24 1.2 (70)

133 39 42 24 1.6 1.3 24 1.2 28 23 1.1 67

131 37 40 24 1.6 1.3 24 1.2 28 23 1.1 64

129 (35) 38 23 1.5 1.3 24 1.1 27 22 1.1 61

127 34 36 23 1.5 1.3 24 1.1 27 22 1.1 58

125 33 35 23 1.4 1.2 24 1.1 27 22 1.1 55

123 32 34 23 1.4 1.2 24 1.1 27 22 1.1 53

121 31 33 23 1.3 1.2 24 1.1 27 21 1.1 50

119 30 32 23 1.3 1.2 24 1.1 27 21 1.1 48

117 30 31 23 1.3 1.2 24 1.1 27 21 1.1 46

115 29 30 23 1.2 1.1 24 1.1 27 21 1.0 45

113 29 29 23 1.2 1.1 24 1.1 27 21 1.0 43

111 29 29 23 1.2 1.1 24 1.0 27 21 1.0 42

109 28 28 23 1.2 1.1 24 1.0 27 21 1.0 41

107 28 (28) 23 1.1 1.1 24 1.0 27 21 1.0 40

105 28 28 23 1.1 1.1 24 1.0 26 21 1.0 39

103 28 27 23 1.1 1.1 24 1.0 26 21 1.0 38

39 27 26 23 1.0 1.0 24 1.0 26 20 1.0 34

A

P

B

Advanced

Ratings

Basic

Ratings
874

J.6 RESULTS OF THE 1998 PILOT STUDIES

Fifty-three panelists representing the four NAEP regions were selected from the 329 nominees

and invited to participate in the civics pilot. Sixty panelists selected in the same way were invited to

participate in the writing pilot from the 419 nominated. The panelists represented teachers at grades 4, 8,

and 12, nonteacher educators, and members of the noneducator (general public) community. The group

was balanced by gender, race/ethnicity, NAEP regions, community type (i.e., low SES or not low SES),

district size, and school type (i.e., public or nonpublic).

Tables J-1 and J-2 display the results of the pilot study cut scores and the standard deviations for

civics and writing in grades 4, 8, and 12. The results are on the ACT NAEP-like scale score, having a

effective range from 0-to-300, with a mean of 155 and a standard deviation of 14. Further details of the

pilot studies can be found in the contractor’s final reports (ACT 1999a, 1999e). It is worthy to note that

unlike other standard-setting studies, the civics pilot cut scores for all grades and all levels increased

from round to round. Additionally, cut scores for dichotomous and polytomous items became closer from

round to round.

Table J-1

Pilot Study Cut Scores (Standard Deviations) on the 1998 Civics NAEP

Grade Achievement Level Round 1 Round 2 Round 3 Final

4 Basic 144.7 (15.9) 146.0 (8.3) 149.3 (5.2) 148.9 (3.6)

Proficient 161.5 (6.9) 162.9 (5.1) 165.0 (3.9) 164.1 (3.7)

Advanced 174.2 (7.6) 176.0 (6.5) 178.8 (5.5) 176.2 (5.1)

8 Basic 152.2 (9.5) 153.3 (6.8) 154.2 (5.7) 154.1 (5.5)

Proficient 165.5 (5.2) 166.1 (5.1) 167.3 (4.2) 167.1 (4.1)

Advanced 176.9 (5.9) 177.6 (5.8) 179.2 (4.7) 179.6 (4.6)

12 Basic 147.6 (6.0) 148.4 (3.7) 149.0 (3.3) 149.3 (3.2)

Proficient 163.0 (3.6) 164.1 (2.7) 164.5 (2.4) 164.6 (2.4)

Advanced 173.8 (5.6) 175.9 (5.1) 176.7 (4.8) 177.5 (4.8)

Table J-2

Pilot Study Cut Scores (Standard Deviations) on the 1998 Writing NAEP

Grade Achievement Level Round 1 Round 2 Round 3 Final

4 Basic 141.6 (5.2) 144.5 (4.0) 145.0 (3.3) 145.3 (2.6)

Proficient 165.1 (9.2) 167.5 (4.0) 168.0 (3.9) 167.1 (3.0)

Advanced 186.7 (8.0) 189.3 (3.5) 189.1 (4.2) 186.6 (3.1)

8 Basic 140.2(10.1) 145.6 (9.9) 149.8 (7.5) 151.2 (5.0)

Proficient 165.0 (8.4) 171.0 (5.9) 172.3 (5.2) 170.9 (4.3)

Advanced 186.1 (4.7) 189.5 (6.5) 190.7 (5.2) 188.6 (4.9)

12 Basic 135.9 (4.6) 137.0 (3.5) 137.5 (2.7) 138.3 (2.1)

Proficient 156.1 (5.5) 159.1 (3.5) 157.9 (5.2) 158.9 (2.3)

Advanced 179.9 (8.4) 182.2 (4.3) 182.8 (4.0) 181.7 (3.7)
875

J.7 1998 LEVEL-SETTING PANELS

Eighty-eight panelists representing the four NAEP regions were selected from the 422 nominees

and invited to participate in the writing level-setting process. In civics, 87 panelists participated, selected

from a nominee pool of 329 persons. Both panels represented teachers at grades 4, 8, and 12, nonteacher

educators, and members of the noneducator (general public) community. The group was balanced by

gender, race/ethnicity, NAEP regions of the country, community type (i.e., low SES or not low SES),

district size, and school type (i.e., public or nonpublic).

J.8 1998 PROCESS FOR DEVELOPING THE ACHIEVEMENT LEVELS

The 1998 pilot studies were successful as dress rehearsals for the operational standard-setting

meetings. However, some adjustments that were made for the operational standard-setting meetings as a

result of the pilot studies were not trivial. First, the consequences data provided during the pilots only

after Round 3 were provided on two occasions in the operational meetings. Grade-level consequences

data were provided after Round 2, and individual consequences data were provided after Round 3.

Grade-level consequences data were provided in the form of the percentages of students at or above the

cut scores, where the cut scores were based on the mean of all panelists within a grade level group.

Individual consequences data were unique to each panelist and were provided in the form of the

percentages of students at or above the panelists’ individual cut scores. The estimates of the cut scores

made in the final Round 4 (with the availability of consequences data) would become the

recommendations made to the board.

Panelists selected for each subject area were convened on separate occasions for a five-day levelsetting process. Virtually the same agenda was followed for both subjects. In the opening sessions,

panelists were provided “advance organizers” to help them see the complete picture of what they would

be doing for the remaining days. An overview, via a computerized presentation, demonstrated each step

in the process, the reasons for each step, and the interconnections between them. Each panelist was given

a “briefing booklet” that described each task to be performed during each session, purpose of the task,

and how to perform the task.

During the first two days, panelists were given a brief overview of NAEP and NAGB, a

presentation on the policy definitions of the achievement levels, a review of the NAEP assessment

frameworks, and a summary of the factors that influence item difficulty. The purpose of the presentations

was to focus the panelists’ attention on the assessment framework and to emphasize the fact that

panelists’ work was directly related to the NAEP assessment, not to the subject-matter domain as a

whole. In addition, all panelists completed and self-scored an appropriate grade-level form of the NAEP

assessment. The purpose of this exercise was to familiarize panelists with the test content and scoring

protocols—as well as time constraints—before beginning the formal training for the level-setting

activities.

The policy definitions are as follows:

Basic This level represents partial mastery of prerequisite knowledge and skills

that are fundamental for proficient work at each grade.

Proficient This level represents solid academic performance for each grade

assessed. Students reaching this level have demonstrated competency

over challenging subject matter, including subject-matter knowledge,

application of such knowledge to real-world situations, and analytical

skills appropriate to the subject matter.

Advanced This higher level signifies superior performance.
876

Panelists received training in the frameworks and the achievement-level descriptions through a

series of exercises designed to provide them experience in working with the descriptions as

operationalized during framework development and finalized during the planning phases of the project.

These descriptions reflect what students should know and be able to do at each level. In addition,

panelists were expected to become familiar with the various exercise formats, scoring guides, and in the

case of polytomous exercises, the scoring rubrics. They were also given the opportunity to review student

responses to selected constructed-response exercises so that they could begin to crystallize their

conception of borderline performance for each level. These were important as prior activities to the item

rating process. Specific training in each task was provided in a general session to ensure standardization

in instructions. Grade-level facilitators reinforced the large-group training sessions and answered

questions for panelists in grade-level groups. Tasks were then completed as designed. This sequence was

followed for all tasks in the five-day session.

Following training in the modified Angoff method for dichotomously scored items and the mean

estimation method for polytomously-scored items, the judges began the three-round rating process. In

Round 1, and all subsequent rounds, panelists rated about one-half the total number of exercises in the

grade-level pool. When each round was completed, panelists’ ratings were key-entered and analyzed to

produce feedback information during the remaining rounds of ratings. After each round, participants

were given item difficulty data for all items in their rating pool, interjudge consistency information,

Reckase charts, and examples of student booklets at or near their estimated cut scores. These types of

data provided panelists with a reality check against which to compare their ratings. They could then

adjust their ratings in subsequent rounds if they thought an adjustment was necessary.

For the multiple-choice and short constructed-response items that were dichotomously scored,

the judges each rated half of the items in the NAEP pool in terms of the expected probability that a

student at a borderline achievement level would answer the item correctly, based on the judges’

operationalization of the policy definitions and the factors that influence item difficulty. To assist the

judges in generating consistently scaled ratings, the rating process was repeated twice, with feedback.

Information on consistency among different judges and on the difficulty of each item was provided after

both rounds, as well as information on the consistency of each judge’s set of ratings with grade-level

estimates. The third round of ratings permitted the judges to discuss their ratings among themselves to

resolve problematic ratings. The mean judges’ final rating, aggregated across all items, yielded the

threshold values for these items in the percent correct metric. These cut scores were then mapped onto

the NAEP scale (which is defined and scored using item response theory, rather than percent correct).

For extended constructed-response items (e.g., in writing), judges were asked to estimate the

mean score on the rating score scale for the borderline performance at each achievement level. The

panelists’ overall mean was mapped onto the NAEP scale in a manner similar to that used for the items

scored dichotomously.

In civics, the final cut score for each achievement level was a weighted average of the cut score

for the multiple-choice and short constructed-response items and the cut score for the extended

constructed-response items, with the weights being proportional to the information supplied by the two

classes of items.

Following Rounds 2 and 3, panelists were given “consequences data”; that is, panelists were

given close approximations of the percentages of students who would score at or above each achievement

level based on the cut scores that had been set during the earlier round. They were asked to consider

these data as they completed Round 3 and the final round.

Tables J-3 and J-4 display the cut scores for each subject area on the ACT NAEP-like scale, as

well as the “percent correct data” across the grade-level item pool.
877

Table J-3

Civics Achievement-Level Cut Scores and Standard Deviations,

by Rounds and Percent Correct Data

Basic Proficient Advanced

Grade 4

Cut Score

Standard

Deviation Cut Score

Standard

Deviation Cut Score

Standard

Deviation

Round 1 147.4 10.1 163.8 4.9 175.5 5.9

Round 2 148.6 5.7 164.1 3.5 177.0 4.4

Round 3 149.7 5.4 164.6 3.4 177.8 4.5

Final 150.2 4.9 164.7 3.2 177.8 4.0

% Correct 47.8% 65.8% 81.7%

Grade 8

Round 1 148.1 9.6 165.2 3.5 177.1 4.4

Round 2 149.3 6.0 165.2 3.0 177.1 3.8

Round 3 149.7 5.6 165.4 2.9 177.1 3.8

Final 149.2 5.3 165.4 2.8 177.9 3.0

% Correct 43.6% 64.6% 82.8%

Grade 12

Round 1 150.6 7.1 163.6 4.0 174.2 5.8

Round 2 150.4 5.2 163.9 3.6 174.8 3.6

Round 3 150.9 5.1 164.1 3.6 175.2 3.6

Final 151.2 3.9 164.1 3.0 175.2 3.4

% Correct 48.1% 67.2% 84.3%

Note: Percent correct data are estimates of the percentage of possible points required for a score at the lower borderline of

each achievement level. Read: "Students would have to get at least 84.3 percent of the possible points on the items to score at

the advanced level in grade 12."

J.9 MAPPING THE LEVELS ONTO THE NAEP SCALE

The process of mapping panelists’ ratings to the NAEP scales used item response theory (IRT).

IRT provided statistically sophisticated methods for determining the expected performance of examinees

on particular test items in terms of an appropriate measurement scale. The same measurement scale

simultaneously described the characteristics of the test items and the performance of the examinees. Once

the item characteristics were set, it was possible to determine precisely how examinees were likely to

perform on the test items at different points of the measurement scale.

The panelists’ ratings of the NAEP test items were likewise linked, by definition, to the expected

performance of examinees at the theoretical achievement-level cut points. It was therefore feasible to use

the IRT item characteristics to calculate the values on the measurement scale corresponding to each

achievement level. This was done by averaging the item ratings over panelists for each achievement level

and then simply using the item characteristics to find the corresponding achievement-level cut points on

the IRT measurement scale. This process was repeated for each of the NAEP civics and writing scales

within each grade (4, 8, and 12).

In the final stage of the mapping process, the achievement-level cut points on the IRT

measurement scale were combined over content areas and rescaled to the NAEP score scale. Weighted

averages of the achievement-level cut points were computed. The weighting constants accounted for the

measurement precision of the test items evaluated by the panelists, the proportion of items belonging to
878

each NAEP content area, and the linear NAEP scale transformations. These weighted averages produced

the final cut points for the basic, proficient, and advanced achievement levels within each grade.

Table J-4

Writing Achievement-Level Cut Scores and Standard Deviations,

by Rounds and Percent Correct Data

Basic Proficient Advanced

Grade 4

Cut Score

Standard

Deviation Cut Score

Standard

Deviation Cut Score

Standard

Deviation

Round 1 137.6 5.4 163.1 5.2 185.6 5.4

Round 2 138.7 4.2 164.9 3.9 186.8 4.6

Round 3 139.2 3.8 164.9 3.4 185.6 4.4

Final 139.5 3.4 164.9 3.2 184.8 4.0

% Correct 45.3% 67.6% 86.2%

Grade 8

Round 1 138.5 6.5 163.6 5.7 185.3 4.2

Round 2 139.7 3.6 164.0 2.5 185.2 2.3

Round 3 139.7 3.2 163.8 2.2 184.9 2.2

Final 139.7 3.0 163.7 2.1 184.9 2.2

% Correct 46.3% 68.0% 87.4%

Grade 12

Round 1 141.8 6.4 164.9 6.5 189.3 8.1

Round 2 142.6 3.5 165.6 3.2 189.7 4.5

Round 3 142.8 3.4 165.8 2.7 187.7 4.7

Final 143.1 3.3 165.8 2.4 186.8 4.1

% Correct 54.7% 74.2% 89.7%

Note: Percent correct data are estimates of the percentage of possible points required for a score at the lower borderline of

each achievement level. Read: "Students would have to get at least 89.7 percent of the possible points on the items to score at

the advanced level in grade 12."

J.10 ADDITIONAL ANALYSIS OF THE 1998 DATA

Additional analyses were completed to examine the effects of item type, panelist type, panelists’

demographics, common blocks, “extreme raters” and other patterns detected through Reckase charts,

effect of consequences data on panelists, responses to specific questionnaire items, and ratinggroup/table-group membership on the item ratings. Mean cut scores were analyzed by grade level for

differences by subgroups. Some notable significant differences for each subject area by subgroup are

described below.

Writing. Among all the comparisons by rating group (i.e., one-half of the grade-level group), no

significant differences were found across all rounds for grades 4 and 8, and grade 12 at the proficient and

advanced levels. However, for grade 12 basic, there were significant differences between the two rating

groups (A and B)2

 across all four rounds. In subsequent analyses using a multiple comparisons procedure

 

2 Both panelists and item pool are divided in half for purposes of conducting the ratings. The criteria for dividing panelists are the

background characteristics such as gender, race/ethnicity, and type of district. The criteria for dividing item pool are item

formats, item difficulty, and numbers of items, ensuring there are some item blocks in common across rating groups. The purpose

of this design is to allow a direct estimation of the standard error using Brennan’s generalizability coefficient.
879

and controlling for other variables (i.e., table group, panelist type, gender, ethnicity, and region) there

were no significant differences among the rating groups at grade 12 basic. There were no significant

gender or regional differences for all grades and all levels. Some modest differences were noted for

ethnicity and panelist type (e.g., teacher, nonteacher educator, or general public). Table J-5 and J-6

display some of these results.

Table J-5

Mean Cut Scores and Standard Deviations in Writing, by Panelist Type*

Basic Proficient Advanced

Type n Mean

Standard

Deviation Mean

Standard

Deviation Mean

Standard

Deviation

Grade 4

Round 1 Teacher 16 138.6 3.7 163.8 4.6 185.8 5.8

Nonteacher Ed. 5 137.5 5.9 163.2 2.2 186.7 3.4

General Public 8 136.4 7.9 161.0 7.5 185.8 6.1

Round 2 Teacher 16 139.2 3.8 166.0 2.6 187.9 5.1

Nonteacher Ed. 5 139.5 3.7 165.1 1.8 187.6 4.8

General Public 8 137.8 5.4 162.1 5.6 185.8 3.4

Round 3 Teacher 16 139.7 3.2 165.8 1.8 186.2 4.9

Nonteacher Ed. 5 140.4 2.6 165.7 1.8 185.3 4.5

General Public 8 137.9 5.4 162.5 5.4 186.2 3.9

 Final Teacher 16 139.9 2.7 165.7 1.9 185.3 4.8

Nonteacher Ed. 5 140.4 2.3 165.6 1.7 183.4 2.7

General Public 8 138.1 5.1 162.9 5.1 184.9 3.0

Grade 8

Round 1 Teacher 19 138.3 6.7 163.4 6.7 186.4 4.0

Nonteacher Ed. 4 141.0 5.9 164.2 3.5 183.5 2.1

General Public 7 138.1 7.1 163.9 3.9 184.4 5.0

Round 2 Teacher 19 139.8 3.6 163.7 2.7 185.4 2.2

Nonteacher Ed. 4 140.2 4.2 164.6 2.2 184.5 0.6

General Public 7 139.6 4.1 164.4 2.4 185.3 3.3

Round 3 Teacher 19 139.7 3.2 163.6 2.2 185.1 2.0

Nonteacher Ed. 4 139.7 3.6 163.5 1.4 184.1 0.5

General Public 7 139.9 3.6 164.3 2.7 185.1 3.3

Final Teacher 19 139.5 2.7 163.5 2.2 185.1 2.1

Nonteacher Ed. 4 139.8 3.9 163.8 1.5 182.5 2.4

General Public 7 140.0 3.7 164.1 2.1 185.6 2.0

* Comparisons (mean differences) significant at the 0.05 level are bold-faced.

(continued)
880

Table J-5 (continued)

Mean Cut Scores and Standard Deviations in Writing, by Panelist Type*

Basic Proficient Advanced

Type n Mean

Standard

Deviation Mean

Standard

Deviation Mean

Standard

Deviation

Grade 12

Round 1 Teacher 15 143.5 6.4 165.8 6.6 191.2 7.9

Nonteacher Ed. 6 142.4 5.2 163.4 7.1 190.7 7.8

General Public 8 139.1 7.0 164.3 6.5 188.6 9.4

Round 2 Teacher 15 143.0 3.9 165.6 3.4 190.5 4.7

Nonteacher Ed. 6 142.9 2.0 164.8 3.0 190.1 3.9

General Public 8 141.8 3.8 166.3 2.9 189.1 4.8

Round 3 Teacher 15 143.0 3.6 165.8 2.8 188.8 4.6

Nonteacher Ed. 6 143.1 2.3 165.4 2.9 187.0 5.3

General Public 8 142.2 4.0 166.1 2.6 187.6 4.6

Final Teacher 15 143.2 3.3 165.5 2.8 186.7 4.2

Nonteacher Ed. 6 144.2 2.2 166.5 0.5 186.7 4.3

General Public 8 142.1 4.0 166.0 2.6 187.1 4.5

* Comparisons (mean differences) significant at the 0.05 level are bold-faced.
881

Table J-6

Mean Cut Scores and Standard Deviations in Writing, by Ethnicity

Basic Proficient Advanced

Ethnicity n Mean

Standard

Deviation Mean

Standard

Deviation Mean

Standard

Deviation

Grade 4

Round 1 White 22 138.1 5.7 163.6 4.8 185.7 5.5

Black 5 135.7 4.8 157.9 5.1 184.7 4.7

Hispanic 1 142.0 — 165.1 — 193.9 —

Other 1 137.3 — 169.7 — 190.1 —

Round 2 White 22 139.1 4.3 165.4 3.1 187.1 3.8

Black 5 137.0 3.0 161.0 5.4 185.1 2.8

Hispanic 1 145.9 — 168.9 — 202.5 .

Other 1 137.7 — 166.0 — 187.2 .

Round 3 White 22 139.6 3.8 165.4 2.6 185.7 4.2

Black 5 137.1 2.6 161.2 4.8 184.7 2.8

Hispanic 1 145.9 — 168.9 — 197.6 —

Other 1 138.2 — 166.2 — 187.3 —

 Final White 22 139.8 3.3 165.4 2.4 184.5 3.6

Black 5 137.0 2.5 161.6 4.7 184.0 1.4

Hispanic 1 146.0 — 169.0 — 198.0 —

Other 1 138.0 — 166.0 — 184.0 —

Grade 8

Round 1 White 24 139.5 5.7 164.4 5.1 186.2 4.4

Black 3 129.1 7.2 157.5 10.0 182.6 1.3

Asian/Pacific 3 141.1 5.2 162.8 2.4 183.2 0.3

Round 2 White 24 139.9 3.5 164.2 2.6 185.6 2.3

Black 3 136.4 0.4 161.7 1.9 184.2 1.8

Asian/Pacific 3 142.3 4.3 164.0 1.8 183.0 0.7

Round 3 White 24 139.8 3.1 163.9 2.3 185.3 2.3

Black 3 136.8 0.6 162.1 1.7 184.6 1.5

Asian/Pacific 3 142.1 4.3 163.8 1.0 182.8 0.5

Final White 24 140.0 3.1 163.9 2.2 185.2 2.3

Black 3 136.7 0.6 162.0 1.7 184.7 1.5

Asian/Pacific 3 140.3 3.1 164.0 1.0 182.7 0.6

Grade 12

Round 1 White 20 141.4 6.0 164.2 5.2 190.3 8.0

Black 5 144.3 8.4 165.3 6.5 187.0 6.3

Asian/Pacific 2 137.2 4.3 159.5 10.9 191.4 16.0

Other 2 148.8 0.4 176.4 7.8 198.6 4.5

Round 2 White 20 142.4 3.8 165.1 3.4 189.7 4.5

Black 5 142.9 3.4 165.7 1.8 188.1 3.7

Asian/Pacific 2 141.3 1.1 166.6 1.2 193.6 4.5

Other 2 146.1 0.3 169.0 3.5 194.5 4.9

Round 3 White 20 142.5 3.6 165.4 2.9 187.5 4.5

Black 5 143.3 3.1 166.1 1.3 188.5 2.1

Asian/Pacific 2 141.8 1.0 166.6 0.9 192.3 3.8

Other 2 146.1 0.3 168.9 3.4 188.5 12.2

Final White 20 142.9 3.6 165.5 2.7 186.7 4.1

Black 5 143.4 3.2 166.2 1.5 186.0 2.1

Asian/Pacific 2 141.5 0.7 166.5 0.7 190.0 2.8

Other 2 146.0 0.0 168.0 1.4 187.5 10.6

Note: Comparisons (mean differences) significant at the 0.05 level are bold-faced.

Civics. Similar findings were obtained in civics as in writing. Multiple comparison tests showed

significant differences among rating groups only for grade 4 basic (Round 1 only) and grade 12 advanced
882

(Round 2 only). There were no significant gender differences; however, there were some modest

differences by region, ethnicity, and panelist type as shown in Tables J-7 and J-8. A full description of

these analyses and the results can be found in ACT’s final reports (ACT, 1999b; 1999d).

Table J-7

Mean Cut Scores and Standard Deviations in Civics, by Ethnicity

Basic Proficient Advanced

Ethnicity n Mean

Standard

Deviation Mean

Standard

Deviation Mean

Standard

Deviation

Grade 4

Round 1 White 24 147.0 9.7 163.7 4.7 175.7 5.8

Black 3 155.2 2.7 169.0 3.5 179.2 5.9

Asian/Pacific 2 156.0 12.1 163.0 2.3 172.6 4.0

Native 1 132.7 — 153.3 — 164.3 —

Hispanic 1 135.4 — 162.9 — 178.1 —

Round 2 White 24 148.1 5.3 164.0 3.5 177.5 4.5

Black 3 156.5 2.8 168.1 1.4 177.0 5.8

Asian/Pacific 2 146.6 0.1 162.2 1.3 173.3 4.2

Native 1 145.5 — 159.3 — 174.4 —

Hispanic 1 138.9 — 161.8 — 174.4 —

Round 3 White 24 149.5 5.2 164.7 3.4 178.5 4.5

Black 3 156.4 2.8 167.9 1.2 177.1 5.6

Asian/Pacific 2 146.8 0.5 162.2 0.8 173.5 3.7

Native 1 145.3 — 159.4 — 173.9 —

Hispanic 1 140.8 — 162.5 — 175.1 —

 Final White 24 150.2 4.7 164.8 3.2 178.6 3.7

Black 3 156.3 2.9 168.0 1.0 177.0 5.6

Asian/Pacific 2 146.5 0.7 162.5 0.7 173.5 3.5

Native 1 145.0 — 159.0 — 174.0 —

Hispanic 1 145.0 — 163.0 — 175.0 —

Note: Comparisons (mean differences) significant at the 0.05 level are bold-faced.

(continued)
883

Table J-7 (continued)

Mean Cut Scores and Standard Deviations in Civics, by Ethnicity

Basic Proficient Advanced

Ethnicity n Mean

Standard

Deviation Mean

Standard

Deviation Mean

Standard

Deviation

Grade 8

Round 1 White 22 147.1 7.9 164.8 3.2 177.1 4.8

Black 4 142.8 17.4 165.0 4.6 177.7 2.1

Hispanic 2 156.7 8.6 168.7 5.9 181.5 4.2

Round 2 White 1 153.5 — 167.1 — 177.9 —

Black 22 148.6 5.6 165.0 2.6 177.0 4.0

Hispanic 4 147.2 5.4 164.7 4.0 177.6 1.3

Other 2 157.6 9.6 169.6 5.1 181.9 1.6

Round 3 White 1 148.2 — 164.1 — 174.8 —

Black 22 149.2 5.0 165.2 2.4 177.1 3.9

Hispanic 4 146.9 5.0 164.7 3.8 177.5 1.2

Other 2 158.0 9.8 170.1 4.6 182.9 2.5

Final White 1 146.7 — 163.3 — 174.5 —

Black 22 148.9 4.6 165.3 2.3 177.7 2.9

Hispanic 4 147.0 5.0 164.8 4.0 177.3 1.3

Other 2 158.0 9.9 170.0 4.2 183.0 2.8

Grade 12

Round 1 White 23 150.6 6.3 163.9 4.0 175.3 6.1

Black 3 153.2 12.7 164.0 4.2 172.7 4.0

Hispanic 1 141.2 — 158.0 — 172.3 —

Round 2 White 23 151.0 4.5 164.3 3.6 175.5 3.7

Black 3 145.6 5.7 162.7 1.6 172.3 1.6

Hispanic 1 139.3 — 158.1 — 174.3 —

Round 3 White 23 151.2 4.6 164.5 3.7 175.9 3.7

Black 3 149.5 5.2 163.2 1.6 172.5 1.5

Hispanic 1 138.6 — 158.4 — 175.1 —

Final White 23 151.3 4.1 164.3 3.2 175.6 3.4

Black 3 149.7 4.9 163.0 1.7 172.3 1.5

Hispanic 1 150.0 — 163.0 — 175.0 —

Note: Comparisons (mean differences) significant at the 0.05 level are bold-faced.
884

Table J-8

Mean Cut Scores and Standard Deviations in Civics, by Panelist Type

Basic Proficient Advanced

Ethnicity n Mean

Standard

Deviation Mean

Standard

Deviation Mean

Standard

Deviation

Grade 4

Round 1 Teacher 19 145.1 10.5 163.3 5.4 175.1 5.7

Nonteacher Ed. 4 153.3 4.9 165.4 1.6 176.1 7.2

General Public 8 150.4 9.6 164.0 5.0 176.4 6.7

Round 2 Teacher 19 147.6 6.4 163.4 3.7 175.5 3.9

Nonteacher Ed. 4 152.8 3.2 166.6 1.4 179.7 2.5

General Public 8 148.1 4.2 164.3 3.4 179.2 5.1

Round 3 Teacher 19 149.0 5.5 163.9 3.3 176.3 3.7

Nonteacher Ed. 4 154.9 5.9 168.0 3.2 181.3 3.6

General Public 8 148.3 3.8 164.5 3.2 179.7 5.3

 Final Teacher 19 149.6 5.0 164.1 3.0 176.8 3.1

Nonteacher Ed. 4 155.0 5.8 167.5 3.0 181.3 3.6

General Public 8 149.3 3.2 164.6 3.4 178.5 5.1

Grade 8

Round 1 Teacher 16 145.2 11.4 165.1 4.0 178.0 4.3

Nonteacher Ed. 4 156.3 2.6 167.3 2.0 178.7 6.6

General Public 9 147.2 5.2 164.5 3.0 176.2 3.7

Round 2 Teacher 16 148.2 7.2 165.5 3.6 178.3 3.6

Nonteacher Ed. 4 153.3 3.4 164.8 2.0 175.2 5.7

General Public 9 148.7 3.5 165.1 2.4 176.7 2.8

Round 3 Teacher 16 148.9 6.9 165.8 3.5 178.5 3.7

Nonteacher Ed. 4 153.0 3.3 164.6 2.0 174.8 5.5

General Public 9 148.9 3.1 165.1 2.1 176.8 2.7

Final Teacher 16 148.8 6.6 165.7 3.4 178.8 3.2

Nonteacher Ed. 4 151.5 3.0 165.3 1.7 177.3 1.3

General Public 9 149.0 3.2 165.1 2.1 176.7 2.8

Grade 12

Round 1 Teacher 17 150.8 8.2 163.6 4.3 175.0 6.9

Nonteacher Ed. 4 149.1 6.3 161.0 3.2 171.6 2.1

General Public 6 151.1 4.6 165.9 2.3 176.7 2.5

Round 2 Teacher 17 149.2 5.3 163.6 3.2 174.9 3.6

Nonteacher Ed. 4 147.5 5.3 160.5 3.1 172.2 1.5

General Public 6 153.7 3.4 166.9 2.8 177.6 3.3

Round 3 Teacher 17 150.2 5.1 164.0 3.1 175.5 3.6

Nonteacher Ed. 4 147.3 5.5 160.3 3.7 172.4 1.7

General Public 6 153.7 3.4 166.9 2.8 177.7 3.4

Final Teacher 17 151.0 4.0 164.1 2.1 175.0 3.3

Nonteacher Ed. 4 148.0 4.6 160.3 3.6 172.8 2.5

General Public 6 153.2 3.1 166.7 2.4 177.3 3.3

Note: Comparisons (mean differences) significant at the 0.05 level are bold-faced.
885

In past standard-setting activities, significant differences in cut scores were found between the

dichotomously and polytomously scored exercises. Table J-9 displays the means and standard deviations

estimated from panelists’ ratings for each item type on the 1998 civics. Pairwise comparisons of cut

scores show that some of these differences are still significant. However, real differences are very much

reduced over prior standard-setting efforts. There is no direct empirical evidence to suggest why this is

the case. However, it is hypothesized that panelists were more aware of the relationship between and

among various items (both dichotomous and polytomous) as they worked with the Reckase charts. This

feature impacted the panelists’ ratings in such a way as to reduce differences across item types.

Table J-9

Mean Differences Between Polytomous and Dichotomous Cut Scores for Civics

Grade

4 (n=31) 8 (n=29) 12 (n=27)

Achievement

Level Item Type Mean

Standard

Deviation Mean

Standard

Deviation Mean

Standard

Deviation

Basic Dichotomous 147.2 5.4 147.0 6.7 147.8 7.9

Polytomous 151.1 7.5 150.3 6.0 151.4 5.9

Both 149.5 5.4 149.4 5.6 150.5 5.1

Proficient Dichotomous 161.9 3.3 163.9 3.6 163.4 4.0

Polytomous 169.5 5.9 167.5 3.3 165.6 5.8

Both 164.6 3.4 165.4 2.9 164.1 3.6

Advanced Dichotomous 175.0 3.8 176.5 4.2 175.0 3.7

Polytomous 186.4 10.7 180.0 4.4 178.7 9.1

Both 177.8 4.5 177.5 3.8 175.5 3.6

Note: These grade-level cut points were aggregated outcomes from individual cut points. They were different from the group

cut points reported to the panelists on site. They were only used to show the combined average to compare the dichotomous

and polytomous cut points.

J.11 SELECTING EXEMPLAR ITEMS

On the final day of the achievement level-setting process, panelists reviewed all items from the

item pools in civics or writing that were marked for release. This process was implemented after the final

round of ratings so that the recommended cut scores could be used to judge whether or not the released

exercises or exemplary student responses met the statistical criterion. Since the process for selecting

exemplars is different for civics and writing (due to the nature of the assessment) the procedures will be

described separately.

Civics. In civics, exercises are organized in blocks consisting of several items, usually employing

each of the three item formats, (i.e., multiple choice, short constructed response, and extended

constructed response). Before the review process, potential released exercises were categorized using

statistical criteria recommended by the Technical Advisory Committee on Standard Setting (TACSS), the

group that advises ACT on technical decisions throughout the process. Items having an average rp=.50

for scores within the achievement-level ranges were included in the list of items submitted to panelists

for their consideration. Further, items were ranked according to their discrimination indices and all items

at or above the 40th percentile that met the statistical criterion were identified as potential exemplars.

Items were listed at the lowest level for which they met the criterion. Constructed-response items were

treated as unique items at each score point, and thus, could meet the criterion n-1 times, where n = the

number of score categories. Items could be recommended as exemplars at a higher level than the
886

statistical criterion placed them (based on content), but could not be placed at a lower level, since the

mean rp would fall below .50 at a lower level. In the actual process for identifying exercises, panelists

are instructed to veto any items that they feel do not meet the content criterion; that is, items that do not

reflect the achievement-level descriptions may be discarded from consideration as appropriate exemplars

for the assessment. The purpose for this veto process is to encourage the inclusion of as many items as

possible for reporting the assessment results.

Writing. In writing, the exemplars consist of not only the prompt and the scoring rubric, but

exemplary student responses as well. Therefore, all potential released prompts were considered by

panelists along with selected anchor papers that had been used in both training scorers and in the

standard-setting process, and met the criterion of representing student performance at the appropriate

level. Panelists were instructed to veto those responses that, in their judgment, did not meet the content

criterion, that is, consistency with the achievement-level descriptions.

J.12 1998 RESEARCH STUDIES ON THE ACHIEVEMENT LEVELS

In 1998, two studies that were conducted independently in 1994 to examine the various aspects

of the validity of the NAEP achievement levels were combined into a single study. The first, the

similarity classification study (SCS), was designed to compare the classifications of students according to

the achievement-level descriptions by students’ teachers with the classification of the same students

according to their performance on a specially designed version of NAEP that yielded individual scores.

The second, the booklet classification study (BCS), was designed to compare the performances of

students (as demonstrated in their NAEP booklets) with the knowledge and skills described in the NAEP

achievement level descriptions.

The purpose of these two studies conducted in tandem was to overcome some of the shortcoming

of the 1994 studies. In fact, findings from the 1994 SCS study were countered by the results from the

1994 BCS study. The BCS in history and geography indicated that the achievement levels may have been

set too low, while the SCS study indicated just the opposite, that is, the levels may have been set too

high. The design of the earlier study did not allow any rational hypothesis to be entertained since

different groups of panelists were involved, and the studies were conducted at different times in the

process. The intent of the current design was to overcome these shortcomings. The current design

included the same panelists to classify expected performance of their students and to classify student

booklets (some of which were also their students). Further, the special form of NAEP designed for the

study included enough items to provide a reliable individual score estimate. Further design features in the

selection of booklets eliminated the need to deal with “not reached” items as “not administered.”

The logic of the SCS study was to explore whether teachers who participated in the ALS studies,

and who had been well-trained in the use of the achievement level descriptions, could indeed apply those

descriptions to the task of classifying their own students, when the empirical performance of their

students was known from the students’ performance on the special form of NAEP. In addition, the BCS

study was designed to test whether those same teachers could examine booklets of student work (some of

which were written by their own students) and, using the achievement levels descriptions, classify the

student performances according to the levels.

The SCS component was conducted with only grade 8 students and their teachers who

participated in the either the 1998 civics ALS pilot or the ALS meeting. Thirteen teachers and 461

students participated in the study. Each student was administered a special form of the NAEP (four

blocks) requiring 100 minutes of testing time. The special form was developed to meet certain minimum

criteria, was administered by Westat under the same conditions as a standardized NAEP administration,

and scored by NCS using scoring procedures identical to those used to score the 1998 NAEP
887

administration. The purpose of the double-length NAEP was to be able to estimate directly the NAEP

scores for students without having to use the conditioning model and plausible values technology.

 Students’ performances were classified by their teachers based on their knowledge of the

students relative to the assessment framework and the achievement-level descriptions. Students’ actual

performances were subsequently classified according to their scores on the extended NAEP assessment.

The results of these two classifications were compared.

The second component of the study, BCS, was conducted with the same panelists. The panelists

examined 40 student double-length booklets that had been used in the SCS component of the study and

were chosen according to a set of criteria appropriate to meet the goals of the study. Booklets from the

individual panelists’ students were embedded in the set of 40 booklets. Prior to the panel meeting, all

booklets in the study had been classified according to the achievement-levels cut scores as either basic,

proficient, or advanced. Panelists were asked to classify the booklets in the same way, but without having

the knowledge of the empirical classification provided through the scoring for each booklet.

The details of the design of this study and the results may also be found in the ACT research

report that will be published at the completion of the project in 2000. As in any study of this nature, the

results are subject to many caveats, and ACT points out several of these in their report. These studies

certainly are not intended to be definitive of the validity of the achievement levels. They are an

indication, however, that additional data analyses need to be completed in order to probe more fully the

technical characteristics of the levels as adopted by the NAGB.
888

Figure J-3

Achievement-Level Descriptions for Civics

Grade 4

Basic

XXX

Fourth-grade students performing at the basic level should have an understanding of what

government is and what it does, and they should be able to identify some things that government is

not allowed to do. These students should have some understanding of the foundations of the

American political system. In the context of their school and community, they should understand

rules and laws, rights and responsibilities, and ways to participate in governing. These students

should know that the world is divided into many countries.

Fourth-grade students performing at the basic level should have some understanding of what

government is and what it does, and they should be able to identify some things that government is

not allowed to do. They should be able to explain purposes of rules in the school and the

community, and to describe what happens when people break laws. These students should

understand how national holidays and symbols such as the flag, the Statue of Liberty, and the

Fourth of July reflect shared American values, and they should be able to identify different types of

diversity in American society. They should be able to describe ways to settle disagreements or

conflicts peacefully. They should be able to name the president and their state governor and to

identify the rights and responsibilities of a citizen. They should know some ways that students can

participate in governing their school and community, and they should be able to describe qualities

of a good leader. Finally, these students should know that the world is divided into many countries.

Grade 4

Proficient

XXX

Fourth-grade students performing at the proficient level should have a good understanding of

what the American government does and of why it is not allowed to act in certain ways. These

students should have an age-appropriate understanding of the foundations of the American

political system. They should understand purposes of laws, ways shared beliefs unify Americans,

what it means to be a citizen, and rights and responsibilities of citizens, and the idea of public

participation in governing. These students should be able to describe ways in which countries

interact with one another.

Fourth-grade students performing at the proficient level should have a good understanding of what

the American government does and of why it is not allowed to act in certain ways. They should be

able to explain why we have laws. These students should be able to recognize diversity in

American society and that Americans are united by shared beliefs and principles. They should

know that the Constitution and the Declaration of Independence are founding documents of

American democracy. They should be able to explain how people make decisions about the ways

they live together in a democracy and how groups in schools and communities can manage conflict

peacefully. They should know what it means to be a citizen of their state and the nation, and they

should be able to distinguish between rights and responsibilities of citizens. They should

understand why it is important for people to participate in governing their school and community.

Finally, these students should be able to describe ways in which countries interact with one another.

(continued)
889

Figure J-3 (continued)

Achievement-Level Descriptions for Civics

Grade 4

Advanced

XXX

Fourth-grade students performing at the advanced level should understand and be able to explain

some purposes of government. When given age-appropriate examples, they should recognize

differences between power and authority and between limited and unlimited government. They

should be able to explain the importance of shared values in American democracy, to identify

ways citizens can participate in governing, and to understand that with rights comes

responsibilities. They should be able to explain how nations benefit when they resolve conflicts

peacefully.

Fourth-grade students performing at the advanced level should understand and be able to explain

some purposes of government. They should recognize differences between power and authority

when given examples and should understand differences between limited and unlimited

government. These students should be able to explain why it is important that citizens share a

commitment to the values of American democracy, and they should be aware of the benefits and

challenges of both unity and diversity in American society. They should be able to distinguish

between services provided by local and state levels of government. These students should be able

to describe how government can make it possible for people to accomplish goals they could not

achieve alone. They should be able to identify ways in which citizens can keep track of their

government’s actions, and they should understand the connection between rights and

responsibilities of a citizen. Finally, they should be able to explain how nations benefit when they

resolve conflicts peacefully.

Grade 8

Basic

XXX

Eighth-grade students performing at the basic level should have some understanding of competing

ideas about purposes of government, and they should be able to describe advantages of limited

government. They should be able to define government, constitution, the rule of law, and politics.

They should be able to identify the fundamental principles of American democracy and the

documents from which they originate, and they should understand the importance of a shared

commitment to the core values of American democracy. They should recognize the components of

the political process and understand personal, political, and economic rights and responsibilities.

They should be able to describe the purposes of some international organizations.

Eighth-grade students performing at the basic level should have some understanding of competing

ideas about purposes of government, and they should be able to describe advantages of limited

government. They should be able to define what is meant by government, constitution, the rule of

law, and politics. These students should be able to identify fundamental principles and values of

American democracy, such as federalism, the separation of powers, checks and balances,

government by the consent of the governed, and individual rights. They should understand that the

Declaration of Independence and the United States Constitution including the Bill of Rights and

other Amendments are sources of these ideas. These students should be able to explain why it is

important that citizens share the values and principles expressed in the nation’s core documents,

and they should understand functions of elections, political parties, and interest groups in a

democratic society. They should know that American citizenship is attained by birth or through

naturalization. They should be able to identify personal, political, and economic rights of

Americans and should understand the responsibilities that these rights imply. Finally, these students

should be able to describe purposes of international organizations to which the United States

belongs.

(continued)
890

Figure J-3 (continued)

Achievement-Level Descriptions for Civics

Grade 8

Proficient

XXX

Eighth-grade students performing at the proficient level should understand and be able to

explain purposes that government should serve. These students should have a good

understanding of differences between government and civil society and of the importance of the

rule of law. They should recognize discrepancies between American ideals and reality and be

able to describe continuing efforts to address them. They should understand the separation and

sharing of powers among branches of government and between federal and state governments,

and they should be able to explain how citizens influence government. They should be able to

describe events within the United States and other countries that have international

consequences.

Eighth-grade students performing at the proficient level should have a good understanding of the

purposes that government should serve, and they should be able to explain why government

should serve those purposes. These students should understand differences between government

and civil society, and they should be able to explain the importance of the rule of law. They

should be able to point out ways in which ideals expressed in the nation’s core documents differ

from reality and to identify ways in which these differences continue to be addressed. They

should be able to explain how and why legislative, executive, and judicial powers are separate,

shared, and limited in the American constitutional government, and they should understand how

and why powers are divided and shared between the national and state governments. They

should be able to discuss ways that citizens can use the political process to influence

government. These students should be able to provide simple interpretations of non-text-based

information, like maps, charts, tables, graphs, and cartoons. Finally, these students should be

able to describe events in the United States that have influenced other nations, as well as events

in other nations that have affected American policy.

Grade 8

Advanced

XXX

Eighth-grade students performing at the advanced level should have a developed understanding

of how civil society helps to maintain limited government and why the rule of law is important.

These students should have a clear understanding of issues in which democratic values are in

conflict and of past efforts to address the discrepancies between American ideals and reality.

They should understand how citizens can monitor and influence government and how

responsible citizens support democracy. They should recognize the impact of American

democracy on other countries, as well as other countries’ impact on American politics and

society.

Eighth-grade students performing at the advanced level should have a developed understanding

of why civil society plays a key role in maintaining a limited government and of the importance

of the rule of law in civil society and government. These students should be able to take

positions on issues in which fundamental values are in conflict, liberty and equality, individual

rights and the common good, and majority rule and minority rights, for example, and they should

be able to defend their positions. They should be able to evaluate results of past efforts to

address discrepancies between American ideals and national reality and to explain how citizens

can monitor and influence local, state, and national government. These students should

understand how laws can achieve purposes of American constitutional government, such as

promoting the common good and protecting rights of individuals. They should understand how

civic dispositions such as civility, tolerance, and respect for law promote the healthy functioning

of American constitutional democracy. Finally, these students should understand the impact of

American democracy on other countries, as well as the impact of other countries on American

politics and society.

(continued)
891

Figure J-3 (continued)

Achievement Level-Descriptions for Civics

Grade 12

Basic

XXX

Twelfth-grade students performing at the basic level should have an understanding of what is meant

by civil society, constitutional government, and politics. They should know that constitutional

governments can take different forms, and they should understand the fundamental principles of

American constitutional government and politics, including functions of political parties and other

organizations. They should understand both rights and responsibilities in a democratic society, and

they should recognize the value of political participation. They should be familiar with

international issues that affect the United States.

Twelfth-grade students performing at the basic level should have an understanding of what is meant

by civil society, constitutional government, and politics. They should know that constitutional

governments can take different forms, and they should understand the fundamental principles of

American constitutional government. These students should be able to explain ways that political

parties, interest groups, and the media contribute to elections, and they should be able to point out

sources of information about public policy issues. They should understand that both power and

rights must be limited in a free society. They should be able to identify those traits that make people

responsible citizens, and they should be able to describe forms of political participation available in

a democracy and recognize reasons that such participation is important. These students should be

able to provide simple interpretations of non-text-based information, like maps, charts, tables,

graphs, and cartoons. Finally, they should be familiar with international issues that affect the United

States.

Grade 12

Proficient

XXX

Twelfth-grade students performing at the proficient level should have a good understanding of how

constitutions can limit the power of government and support the rule of law. They should be able to

describe similarities and differences among constitutional systems of government, and they should

be able to explain fundamental American democratic values, their applications, and their

contribution to expanding political participation. They should understand the structure of American

government and be able to evaluate activities of political parties, interest groups, and media in

public affairs. They should be able to explain the importance of political participation, public

service, and political leadership. They should be able to describe major elements of American

foreign policy and the performance of major international organizations.

Twelfth-grade students performing at the proficient level should have a good understanding of how

constitutions can limit the power of government and support the rule of law. They should be able to

distinguish between parliamentary systems of government and those based on separate and shared

powers, and they should be able to describe the structure and functions of American government.

These students should be able to identify issues in which fundamental democratic values and

principles are in conflict, liberty and equality, individual rights and the common good, and majority

rule and minority rights, for example, and they should be able to take and defend positions on these

issues. They should be able to evaluate ways that law protects individual rights and promotes the

common good in American society. They should understand how the application of fundamental

principles of American constitutional democracy has expanded participation in public life, and they

should be able to explain how citizens can work individually and collectively to monitor and

influence public policy. These students should understand the importance and means of participation

in political life at the national, state, and local levels. They should be able to evaluate contributions

made by political parties, interest groups, and the media to the development of public policy, and

they should be able to explain how public service and political leadership contribute to American

democracy. They should understand how American foreign policy is made and carried out, and they

should be able to evaluate the performance of major international organizations. Finally, these

students should be able to discuss reasons for and consequences of conflicts that arise when

international disputes cannot be resolved peacefully.

(continued)
892

Figure J-3 (continued)

Achievement-Level Descriptions for Civics

Grade 12

Advanced

XXX

Twelfth-grade students performing at the advanced level should have a thorough and mature

understanding of the strengths and weaknesses of various forms of constitutional democracy. They

should be able to explain fully the structure of American government and the political process. They

should understand differences between American ideals and realities, and they should be able to

explain past and present responses to those differences. They should understand why civic

dispositions and individual and collective political actions sustain democracy. They should be able

to explain objectives and consequences of American foreign policy.

Twelfth-grade students performing at the advanced level should have a thorough and mature

understanding of the strengths and weaknesses of various forms of constitutional democracy. They

should be able to discuss advantages and disadvantages of confederal, federal, and unitary systems

of government, as well as strengths and weaknesses of parliamentary systems of government when

compared with those based on separate and shared powers. These students should be able to explain

how the structure of American government and the nation’s social and political cultures serve one

another. They should know which level and agency of government to contact to express their

opinions or influence public policy. They should be able to explain and evaluate past and present

individual and collective political actions aimed at narrowing the gap between American ideals and

national reality. They should understand how elections help determine public policies, and they

should be able to evaluate public policy issues in which fundamental values and principles are in

conflict, liberty and equality, individual rights and the common good, and majority rule and minority

rights, for example. These students should be able to evaluate the validity and emotional appeal of

past and present political communication. They should be able to explain how civic dispositions

such as civility, tolerance, and respect for law are important for preserving democracy, and they

should be able to evaluate the many forms of participation in public affairs. Finally, they should be

able to explain how American foreign policy is made and carried out and to evaluate its

consequences.
893

Figure J-4

Achievement-Level Descriptions for Writing

The following statements describe the kinds of things fourth-grade students should be able to do in writing at each

level of achievement. These statements should be interpreted with the constraints of the National Assessment of

Educational Progress in mind. Student performances reported with respect to these descriptions are in response to

two age-appropriate writing tasks completed within 25 minutes each. Students are not advised of the writing tasks in

advance nor engaged in prewriting instruction and preparation; however, they are given a set of “ideas for planning

and evaluating” their writing for the assessment. Although the writing NAEP cannot fully assess students’ abilities to

produce a polished piece of writing, the results do provide valuable information about students’ abilities to generate

writing in response to a variety of purposes, tasks, and audiences within a rather limited period of time.

Grade 4

Basic

YYY

Fourth-grade students performing at the basic level should be able to produce a somewhat

organized and detailed response within the time allowed that shows a general grasp of the

writing task they have been assigned.

Fourth-grade students performing at the basic level should be able to produce a somewhat

organized response within the time allowed that shows a general grasp of the writing task they

have been assigned. Their writing should include some supporting details. Its grammar, spelling,

and capitalization should be accurate enough to communicate to a reader, although there may be

mistakes that get in the way of meaning.

Grade 4

Proficient

YYY

Fourth-grade students performing at the proficient level should be able to produce an organized

response within the time allowed that shows an understanding of the writing task they have been

assigned. Their writing should include details that support and develop their main idea, and it

should show that these students are aware of the audience they are expected to address.

Fourth-grade students performing at the proficient level should be able to produce an organized

response within the time allowed that shows an understanding of the writing task they have been

assigned. Their writing should include details that support and develop the main idea of the

piece, and its form, content, and language should show that these students are aware of the

audience they are expected to address. The grammar, spelling, and capitalization in the work

should be accurate enough to communicate to a reader; there may be some mistakes, but these

should not get in the way of meaning.

Grade 4

Advanced

YYY

Fourth-grade students performing at the advanced level should be able to produce an effective,

well-developed response within the time allowed that shows a clear understanding of the writing

task they have been assigned and the audience they are expected to address. Their writing

should include details and be clearly organized, should use precise and varied language, and

may show signs of analytical, evaluative, or creative thinking.

Fourth-grade students performing at the advanced level should be able to produce an effective,

well-developed response within the time allowed that shows a clear understanding of the writing

task they have been assigned. Their writing should be clearly organized, making use of

techniques such as consistency in topic or theme, sequencing, and a clearly marked beginning

and ending. It should make use of precise and varied language to speak to the audience the

students are expected to address, and it should include details and elaboration that support and

develop the main idea of the piece. Their writing may also show signs of analytical, evaluative,

or creative thinking. The grammar, spelling, and capitalization in the work should be accurate

enough to communicate clearly; mistakes should be so few and so minor that a reader can easily

skim over them.

(continued)
894

Figure J-4 (continued)

Achievement-Level Descriptions for Writing

The following statements describe the kinds of things eighth-grade students should be able to do in writing at each

level of achievement. These statements should be interpreted with the constraints of the National Assessment of

Educational Progress in mind. Student performances reported with respect to these descriptions are in response to two

age-appropriate writing tasks completed within 25 minutes each. Students are not advised of the writing tasks in

advance nor engaged in prewriting instruction and preparation; however, they are given a set of “ideas for planning

and evaluating” their writing for the assessment. Although the writing NAEP cannot fully assess students’ abilities to

produce a polished piece of writing, the results do provide valuable information about students’ abilities to generate

writing in response to a variety of purposes, tasks, and audiences within a rather limited period of time.

Grade 8

Basic

YYY

Eighth-grade students performing at the basic level should be able to produce an effective response within

the time allowed that shows a general understanding of the writing task they have been assigned. Their

writing should show that these students are aware of the audience they are expected to address, and it

should include supporting details in an organized way.

Eighth-grade students performing at the basic level should be able to produce an effective response within the

time allowed that shows a general understanding of the writing task they have been assigned. Their writing

should show that these students are aware of the audience they are expected to address, and it should include

supporting details in an organized way. The grammar, spelling, punctuation, and capitalization in the work

should be accurate enough to communicate to a reader, although there may be mistakes that get in the way of

meaning.

Grade 8

Proficient

YYY

Eighth-grade students performing at the proficient level should be able to produce a detailed and organized

response within the time allowed that shows an understanding of both the writing task they have been

assigned and the audience they are expected to address. Their writing should include precise language and

varied sentence structure, and it may show analytical, evaluative, or creative thinking.

Eighth-grade students performing at the proficient level should be able to produce an effective response

within the time allowed that shows an understanding of both the writing task they have been assigned and

the audience they are expected to address. Their writing should be organized, making use of techniques

such as sequencing or a clearly marked beginning and ending, and it should make use of details and some

elaboration to support and develop the main idea of the piece. Their writing should include precise

language and some variety in sentence structure, and it may show analytical, evaluative, or creative

thinking. The grammar, spelling, punctuation, and capitalization in the work should be accurate enough to

communicate to a reader; there may be some errors, but these should not get in the way of meaning.

Grade 8

Advanced

YYY

Eighth-grade students performing at the advanced level should be able to produce a fully developed

response within the time allowed that shows a clear understanding of both the writing task they have been

assigned and the audience they are expected to address. Their writing should show some analytical,

evaluative, or creative thinking and may make use of literary strategies to clarify a point. At the same time,

the writing should be clearly organized, demonstrating precise word choice and varied sentence structure.

Eighth-grade students performing at the advanced level should be able to produce an effective and fully

developed response within the time allowed that shows a clear understanding of both the writing task they

have been assigned and the audience they are expected to address. Their writing should show some

analytical, evaluative, or creative thinking, and should demonstrate precise word choice and varied sentence

structure. Their work should include details and elaboration that support and develop the main idea of the

piece, and it may make use of strategies such as analogies, illustrations, examples, anecdotes, or figurative

language to clarify a point. At the same time, the writing should show that these students can keep their work

clearly and consistently organized. Writing by eighth-grade students performing at the advanced level should

contain few errors in grammar, spelling, punctuation, capitalization, and sentence structure. These writers

should demonstrate good control of these elements and may use them for stylistic effect in their work.

(continued)
895

Figure J-4 (continued)

Achievement-Level Descriptions for Writing

The following statements describe the kinds of things twelfth-grade students should be able to do in writing at each

level of achievement. These statements should be interpreted with the constraints of the National Assessment of

Educational Progress in mind. Student performances reported with respect to these descriptions are in response to

two age-appropriate writing tasks completed within 25 minutes each. Students are not advised of the writing tasks in

advance nor engaged in prewriting instruction and preparation; however, they are given a set of “ideas for planning

and evaluating” their writing for the assessment. Although the writing NAEP cannot fully assess students’ abilities to

produce a polished piece of writing, the results do provide valuable information about students’ abilities to generate

writing in response to a variety of purposes, tasks, and audiences within a rather limited period of time.

Grade 12

Basic

YYY

Twelfth-grade students performing at the basic level should be able to produce a

Well-organized response within the time allowed that shows an understanding of both the

writing task they have been assigned and the audience they are expected to address. Their

writing should show some analytical, evaluative, or creative thinking, and it should include

details that support and develop the main idea of the piece.

Twelfth-grade students performing at the basic level should be able to produce an effective

response within the time allowed that shows an understanding of both the writing task they

have been assigned and the audience they are expected to address. Their writing should show

some analytical, evaluative, or creative thinking. It should include details that support and

develop the central idea of the piece, and it should be clearly organized, making use of

techniques such as a consistency in topic or theme, sequencing, and a clear introduction and

conclusion. The grammar, spelling, punctuation, and capitalization in these students’ work

should be accurate enough to communicate to a reader; there may be some errors, but these

should not get in the way of meaning.

Grade 12

Proficient

YYY

Twelfth-grade students performing at the proficient level should be able to produce an

effectively organized and fully developed response within the time allowed that uses

analytical, evaluative, or creative thinking. Their writing should include details that support

and develop the main idea of the piece, and it should show that these students are able to use

precise language and variety in sentence structure to engage the audience they are expected

to address.

Twelfth-grade students performing at the proficient level should be able to produce an

effective and fully developed response within the time allowed that uses analytical, evaluative,

or creative thinking. Their writing should be organized effectively, and it should show that

these students have a clear understanding of the writing task they have been assigned. It should

be coherent, making use of techniques such as a consistent theme, sequencing, and a clear

introduction and conclusion, and it should include details and elaboration that support and

develop the main idea of the piece. The writing should show that these students are able to use

precise language and variety in sentence structure to engage the audience they are expected to

address. Writing by twelfth-grade students performing at the proficient level should contain

few errors in grammar, spelling, punctuation, capitalization, and sentence structure. These

writers should demonstrate a command of these elements and may use them for stylistic effect

in their work.

(continued)
896

Figure J-4 (continued)

Achievement-Level Descriptions for Writing

Grade 12

Advanced

YYY

Twelfth-grade students performing at the advanced level should be able to produce a mature

and sophisticated response within the time allowed that uses analytical, evaluative, or

creative thinking. Their writing should be detailed and fully developed, and it should show

that these students are able to use literary strategies to develop their ideas. At the same time,

the writing should be well crafted and coherent, and it should show that these students are

able to engage the audience they are expected to address through rich and compelling

language, precise word choice, and variety in sentence structure.

Twelfth-grade students performing at the advanced level should be able to produce a mature

and sophisticated response within the time allowed that uses analytical, evaluative, or creative

thinking. Their writing should be fully developed, incorporating details and elaboration that

support and extend the main idea of the piece. It should show that these students can use

literary strategies anecdotes and repetition, for example, to develop their ideas. At the same

time, the writing should be well crafted, organized, and coherent, and it should incorporate

techniques such as a consistency in topic or theme, sequencing, and a clear introduction and

conclusion. It should show that these writers can engage the audience they are expected to

address through rich and compelling language, precise word choice, and variety in sentence

structure. Writing by twelfth-grade students performing at the advanced level should contain

few errors in grammar, spelling, punctuation, capitalization, and sentence structure. These

writers should demonstrate a sophisticated command of these elements and may use them for

stylistic effect in their work.
897

Figure J-5

Meeting Participants, NAEP Civics Achievement Level Setting Pilot Study

St. Louis, Missouri, August 13–17, 1998

Peggy Allan

Grade 8 Teacher

Greenville Junior High School

Greenville, IL 62246

Vicki Allen

Grade 4 Teacher

Minooka Community Consolidated

SD #201

Minooka, IL 60447

Penny Andrew

Grade 12 Teacher

Centennial High School

Boise, ID 83713

Joe Andrews

Grade 12 General Public

Office of the Mayor

San Francisco, CA 94102

Toni Apoldo

Grade 4 Teacher

Central School

East Brunswick, NJ 08816

Susan Barrow

Grade 12 General Public

Former School Board Chair

Billings, MT 59102

Regina Bell

Grade 8 General Public

Educational Coordinator

Cox Communications

West Warwick, RI 02893

Michael Carroll

Grade 8 General Public

School Board Chair

Akeley, MN 56433

Mary Carter

Grade 8 Teacher

Alma Middle School

Alma, AR 72921

Susan Churchill

Grade 4 General Public

Anchorage Community YMCA

Anchorage, AK 99507

Colleen Clark-Sutton

Grade 12 Teacher

Garfield Heights High School

Garfield Heights, OH 44125

Susannah Jones Cleveland

Grade 12 Teacher

Prattville High School

Prattville, AL 36067

Frank Collins

Grade 12 General Public

Retired Vice President

Organization and Management Development

Hartford, CT 06108

Jose Colon

Grade 8 Teacher

Millikan Middle School

Sherman Oaks, CA 91423

Amy Dillon

Grade 12 Teacher

Wichita Northwest High School

Wichita, KS 67212

Ronda Ely

Grade 8 Teacher

North Davie Middle School

Mocksville, NC 27028

Gene Ford

Grade 4 Teacher

Sutterville Elementary

Sacramento, CA 95822

(continued)
898

Figure J-5 (continued)

Meeting Participants, NAEP Civics Achievement Level Setting Pilot Study

St. Louis, Missouri, August 13–17, 1998

Joshua Fradel

Grade 8 Teacher

Centreville Middle School

Centreville, MD 21617

John Hall

Grade 8 General Public

Town Manager

Town of St Johnsbury

St Johnsbury, VT 05819

Christine Hamilton

Grade 4 Teacher

Ottawa River Elementary

Toledo, OH 43611

Kathy Hand

Grade 8 Nonteacher

State Director

Center for Civic Education

Seattle, WA 98148

Craig Haugaard

Grade 12 General Public

Minnesota Extension Office

Benson, MN 56215

Beverly Hoffmaster

Grade 4 Teacher

Berkeley County Schools

Martinsburg, WV 25401

Kerry Horn

Grade 8 General Public

Minister, First Baptist Church

Covington, TX 76636

Denny Hurtado

Grade 8 Nonteacher

Office of Superintendent of Public Instruction

Olympia, WA 98504-7200

Myrna Jensen

Grade 8 Teacher

Service Center

Wenatchee, WA 98801

Barbara Jo Johnson

Grade 4 Teacher

Rolling Green School

Rockford, IL 61108

Stanley Keene

Grade 8 Teacher

Hopwood Junior High School

Saipan, MP 96950

Harriet Kopp

Grade 12 Teacher

Musselman High School

Gerrardstown, WV 25420

Kim Kozbial-Hess

Grade 4 Teacher

Toledo Public Schools

Toledo, OH 43608

Leslie Lee

Grade 8 Teacher

Hammocks Middle School

Miami, FL 33196

Hsien-Tung Liu

Grade 12 Nonteacher

Bloomsburg University

Bloomsburg, PA 17818

Garnet Lynch

Grade 4 Teacher

Whittier Elementary

Frederick, MD 21702

Ed Markarian

Grade 12 Teacher

Franklin High School

Los Angeles, CA 90042

Russell Maruna

Grade 8 Nonteacher

Cleveland City School District

Cleveland, OH 44114

(continued)
899

Figure J-5 (continued)

Meeting Participants, NAEP Civics Achievement Level Setting Pilot Study

St. Louis, Missouri, August 13–17, 1998

Phyllis Mattingly

Grade 8 General Public

McCauley & Mattingly Attorneys

Versailles, KY 40383

Dean Moore

Grade 12 Nonteacher

Hamilton County Educational Service Center

Cincinnati, OH 45231

Gwen Parks

Grade 12 Teacher

Newtown High School

Sandy Hook, CT 06482

Debbie Pascal

Grade 12 General Public

Jefferson County Chamber of Commerce

Mount Vernon, IL 62864

Mary Peschel

Grade 4 General Public

Administrator/Clerk/Treasurer

City of Schuyler

Schuyler, NE 68661

Robert Pickus

Grade 12 General Public

President, World Without War Council, Inc.

Berkeley, CA 94709

Jennifer Powell

Grade 8 Teacher

Brunswick Middle School

Brunswick, MD 21716

Linda Richards

Grade 4 General Public

School Board President

Bradley, CA 93426

Robby Sauer

Grade 12 Teacher

Bliss School

Bliss, ID 83314

Eileen Sudock

Grade 4 Teacher

Washington Elementary School

Edison, NJ 08817

Gary Swalley

Grade 8 Teacher

Edwardsville Middle School

Edwardsville, IL 62025

Mary Talanay

Grade 4 Teacher

Chatsworth Park Elementary School

Chatsworth, CA 91311

Barbara Tranquilla

Grade 12 Teacher

Miami Killian Senior

Miami, FL 33176

Lynne Unice

Grade 4 Teacher

Hammarskjold School

East Brunswick, NJ 08816

Debbie Waitekus

Grade 8 General Public

Past Member, Board of Education

Matteson, IL 60443

Elwood Williams

Grade 4 General Public

President, Broad Creek Civic League

Norfolk, VA 23523

Jennifer Yoo

Grade 4 Teacher

Welby Way Elementary

Canoga Park, CA 91307
900

Figure J-6

Meeting Participants, NAEP Writing Achievement Level Setting Pilot Study

St. Louis, Missouri, October 1–5, 1998

Darrell Barrett

Grade 12 Teacher

Greenway High School

Phoenix, AZ 85023

Paul Batesel

Grade 4 Teacher

Maryville State University

Mayville, ND 58257–1299

Pat Belanoff

Grade 4 Nonteacher

SUNY at Stony Brook

Stony Brook, NY 11794

Jack Bell

Grade 12 General Public

Chair, City Zoning/Finance Committee

Rupert, ID 83350

Jane Bock

Grade 8 Nonteacher

University of Wisconsin – Green Bay

Green Bay, WI 54311

Michael Boyles

Grade 8 Teacher

Lincoln Junior High

Burns, OR 97720

Dixie Bruhner

Grade 8 General Public

Southern Utah News

Kanab, UT 84741

Phyllis Campagna

Grade 12 General Public

XSEL Performance Strategies

West Dundee, IL 60118

Jim Campbell

Grade 4 General Public

The Sun Newspaper

Bremerton, WA 98337

Judy Chandler

Grade 12 Teacher

Bullitt East High School

Mount Washington, KY 40047

Lucia Greene Connolly

Grade 4 General Public

Freelance Writer/Editor

Woodbridge, CT 06524

DonnaLynn Cooper

Grade 4 Teacher

Keene Mill Elementary School

Springfield, VA 22152

Claude Bain Culver

Grade 4 General Public

President, Board of Education

Sylacauga, AL 35151

James Davis

Grade 4 Nonteacher

Lansing Public Schools

Lansing, MI 48906

Gary Dunlap

Grade 12 General Public

Process Engineer

American Down and Textile

Onalaska, WI 54650

Susan Edmunds

Grade 4 General Public

Writes Children’s Stories

Kanab, UT 84741

Larry Elsaner

Grade 12 General Public

Columnist-Local Newspaper

Occidental, CA 95465

Rose Furr

Grade 4 Nonteacher

Garrett Park Elementary School

Garrett Park, MD 20896

Patricia Goetz

Grade 4 Teacher

Ludlow Elementary School

Ludlow, KY 41016
901

Figure J-6 (continued)

Meeting Participants, NAEP Writing Achievement Level Setting Pilot Study

St. Louis, Missouri, October 1–5, 1998

Lucia Graham

Grade 4 General Public

Chair, Academic Affairs Committee

McKendree College

O’Fallon, IL 62269

Bonna Greene

Grade 8 General Public

Member, Board of Education

Woodbridge, CT 06525

Richard Harris

Grade 12 Teacher

Danbury High School

Danbury, CT 06811–4599

Roseanne Hiatt Harris

Grade 8 Nonteacher

New Jersey Department of Education

Trenton, NJ 08625–0500

Mark Holding

Grade 12 Teacher

Merrimack High School

Merrimack, NH 03054

Jeffrey Honiz

Grade 4 Teacher

St Philomena School

Portsmouth, RI 02871

Nioka Houston

Grade 4 Teacher

Wolott Elementary

Wolcott, VT 05680

Michael Johnson

Grade 12 Nonteacher

Alameda Unified School District

Alameda, CA 94501

Bonnie Josewski

Grade 4 Teacher

Sargent Central

Forman, ND 58032

Michael Kelly

Grade 8 Teacher

Brittany Hill Middle School

Blue Springs, MO 64015

Antonia Lee

Grade 8 Teacher

Shawnee Middle School

Shawnee, OK 74801

Beverly Lee

Grade 4 General Public

Lt. Colonel/Public Affairs Officer

U.S. Air Force Reserves

Yokota Air Base, Japan

Tommy Martin

Grade 8 General Public

Executive Vice President

People's Bank

Clifton, TN 38425

Bill Mason

Grade 12 Teacher

Wichita Southeast High

Wichita, KS 67218–3397

James Matschulat

Grade 8 General Public

Vice President, The Country School Board

Madison, CT

Barbara Mays

Grade 8 Nonteacher

Detroit Public Schools

Detroit, MI 48202

Maxine McElroy

Grade 8 Teacher

Almeria Middle School

Fontana, CA 92335

Salvatore Menzo

Grade 8 Teacher

Windham Middle School

Willimantic, CT 06226

M. Lynne Murchison

Grade 12 Teacher

Oxford High School

Oxford, MS 38655

(continued)
902

Figure J-6 (continued)

Meeting Participants, NAEP Writing Achievement Level Setting Pilot Study

St. Louis, Missouri, October 1–5, 1998

Mable Hoskins Oatis

Grade 12 Teacher

Meridian High School

Meridian, MS 39305

Rosetta Oyeneyin

Grade 4 Teacher

Langston Hughes Elementary School

Chicago, IL 60628

Margie Elaine Rybarsyk

Grade 8 Teacher

Bellaire Public School

Bellaire, MI 49615

Jonell Safford

Grade 8 Teacher

Rockwood South Middle School

Fenton, MO 63026

John Scheeler

Grade 12 Teacher

Minot High School – Magic City Campus

Minot, ND 58701

Cindi Schmitz

Grade 4 Teacher

Robert Frost Elementary

Silverton, OR 97381

Janetta Scott

Grade 12 Teacher

Del City High School

Del City, OK 73115

Craig Shwery

Grade 4 Nonteacher

Fort Hays State University

Hays, KS 67601

Vaughn Stratford

Grade 8 General Public

San Rafael Public Library

San Rafael, CA 94901

Daniel Sullivan

Grade 12 General Public

Business and Industry Liaison

School-to-Work Initiative

Attleboro, MA 02703

Judy Szymialis

Grade 8 Teacher

Bridge Street Middle School

Elm Grove, WV 26003

Debra Taylor

Grade 8 Teacher

Binford Model Middle School

Richmond, VA 23220

Richard Thompson

Grade 4 Teacher

Canyon Elementary

Hungry Horse, MT 59919

Bevan Trembly

Grade 4 Nonteacher

Edutek

Sebastopol, CA 95472

Terri Vazquez

Grade 4 Teacher

Bear Gulch

Rancho Cucamonga, CA 91730

Inga Wiehl

Grade 12 Nonteacher

Yakima Valley Community College

Yakima, WA 98907–2520

Goldie Wood

Grade 12 Teacher

Dodge City High School

Dodge City, KS 67801

Erik Wooster

Grade 8 Teacher

Castle Rock Middle School

Billings, MT 59102

Joseph Stafford York

Grade 12 Teacher

Munford High School

Munford, TN 38058
903

Figure J-7

Meeting Participants, NAEP Civics Achievement Level Setting

St. Louis, Missouri, November 12–16, 1998

Joseph Glenn Amos

Grade 12 Teacher

Blue Valley North High School

Overland Park, KS 66209

Allison Barker

Grade 4 Teacher

Ethel Schoeffner Elementary School

Destrehan, LA 70047

Mary Bates

Grade 4 General Public

School Board President

Carlinville, IL 62626

Robert Beck

Grade 4 Teacher

Benkelman Elementary

Benkelman, NE 69021

Angie Bessenbacher

Grade 4 General Public

PTA Officer

Folsom, CA 95630

Peggy Braam

Grade 4 Nonteacher

Sugar Creek Elementary School

Verona, WI 53593

Bruce Breedlove

Grade 4 Teacher

Walnut Grove Elementary

Covington, GA 30014

Cynthia Brendle

Grade 12 Teacher

Zephyrhills High School

Zephryhills, FL 33540

Andrew Brewer

Grade 4 Teacher

Gardendale Elementary

Merritt Island, FL 32953

Jo Ann Brou

Grade 8 Teacher

East Thibodaux

Thibodaux, LA 70301

Bruce Brousseau

Grade 4 Nonteacher

Michigan Department of Education

Lansing, MI 48933

Misty Bruckner

Grade 4 General Public

Citizen/Neighborhood Services Coordinator

City of Wichita

Wichita, KS 67202

Marleni Burns

Grade 8 Teacher

Multicultural Magnet School

Bridgeport, CT 06610

Preston Castille

Grade 4 General Public

Taylor, Porter, Brooks and Phillips LLP

Baton Rouge, LA 70821

Dee Coles

Grade 4 Teacher

Rawson School

Hartford, CT 06112

Beverly D’Amico

Grade 4 Teacher

Central Avenue School

Madison, NJ 07940

Lynn Davidson

Grade 12 General Public

President, Board of Education

Salt Lake City, UT 84106

Leslie Davis

Grade 8 General Public

Case Manager

Suffolk County Community College

Selden, NY 11784

(continued)
904

Figure J-7 (continued)

Meeting Participants, NAEP Civics Achievement Level Setting

St. Louis, Missouri, November 12–16, 1998

Nancy Downing

Grade 4 Teacher

South Side Elementary

Picayune, MS 39466

Robert Downing

Grade 12 General Public

Chief Judge, 19th Judicial Court

Baton Rouge, LA 70802

Michael Drahos

Grade 12 Teacher

Sherburne–Earlville Central

Sherburne, NY 13460

Phyllis Dramer

Grade 4 Teacher

Wilder Elementary

Woodinville, WA 98072

R. Eugene Earsom

Grade 12 Nonteacher

Oklahoma State Department of Education

Oklahoma City, OK 73105–4599

Janice Edmond

Grade 4 Teacher

Hutchinson Elementary School

Atlanta, GA 30315

Carolyn Eslick

Grade 8 General Public

City Councilperson

Sultan, WA 98294

Ronald Felder

Grade 8 General Public

Queen City Foundation

Cincinnati, OH 45201

Vicki Foley

Grade 8 Teacher

Campbell County Middle School

Alexandria, KY 41001

Linda Gohlke

Grade 12 Nonteacher

Commission on Student Learning

Seattle, WA 98318

Jerry Graves

Grade 8 Teacher

South Junior High School

Boise, ID 83705

Gretchen Gundlach

Grade 8 Teacher

EJD Middle School

Phoenix, NY 13135

Brian Gustafson

Grade 8 Teacher

Eisenhower Middle School

Rockford, IL 61107

Richard Gutierrez

Grade 12 Teacher

Montwood High School

El Paso, TX 79936

Lorraine Hall

Grade 12 Teacher

Capital High School

Charleston, WV 25311

William Hardin

Grade 8 Teacher

Wayne Middle School

Wayne, WV 25570

Barbara Hoekstra

Grade 12 General Public

Langlais & Schumacher, P.A.

Hastings, MN 55033

Linda Hofler

Grade 12 Teacher

Gates County High School

Gatesville, NC 27938

Rhonda Holtan

Grade 4 Teacher

Carver Elementary School

Montgomery, AL 36108

Jim Horton

Grade 8 General Public

CDI Engineering Group

Baton Rouge, LA 70827

(continued)
905

Figure J-7 (continued)

Meeting Participants, NAEP Civics Achievement Level Setting

St. Louis, Missouri, November 12–16, 1998

Ann Huxtable–Scates

Grade 4 Teacher

Yates Academy

Omaha, NE 68104

Jackie Johnson

Grade 8 Teacher

Campus Middle School

Englewood, CO 80111

Sheila Keller–Powell

Grade 4 Teacher

Monte Vista Elementary

South Jordan, UT 84095

Eugene Kirk

Grade 4 General Public

Office of Navajo Labor Relations

Window Rock, AZ 86515

Michael Kraus

Grade 4 Nonteacher

Country View Elementary School

Verona, WI 53593

Rolland Larson

Grade 8 Nonteacher

Retired

New Salem, ND 58563

Bill Marcy

Grade 12 Teacher

Danbury High School

Danbury, CT 06811

Mary Beth Margolis

Grade 8 General Public

Chairperson, Melrose School Committee

Winchester, MA 01890

Charles McAlister

Grade 8 Teacher

Hopwood Junior High

Saipan, MP 96950

Kris McGee

Grade 4 Teacher

Boonesboro Elementary School

Boonesboro, MD 21713

Mirv Metzger

Grade 4 Teacher

Prettyboy Elementary School

Freeland, MD 21053

Eileen Miles

Grade 12 Teacher

Academy of the Holy Cross

Kensington, MD 20895

Rosemary Miller

Grade 12 Teacher

South River High School

Edgewater, MD 21037

Cheryl Montijo

Grade 8 General Public

Univ. of Arizona La Paz Cooperative Extension

Parker, AZ 85344

Evelyn Nash

Grade 8 Nonteacher

Detroit Public Schools (retired)

Detroit, MI 48154

Denise Nickens

Grade 8 Teacher

Robinson Middle School

Wichita, KS 67208

Jon Noon

Grade 8 General Public

Farmer/Parent Volunteer

Walnut, IA 51577

Susan Palmer

Grade 4 Teacher

Conover Road School

Colts Neck, NJ 07722

Ilene Pearson

Grade 4 General Public

Past PTO President

Melrose, MA 02176

Lynnette Poag

Grade 8 Teacher

Memorial Middle School

Vineland, NJ 08360
906

Figure J-7 (continued)

Meeting Participants, NAEP Civics Achievement Level Setting

St. Louis, Missouri, November 12–16, 1998

Jane Politis

Grade 12 Teacher

Newburyport High School

Newburyport, MA 01950

Jimmy Puckett

Grade 12 Teacher

Aberdeen School District

Aberdeen, MS 39730

Susan Putnam

Grade 8 Teacher

Laurens Junior High

Laurens, SC 29360

Vance Randall

Grade 4 General Public

President, Salome Elementary School Board

Salome, AZ 85348

Carol Roberson

Grade 8 Teacher

Ahwahnee Middle School

Fresno, CA 93710

Carmen Sanchez

Grade 4 Teacher

Greenbriar Elementary

Fort Worth, TX 76134

Robert Shamy

Grade 12 Teacher

Monroe Township High School

Jamesburg, NJ 08831

Janet Smith

Grade 8 Teacher

Kenneth Henderson Middle School

Garden City, KS 67846

Evelyn Spraggins

Grade 4 Teacher

Lopez Elementary School

Biloxi, MS 39530

J. Kelli Sweet

Grade 4 Nonteacher

Kalamazoo Public Schools

Kalamazoo, MI 49008

Marcie Taylor Thoma

Grade 8 Nonteacher

Maryland State Department of Education

Baltimore, MD 21201

Thomas Thompson

Grade 12 General Public

PTA Member

Citrus Heights, CA 95621

Charles Tillery

Grade 4 General Public

Retired

Coalgate, OK 74538

Karen Todorov

Grade 8 Nonteacher

Michigan Department of Education

Lansing, MI 48909

Louise Uchaczyk

Grade 12 Teacher

J.A. Foran High School

Milford, CT 06460

Jackie Viana

Grade 8 Teacher

Hialeah Middle School

Hialeah, FL 33013

Kathleen Vick-Martin

Grade 12 Teacher

Rhinelander High School

Rhinelander, WI 54501

Anita Walter

Grade 8 General Public

Central Coast Trains

Atascadero, CA 93422

David Weinstock

Grade 12 Nonteacher

Rio Americano

Sacramento, CA 95864

Linda Weissler

Grade 4 Teacher

Welby Way Elementary

Canoga Park, CA 91307

(continued)
907

Figure J-7 (continued)

Meeting Participants, NAEP Civics Achievement Level Setting

St. Louis, Missouri, November 12–16, 1998

Jake Wells

Grade 12 General Public

Gaston Chamber of Commerce

Gastonia, NC 28053

David Wilder

Grade 8 General Public

School Board Member

Port Alsworth, AK 99653

Jerome Williams

Grade 12 General Public

Witchita Board of Education

Wichita, KS 67201–0085

Karen Williams

Grade 12 Nonteacher

Wayne County Regional Educational

Service Agency

Wayne, MI 48184

Mary Williams

Grade 12 Teacher

South Atlanta High School

Atlanta, GA 30315

Mary Williams

Grade 8 Teacher

Hamlin Middle School

Corpus Christi, TX 78411

Alan Wind

Grade 12 Teacher

Chattahoochee High School

Alpharetta, GA 30202

Barbara Yingling

Grade 4 Teacher

Villa Cresta Elementary School

Baltimore, MD 21234

Michael Young

Grade 12 Teacher

Burke High School

Omaha, NE 68154
908

Figure J-8

Meeting Participants, NAEP Writing Achievement Level Setting

St. Louis, Missouri, December 9–13, 1998

Anne Atkins-Hall

Grade 4 Teacher

Green T. Lindon Elementary

Youngsville, LA 70592

Mary Kay Baltes

Grade 4 Teacher

Tanglewood Elementary

Baton Rouge, LA 70818

Joan Beal

Grade 4 Teacher

Breckinridge Elementary School

Louisville, KY 40204

Roger Baumgardner

Grade 12 Teacher

Alcona High School

Lincoln, MI 48541

Maxine Blom

Grade 4 General Public

Homemaker/Classroom Volunteer

Dallas, TX 75211

Susan Broussard

Grade 4 Teacher

Pinewood Elementary

DeRidder, LA 70634

Rae Bruce

Grade 12 Teacher

Merrimack High School

Merrimack, NH 03054

Dennis Carpenter

Grade 8 Teacher

North Forsyth Middle School

Cumming, GA 30040

Rose Carter

Grade 12 Teacher

Capitol High School

Baton Rouge, LA 70802

Kim Castagna

Grade 8 Teacher

Strafford School

Center Strafford, NH 03815

Bobbie Caton

Grade 8 Teacher

Northside Middle School

Roanoke, VA 24019

Gidget Coates

Grade 8 Teacher

Glasgow Middle School

Baton Rouge, LA 70808

Maurene Coe

Grade 4 Teacher

West Elementary School

Youngstown, OH 44509

Jeanne Wells Cook

Grade 12 Nonteacher

Hinds Community College

Raymond, MS 39154–9799

Mary Jane Currie

Grade 8 Teacher

North Middle School

Saginaw, MI 48602

Margaret Davidson

Grade 4 Teacher

Butcher Children's School

Emporia, KS 66801–5087

Mark Dressman

Grade 4 Nonteacher

University of Houston

Houston, TX 77204

Karen Elston

Grade 4 General Public

Past PTA President/Classroom Volunteer

Sylacauga, AL 35150

Susan Emerick

Grade 4 General Public

Public Relations Assistant

Detroit Pistons

Auburn Hills, MI 48326

Peggy Euwins

Grade 8 Teacher

Chickasha Junior High School

Chickasha, OK 73018

(continued)
909

Figure J-8 (continued)

Meeting Participants, NAEP Writing Achievement Level Setting

St. Louis, Missouri, December 9–13, 1998

Barbara Ferguson

Grade 12 General Public

Past Columnist

Sleepy Hollow, IL 60118

Maria Ferrandini

Grade 4 Teacher

Cashman Elementary School

Amesbury, MA 01913

Neil Fisher

Grade 12 General Public

Vice President & Chief Operating Officer

Kansas Building Systems, Inc.

Topeka, KS 66675–0555

Paula Flemming

Grade 4 Teacher

Conval School District

Peterborough, NH 03458–1109

Beverly Gaffney

Grade 4 General Public

Owner, Executive Secretarial Service

University Heights, OH 44118

Lisa Goins

Grade 4 Teacher

Western Branch Intermediate School

Chesapeake, VA 23321

Karen Grady–Glass

Grade 4 Teacher

Ford Elementary C.E.C.

St Louis, MO 63112

Nancy Guss

Grade 8 Teacher

Pineview Middle School

Land O' Lakes, FL 34639

Joan Vohl Hamilton

Grade 8 Teacher

Fairview Veterans Memorial Middle

School

Chicopee, MA 01020

Cassandra Hansbrough

Grade 12 Teacher

Amanda Elzy High School

Greenwood, MS 38930

Anne Hanson

Grade 8 Teacher

Ingleside Middle School

Phoenix, AZ 85018

Demetrice Hardman

Grade 4 Teacher

Kanawha City Elementary

Charleston, WV 25304

Roxanne Hawrylewicz

Grade 8 General Public

Scorer, Local Writing Assessment

Carpentersville, IL 60110

Dewey Hensley

Grade 12 Teacher

South Oldham High School

Crestwood, NY 40014

Elizabeth Holcomb

Grade 12 Teacher

Jackson, MS 39212

Linda Holt

Grade 4 Teacher

Ha'iku Elementary School

Maui, HI 96708

Susan Horn

Grade 8 Nonteacher

Adams Cheshire Regional School District

Adams, MA 01220

Janice Hughes

Grade 4 Teacher

Ben Hill Elementary

Fitzgerald, GA 31750

Bill Irmen

Grade 8 Teacher

Jim Hill Middle School

Minot, ND 58701

Judy Jenlink

Grade 12 Teacher

Timberlake High School

Helena, OK 73741
910

Figure J-8 (continued)

Meeting Participants, NAEP Writing Achievement Level Setting

St. Louis, Missouri, December 9–13, 1998

Karla Johnson

Grade 12 General Public

Past PTA President

Kanab, UT 84741

Marilyn Johnston

Grade 12 General Public

Director, Oregon Governor’s School

Salem, OR 97301

Jennifer Jones

Grade 8 Nonteacher

Center for Research on Learning

Lawrence, KS 66045

Jean Jonker

Grade 12 Teacher

William J. Dean Technical High School

Holyoke, MA 01040

Alice Kawazoe

Grade 12 Nonteacher

Oakland Unified School District

Oakland, CA 94606

Jeanne Kent

Grade 12 General Public

Past Educational Consultant

Columbia, VA 23038

Milton Kimura

Grade 8 Teacher

Jarrett Middle School

Honolulu, HI 96816

Noelle Kodroff

Grade 12 Teacher

Fels High School

Philadelphia, PA 19111

Gary Kwong

Grade 8 General Public

Gifted/Talented Advisory Council

Falcon Heights, MN 55113

Gail Ann Lee

Grade 12 Nonteacher

Mililani High School

Mililani, HI 96789

Janice Lindsay

Grade 8 Nonteacher

Detroit Public Schools

Detroit, MI 48235

Dan Lloyd

Grade 4 General Public

Assistant Manager

D.L. Evans Bank

Burley, ID 83318

William Lynch

Grade 8 Teacher

Lindale/Brooklyn Park Middle

Linthicum, MD 21090

Jeff Maas

Grade 4 Teacher

Lincoln Elementary School

Madison, WI 53713

Sharon Manson

Grade 8 General Public

Member, Board of Education

Waverly, OH 45690

Thomas McGuire

Grade 8 Teacher

Irma C. Ruiz

Chicago, IL 60608

Nancy McKay

Grade 12 General Public

PTA President

Sylacauga, AL 35150

Ruth Ann McKenna

Grade 12 Nonteacher

New Haven Unified School District

Union City, CA 94587

Stacey Millett

Grade 4 General Public

Grant Writer

St. Paul, MN 55101

Jeff Miner

Grade 8 Teacher

Piligrim School

Warwick, RI 02888

(continued)
911

Figure J-8 (continued)

Meeting Participants, NAEP Writing Achievement Level Setting

St. Louis, Missouri, December 9–13, 1998

Jacqueline Mink

Grade 12 Teacher

C.M. Russell High School

Great Falls, MT 59404

Irene Mlynar

Grade 8 Teacher

New Fairfield, CT 06812

Susan Morehouse

Grade 4 Nonteacher

Alfred University

Alfred, NY 14802

Danna Morrison

Grade 8 Teacher

Hebron Middle School

Shepherdsville, KY 40165

William Newton

Grade 8 General Public

Vice President, Human Resources

Palace Sports & Entertainment

Auburn Hills, MI 48326

Christa O'Keefe–Hutchison

Grade 4 Teacher

Center/Sylvester School

Hanover, MA 02339

Kathleen O'Shaughnessy

Grade 8 Teacher

Broussard Middle School

Broussard, LA 70518

Aileen Ortega

Grade 8 Teacher

F.B. Leon Guerrero Middle School

Agana, GU 96912

Tara Oursler

Grade 12 General Public

Public Relations Specialist

Baltimore County Government

Towson, MD 21204

Dan Pelletier

Grade 4 General Public

Vice President/General Manager

Pelletier's Automotive/Brownwell

Industrial Supply

Attleboro, MA 02703

Claudia Peters

Grade 4 Nonteacher

Nashoha Regional School District

Lancaster, MA 01523

Rochelle Pokorny

Grade 4 Teacher

Cesar Chavez Elementary

Corona, CA 91720

Grace Polivka

Grade 8 Teacher

North Branford Int. School

North Brandford, CT 06471

Barb Rodriguez

Grade 4 Teacher

Kendrick Lakes Elementary

Lakewood, CO 80226

Rosa Sailes

Grade 12 Teacher

CPS Teachers Academy

Chicago, IL 60643

Annette Sample

Grade 12 Teacher

Carver High School

Philadelphia, PA 19121

Barbara Singleton

Grade 4 Nonteacher

Harrison County School District

Gulfport, MS 39503

Stephen Smarjesse

Grade 12 Teacher

Osseo Senior High School

Osseo, MN 55339

Robert Smith

Grade 12 Teacher

Woodstock Academy

Woodstock, CT 06281-2303

Maria Stashak

Grade 8 General Public

Parent Volunteer

Sebastopol, CA 95472
912

Figure J-8 (continued)

Meeting Participants, NAEP Writing Achievement Level Setting

St. Louis, Missouri, December 9–13, 1998

Ronald Sudol

Grade 12 Nonteacher

Oakland University

Rochester, MI 48309

Sandra Thompson

Grade 12 Teacher

Cleveland High School

Cleveland, MS 38732

Troi Tyner

Grade 12 Nonteacher

Liberty–Benton Local Schools

Findlay, OH 45840

Rick Varner

Grade 12 General Public

Attorney Law Offices of

John H. Anderson

San Clemente, CA 92672

Sheila Vaughn

Grade 8 General Public

Psychologist and Lecturer

Irvine, CA 92618

Sue Wade

Grade 8 General Public

Youth Services Librarian

St Albans Free Library

St Albans, VT 05478

Janis Weeks

Grade 4 General Public

Senior Labor Relations Manager

State of Oregon

Salem, OR 97310
913

Figure J-9

Meeting Participants, Civics NAEP Similarities Classification Study

Validation Meeting, St. Louis, Missouri, July 9–11, 1999

Marleni Burns

Multicultural Magnet School

Bridgeport, CT 06610

Mary Carter

Alma Middle School

Alma, AR 72921

Vicki Foley

Campbell Co. Middle School

Alexandria, KY 41001

Brian Gustafson

Eisenhower Middle School

Rockford, IL 61107

Gretchen Gundlach

EJD Middle School

Phoenix, NY 13135

William Hardin

Wayne Middle School

Wayne, WV 25570

Leslie Lee

Hammocks Middle School

Miami, FL 33196

Denise Nickens

Robinson Middle School

Wichita, KS 67208

Lynnette Poag

Memorial Middle School

Vineland, NJ 08360

Susan Putnam

Laurens Junior High

Laurens, SC 29360

Janet Smith

Kenneth Henderson Middle School

Garden City, KS 67846

915

Appendix K

PARTICIPANTS IN THE OBJECTIVES

AND ITEM DEVELOPMENT PROCESS

The National Assessment of Educational Progress extends its deep appreciation to all those

individuals who participated in the development of the framework, objectives, and items for the 1998

national assessment.

Figure K-1

1998 NAEP Reading Item Development Committee

Professor Peter Afflerbach

Department of Curriculum and Instruction

University of Maryland

College Park, MD

Professor Rosalinda Barrera

Department of Curriculum and Instruction

University of Illinois at Urbana-Champaign

Champaign, IL

Marshe Delain

Delaware State Department of Public Instruction

Dover, DE

Professor Jan Dole

University of Utah

Salt Lake City, UT

Professor Violet Harris

Department of Curriculum and Instruction

University of Illinois at Urbana

Urbana, IL

Patricia McGonegal

Mount Mansfield High School

Jericho, VT

Professor Susan Neuman

Philadelphia, PA

Jack Pikulski

University of Delaware

Newark, DE

Terry Salinger

International Reading Association

Warren Simmons

The Philadelphia Education Fund

Philadelphia, PA

Karen Wixson

University of Michigan

Ann Arbor, MI
916

Figure K-2

1998 NAEP Reading/Writing Standing Committee

Elyse Eidman-Aadahl

National Writing Project

University of California at Berkeley

Berkeley, CA

Arthur Applebee

School of Education

State University of New York

Albany, NY

John Guthrie

National Reading Research Center

University of Maryland

College Park, MD

Kris Gutierrez

Graduate School of Education

University of California

Los Angeles, CA

Violet Harris

Dept of Curriculum and Instruction

University of Illinois at Urbana-Champaign

Champaign, IL

Barbara Kapinus

Hyattsville, MD

Judith Langer

School of Education

State University of New York

Albany, NY

Miles Myers

National Council of Teachers of English

Urbana, IL

Susan Neuman

Lexington, MA

Timothy Shanahan

College of Education

University of Illinois at Chicago

Chicago, IL

Richard Sterling

National Writing Project

University of California at Berkeley

Berkeley, CA

Daisy Vickers

North Carolina Public Instruction Dept.

Raleigh, NC
917

Figure K-3

1998 NAEP Writing Item Development Committee

Arthur Applebee

School of Education

State University of New York

Albany, NY

Katherine Au

Honolulu, HI

Noreen Duncan

Lawrenceville, NJ

Elyse Eidman-Aadahl

University of California at Berkeley

Berkeley, CA

Tammy Elser

Arlee Schools

Missoula, MT

Sylvia Flores

Artesia, NM

Kris Gutierrez

University of California at Los Angeles

Graduate School of Education

Los Angeles, CA

Geraldine Guttwein

Dillsburg, PA

Charlotte Higuchi

CRESST

University of California at Los Angeles

Los Angeles, California

Brian Huot

Louisville, KY

James Marshall

NCTE/Department of Education,

Curriculum & Instruction

University of Iowa

Iowa City, IA

Peter Mosenthal

Department of Reading and Lang. Arts

Syracuse University

Syracuse, NY

Patricia Porter

Texas Education Agency

Austin, TX

Darian Walker

Pinellas County Schools

Largo, FL
918

Figure K-4

1998 NAEP Civics Item Development Committee

Sandy Baker

Elliott School

Munster, IN

Richard Brody

Stanford University

Stanford, CA

Elaine Craig

Center for Civic Education

Los Angeles, CA

Rosie Heffernan

Our Lady of Lourde’s Academy

Miami, FL

Milton Kato

Tulare County Office of Education

Visalia, CA

Linda Levstik

University of Kentucky

Lexington, KY

Peter Litchka

Battle Creek Central Schools

Battle Creek, MI

Lori Morton

Westlawn Elementary

Laurel, MD

Richard Niemi

Department of Political Science

University of Rochester

Rochester, NY

John Patrick

Smith Research Center

Indiana University

Bloomington, IN

Sharon Shiraiwa

East Hartford-Glastonbury Magnet School

East Hartford, CT

Judith Torney-Purta

University of Maryland

College Park, MD

Donald Vetter

Council of Chief State School Officers

Washington, DC
919

Figure K-5

1998 NAEP Civics Standing Committee

John Eyster

Parker High School

Janeville, WI

David Harris

Oakland Schools

Waterford, MI

Milton Kato

Jefferson School

Madera, CA

Sheilah Mann

American Political Science Association

Washington, DC

James Marran

New Trier Township High School

Winnetka, IL

Mabel McKinney-Browning

American Bar Association

Chicago, IL

Richard Niemi

University of Rochester

Rochester, NY

John Patrick

Indiana University

Bloomington, IN

921

REFERENCES CITED IN TEXT

Abedi, J. (1996). The Interrater/Test Reliability System. (ITRS). Multivariate Behavioral

Research, 31(4), 409–417.

Allen, N. L., & Donoghue, J. R. (1994, April). Differential item functioning based on complex

samples of dichotomous and polytomous items. Paper presented at the annual meeting of the American

Educational Research Association, New Orleans, LA.

Allen, N. L., & Donoghue, J. R. (1996). Applying the Mantel-Haenszel procedure to complex

samples of items. Journal of Educational Measurement, 33, 231–251.

Allen, N. L., Jenkins, F., Kulick, E., & Zelenak, C. A. (1997). Technical report of the NAEP

1996 state assessment program in mathematics (NCES 97–951). Washington, DC: National Center for

Education Statistics.

Allen, N. L., Kline, D. L., & Zelenak, C. A. (1996). The NAEP 1994 technical report (NCES

97–897). Washington, DC: National Center for Education Statistics.

Allen, N. L., Mazzeo, J., Ip, E. H. S., Swinton, S., Isham, S. P., & Worthington, L. (1995). Data

analysis and scaling for the 1994 Trial State Assessment in reading. In J. Mazzeo, N. L. Allen, & D. L.

Kline, Technical report of the NAEP 1994 Trial State Assessment in reading (pp. 169–219). Washington,

DC: National Center for Education Statistics.

Allen, N. L., Mazzeo, J., Isham, S. P., Fong, Y. F., & Bowker, D. W. (1994). Data analysis and

scaling for the 1992 Trial State Assessment in reading. In E. G. Johnson, J. Mazzeo, & D. L. Kline,

Technical Report of the 1992 Trial State Assessment program in reading (pp. 147–149). Washington,

DC: National Center for Education Statistics.

Allen, N. L., Swinton, S., Isham, S. P., & Zelenak, C. A. (1998). Technical report of the state

assessment program in science. Washington DC: National Center for Education Statistics.

American College Testing. (1993). Setting achievement levels on the 1992 National Assessment

of Educational Progress in mathematics, reading, and writing: A technical report on reliability and

validity. Iowa City, IA: Author.

American College Testing. (1998). Developing achievement levels on the 1998 NAEP in civics: 

Field trial report. Iowa City, IA: Author.

American College Testing. (1999a). Developing achievement levels on the 1998 NAEP in civics:

 Pilot study report. Iowa City, IA: Author.

American College Testing. (1999b). Developing achievement levels on the 1998 NAEP in civics:

 Final report. Iowa City, IA: Author.

American College Testing. (1999c). Developing achievement levels on the 1998 NAEP in

writing: Field trial report. Iowa City, IA: Author.

American College Testing. (1999d). Developing achievement levels on the 1998 NAEP in

writing: Final report. Iowa City, IA: Author.
922

American College Testing. (1999e). Developing achievement levels on the 1998 NAEP in

writing: Pilot study report. Iowa City, IA: Author.

Andersen, E. B. (1980). Comparing latent distributions. Psychometrika, 45, 121–134.

Angoff, W. H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.),

Educational Measurement (2nd ed., pp. 508–600). Washington DC: American Council on Education.

Ballator, N., & Jerry, L. (1999a). NAEP 1998 reading state report for [state], NCES–460.

Washington, DC: National Center for Education Statistics.

Ballator, N., & Jerry, L. (1999b). NAEP 1998 writing state report for [state], NCES–460.

Washington, DC: National Center for Education Statistics.

Beaton, A. E., & Johnson, E. G. (1990). The average response method of scaling. Journal of

Educational Statistics, 15, 9–38.

Beaton, A. E., & Johnson, E. G. (1992). Overview of the scaling methodology used in the

National Assessment. Journal of Educational Measurement, 26(2), 163–175.

Beaton, A. E., & Zwick, R. (1990). The effect of changes in the National Assessment: 

Disentangling the NAEP 1985-86 reading anomaly. (No. 17–TR–21) Princeton, NJ: Educational Testing

Service, National Assessment of Educational Progress.

Benjamini, Y., & Hochberg, Y. (1994). Controlling the false discovery rate: A practical and

powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B, 57(1), 289–300.

Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In

F. M. Lord & M. R. Novick, Statistical theories of mental test scores. Reading, MA: Addison-Wesley.

Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in

two or more latent categories. Psychometrika, 37, 29–51.

Bourque, M. L., Champagne, A. B., & Crissman, S. (1997). NAEP 1996 science performance

standards: Achievement results for the nation and the states. Washington, DC: National Assessment

Governing Board.

Bourque, M. L., & Garrison, H. (1991). The levels of mathematics achievement: Initial

performance standards for the 1990 NAEP Mathematics Assessment (Vols. I & II). Washington, DC: 

National Assessment Governing Board.

Broughman, S. P., & Colaciello, L.A. (1998). Private school universe study, 1995-96, (NCES

98-229), Washington, DC: National Center for Education Statistics.

Calderone, J., King, L. M., & Horkay, N. (Eds.). (1997). The NAEP guide: A description of the

content and methods of the 1997 and 1998 assessments, (NCES 97–990). Washington, DC: National

Center for Education Statistics.

Carlson, J. E. (1993). Dimensionality of NAEP instruments that incorporate polytomously scored

items. Paper presented at the annual meeting of the American Educational Research Association, Atlanta,

GA.
923

Carlson, J. E., & Jirele, T. (1992, April). Dimensionality of 1990 NAEP mathematics data. Paper

presented at the meeting of the American Educational Research Association, San Francisco, CA.

Center for Civic Education. (1994). National standards for civics and government. Calabasas,

CA: Author.

Center for Research on Evaluation, Standards, and Student Testing (CRESST). (1996). Writing

framework and specifications for the 1998 National Assessment of Educational Progress (Contract

RS89174001). Washington DC: National Assessment Governing Board.

Chang, H. H., Mazzeo, J., & Roussos, L. (1995). Detecting DIF for polytomously scored items: 

An adaptation of the SIBTEST procedure, (ETS Research Report RR-95-05). Princeton, NJ: Educational

Testing Service.

Cizek, G. (1993). Reactions to National Academy report: Setting performance standards for

student achievement. Washington, DC: National Assessment Governing Board.

Cochran, W. G. (1977). Sampling techniques. New York, NY: John Wiley & Sons.

Cohen, J. (1968). Weighted kappa: Nominal scale agreement with provision for scaled

disagreement or partial credit. Psychological Bulletin, 70(4), 213–220.

Council of Chief State School Officers. (1996). Civics framework for the 1998 National

Assessment of Educational Progress. Washington, DC: National Assessment Governing Board.

Curry, L. (1987, April). Group decision process in setting cut-off scores. Paper presented at the

annual meeting of the American Educational Research Association, Washington, DC.

Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete

data via the EM algorithm (with discussion). Journal of the Royal Statistical Society, Series B, 39, 1–38.

Donoghue, J. R. (1993). An empirical examination of the IRT information in polytomously

scored reading items. (ETS Research Report RR-95-05). Princeton, NJ: Educational Testing Service.

Donoghue, J. R. (1995, April). Assessing some of the measurement properties of longer reading

blocks: An application of the bootstrap to structural equation models. Paper presented at the annual

meeting of the American Educational Research Association, San Francisco, CA.

Donoghue, J. R. (1998a). Detecting DIF for polytomously scored items: An adaptation of the

SIBTEST procedure, [computer program] Princeton, NJ: Educational Testing Service.

Donoghue, J. R. (1998b). NAEP SIBTEST [computer program]. Princeton, NJ: Educational

Testing Service.

Donoghue, J. R. (2000) On the derivation of an effect size measure for polytomous DIF [draft

manuscript]. Princeton, NJ: Educational Testing Service.

Donoghue, J. R., Holland, P. W., & Thayer, D. T. (1993). A Monte Carlo study of factors that

affect the Mantel-Haenszel and standardization measures of differential item functioning. In P. W.

Holland and H. Wainer (Eds.) Differential item functioning (pp. 137–166). Hillsdale, NJ: Erlbaum.
924

Donoghue, J. R. & Hombo, C. M. (June, 1999) Some asymptotic results on the distribution of an

IRT measure of item fit. Paper presented at the annual meeting of the Psychometric Society, Lawrence,

KS.

Donoghue, J. D. & Isham, S. P. (1998). A comparison of procedures to detect item parameter

drift (with S. P.). Applied Psychological Measurement, 22, 33-51.

Donoghue, J. R., & Mazzeo, J. (1995). Assessing some of the properties of longer blocks in the

1992 NAEP reading assessment, (ETS Research Report RR-95-28). Princeton, NJ: Educational Testing

Service.

Donahue, P. L., Voelkl, K. E., Campbell, J. R., & Mazzeo, J. (1999). NAEP 1998 reading report

card for the nation and the states. (NCES 1999–500). Washington, DC: National Center for Education

Statistics.

Dorans, N., & Kulick, E. (1986). Demonstrating the utility of the standardization approach to

assessing unexpected differential item performance on the Scholastic Aptitude Test. Journal of

Educational Measurement, 23, 355–368.

Educational Testing Service. (1992). Innovations and ingenuity: A foundation for the future.

Application for cooperative agreement for NAEP (CFDA Number: 84.999E). Princeton, NJ: Author.

Educational Testing Service. (1987). ETS standards of quality and fairness. Princeton, NJ: 

Author.

Engelen, R. J. H. (1987). Semiparametric estimation in the Rasch model. Research Report 87-1.

Twente, the Netherlands: Department of Education, University of Twente.

Fitzpatrick, A. R. (1989). Social influences in standard-setting: The effects of social interaction

on group judgments. Review of Educational Research, 59, 315–328.

Friedman, C. B., & Ho, K. T. (1990, April). Interjudge consensus and intrajudge consistency: Is

it possible to have both on standard setting? Paper presented at the annual meeting of the National

Council for Measurement in Education, Boston, MA.

Grant, E. L. (1964). Statistical quality control.Washington, DC: McGraw Hill, 211-212.

Gray, L. M., Krenzke, T., & Wallace, L. (2000). Sampling activities and field operations for

1998 NAEP. Rockville, MD: Westat.

Greenwald, E. A., Persky, H. R., Campbell, J. R., & Mazzeo, J. (1999). NAEP 1998 writing

report card for the nation and the states. (NCES 1999–462). Washington, DC: National Center for

Education Statistics.

Habing, B. & Donoghue, J. R. (in press). Local dependence assessment for exams with

polytomous items and incomplete item-examinee layouts. Journal of Educational and Behavioral

Statistics.

Hambleton, R. K., & Bourque, M. L. (1991). The levels of mathematics achievement. Vol. II,

technical report. Washington, DC: National Assessment Governing Board.

Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple tests of significance.

Biometrika, 75, 800–802.
925

Hoijtink, H. (1991). Estimating the parameters of linear models with a latent dependent variable

by nonparametric maximum likelihood. Research Bulletin HB-91-1040-EX. Groningen, The Netherlands:

 Psychological Institute, University of Groningen.

Holland, P. W., & Thayer, D. T. (1988). Differential item performance and the Mantel-Haenszel

procedure. In H. Wainer & H. I. Braun (Eds.), Test validity. Hillsdale, NJ: Erlbaum.

Hombo, C. M., & Donoghue, J. R. (2000). Some properties of the distribution of an IRT measure

of item fit. Paper presented at the annual meeting of the Psychometric Society, Vancouver, British

Columbia, Canada.

Huynh, H. (1994). Some technical aspects of standard setting. Paper presented at the Joint

Conference on Standard Setting for Large-Scale Assessments. Washington, D.C.

Huynh, H. (1998). On score locations of binary and partial credit items and their applications to

item mapping and criterion-referenced interpretation. Journal of Educational and Behavioral Statistics,

23(1), 38–58.

Improving America’s Schools Act of 1994. Pub. L. No. 103–382, Title I, Part A, 108 Stat. 3519

(1994).

Jenkins, F., Kulick, E., Kaplan, B. A., Wang, S., Qian, J., & Wang, X. (1997). Data analysis and

scaling for the 1996 state assessment program in mathematics. In N. L.Allen, F. Jenkins, E. Kulick, & C.

A. Zelenak. (1997). Technical report of the NAEP 1996 state assessment program in mathematics

(NCES 97–951). Washington, DC: National Center for Education Statistics.

Johnson, E. G. (1989). Considerations and techniques for the analysis of NAEP data. Journal of

Educational Statistics, 14(4), 303–334.

Johnson, E. G., & Carlson, J. E. (1994). The NAEP 1992 technical report. Washington, DC: 

National Center for Education Statistics.

Johnson, E. G., & King, B. F. (1987). Generalized variance functions for a complex sample

survey. Journal of Official Statistics, 3, 235–250.

Johnson, E. G., & Rust, K. F. (1992). Population inferences and variance estimation for NAEP

data. Journal of Educational Statistics, 17, 175–190.

Johnson, E. G. & Rust, K. F. (1993). Effective degrees of freedom for variance estimates from a

complex sample survey. American Statistical Association 1993 Proceedings: Survey research methods

section, 863–866.

Kane, M. (1993). Comments on the NAE evaluation of the NAGB achievement levels.

Washington, DC: National Assessment Governing Board.

Kaplan, B. A., Beaton, A. E., Johnson, E. G., & Johnson, J. R. (1988). National Assessment of

Educational Progress: 1986 bridge studies. Princeton, NJ: Educational Testing Service, National

Assessment of Educational Progress.

Kaplan, B. A., & Johnson, E. G. (1992, April). Reliability of professionally scored data: NAEP

related issues. Paper presented at the annual meeting of the American Educational Research Association,

San Francisco, CA.
926

Kaplan, D. (1995). The impact of BIB spiraling-induced missing data patterns on goodness-of-fit

tests in factor analysis. Journal of Educational and Behavioral Statistics, 20, 69–82.

Keyfitz, N. (1951). Sampling with probability proportional to size; adjustment for changes in

probabilities. Journal of the American Statistical Association, 46, 105–109.

Kish, L. (1965). Survey Sampling. New York: John Wiley & Sons.

Kish, L., & Frankel, M. R. (1974). Inference from complex samples. Journal of the Royal

Statistical Society, Series B, 36, 1–22.

Krewski, D., & Rao, J. N. K. (1981). Inference from stratified samples: Properties of

linearization, jackknife and balanced repeated replication. Annals of Statistics, 9, 1010–1019.

Laird, N. M. (1978). Nonparametric maximum likelihood estimation of a mixing distribution.

Journal of the American Statistical Association, 73, 805–811.

Langer, J. A. (1989). The process of understanding literature. Report Series 2.1. Albany, NY: 

Center for the Learning and Teaching of Literature, State University of New York.

Langer, J. A. (1990). The process of understanding: Reading for literary and informative

purposes. Research in the Teaching of English, 24(3), 229–260.

Lindsey, B., Clogg, C. C., & Grego, J. (1991). Semiparametric estimation in the Rasch model and

related exponential response models, including a simple latent class model for item analysis. Journal of

the American Statistical Association, 86, 96–107.

Little, R. J. A., & Rubin, D. B. (1983). On jointly estimating parameters and missing data.

American Statistician, 37, 218–220.

Little, R. J. A., & Rubin, D. B. (1987). Statistical analysis with missing data. New York, NY: 

John Wiley & Sons.

Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale,

NJ: Lawrence Erlbaum Associates.

Lutkus, A. D., Weiss, A. R., Campbell, J. R., Mazzeo, J., & Lazer, S. (1999). NAEP 1998 civics

report card for the nation. (NCES 2000–457). Washington, DC: National Center for Education

Statistics.

Mantel, N. (1963). Chi-square tests with one degree of freedom: Extensions of the MantelHaenszel procedure. Journal of the American Statistical Association, 58, 690–700.

Mantel N., & Haenszel, W. M. (1959). Statistical aspects of the analysis of data from

retrospective studies of disease. Journal of the National Cancer Institute, 22, 719-748.

Mazzeo, J. (1991). Data analysis and scaling. In S. L. Koffler, The technical report of NAEP’s

1990 Trial State Assessment program (pp. 138–182) [No. ST–21–01]. Washington, DC: National

Center for Education Statistics.

Mazzeo, J., Allen, N. L., & Kline, D. L. (1995). Technical report of the NAEP 1994 Trial State

Assessment program in reading. Washington, DC: National Center for Education Statistics.
927

Mazzeo, J., Carlson, J.E., Voelkl, K.E., & Lutkus, A.D. (1999). Increasing the participation of

special needs students in NAEP: A report on 1996 NAEP research activities. (NCES 2000-473).

Washington, DC: National Center for Education Statistics.

Mazzeo, J., Chang, H., Kulick, E., Fong, Y. F., & Grima, A. (1993). Data analysis and scaling

for the 1992 Trial State Assessment. In E. G. Johnson, J. Mazzeo, & D. L. Kline, Technical report of the

NAEP 1992 Trial State Assessment program in mathematics (NCES 23–ST05). Washington, DC: 

National Center for Education Statistics.

Mazzeo, J., Johnson, E. G., Bowker, D., & Fong, Y. F. (1992). The use of collateral information

in proficiency estimation for the Trial State Assessment. Paper presented at the annual meeting of the

American Educational Research Association, San Francisco, CA.

Mislevy, R. J. (1985). Estimation of latent group effects. Journal of the American Statistical

Association, 80, 993–997.

Mislevy, R. J. (1990). Scaling procedures. In E. G. Johnson and R. Zwick, Focusing the new

design: The NAEP 1988 technical report (pp. 229–250) [No. 19–TR–20]. Princeton, NJ: Educational

Testing Service, National Assessment of Educational Progress.

Mislevy, R. J. (1991). Randomization-based inference about latent variables from complex

samples. Psychometrika, 56, 177–196.

Mislevy, R. J., Beaton, A. E., Kaplan, B., & Sheehan, K. M. (1992). Estimating population

characteristics from sparse matrix samples of item responses. Journal of Educational Measurement,

29(2), 133–161.

Mislevy, R. J., & Bock, R. D. (1982). BILOG: Item analysis and test scoring with binary logistic

models [Computer program]. Mooresville, IN: Scientific Software.

Mislevy, R. J., Johnson, E. G., & Muraki, E. (1992). Scaling procedures in NAEP. Journal of

Educational Statistics, 17(2), 131–154.

Mislevy, R. J., & Sheehan, K. M. (1987). Marginal estimation procedures. In A. E. Beaton,

Implementing the new design: The NAEP 1983-84 technical report, [No. 15–TR–20] (pp. 293–360).

Princeton, NJ: Educational Testing Service, National Assessment of Educational Progress.

Mislevy, R. J., & Stocking, M. L. (1989). A consumer’s guide to LOGIST and BILOG. Applied

Psychological Measurement, 13(1), 57–75.

Mislevy, R. J., & Wu, P. K. (1988). Inferring examinee ability when some item responses are

missing (ETS Research Report RR-88-48-ONR). Princeton, NJ: Educational Testing Service.

Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied

Psychological Measurement, 16(2), 159–176.

Muraki, E. (1993). Information functions of the generalized partial credit model. Applied

Psychological Measurement, 17(4), 351–362.

Muraki, E., & Bock, R. D. (1991). PARSCALE: Parameter scaling of rating data. Chicago, IL: 

Scientific Software, Inc.
928

Muraki, E., & Bock, R. D. (1997). PARSCALE: IRT item analysis and test scoring for ratingscale data. Chicago, IL: Scientific Software International.

Muthén, B. (1991, November). Issues in using NAEP mathematics items to study achievement

dimensionality, within-grade differences, and across-grade growth. Report presented to the Design and

Analysis Committee of the National Assessment of Educational Progress, Washington, DC.

National Academy of Education. (1992). Assessing student achievement in the states. The first

report of the National Academy of Education Panel on the evaluation of the NAEP Trial State

Assessment: 1990 Trial State Assessment. Stanford, CA: Author.

National Academy of Education. (1993a). Setting performance standards for student

achievement. Stanford, CA: Author.

National Academy of Education. (1993b). The Trial State Assessment: prospects and realities.

Stanford, CA: Author.

National Academy of Sciences. (1998). Grading the nation’s report card: Evaluating NAEP and

transforming the Assessment of Educational Progress. Washington, DC: Author.

National Assessment Governing Board. (1989). Setting achievement goals on the National

Assessment of Educational Progress: A draft policy statement. Washington, DC: Author.

National Assessment Governing Board. (1990). Reading framework for the National Assessment

of Educational Progress, 1992-1998. Washington, DC: Author.

National Assessment Governing Board. (1996a). Civics framework for the 1998 National

Assessment of Educational Progress. Washington, DC: Author.

National Assessment Governing Board. (1996b). Writing framework and specifications for the

1998 National Assessment of Educational Progress, 1992-1998. Washington, DC: Author.

National Assessment of Educational Progress. (1996). The NAEP guide: A description of the

content and methods of the 1994 and 1996 assessments. Washington, DC: National Center for Education

Statistics.

National Assessment of Educational Progress Improvement Act of 1988. Pub L. 100–297,

U.S.C.A.  1221 et seq. (1988).

National Computer Systems. (1998). 1998 NAEP Assessment: Report of processing and

professional scoring activities. Iowa City, IA: Author.

Orlando, M. & Thissen, D. (2000). Likelihood-based item-fit indices for dichotomous item

response theory models. Applied Psychological Measurement, 24, 50-64.

Petersen, N. (1988). DIF procedures for use in statistical analysis. Internal memorandum.

Rock, D. A. (1991, November). Subscale dimensionality. Paper presented at the meeting of the

Design and Analysis Committee of the National Assessment of Educational Progress, Washington, DC.

Rogers, A. M., Kokolis, G. A., Stoeckel, J. J., & Kline, D. L. (2000). National Assessment of

Educational Progress: 1998 civics assessment secondary-use data files data companion. Princeton, NJ: 

Educational Testing Service, National Assessment of Educational Progress.
929

Rogers, A. M., Kokolis, G. A., Stoeckel, J. J., & Kline, D. L. (2000). National Assessment of

Educational Progress: 1998 writing assessment secondary-use data files data companion. Princeton,

NJ: Educational Testing Service, National Assessment of Educational Progress.

Rogers, A. M., Kokolis, G. A., Stoeckel, J. J., & Kline, D. L. (2000). National Assessment of

Educational Progress: 1998 reading assessment secondary-use data files data companion. Princeton,

NJ: Educational Testing Service, National Assessment of Educational Progress.

Roussos, L., & Stout, W. (1996). A multidimensionality-based DIF analysis paradigm. Applied

Psychological Measurement, 20(4), 355-71.

Roussos. L. A., Stout, W. F., & Marden, J. I. (1998). Using new proximity measures with

hierarchical cluster analysis to detect multidimensionality. Journal of Educational Measurement, 35.

1-30.

Rubin, D. B. (1987). Multiple imputation for nonresponse in surveys. New York, NY: John

Wiley & Sons.

Rubin, D. B. (1991). EM and beyond. Psychometrika, 56, 241–254.

Rust, K. (1985). Variance estimation for complex estimators in sample surveys. Journal of

Official Statistics, 1(4), 381–397.

Rust, K. F., Bethel, J., Burke, J., & Hansen, M. H. (1990). The 1988 National Assessment of

Educational Progress: Sampling and weighting procedures, final report. Rockville, MD: Westat.

Rust, K. F., Burke, J., & Fahimi, M. (1992). 1990 National Assessment of Educational Progress

sampling and weighting procedures, part 2: National assessment, final report. Rockville, MD: Westat

Rust, K. F., & Johnson, E. G. (1992). Sampling and weighting in the national assessment.

Journal of Educational Statistics, 17(2), 111–129.

Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores.

Psychometrika, 34(4), 129–301.

Satterthwaite, F. E. (1941). Synthesis of variance. Psychometrika, 6, 309–316.

Shaffer, J. P. (1994). Multiple hypothesis testing: A review (Tech. Rep. No. 23). Research

Triangle Park, NC: National Institute of Statistical Sciences.

Shealy, R., & Stout, W. (1993). A model-based standardization approach that separates true

bias/DIF from group ability differences and detects test bias/DIF as well as item bias/DIF. Psychometrika,

58, 159–194.

Sheehan, K. M. (1985). MGROUP: Estimation of group effects in multivariate models

[Computer program]. Princeton, NJ: Educational Testing Service.

Smith, R. L., & Smith, J. K. (1988). Differential use of item information by judges using Angoff

and Nedelsky procedures. Journal of Educational Measurement, 25, 259–274.

Stone, C. A., Ankenmann, R. D., Lane, S., & Liu, M. (1993, April). Scaling QUASAR’s

performance assessments. Paper presented at the annual meeting of the American Educational Research

Association, Atlanta, GA.
930

Stone, C. A., Mislevy, R. J., & Mazzeo, J. (1994, April). Misclassification error and goodnessof-fit in IRT models. Paper presented at the annual meeting of the American Educational Research

Association, New Orleans, LA.

Tanner, M., & Wong, W. (1987). The calculation of posterior distributions by data augmentation

(with discussion). Journal of the American Statistical Association, 82, 528–550.

Thomas, N. (1993a). Asymptotic corrections for multivariate posterior moments with factored

likelihood functions. Journal of Computational and Graphical Statistics, 2, 309–322.

Thomas, N. (1993b). The E-step of the MGROUP EM algorithm. (Program Statistics Research,

ETS Research Report RR-95-05). Princeton, NJ: Educational Testing Service.

Thomas, N. (1994). CGROUP and BGROUP: Modifications of the MGROUP program to

estimate group effects in multivariate models [Computer programs]. Princeton, NJ: Educational Testing

Service.

Tukey, J. W. (1977). Exploratory data analysis. Reading, MA: Addison-Wesley Publishing

Company.

Wainer, H. (1974). The suspended rootogram and other visual displays: An empirical validation.

American Statistician, 28(4), 143–145.

Wallace, L., & Rust, K. F. (1999). Sample design. In N. L Allen, J. E. Carlson, & C. A. Zelenak.

The NAEP 1996 technical report (NCES 1999–452). Washington, DC: National Center for Education

Statistics.

Weiss, A. R., Lutkus A. D., Grigg, W. S., & Niemi, R. G. (2000). The next generation of

citizens: NAEP trends in civics, 1988 to 1998 (NCES 2000-494). Washington, DC: National Center for

Education Statistics.

Westat. (1998). Report on data collection activities for all states. Rockville, MD: Author.

Williams, V. S. L., Jones, L. V., & Tukey, J. W. (1999). Controlling error in multiple

comparisons, with examples from state-to-state differences in educational achievement. Journal of

Educational and Behavioral Statistics, 24(1), 42–69.

Wingersky, M., Kaplan, B. A., & Beaton, A. E. (1987). Joint estimation procedures. In A. E.

Beaton, Implementing the new design: The NAEP 1983-84 technical report (pp. 285–292). (No. 15–

TR–20) Princeton, NJ: Educational Testing Service, National Assessment of Educational Progress.

Wolter, K. M. (1985). Introduction to variance estimation. New York, NY: John Wiley & Sons.

Yamamoto, K. (1988). Science data analysis. In A. E. Beaton, Expanding the new design: The

NAEP 1985-86 technical report (pp. 243–255). (No. 17–TR–20) Princeton, NJ: Educational Testing

Service, National Assessment of Educational Progress.

Yamamoto, K., & Mazzeo, J. (1992). Item response theory scale linking in NAEP. Journal of

Educational Statistics, 17(2), 155–173.

Yen, W. M. (1993). Scaling performance assessments: Strategies for managing local item

dependence. Journal of Educational Measurement, 30, 187-213.
931

Zhang, J. & Stout, W. (1999). The theoretical DETECT index of dimensionality and its

application to approximate simple structure. Psychometrika, 64, 213-249.

Zieky, M. (1993). Practical questions in the use of DIF statistics. In P. W. Holland & H. Wainer

(Eds.), Differential item functioning. Hillsdale, NJ: Lawrence Erlbaum Associates.

Zwick, R. (1986). Assessment of the dimensionality of NAEP year 15 reading data. (ETS

Research Report No. 86–4). Princeton, NJ: Educational Testing Service.

Zwick, R. (1987). Assessment of the dimensionality of NAEP year 15 reading data. Journal of

Educational Measurement, 24(4), 293–308.

Zwick, R. (1988). Professionally scored items. Technical memorandum.

Zwick, R., Donoghue, J. R., & Grima, A. (1993). Assessment of differential item functioning for

performance tasks. Journal of Educational Measurement, 30, 233–251.

Zwick, R., & Grima, A. (1991). Policy for differential item functioning (DIF) analysis in NAEP.

Technical memorandum.

Zwinderman, A. H. (1991). Logistic regression Rasch models. Psychometrika, 56, 589–600.
Standard Mail (B)

Postage and Fees Paid

U.S. Department of Education

Permit No. G-17

United States

Department of Education

ED Pubs

8242-B Sandy Court

Jessup, MD 20794-1398

Official Business Only

Penalty for Private Use, $300