U.S. Department of Education 

Institute of Education Sciences 

NCES 2006-037 

Early Childhood 

Longitudinal Study, 

Kindergarten Class of 

1998–99 (ECLS-K) 

Fifth-Grade 

Methodology Report 

December 2005 

Karen Tourangeau 

Thanh Lê 

Christine Nord 

Westat 

Elvira Germino Hausken 

Project Officer 

National Center for 

Education Statistics 
U.S. Department of Education

Margaret Spellings 

Secretary

Institute of Education Sciences 

Grover J. Whitehurst 

Director

National Center for Education Statistics

Mark Schneider 

Commissioner

The National Center for Education Statistics (NCES) is the primary federal entity for collecting, analyzing, and 

reporting data related to education in the United States and other nations. It fulfills a congressional mandate to 

collect, collate, analyze, and report full and complete statistics on the condition of education in the United States; 

conduct and publish reports and specialized analyses of the meaning and significance of such statistics; assist state 

and local education agencies in improving their statistical systems; and review and report on education activities in 

foreign countries. 

NCES activities are designed to address high-priority education data needs; provide consistent, reliable, complete, 

and accurate indicators of education status and trends; and report timely, useful, and high-quality data to the U.S. 

Department of Education, the Congress, the states, other education policymakers, practitioners, data users, and the 

general public. Unless specifically noted, all information contained herein is in the public domain. 

We strive to make our products available in a variety of formats and in language that is appropriate to a variety of 

audiences. You, as our customer, are the best judge of our success in communicating information effectively. If you 

have any comments or suggestions about this or any other NCES product or report, we would like to hear from you. 

Please direct your comments to: 

 National Center for Education Statistics 

 Institute of Education Sciences 

 U.S. Department of Education 

 1990 K Street NW 

 Washington, DC 20006-5651 

December 2005 

The NCES World Wide Web Home Page address is http://nces.ed.gov. 

The NCES World Wide Web Electronic Catalog is http://nces.ed.gov/pubsearch. 

This publication is only available online. To download, view, and print the report as a PDF file, go to the NCES World 

Wide Web Electronic address shown above. 

Suggested Citation 

Tourangeau, K., Lê, T., and Nord, C. (2005). Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 

(ECLS–K), Fifth-Grade Methodology Report (NCES 2006–037). U.S. Department of Education. Washington, DC: 

National Center for Education Statistics. 

Content Contact: 

Elvira Germino Hausken 

(202) 502-7352 

elvira.hausken@ed.gov

iii 

TABLE OF CONTENTS 

Chapter Page

1 INTRODUCTION ........................................................................................... 1-1 

1.1 Background......................................................................................... 1-4 

1.2 Conceptual Model............................................................................... 1-5 

1.3 Study Components.............................................................................. 1-5 

1.4 Contents of Report.............................................................................. 1-7 

2 DEVELOPMENT OF SURVEY INSTRUMENTS........................................ 2-1 

2.1 Review Panels..................................................................................... 2-2 

2.1.1 Technical Review Panel...................................................... 2-2 

2.1.2 Content Review Panel ......................................................... 2-3 

2.2 Modifications to the Parent Interview................................................. 2-3 

2.3 Timing Study ...................................................................................... 2-6 

2.4 Child Food Consumption Questions................................................... 2-8 

2.5 Modifications to Teacher Questionnaires ........................................... 2-9 

2.6 Modifications to School Administrator Questionnaire ....................... 2-11 

3 SAMPLE DESIGN AND IMPLEMENTATION............................................ 3-1 

3.1 Precision Requirements and Achieved Sample Sizes......................... 3-3 

3.2 Base Year Sample............................................................................... 3-6 

3.2.1 Selecting the Area Sample .................................................. 3-7 

3.2.2 Selecting the School Sample............................................... 3-11 

3.2.2.1 School Frame Construction ................................. 3-11 

3.2.2.2 School Measure of Size ....................................... 3-12 

3.2.2.3 School Allocation ................................................ 3-13 

3.2.2.3.1 Public Schools.................................... 3-13 

3.2.2.3.2 Private Schools .................................. 3-14 

3.2.2.4 Clustering of Small Schools ................................ 3-16 

3.2.2.4.1 Public Schools.................................... 3-16 

3.2.2.4.2 Private Schools .................................. 3-17 
iv 

TABLE OF CONTENTS (continued) 

Chapter Page

3.2.2.5 Implicit Stratification of Schools/Clusters 

of Schools ............................................................ 3-18 

3.2.2.5.1 Public Schools.................................... 3-19 

3.2.2.5.2 Private Schools .................................. 3-19 

3.2.2.6 School Selection .................................................. 3-19 

3.2.2.7 The ECLS-K Main School Sample ..................... 3-20 

3.2.2.8 Supplemental School Sample .............................. 3-20 

3.2.2.8.1 Public Schools.................................... 3-20 

3.2.2.8.2 Private Schools .................................. 3-22 

3.2.2.8.3 Non-Catholic Private Schools............ 3-22 

3.2.3 Sampling Children, Parents, and Teachers 

Within Schools.................................................................... 3-24 

3.3 Fall-First Grade Subsample ................................................................ 3-24 

3.3.1 PSU Sample ........................................................................ 3-25 

3.3.2 School Sample..................................................................... 3-25 

3.3.3 Child Sample....................................................................... 3-25 

3.4 Spring-First Grade Sample ................................................................. 3-27 

3.4.1 Subsampling Movers........................................................... 3-27 

3.4.2 Sample Freshening.............................................................. 3-28 

3.5 Spring-Third Grade Sample................................................................ 3-29 

3.5.1 Estimates from Spring-First Grade ..................................... 3-29 

3.5.2 Third-Grade Sample Design ............................................... 3-32 

3.5.3 Expected Sample Size ......................................................... 3-33 

3.5.4 Protecting the Language Minority Children........................ 3-36 

3.5.5 Precision Requirements....................................................... 3-38 

3.5.6 Spring-Third Grade Sampling Outcome ............................. 3-39 

3.6 Spring-Fifth Grade Sample................................................................. 3-44 

3.6.1 Options for Subsampling Movers ....................................... 3-44 

3.6.2 Expected Sample Size ......................................................... 3-48 

3.6.3 Sample Outcome and Precision Requirements ................... 3-48 
v 

TABLE OF CONTENTS (continued) 

Chapter Page

4 DATA COLLECTION METHODS ................................................................ 4-1 

4.1 Overview of Data Collection Methods ............................................... 4-1 

4.2 Roles and Responsibilities in the ECLS-K Study............................... 4-2 

4.2.1 School’s Role ...................................................................... 4-2 

4.2.2 School Coordinator’s Role.................................................. 4-3 

4.2.3 Supervisor’s Role................................................................ 4-3 

4.2.4 Assessor’s Role ................................................................... 4-4 

4.2.5 Field Manager’s Role.......................................................... 4-4 

4.3 Field Staff Training............................................................................. 4-5 

4.3.1 Advance Contact and Recruitment Training....................... 4-6 

4.3.2 Spring-Fifth Grade Training ............................................... 4-7 

4.3.2.1 Trainers’ Training and Certification.................... 4-7 

4.3.2.2 Certification of the Child Assessors .................... 4-19 

4.3.2.3 Parent Interview Training and Certification........ 4-32 

4.4 Fall Preassessment School Contact..................................................... 4-33 

4.4.1 Advance Mailings ............................................................... 4-34 

4.4.2 Fall Preassessment School Coordinator Contact................. 4-35 

4.4.3 Fall Preassessment Contact Results .................................... 4-40 

4.4.4 Tracing Activities during Fifth-Grade Data Collection ...... 4-42 

4.5 Spring-Fifth Grade Data Collection.................................................... 4-46 

4.5.1 Spring Preassessment Activities.......................................... 4-47 

4.5.2 Conducting the Direct Child Assessments.......................... 4-47 

4.5.3 Conducting the Parent Interview......................................... 4-50 

4.5.4 Conducting Data Collection on Children Who 

Withdrew From Their Previous Round School................... 4-53 

4.5.5 Teacher and School Data Collection................................... 4-57 

4.5.5.1 Hard-Copy Data Retrieval................................... 4-59 

4.5.6 Incentives in the ECLS-K ................................................... 4-61 
vi 

TABLE OF CONTENTS (continued) 

Chapter Page

4.6 Data Collection Quality Control ......................................................... 4-64 

4.6.1 Maintaining Reliability on the Child Assessment............... 4-64 

4.6.1.1 Assessment Observation Form ............................ 4-65 

4.6.1.2 Assessor Interrater Reliability ............................. 4-72 

4.6.2 Validation of Parent Interviews........................................... 4-74 

4.6.3 Validations of School Visits................................................ 4-74 

5 DATA PREPARATION AND EDITING ....................................................... 5-1 

5.1 Coding and Editing Specifications for Computer-Assisted 

Interviews (CAI)................................................................................. 5-1 

5.1.1 Range Specifications........................................................... 5-1 

5.1.2 Logical Consistency Checks (Logical Edits) ...................... 5-2 

5.1.3 Coding................................................................................. 5-3 

5.1.4 Editing the Household Roster in the Parent Interview........ 5-10 

5.2 Coding and Editing Specifications for Hard-Copy Questionnaires .... 5-11 

5.2.1 Receipt Control ................................................................... 5-11 

5.2.2 Scan Edit Procedures........................................................... 5-12 

5.2.3 Data Entry ........................................................................... 5-14 

5.2.4 Coding................................................................................. 5-15 

5.3 Data Editing Management Process ..................................................... 5-15 

5.3.1 Collection Database ............................................................ 5-17 

5.3.2 Holding Database................................................................ 5-17 

5.3.3 Editing Database ................................................................. 5-18 

5.3.4 Delivery Database ............................................................... 5-19 

5.4 Data Editing ........................................................................................ 5-20 

5.4.1 Range Specifications........................................................... 5-20 

5.4.2 Consistency Checks (Logical Edits) ................................... 5-20 

5.4.3 Teacher Responses to Key Child Items............................... 5-21 

5.4.4 Frequency and Cross-Tabulation Review ........................... 5-22 

5.5 Creation of the Socioeconomic Status (SES) Variable....................... 5-23 

5.6 Imputation of the School Lunch Composites...................................... 5-39 
vii 

TABLE OF CONTENTS (continued) 

Chapter Page

6 RESPONSE RATES........................................................................................ 6-1 

6.1 Definition of Response and Completion Rates................................... 6-1 

6.2 Completion Rates................................................................................ 6-3 

6.2.1 Children Sampled in Kindergarten...................................... 6-4 

6.2.2 Children Sampled in First Grade......................................... 6-25 

6.2.3 Spring-Fifth Grade Completion Rates for All Children...... 6-28 

6.3 Unit Nonresponse Bias Analysis ........................................................ 6-33 

6.4 Item Response Rates........................................................................... 6-36 

7 WEIGHTING AND VARIANCE ESTIMATION.......................................... 7-1 

7.1 Types of Weights................................................................................ 7-1 

7.2 Computation of the Fifth-Grade Weights ........................................... 7-5 

7.2.1 Initial Child Weights........................................................... 7-7 

7.2.1.1 Base Year Nonresponse-Adjusted School 

Weights................................................................ 7-8 

7.2.1.2 Base Year Child Weights .................................... 7-11 

7.2.1.2.1 Base Year Child Weights for 

 Base Year Respondents...................... 7-11 

7.2.1.2.2 Base Year Child Weights for 

 Children Sampled in First Grade ....... 7-12 

7.2.2 Adjustment for Movers Between the Base Year and 

Fifth Grade .......................................................................... 7-17 

7.2.3 Adjustment for Fifth-Grade Nonresponse........................... 7-19 

7.2.4 Raking to Sample-Based Control Totals............................. 7-21 

7.2.5 Additional Adjustment for Child-Parent-Teacher 

Cross-Sectional Weights ..................................................... 7-22 

7.2.6 Replicate Weights ............................................................... 7-25 

7.2.6.1 Replicate Weights for Samples Not Involving 

Fall-First Grade ................................................... 7-26 

7.2.6.2 Replicate Weights for Samples Involving 

Fall-First Grade ................................................... 7-27 
viii 

TABLE OF CONTENTS (continued) 

Chapter Page

7.3 Variance Estimation............................................................................ 7-28 

7.3.1 Jackknife Method................................................................ 7-28 

7.3.2 Taylor Series Method.......................................................... 7-28 

 REFERENCES ................................................................................................ R-1 

List of Appendixes 

Appendix

A Mover Adjustment Cells.................................................................................. A-1 

B Unknown Eligibility and Nonresponse Adjustment Cells ............................... B-1 

C Raking Dimensions.......................................................................................... C-1 
ix 

TABLE OF CONTENTS (continued) 

List of Tables 

Table Page

2-1 Interviewer timings for the revised ECLS-K third-grade parent 

 interview, by interviewer and by section: School year 2002–03 ..................... 2-7 

3-1 ECLS-K sample size from the base year through fifth grade: 

 School years 1998–99, 1999–2000, 2001–02, and 2003–04 ........................... 3-6 

3-2 Stratum definitions for the 38 non-self-representing strata: 

 School year 1998–99 ....................................................................................... 3-9 

3-3 Distribution of the ECLS-K primary sampling unit (PSU) sample by 

 self-representing (SR) status, metropolitan statistical area (MSA) status, 

 and census region: School year 1998–99......................................................... 3-11 

3-4 Estimates of the number of kindergarten schools and students, by primary 

 sampling unit (PSU) status: School year 1998–99........................................... 3-12 

3-5 Expected number of clusters, schools, and students—public schools: 

 School year 1998–99 ....................................................................................... 3-14 

3-6 Expected number of clusters, schools, and students—private schools: 

 School year 1998–99 ....................................................................................... 3-16 

3-7 Number of clusters and schools in the public school frame: 

 School year 1998–99 ....................................................................................... 3-17 

3-8 Number of clusters and schools in the private school frame: 

 School year 1998–99 ....................................................................................... 3-18 

3-9 Number of sample schools, by region and other characteristics: 

 School year 1998–99 ....................................................................................... 3-21 

3-10 Characteristics of base year cooperating schools selected for 

 fall-first grade: School year 1999–2000 .......................................................... 3-26 

3-11 Spring-first grade data collection results by mover status: 

 School year 1999–2000 ................................................................................... 3-30 

3-12 Expected sample size for selected subgroups for third grade, 

 by mover status: School year 2001–02 ............................................................ 3-34 
x 

TABLE OF CONTENTS (continued) 

List of Tables (continued) 

Table Page

3-13 Expected sample size for third grade, by mover status, if language 

 minority children are preserved: School year 2001–02 ................................... 3-37 

3-14 Characteristics of third-grade respondents—number of third-graders 

 by subgroup: School year 2001–02.................................................................. 3-39 

3-15 Characteristics of third-grade respondents—percent distribution by 

 subgroup: School year 2001–02....................................................................... 3-41 

3-16 Spring-third grade data collection results by mover status: School year 

 2001–02 ........................................................................................................... 3-43 

3-17 Number of children eligible after the base year but excluded from the 

 fifth-grade data collection: School year 2003–04............................................ 3-45 

3-18 ECLS-K options for subsampling movers for fifth grade: School year 

 2003–04 ........................................................................................................... 3-47 

3-19 Expected sample size for selected subgroups for fifth grade, by mover 

 status: School year 2003–04 ............................................................................ 3-48 

3-20 Characteristics of fifth-grade respondents—number of fifth-graders by 

 subgroup: School year 2003–04....................................................................... 3-50 

3-21 Characteristics of fifth-grade respondents—percent distribution by 

 subgroup: School year 2003–04....................................................................... 3-51 

3-22 Spring-fifth-grade data collection results by mover status: School year 

 2003–04 ........................................................................................................... 3-53 

4-1 Results of child assessment certification, fifth grade: School year 2003–04... 4-31 

4-2 Results of parent interview certification, fifth grade: School year 2003–04 ... 4-33 

4-3 Status of movers identified during fall preassessment contact: School 

 year 2003–04.................................................................................................... 4-41 

4-4 Fall preassessment contact productivity report, by week: School year 

 2003–04 ........................................................................................................... 4-42 
xi 

TABLE OF CONTENTS (continued) 

List of Tables (continued) 

Table Page

4-5 Fall preassessment contact production report—original schools: School 

 year 2003–04.................................................................................................... 4-43 

4-6 Fall preassessment contact production report—transfer schools, by 

 week: School year 2003–04............................................................................. 4-44 

4-7 Tracing results for spring 2003........................................................................ 4-45 

4-8 Results of the Telephone Research Center’s locating efforts for cases 

 with a status of unlocatable at the beginning of fifth grade: School 

 year 2003–04.................................................................................................... 4-45 

4-9 Results of the fall household tracing contact: School year 2003–04 .............. 4-46 

4-10 Weekly completion of the child assessments: School year 2003–04............... 4-48 

4-11 Completed child assessments by round of data collection: School 

 years 1998–99, 1999–2000, 2001–02, and 2003–04 ....................................... 4-49 

4-12 Number of children excluded from or accommodated in the spring-fifth 

 grade assessments: School year 2003–04 ........................................................ 4-50 

4-13 Weekly completion of parent interviews, spring-fifth grade: School year 

 2003–04 ........................................................................................................... 4-51 

4-14 Number and percent of completed parent interviews by data collection 

 mode, language, and wave of data collection: School years 1998–99, 

 1999–2000, 2001–02, and 2003–04................................................................. 4-52 

4-15 Number of children who moved in spring-fifth grade by completion 

 category: School year 2003–04........................................................................ 4-54 

4-16 Number of children who moved by completion category: School years 

 1998–99, 1999–2000, 2001–02 and 2003–04.................................................. 4-56 

4-17 Children moving in fifth grade: School year 2003–04 .................................... 4-57 

4-18 Number of child-level questionnaires and percent of teachers: School 

 year 2003–04.................................................................................................... 4-58 
xii 

TABLE OF CONTENTS (continued) 

List of Tables (continued) 

Table Page

4-19 Total number and percent of the final results of critical item retrieval: 

 School year 2003–04 ....................................................................................... 4-62 

4-20 Results of the child assessment observations, fifth-grade data collection: 

 School year 2003–04 ....................................................................................... 4-72 

4-21 Interrater reliability on child assessment validation items: School year 

 2003–04 ........................................................................................................... 4-73 

4-22 Results of parent interview validations: School year 2003–04........................ 4-75 

4-23 Results of school validation calls: School year 2003–04................................. 4-76 

5-1 “Other, specify” codes added during fifth grade: School year 2003–04.......... 5-3 

5-2 “Other, specify” items added to the coding system: School year 2003–04 ..... 5-4 

5-3 Number of text strings by “Other, specify” item, fifth-grade parent 

 interview: School year 2003–04 ...................................................................... 5-4 

5-4 Number and percent of occupations coded, by coding status: School 

 year 2003–04.................................................................................................... 5-9 

5-5 Data editing of the parent interview household roster, fifth-grade data 

 collection: School year 2003–04...................................................................... 5-10 

5-6 Number of questionnaires receipted by week, fifth-grade data collection: 

 School year 2003–04 ....................................................................................... 5-14 

5-7 Hard-copy editing progress report, fifth-grade data collection: School 

 year 2003–04.................................................................................................... 5-21 

5-8 Recode of occupation values to 1989 GSS prestige scores, spring-fifth 

 grade: School year 2003–04............................................................................. 5-24 

5-9 Parents or households with missing value of the SES components, 

 spring-fifth grade: School year 2003–04 ......................................................... 5-25 

5-10 Components of the SES variable, spring-fifth grade: School year 

 2003–04 ........................................................................................................... 5-26 
xiii 

TABLE OF CONTENTS (continued) 

List of Tables (continued) 

Table Page

5-11 Demographic characteristics used in creating imputation cells, spring- 

 fifth grade: School year 2003–04..................................................................... 5-28 

5-12 Imputation of SES components, spring-fifth grade: School year 2003–04 ..... 5-31 

5-13 Mother’s education, before and after imputation, spring-fifth grade: 

 School year 2003–04 ....................................................................................... 5-32 

5-14 Father’s education, before and after imputation, spring-fifth grade: 

 School year 2003–04 ....................................................................................... 5-32 

5-15 Mother’s labor force status, before and after imputation, spring-fifth 

 grade: School year 2003–04............................................................................. 5-33 

5-16 Father’s labor force status, before and after imputation, spring-fifth 

 grade: School year 2003–04............................................................................. 5-33 

5-17 Mother’s occupation, before and after imputation, spring-fifth grade: 

 School year 2003–04 ....................................................................................... 5-34 

5-18 Father’s occupation, before and after imputation, spring-fifth grade: 

 School year 2003–04 ....................................................................................... 5-35 

5-19 Household income, before and after imputation, spring-fifth grade: 

 School year 2003–04 ....................................................................................... 5-36 

5-20 Distribution of socioeconomic status (SES) values, spring-fifth grade: 

 School year 2003–04 ....................................................................................... 5-38 

5-21 Unweighted frequencies of the socioeconomic status (SES) 

 variable, spring-fifth grade: School year 2003–04........................................... 5-38 

5-22 Public schools and child-parent respondents with missing value of the 

 school lunch composites, spring-fifth grade: School year 2003–04 ................ 5-39 

5-23 Imputation of school lunch composites at the school level, spring-fifth 

 grade: School year 2003–04............................................................................. 5-41 

5-24 Imputation of school lunch composites at the child level, spring-fifth 

 grade: School year 2003–04............................................................................ 5-41 
xiv 

TABLE OF CONTENTS (continued) 

List of Tables (continued) 

Table Page

5-25 Free lunch composite at the school level, before and after imputation, 

 spring-fifth grade: School year 2003–04 ......................................................... 5-42 

5-26 Free lunch composite at the child level, before and after imputation, 

 spring-fifth grade: School year 2003–04 ......................................................... 5-43 

5-27 Reduced-price lunch composite at the school level, before and after 

 imputation, spring-fifth grade: School year 2003–04 ...................................... 5-44 

5-28 Reduced-price lunch composite at the child level, before and after 

 imputation, spring-fifth grade: School year 2003–04 ...................................... 5-45 

6-1 Number of completed child-level cases and child-level completion rates 

 for the child assessment and parent interview for children sampled in the 

 base year, by school characteristics: School year 2003–04 ............................. 6-5 

6-2 Number of completed child-level cases and child-level completion rates 

 for the school administrator questionnaire and facilities checklist for children 

 sampled in the base year, by school characteristics: School year 2003–04..... 6-8 

6-3 Number of completed child-level cases and child-level completion rates for 

 the student records abstract and teacher-level questionnaire for children 

 sampled in the base year, by school characteristics: School year 2003–04..... 6-11 

6-4 Number of completed child-level cases and child-level completion rates 

 for the child-level teacher questionnaires for children sampled in the base 

 year, by school characteristics: School year 2003–04 ..................................... 6-13 

6-5 Number of completed child-level cases and child-level completion rates 

 for the child assessment and parent interview for children sampled in the 

 base year, by child’s mover status: School year 2003–04 ............................... 6-15 

6-6 Number of completed child-level cases and child-level completion rates 

 for the school administrator questionnaire and facilities checklist for 

 children sampled in the base year, by child’s mover status: 

 School year 2003–04 ....................................................................................... 6-17 

6-7 Number of completed child-level cases and child-level completion rates 

 for the student records abstract and teacher-level questionnaire for 

 children sampled in the base year, by child’s mover status: 

 School year 2003–04 ....................................................................................... 6-18 
xv 

TABLE OF CONTENTS (continued) 

List of Tables (continued) 

Table Page

6-8 Number of completed child-level cases and child-level completion rates 

 for the child-level teacher questionnaires for children sampled in the base 

 year, by child’s mover status: School year 2003–04 ....................................... 6-19 

6-9 Number of completed child-level cases and child-level completion rates 

 for the child assessment and parent interview for children sampled in the 

 base year, by child characteristics: School year 2003–04................................ 6-21 

6-10 Number of completed child-level cases and child-level completion rates 

 for the school administrator questionnaire and facilities checklist for 

 children sampled in the base year, by child characteristics: 

 School year 2003–04 ....................................................................................... 6-22 

6-11 Number of completed child-level cases and child-level completion rates 

 for the student records abstract and teacher-level questionnaire for children 

 sampled in the base year, by child characteristics: School year 2003–04 ....... 6-23 

6-12 Number of completed child-level cases and child-level completion rates 

 for the child-level teacher questionnaires for children sampled in the base 

 year, by child characteristics: School year 2003–04........................................ 6-24 

6-13 Number of completed instruments and child-level completion rates for 

 the special education teacher questionnaires for children sampled in the 

 base year: School year 2003–04 ...................................................................... 6-25 

6-14 Number of completed child-level cases and child-level completion rates 

 for children sampled in first grade: School year 2003–04 ............................... 6-27 

6-15 Number of completed child-level cases and child-level completion rates, 

 for children sampled in kindergarten and first grade, by survey instruments: 

 School year 2003–04 ....................................................................................... 6-30 

6-16 Number of completed child-level cases and child-level completion rates, 

 for children with scorable reading, mathematics or science assessment or children 

 not assessed due to disabilities, by survey instruments: School year 2003–04 6-32 

6-17 Number of completed child-level cases and child-level completion rates, 

 if excluded children were fielded, by survey instruments: 

 School year 2003–04 ....................................................................................... 6-34 
xvi 

TABLE OF CONTENTS (continued) 

List of Tables (continued) 

Table Page

6-18 Number of survey items and percent of nonresponse values: School year 

 2003–04 ........................................................................................................... 6-41 

6-19 Item response rates for items on the ECLS-K fifth-grade restricted-use 

 data file: School year 2003–04 ........................................................................ 6-42 

6-20 Items on the ECLS-K fifth-grade restricted-use data file with less than 

 85 percent response rates: School year 2003–04 ............................................. 6-43 

7-1 Size classification for school nonresponse adjustment: School year 1998–99 7-10 

7-2 Groups of children defined for mover adjustments: School year 1999–2000 . 7-16 

7-3 Groups of children defined for nonresponse adjustments: School year 

 2003–04 ........................................................................................................... 7-19 

7-4 Number of children with direct child assessment, parent interview, and 

 child-level data from mathematics teacher, by number of teachers who filled out 

 teacher-level questionnaire: School year 2003–04 .......................................... 7-23 

7-5 Number of children with direct child assessment, parent interview, and 

 child-level data from science teacher, by number of teachers who filled out 

 teacher-level questionnaire: School year 2003–04 .......................................... 7-24 
xvii 

TABLE OF CONTENTS (continued) 

Appendix Tables 

Table Page

A-1 Mover adjustment cells for cross-sectional weights C6CW0, C6PW0 

 and C6CPTR0: School year 2003–04.............................................................. A-1 

A-2 Mover adjustment cells for cross-sectional weight C6CPTM0: School 

 year 2003–04.................................................................................................... A-1 

A-3 Mover adjustment cells for cross-sectional weight C6CPTS0: School 

 year 2003–04.................................................................................................... A-1 

A-4 Mover adjustment cells for longitudinal weights not involving fall-first 

 grade: School year 2003–04............................................................................ A-2 

A-5 Mover adjustment cells for longitudinal weights involving fall-first 

 grade: School year 2003–04............................................................................. A-2 

B-1 Unknown eligibility and nonresponse adjustment cells for C6CW0: 

 School year 2003–04 ....................................................................................... B-3 

B-2 Unknown eligibility and nonresponse adjustment cells for C6PW0: School 

 year 2003–04.................................................................................................... B-3 

B-3 Unknown eligibility and nonresponse adjustment cells for C6CPTR0: 

 School year 2003–04 ....................................................................................... B-4 

B-4 Unknown eligibility and nonresponse adjustment cells for C6CPTM0: 

 School year 2003–04 ....................................................................................... B-4 

B-5 Unknown eligibility and nonresponse adjustment cells for C6CPTS0: 

 School year 2003–04 ....................................................................................... B-5 

B-6 Unknown eligibility and nonresponse adjustment cells for C56CW0: 

 School year 2003–04 ....................................................................................... B-6 

B-7 Unknown eligibility and nonresponse adjustment cells for C56PW0: 

 School year 2003–04 ....................................................................................... B-6 

B-8 Unknown eligibility and nonresponse adjustment cells for C456CW0: 

 School year 2003–04 ....................................................................................... B-6 

B-9 Unknown eligibility and nonresponse adjustment cells for C456PW0: 

 School year 2003–04 ....................................................................................... B-7 
xviii 

TABLE OF CONTENTS (continued) 

Appendix Tables (continued) 

Table Page

B-10 Unknown eligibility and nonresponse adjustment cells for C2_6FC0: 

 School year 2003–04 ....................................................................................... B-8 

B-11 Unknown eligibility and nonresponse adjustment cells for C2_6FP0: 

 School year 2003–04 ....................................................................................... B-8 

B-12 Unknown eligibility and nonresponse adjustment cells for C1_6FC0: 

 School year 2003–04 ....................................................................................... B-9 

B-13 Unknown eligibility and nonresponse adjustment cells for C1_6FP0: 

 School year 2003–04 ....................................................................................... B-9 

B-14 Unknown eligibility and nonresponse adjustment cells for C1_6SC0: 

 School year 2003–04 ....................................................................................... B-10 

B-15 Unknown eligibility and nonresponse adjustment cells for C1_6SP0: 

 School year 2003–04 ....................................................................................... B-10 

C-1 Raking dimension 1—sex by age: School year 2003–04 ................................ C-1 

C-2 Raking dimension 2—region by locale: School year 2003–04........................ C-1 

C-3 Raking dimension 3—race/ethnicity by SES quintile: School year 2003–04.. C-2 

C-4 Raking dimension 4—school affiliation: School year 2003–04 ...................... C-2 

C-5. Raking dimension 5—language minority status: School year 2003–04 .......... C-3 

C-6 Raking dimension 6—mover status in spring-first grade: School year 

 2003–04 ........................................................................................................... C-3 
xix 

TABLE OF CONTENTS (continued) 

List of Exhibits 

Exhibit Page

1-1 ECLS-K waves of data collection: School years 1998–99, 1999–2000, 

 2001–02, and 2003–04..................................................................................... 1-2 

1-2 ECLS-K conceptual model: School years 1998–2004..................................... 1-6 

1-3 Instruments used in the ECLS-K, by round of data collection: School 

 years 1998–99, 1999–2000, 2001–02, and 2003–04 ...................................... 1-9 

1-4 Direct child assessments, by domain and round of data collection: 

 School years 1998–99, 1999–2000, 2001–02, and 2003–04 ........................... 1-10 

2-1 Sections comprising the spring-fifth grade parent interview: 2003–04........... 2-4 

4-1 Timeline of fifth-grade data collection: School year 2003–04 ........................ 4-2 

4-2 Fifth-grade field organization .......................................................................... 4-5 

4-3 Spring-fifth grade Trainer Training Agenda: School year 2003–04................ 4-8 

4-4 Spring-fifth grade field supervisor training agenda: School year 2003–04 ..... 4-11 

4-5 Spring-fifth grade assessor training: School year 2003–04 ............................. 4-13 

4-6 Section 1 of the Assessment Certification Form: School year 2003–04.......... 4-21 

4-7 Instructions for Section 2 of the Assessment Certification Form: School 

 year 2003–04.................................................................................................... 4-22 

4-8 Quality review screen: School year 2003–04 .................................................. 4-23 

4-9 Sample question from Assessment Certification Form: School year 

 2003–04 ........................................................................................................... 4-24 

4-10 Certification Scoring Form: School year 2003–04 .......................................... 4-26 

4-11 Example of Form A for Trainee 1: School year 2003–04................................ 4-27 

4-12 Total Possible Points Chart: School year 2003–04.......................................... 4-28 

4-13 Example of Form B: School year 2003–04...................................................... 4-29 
xx 

TABLE OF CONTENTS (continued) 

List of Exhibits (continued) 

Exhibit Page

4-14 Proportion Correct Chart: School year 2003–04.............................................. 4-30 

4-15 Example of Proportion Correct Chart: School year 2003–04 .......................... 4-31 

4-16 Certification Scoring Form: School year 2003–04 .......................................... 4-32 

4-17 Child Work Grid, spring-fifth grade: School year 2003–04 ............................ 4-36 

4-18 Individualized Education Program (IEP) process, spring fifth-grade: 

 School year 2003–04 ....................................................................................... 4-39 

4-19 Critical items by questionnaire type: School year 2003–04 ........................... 4-60 

4-20 Types of incentives used in the ECLS-K: School year 2003–04..................... 4-63 

4-21 Section 1 of the Assessment Certification Form: School year 2003–04.......... 4-65 

4-22 Instructions for Section 2 of the Assessment Observation Form: School 

 year 2003–02.................................................................................................... 4-66 

4-23 Sample question from Assessment Certification Form: School year 

 2003–04 ........................................................................................................... 4-67 

4-24 Observation Scoring Form: School year 2003–04 .......................................... 4-68 

4-25 Example of Form A: School year 2003–04 .................................................... 4-69 

4-26 Example of Form B: School year 2003–04...................................................... 4-70 

4-27 Proportion Correct Chart: School year 2003–04.............................................. 4-71 

4-28 Example of Proportion Correct Chart: School year 2003–04 .......................... 4-72 

5-1 Aggregated occupation coding categories in the ECLS-K: School 

 years 1998–99, 2001–02, and 2003–04 ........................................................... 5-5 

5-2 Grade 5 critical items for questionnaires: School year 2003–04 ..................... 5-13 
xxi 

TABLE OF CONTENTS (continued) 

List of Exhibits (continued) 

Exhibit Page

5-3 ECLS-K hard-copy data editing, fifth-grade data collection: School 

 year 2003–04.................................................................................................... 5-16 

5-4 Status codes assigned for data management databases, fifth-grade data 

 collection: School year 2003–04...................................................................... 5-19 

6-1 Relationship between the student record abstract response status and school 

 characteristics: School year 2003–04............................................................... 6-37 

7-1 ECLS-K fifth-grade cross-sectional weights: School year 2003–04 ............... 7-3 

7-2 ECLS-K: K–5 longitudinal weights, spring-fifth grade: School year 2003–04 7-4 

7-3 Movers and nonmovers by retention status: School year 2003–04.................. 7-18 
xxii 

This page is intentionally left blank. 
1-1 

1. INTRODUCTION 

This methodology report provides technical information about the development, design, and 

conduct of the fifth-grade1

 data collection of the Early Childhood Longitudinal Study, Kindergarten Class 

of 1998–99 (ECLS-K). It begins with an overview of the ECLS-K study. Subsequent chapters provide 

information on the development of the instruments, sample design, data collection methods, data 

preparation and editing, response rates, and weighting and variance estimation. 

The ECLS-K focuses on children’s early school experiences, beginning with kindergarten. It 

is a multisource, multimethod study that includes interviews with parents; the collection of data from 

principals, teachers, and student record abstracts; and direct child assessments. The ECLS-K was 

developed under the sponsorship of the U.S. Department of Education, National Center for Education 

Statistics (NCES) in the Institute of Education Sciences. Westat is conducting this study with assistance 

provided by Educational Testing Service (ETS) of Princeton, New Jersey. The Survey Research Center 

and the School of Education at the University of Michigan assisted Westat in conducting the base year 

and first-grade studies. 

The ECLS-K follows a nationally representative cohort of children from kindergarten into 

high school. The base year data were collected in the fall and spring of the 1998–99 school year when the 

sampled children were in kindergarten. A total of 21,260 kindergartners throughout the nation 

participated. 

Two more waves of data were collected in the fall and spring of the 1999–2000 school year 

when most, but not all, of the base year children were in first grade.2

 The fall-first grade data collection 

was limited to a 30 percent subsample of schools. Approximately 27 percent of the base year students 

who were eligible to participate in year 2 attended the 30 percent subsample of schools (see exhibit 1-1). 

The fall-first grade data collection was a design enhancement to enable researchers to measure the extent 

of summer learning loss and the factors that contributed to such loss and to better disentangle school and 

home effects on children’s learning. The spring-first grade data collection, including the full sample, was 

 

1

 The term “fifth grade” is used throughout this document to refer to the data collection that took place in the 2003–2004 school year, at which 

time most of the sampled children—but not all of them—were in fifth grade. 2

 Though the majority of base year children were in first grade during the 1999–2000 school year, about 5 percent of the sampled children were 

retained in kindergarten and a handful of others were in second grade during the 1999–2000 school year. 
1-2 

Exhibit 1-1. ECLS-K waves of data collection: School years 1998–99, 1999–2000, 2001–02, and 

2003–04 

Data collection Date of collection Sample 

Fall-kindergarten Fall 1998 Full sample 

Spring-kindergarten Spring 1999 Full sample 

Fall-first grade Fall 1999 30 percent subsample1 

Spring-first grade Spring 2000 Full sample 

Spring-third grade Spring 2002 Full sample 

Spring-fifth grade Spring 2004 Full sample 

1

 Fall data collection consisted of a 30 percent sample of schools containing approximately 27 percent of the base year students eligible to 

participate in year 2. 

NOTE: See section 1.3 for a description of the study components. More information is provided in the Combined User’s Manual for the ECLS-K 

Fifth-Grade Data Files and Electronic Codebooks (NCES 2006–032) (Tourangeau et al. forthcoming). 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99, spring 2004. 

part of the original study design and can be used to measure annual school progress and to describe the 

first grade learning environment of children in the study. All children assessed during the base year were 

eligible to be assessed in the spring-first grade data collection regardless of whether they repeated 

kindergarten, were promoted to first grade, or were promoted to second grade. In addition, children who 

were not in kindergarten in the United States during the 1998–99 school year and, therefore, did not have 

a chance to be selected to participate in the base year of the ECLS-K were added to the spring-first grade 

sample.3

 Such children included immigrants to the United States who arrived after fall 1998 sampling, 

children living abroad during the 1998–99 school year, children who were in first grade in 1998–99 and 

repeated it in 1999–2000, and children who did not attend kindergarten. Their addition allows researchers 

to make estimates for all first-graders in the United States rather than just for those who attended 

kindergarten in the United States in the previous year. 

A fifth wave of data was collected in the spring of the 2001–02 school year when most, but 

not all, of the sampled children were in third grade. Approximately 89 percent of the children interviewed 

were in third grade during the 2001–02 school year, 9 percent were in second grade, and less than 1 

percent were in fourth grade or higher. In addition to the school, teacher, parent, and child assessment 

data collection components, children were asked to complete a short self-description questionnaire, which 

asked them how they thought and felt about themselves, both socially and academically. The spring-third 

 

3

 The addition of these children is referred to as “freshening” the sample. See chapter 3 for more detail on the freshening process. 
1-3 

grade data collection can be used to measure school progress and to describe the third-grade learning 

environment of children in the study. 

A sixth wave of data was collected in the spring of the 2003–04 school year when most, but 

not all, of the sampled children were in fifth grade.4

 In addition to the school, teacher, parent, and child 

assessment data collection components, children were asked to complete a short self-description 

questionnaire, which asked them how they thought and felt about themselves, both socially and 

academically. They were also asked about their food consumption at school and in the week prior to the 

interview. The spring-fifth grade data collection can be used to measure school progress and to describe 

the fifth-grade learning environment of children in the study. 

The sample of children in the fifth-grade round of data collection of the ECLS-K represents 

the cohort of children who were in kindergarten in 1998–99 or in first grade in 1999–2000. Since the 

sample was not freshened after the first-grade year with third- or fifth-graders who did not have a chance 

to be sampled in kindergarten or first grade (as was done in first grade), estimates from the ECLS-K thirdand fifth-grade data are representative of the population cohort rather than all third-graders in 2001–02 or 

all fifth-graders in 2003–04. The estimated number of third-graders from the third-grade ECLS-K data 

collection is approximately 86 percent of all third-graders. From the fifth-grade ECLS-K data collection, 

the estimated number of fifth-graders is approximately 83 percent of all fifth-graders. While the vast 

majority of children in third grade in the 2001–02 school year and in fifth grade in the 2003–04 school 

year are members of the cohort, third-graders who repeated second or third grade, fifth-graders who 

repeated third or fourth grade, and recent immigrants are not covered. Data were collected from teachers 

and schools to provide important contextual information about the school environment for the sampled 

children. The teachers and schools are not representative of fifth-grade teachers and schools in the country 

in 2003–04. For this reason, the only weights produced from the study are for making statements about 

children, including statements about the teachers and schools of those children. 

The ECLS-K has several major objectives and numerous potential applications. The 

ECLS-K combines (1) a study of achievement in the elementary years; (2) an assessment of the 

developmental status of children in the United States at the start of their formal schooling and at key 

points during the elementary school years; (3) cross-sectional studies of the nature and quality of 

kindergarten programs in the United States; and (4) a study of the relationship of family, preschool, and 

 

4

 Approximately 90 percent of the children interviewed were in fifth grade during the 2003-04 school year, 9 percent were in fourth grade, and 

less than 1 percent were in third or some other grade. 
1-4 

school experiences to children’s developmental status at school entry and their progress from 

kindergarten, through elementary school, and into high school. 

The ECLS-K is part of a longitudinal studies program comprising two cohorts—a 

kindergarten cohort and a birth cohort. The birth cohort (ECLS-B) is following a national sample of 

children born in the year 2001 from birth to kindergarten. The ECLS-B focuses on the characteristics of 

children and their families that influence children’s first experiences with the demands of formal school, 

as well as children’s early health care and in- and out-of-home experiences. Together these cohorts will 

provide the depth and breadth of data required to more fully describe and understand children’s health and 

early learning, development, and education experiences. 

The ECLS-K has both descriptive and analytic purposes. It provides descriptive data on 

children’s status at school entry, their transition into school, and their progress into high school. The 

ECLS-K also provides a rich data set that enables researchers to analyze how a wide range of family, 

school, community, and individual variables affect children’s early success in school; explore school 

readiness and the relationship between the kindergarten experience and later elementary school 

performance; and record children’s cognitive and academic growth as they move through secondary 

school. 

1.1 Background 

Efforts to expand and improve early education will benefit from insights gained through 

analyses drawn from the large scale, nationally representative ECLS-K data and the study’s longitudinal 

design. The ECLS-K database contains information about the types of school programs in which children 

participated, the services they received, and repeated measures of the children’s cognitive skills and 

knowledge. The ECLS-K database also contains measures of children’s physical health and growth, social 

development, and emotional well-being, along with information on family background and the 

educational quality of their home environments. 

As a study of early achievement, the ECLS-K allows researchers to examine how children’s 

progress is associated with such factors as placement in high or low ability groups, receipt of special 

services or remedial instruction, grade retention, and frequent changes in schools attended because of 

family moves. Data on these early school experiences are collected as they occur, with the exception of 
1-5 

their experiences before kindergarten, which were collected retrospectively. This produces a more 

accurate measurement of these antecedent factors and enables inferences to be made about their 

relationship to later academic progress. The longitudinal nature of the study enables researchers to study 

children’s cognitive, social, and emotional growth and to relate trajectories of change to variations in 

children’s experiences in kindergarten and the early grades to later grades. 

The spring-fifth grade data collection can be used to describe the diversity of children in the 

study and the classrooms and schools they attend. It can also be used to study children’s academic gains 

in the years following kindergarten and first grade. The ECLS-K sample includes substantial numbers of 

children from various minority groups. Thus, the ECLS-K data present many possibilities for studying 

cultural and ethnic differences in the educational preferences and literacy practices of families, the 

developmental patterns and learning styles of children, and the educational resources and opportunities 

that different groups are afforded in the United States. 

1.2 Conceptual Model 

The design of the ECLS-K has been guided by a framework of children’s development and 

schooling that emphasizes the interrelationships between the child and family, the child and school, the 

family and school, and the family, school, and community. The ECLS-K recognizes the importance of 

factors that represent the child’s health status and socioemotional and intellectual development and 

incorporates factors from the child’s family, community, and school-classroom environments. The 

ECLS-K conceptual model is depicted in exhibit 1-2. The study has paid particular attention to the role 

that parents and families play in helping children adjust to formal school and in supporting their education 

through the elementary grades. It has also gathered information on how schools prepare for and respond 

to the diverse backgrounds and experiences of the children and families they serve. 

1.3 Study Components 

The emphasis placed on measuring children’s environments and development broadly has 

critical implications for the design of the ECLS-K. The design of the study includes the collection of data 

from the child, the child’s parents/guardians, teachers, and schools. 
1-6 

Exhibit 1-2. ECLS-K conceptual model: School years 1998–2004 

Child 

Characteristics 

Child and Family 

Health 

Parent 

Characteristics 

Parent-Child 

Interactions 

Community 

Structure/ 

Social Support 

Early Childhood 

Nonparental Care/ 

Education Characteristics 

Kindergarten 

Outcomes 

Elementary School 

Characteristics 

Elementary School 

Outcomes 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99. 

 Children participate in various activities to measure the extent to which they exhibit 

those abilities and skills deemed important for success in school. They are asked to 

participate in activities designed to measure important cognitive (i.e., literacy, 

quantitative, and science) skills and noncognitive (fine and gross motor coordination 

[in kindergarten] and socioemotional) skills and knowledge. Most measures of a 

child’s cognitive skills are obtained through an untimed one-on-one assessment of the 

child. Beginning with the third-grade data collection, children report on their own 

perceptions of their abilities and achievement as well as their interest in and 

enjoyment of reading, math, and other school subjects. Children are assessed in each 

round of data collection. Children’s height and weight are also measured in each 

round of data collection. 

 Parents/guardians are an important source of information about the families of the 

children selected for the study and about themselves. Parents provide information 

about children’s development at school entry and their experiences both with family 

members and others. Information is collected from parents each time children are 

assessed using computer-assisted interviews (CAIs). Information is collected from 

parents/guardians in each round of data collection. 

 Teachers, like parents, represent a valuable source of information on themselves, the 

children in their classrooms, and the children’s learning environment (i.e., the 

classroom). Teachers are not only asked to provide information about their own 

backgrounds, teaching practices, and experience, they are also called on to provide 

information on the classroom setting for the sampled children they teach and to 

evaluate each sampled child on a number of critical cognitive and noncognitive 

dimensions. Special education teachers and service providers of sampled children with 
1-7 

disabilities are also asked to provide information on the nature and types of services 

provided to the child. With the exception of the fall-first grade data collection, 

teachers complete self-administered questionnaires each time children are assessed. 

 School administrators, or their designees, are asked to provide information on the 

physical, organizational, and fiscal characteristics of their schools, and on the schools’ 

learning environment and programs. Special attention is paid to the instructional 

philosophy of the school and its expectations for students. School administrators or 

their designees, are also asked to provide basic information about school5

 grade level, 

school type (public or private), length of school year, and attendance recordkeeping 

practices. Information is collected from school administrators via self-administered 

questionnaires during each spring data collection.

 School office staff are asked to complete a student records abstract form and provide 

basic information about the school. The student records abstract form includes 

questions about an individual child’s enrollment and attendance at the school, transfer 

to another school (if applicable), and verifies whether the child has an Individualized 

Education Program (IEP) on record. A student records abstract form is completed for 

each child in the study during each spring data collection. 

Exhibit 1-3 summarizes the instruments that were used in each of the data collection periods 

from kindergarten through spring-fifth grade. Exhibit 1-4 provides additional detail about the direct child 

assessments conducted during each of the data collection periods. Separate psychometric reports have 

been prepared to describe the design and development of the kindergarten through first-grade and thirdgrade assessment batteries. For detailed information about the child assessments, including their 

psychometric properties, see the ECLS-K Psychometric Report for Kindergarten Through First Grade, 

NCES 2002–05 (Rock and Pollack 2002); the ECLS-K Psychometric Report for the Third Grade (NCES 

2005–062) (Pollack et al. 2005); and the ECLS-K Psychometric Report for the Fifth Grade (NCES 

2006–036) (Pollack et al. 2005). 

1.4 Contents of Report 

This report provides detailed technical information about the development, design, and 

conduct of the fifth-grade data collection. Chapter 2 provides an overview of processes used to develop 

the computer-assisted (CAI) and hard-copy survey instruments. Chapter 3 describes the sample design 

and implementation. Chapter 4 describes the data collection methods, including information about the 

training of field staff and quality control procedures. Chapter 5 details the preparation and editing of the 

 

5

 These items were collected in a separate school fact sheet in 3rd grade, but were reintegrated into the school administrator questionnaire in the 

5th grade data collection. Prior to the 3rd grade data collection, the questions had been part of the school administrator questionnaire. 
1-8 

data as it is receipted from the field. Chapter 6 provides information on unit and item response rates. 

Chapter 7 discusses weighting and variance information. 

Because both this report and the ECLS-K Psychometric Report for the Fifth Grade focus on 

the fifth-grade data collection, minimal information is provided about the base year, first-grade, or thirdgrade data. Users who wish to learn more about these data collections should refer to the ECLS-K Base 

Year Public-Use Data Files and Electronic Codebook: User’s Manual (NCES 2001–029r) (Tourangeau, 

Burke, et al. 2004); the User’s Manual for the ECLS-K First Grade Public-Use Data Files and Electronic 

Codebook (NCES 2002–135) (Tourangeau et al. 2002); or the User’s Manual for the ECLS-K Third 

Grade Public-Use Data File and Electronic Code Book (NCES 2004–001) (Tourangeau, Brick, et al. 

2004) Additional information about the ECLS program can be found on the World Wide Web at 

http://nces.ed.gov/ecls. 
1-9 

Exhibit 1-3. Instruments used in the ECLS-K, by round of data collection: School years 1998–99, 

 1999–2000, 2001–02, and 2003–04 

1998–99 

school year 

1999–2000 

school year 

2001–02 

school year 

2003–04 

school year 

Instruments 

Fallkindergarten Springkindergarten Fallfirst grade1 Springfirst grade2 Springthird grade Springfifth grade 

Parent interview X X X X X X 

Child assessments X X X X X X 

Teacher questionnaire part A X X X X2 X 

Teacher questionnaire part B X X X X2 X 

Teacher questionnaire part C X X X X2 X 

Teacher questionnaire (teacher level) X3 

Reading teacher questionnaire X 

Math teacher questionnaire X 

Science teacher questionnaire X 

Special education teacher 

 questionnaire part A X X X X 

Special education teacher 

 questionnaire part B X X X X 

Adaptive Behavior Scale X X 

Self-description questionnaire X X 

Food consumption questionnaire X 

School administrator questionnaire X X4 X X5 

Student record abstract X X X X 

School fact sheet X6 

School facilities checklist X X X X 

Salary and benefits questionnaire7 X 

Head Start verification8 X 

X Round that included the instrument. 

1

 The fall-first grade data collection consisted of a 30 percent subsample of the study schools. See the User’s Manual for the ECLS-K First Grade 

Public-Use Data Files and Electronic Code Book (NCES 2002-135) (Tourangeau, Burke, et al. 2002) for information about the purposes and 

methods of the fall-first grade data collection. 

2

 In spring-first grade, there were two sets of teacher questionnaires⎯one for the teachers of children who had made the transition to the first 

grade or any higher elementary school grade, and the second for teachers of children who were repeating or attending the second year of 

kindergarten. 

3

 In spring-fifth grade, teacher questionnaires part A, B, and C were replaced by a teacher-level questionnaire and questionnaires for reading, 

math, and science teachers. 

4

 In spring-first grade, there were two different school administrator questionnaires⎯one for school administrators in schools new to the study 

and one for school administrators in schools that participated in the base year data collection. 5

 In spring-fifth grade, questions fromthe school fact sheet used in spring-third grade were included in the school administrator questionnaire. 6

 The items in the school fact sheet were included in the school administrator questionnaire in kindergarten and in first grade. These items were 

reintegrated into the school administrator questionnaire in the 5th grade data collection. 7

 The salary and benefits questionnaire collected information on the base salary, merit pay, and health benefit pay of teachers and principals. It 

was completed by the school or district business administrator or by a private school administrator or headmaster. 8

 The Head Start Verification Study confirmed parent and school reports of children’s Head Start participation by matching information on the 

name and location of the Head Start facilities the children were reported to have attended against a database of Head Start centers. For each 

match, the center was contacted to confirm that the child had attended the center in the year before kindergarten. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), fall 1998, spring 1999, fall 1999, spring 2000, spring 2002, and spring 2004.
1-10 

Exhibit 1-4. Direct child assessments, by domain and round of data collection: School years 1998–99, 

1999–2000, 2001–02, and 2003–04 

1998–99 

school year 

1999–2000 

school year 

2001–02 

school year 

2003–04 

school year 

Direct child assessment 

Fallkindergarten Springkindergarten Fallfirst grade Springfirst grade Springthird grade Springfifth grade 

Language screener (Oral Language 

Development Scale[OLDS])1

 X X X X 

Food consumption 

questionnaire (FCQ) X 

Reading (language and literacy) X X X X X X 

Mathematical thinking X X X X X X 

Socioemotional development X X 

General knowledge (science and 

 social studies) X X X X 

Science2

 X X

Psychomotor X 

Height and weight X X X X X X 

X Round that included the instrument. 

1

 OLDS (Oral Language Development Scale) was given to language-minority students new to the study in the spring, or who did not pass the cut 

score in the English version during the previous OLDS administration. The screener determined if the children understood English well enough to 

receive the direct child assessments in English. For further information on the language screener, please refer to the ECLS-K Base Year PublicUse Data Files and Electronic Code Book: User’s Manual (NCES 2001-029r) (Tourangeau; Burke, et al. 2004). The screener was not used in 

third or fifth grade because the vast majority of children passed it by spring-first grade. 2

 In spring-third grade, the general knowledge assessment was replaced with a science assessment. Children received a science assessment in 

third and fifth grade that measured their understanding of science concepts and scientific investigation skills. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), fall 1998, spring 1999, fall 1999, spring 2000, spring 2002, and spring 2004. 
2-1 

2. DEVELOPMENT OF SURVEY INSTRUMENTS 

The Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K) fifthgrade survey collected data on the achievement and home and school experiences of children who had 

attended kindergarten in 1998–99 to provide information on the children’s progress in the middle 

elementary grades. In the design phase of the ECLS-K kindergarten, first-grade, third-grade, and fifthgrade waves of data collection, policymakers, teachers, and researchers were consulted, and relevant 

literature was reviewed to ascertain the specific areas within each of the topical components for which 

national data were needed. Information gathered from these activities guided the formulation of research 

questions deemed most important for the ECLS-K to address. Extant surveys were reviewed to identify 

surveys that had been fielded to answer similar questions. 

The ECLS-K data collection instruments were similar in content and form in all six waves of 

the study. The ECLS-K employed two modes of data collection, computer-assisted and self-administered 

hard-copy instruments. This chapter describes the development of the computer-assisted and hard-copy 

instruments for the fifth-grade data collection. The procedures for developing the child assessment battery 

and indirect rating forms are described in a separate psychometric report. More information on the 

assessment battery and indirect rating forms is found in ECLS-K Psychometric Report for Kindergarten 

Through First Grade (NCES 2002–05) (Rock and Pollack 2002) and in ECLS-K Psychometric Report for 

the Third Grade (NCES 2005–062) (Pollack et al. 2005). 

In spring-fifth grade, several modifications were made to the instruments. Many of the 

changes were based on advice given by the ECLS-K Technical Review Panel (TRP) and Content Review 

Panel (CRP) that was provided for both the spring-third grade and spring-fifth grade data collections 

simultaneously. Modifications were made to the fifth-grade parent interview to reduce its length and add 

other items of interest. A timing study was conducted to assess the effect of these changes. Several 

changes were also made to the child assessment, the teacher questionnaires, and the school administrator 

questionnaire. These are discussed below. 
2-2 

2.1 Review Panels 

Studies with the scope, complexity, and importance of the ECLS-K require consultation with 

a number of individuals and organizations to address the data needs of policymakers and of those 

performing policy studies and educational research. In addition, consultations with practitioners, content 

area experts, and researchers are necessary to ensure that instruments accurately reflected curricular 

standards and practices. The ECLS-K project staff established and sought guidance from the Technical 

Review Panel and the Content Review Panel. 

2.1.1 Technical Review Panel 

The Technical Review Panel was assembled to provide review and comment on such matters 

as the technical design and implementation of the ECLS-K and policy and research topics that are 

appropriate for the ECLS-K third-grade and fifth-grade data collections. The membership of the 

Technical Review Panel represents a broad range of nonfederal and federal experts in elementary 

education, educational, and family research and policy issues. 

 Technical Review Panel Members: 

Karl Alexander, Johns Hopkins University 

Richard Duran, University of California at Santa Barbara 

Nancy Karweit, Johns Hopkins University 

Donna Morrison, Georgetown Public Policy Institute 

Jane Stallings, Texas A&M University 

Deborah Stipek, Stanford University 

U.S. Department of Education: Institute of Education Sciences, National Center 

for Education Statistics 

Office of Special Education Programs, and 

Office of English Language Acquisition, Language Enhancement and 

Academic Achievement for Limited English Proficient Students 

U.S. Department of Agriculture: Economic Research Service, 

and Office of Analysis, Nutrition, and Evaluation 

U.S. Department of Health and Human Services: 

Administration on Children, Youth and Families 
2-3 

2.1.2 Content Review Panel 

In addition to the Technical Review Panel, the Content Review Panel (CRP) was established 

to provide expert review of (1) the validity of the content of the child assessments and (2) the consistency 

of the items in the assessment battery with instructional practice. The panel included subject matter 

experts in reading, mathematics, and science as well as school assessment and evaluation administrators. 

 Content Review Panel Members 

Dr. Gloria Johnson, Superintendent, West Contra Costa Unified School 

District, Richmond, California. Specialty: Reading. 

Dr. Jean Joyner, North Carolina Department of Public Instruction. Specialty: 

Mathematics. 

Helen Lounsbury, Teacher, Berne-Knowx-Westerlo Elementary School, Berne, 

New York. Specialty: Mathematics. 

Dr. Alba Ortiz, Professor, Department of Special Education, University of 

Texas. Specialty: Reading. 

Kathy DiRanna, Director, K-12 Alliance, California. Specialty: Science. 

Iris Weiss, President, Horizon Research, Inc., Chapel Hill, North Carolina. 

Specialty: Science.

2.2 Modifications to the Parent Interview 

Exhibit 2-1 lists the 17 sections comprising the spring-fifth grade parent interview. 

Many items from previous rounds of the ECLS-K were not included in spring-fifth grade 

because they had been used previously and to reduce respondent burden. The following constructs in the 

spring-third grade parent interview were not asked in spring-fifth grade: 

 School practices (PIQ.030a-e); 

 School climate (PIQ.120a-h); 
2-4 

Exhibit 2-1. Sections comprising the spring-fifth grade parent interview: School year 2003-04 

INQ = Introduction 

PIQ = Parent Involvement 

FSQ = Family Structure 

HEQ = Home Environment, Activities, and Cognitive Stimulation 

CFQ = Critical Family Processes 

CCQ = Child Care 

DWQ = Discipline, Warmth, and Emotional Supportiveness 

NRQ = Non-Resident Parent Questions 

COQ = Country of Origin Questions for Non-Resident Biological Parents 

CHQ = Child Health and Well Being 

PPQ = Parent’s Psychological Well Being and Health 

FDQ = Food Security 

PEQ = Parent Education 

EMQ = Parent Employment 

WPQ = Welfare and Other Public Transfers 

PAQ = Parent Income and Assets 

CMQ = Child Mobility and Plans to Move 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 

Kindergarten Class of 1998-99 (ECLS-K), spring 2004. 

 Parent perception of how well child is doing in reading/language arts and math 

(PIQ.090, PIQ.100); 

 Specific home learning activities and resources (HEQ.010, HEQ.015, HEQ.017, 

HEQ.021, HEQ.023, HEQ.030, HEQ.065B, HEQ.094, HEQ.097, HEQ.100, 

HEQ.106); 

 Family breakfast patterns (HEQ.115, HEQ.116, HEQ.117, HEQ.118); 

 Transportation and commute to school (HEQ.124, HEQ.125, HEQ.126, HEQ.127); 

 Problems around house/neighborhood (HEQ.410); 

 Social support (CFQ.020); and 

 Parental depression (PPQ.100, PPQ.110, PPQ.120, PPQ.130, PPQ.140, PPQ.150, 

PPQ.160, PPQ.170, PPQ.180, PPQ.190, PPQ.200, PPQ.210, PPQ.230, PPQ.240). 
2-5 

Several new construct areas were added for fifth grade. These were as follows: 

 Discussions with child about school and friends (HEQ.420a and b); 

 Discussions with child about smoking, sexual activity, drinking alcohol, and using 

other drugs (HEQ.421a–d); 

 Identification of when a diagnosis of a particular disability was made, if applicable 

(CHQ.076, CHQ.077, CHQ.136, CHQ.137, CHQ.186, CHQ.187, CHQ.226, 

CHQ.227, CHQ.314, CHQ.315, CHQ.346, CHQ.347, CHQ.376, CHQ.377); 

 Identification of when cochlear implants were implanted (CHQ.251-CHQ.253); 

 Child’s use of cochlear implants in school (CHQ.254); 

 Identification of when child’s use of therapy services or program for children with 

disabilities ended (CHQ.536, CHQ.537); 

 Reason why the child no longer participated in services for children with special needs 

or special education (CHQ.546); 

 Whether child takes prescription medication for Attention Deficit Disorder (ADD), 

Attention Deficit Hyperactivity Disorder (ADHD), or hyperactivity (CHQ.740); 

 Medications taken for ADD, ADHD, or hyperactivity (CHQ.750a-g); 

 Length of time medications have been taken for ADD, ADHD, or hyperactivity 

(CHQ.760); 

 Receipt of family therapy (CHQ.770); 

 Reason for family therapy (CHQ.780); 

 Type of family therapist seen (CHQ.790); and 

 Number of times family therapist seen (CHQ.800). 

Other questions were added that had been used previously in the ECLS-K for parent figures 

in the household, but in spring-fifth grade were asked about a new group of persons (nonresident 

biological parents). These were: 

 Country of origin, age moved to U.S., and U.S. citizenship for non-resident biological 

parents (section COQ); 
2-6 

In addition, one question was reintroduced from an earlier year of the study: 

 Whether father or mother figure has a high school diploma or its equivalent 

(PEQ.021); 

Finally, some questions do not include new content but were modified from a previous 

round: 

 In spring-fifth grade, interviewers were able to record the time set aside every day for 

children to do homework in hours (HEQ.092b) in addition to minutes (HEQ.093a). 

 Questions about who usually helps the child with his/her reading, language arts, or 

spelling homework or helps with math homework allowed the respondent to volunteer 

who helped (HEQ.095b, HEQ.099). In spring-third grade, respondents were instead 

asked separate questions about each household member and whether he/she helped 

with homework. 

2.3 Timing Study 

As with any study instrument, questionnaire length and respondent burden were issues of 

concern. A timing study was conducted for the draft parent questionnaire. Three Westat staff members 

conducted nine interviews with respondents who had previously volunteered to participate in studies 

being conducted by Westat. No attempt was made to recruit respondents representative of either racial or 

economic groups as the objective was to obtain an estimate of the length of the questionnaire rather than 

to examine how individuals interpreted the questions. Westat did attempt to select people who would go 

through the various questionnaire paths (e.g., married couples, single parents). All of the respondents 

were parents of fifth-grade children. All interviews were conducted over the telephone using a paper 

version of the questionnaire. Interviewers used stopwatches to time the individual sections and to get an 

overall time for the interview. The interviewers stopped the watches for extended interruptions, such as a 

respondent having to take care of the needs of a family member. In most cases, the respondents were 

asked to answer questions in sections that required knowledge of data collected from an earlier wave of 

the data collection as if they had provided the information in a previous round of the survey. In only two 

interviews were respondents asked to complete such sections, as would be the case with a new 

respondent. 

The revised paper version of the questionnaire took an average of 39 minutes and 17 seconds 

to complete. Table 2-1 summarizes the overall and section timings for each interviewer and presents the 
2-7 

average time expended for each section. The initials denote the three interviewers. Each interviewer 

completed three interviews (e.g., interviewer SG completed SG1, SG2, and SG3). One interview was 

done with a parent of twins (NV2). The second twin’s time was included in the overall average time. 

Table 2-1. Interviewer timings for the revised ECLS-K fifth-grade parent interview, by interviewer and 

by section: School year 2002–03 

Section1 SG1 LB1 NV1 LB2 SG2 NV2 

NV2 

(TWIN) LB3 SG3 NV3 

Mean 

time 

 

Total 0:35:53 0:41:18 0:45:10 0:41:15 0:34:23 0:53:39 0:14:34 0:35:14 0:36:33 0:30:07 † 

INQ 0:01:28 0:00:39 0:01:01 0:01:45 0:01:12 0:02:05 0:00:23 0:01:31 0:01:28 0:00:43 0:01:19 

PIQ 0:02:50 0:02:20 0:03:28 0:03:38 0:03:41 0:03:58 0:01:34 0:03:21 0:03:04 0:03:02 0:03:16 

FSQ 0:01:59 0:03:10 0:01:04 0:03:30 0:01:07 0:02:35 0:00:00 0:02:12 0:01:55 0:01:39 0:02:08 

HEQ 0:08:00 0:14:05 0:09:02 0:07:39 0:09:18 0:09:32 0:04:45 0:06:43 0:06:16 0:07:29 0:08:40 

CFQ 0:00:12 0:00:11 0:00:00 0:00:10 0:00:10 0:00:56 † 0:00:12 0:00:10 0:00:09 0:00:14 

CCQ 0:01:25 0:01:00 0:03:29 0:02:30 0:01:53 0:01:30 0:00:10 0:01:14 0:01:30 0:01:34 0:01:47 

DWQ 0:01:09 0:01:10 0:02:12 0:01:02 0:01:18 0:01:15 0:00:34 0:00:54 0:00:59 0:00:35 0:01:10 

NRQ † † 0:03:27 † † † † † 0:02:31 † 0:02:59 

CHQ 0:10:17 0:11:10 0:13:27 0:11:30 0:08:16 0:08:25 0:06:59 0:08:58 0:07:24 0:07:20 0:09:39 

PPQ 0:00:10 0:00:07 0:01:00 0:00:10 0:00:10 0:00:14 † 0:00:10 0:00:10 0:00:07 0:00:15 

FDQ 0:01:04 0:01:00 0:01:00 0:00:51 0:00:48 0:01:00 † 0:00:54 0:02:40 0:00:45 0:01:07 

PEQ 0:00:14 0:00:49 0:01:00 0:01:20 0:00:15 0:01:00 † 0:00:43 0:00:15 0:00:52 0:00:43 

EMQ 0:01:24 0:00:48 0:02:00 0:02:41 0:01:22 0:02:00 † 0:02:35 0:01:59 0:02:09 0:01:53 

WPQ 0:00:43 0:01:00 0:01:00 0:01:13 0:01:03 0:01:00 † 0:01:03 0:01:36 0:00:47 0:01:03 

PAQ 0:01:06 0:01:02 0:01:00 0:00:40 0:00:44 0:01:00 † 0:00:51 0:00:45 0:00:42 0:00:52 

CMQ 0:03:52 0:02:47 0:01:00 0:02:36 0:03:06 0:02:35 0:00:09 0:03:53 0:03:51 0:02:14 0:02:53 

Mean time † † † † † † † † † 0:39:17 

† Not applicable. 

1

 See exhibit 2-1 for full section names. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 fifth-grade data collection, school year 2002–03 timing study. 

Overall, the parent interview required just under 40 minutes to complete. The timings for the 

first interview in the household ranged from a low of 30 minutes to a high of 53 minutes. The approaches 

used to capture the information (update versus obtain new data) and the characteristics of the child and 

household contributed to the variations in the length of interviews. The twin interview required an 

additional 14 minutes. 

No individual section was unduly long. The section that required the most time to administer 

was the Home Environment, Activities and Cognitive Stimulation (HEQ) section. In the first interview 

for respondents (i.e., for child 1), it took between 6 minutes and 14 minutes, depending upon the 

interviewer. Only two respondents completed the Nonresident Parent section of the questionnaire, which 
2-8 

included items about parents who did not live with their child, such as a question about the frequency of 

their contact with the child. The average time for this section was about 3 minutes. The results of the 

timing study suggested that the parent questionnaire could be administered within 40 minutes. 

2.4 Child Food Consumption Questions 

To measure children’s food consumption the ECLS-K assessors administered the Food 

Consumption Questionnaire (FCQ), a questionnaire used to determine the kinds of food the children can 

buy at school and food they have eaten in the past week. The FCQ for children consisted of 19 questions. 

There were also food consumption questions for school administrators. Those are described in section 2.6 

below. 

In the FCQ for children, the first set of questions was about foods that are high in fat, 

sodium, and/or added sugars (e.g., candy, salty snacks, soda pop). Children were asked if they could buy 

these foods at school, and if so, how often they bought the food in the past week and where they bought 

the food (vending machine, cafeteria, or somewhere else in school). In the second set of questions, 

children were asked about whether they ate particular key foods and beverages in the past 7 days, such as 

milk, sweetened beverages (e.g., soft drinks), fruits and vegetables, and fast food. They were asked to 

include food they ate at home, at school, at restaurants, or anywhere else. 

Items for the FCQ were taken mainly from existing surveys, although some were developed 

for the ECLS-K. Two main sources for questions were two surveys by the Center for Disease 

Control/Division of Adolescent and School Health Surveys: the Youth Risk Behavior Surveillance 

Survey (YRBSS) and the School Health Programs and Policies Survey (SHPPS).1

 The question on fastfood meals was taken from the California Children’s Healthy Eating and Exercise Practices Survey 

(CalCheeps). Questions on soft drinks and children’s at-school consumption of snack foods were 

developed at the U.S. Department of Agriculture (USDA), using YRBSS and CalCheeps questions as 

models. 

Assessors read each question of the FCQ to the child, along with the response categories, 

and the child circled his or her answer. The child was asked to tell the assessor what he or she circled so 

the assessor could enter the answer into the computer. At the beginning of the FCQ, there is an example 

 

1

 Information on these CDC surveys is available at http://www.cdc.gov/nccdphp/dash/ 
2-9 

question to show the child what kinds of questions would be asked. The example was also used to show 

the child how to circle a response and to practice telling the assessor what answer had been chosen. After 

the first few questions of the FCQ, if the child appeared to understand the response categories and was in 

one of the higher reading categories in the reading assessment, the child was allowed to read the response 

categories if he or she wanted to do so. 

For children who were homeschooled by their parents or another adult and did not attend 

school, questions about food that could be purchased at school did not apply. For these cases, assessors 

were told to skip questions 1 through 9 and enter “Don’t Know” into CAPI for each of these questions 

and then begin with the statement after question 9. 

2.5 Modifications to Teacher Questionnaires 

The approach for administering teacher questionnaires in spring-fifth grade differed from 

that of previous rounds because many fifth-grade children were expected to have different teachers for 

different subject areas. In the prior rounds of data collection, general education teacher questionnaires 

were designed for a single classroom teacher. All questions pertaining to the core academic subjects were 

asked in a single questionnaire and distributed to one teacher for each sampled child. When children had 

different subject matter teachers, it was left to the child’s main teacher to ask the other teachers to 

complete specific sections of the questionnaire for the subjects they taught. However, as children move 

through the elementary grades, the prevalence of children being taught core academic subjects 

(reading/language arts, mathematics, and science) by a single teacher decreased. Data from the 1999-2000 

Schools and Staffing Survey (SASS) indicated that the proportion of children taught core academic 

subjects by different teachers (i.e., team teaching and departmentalized instruction) was about 41 percent 

in public schools and about 56 percent in private schools. Thus, data collection procedures for spring-fifth 

grade were designed to ensure that the teachers most knowledgeable of the child’s performance in each of 

the core academic subjects (i.e., reading/language arts, mathematics, and science) provided the data 

germane to each child’s classroom environment, instruction in each of the core academic subjects, and the 

core academic teacher’s professional background. 

During the spring-fifth grade data collection, each child’s reading and math or science 

teacher received a self-administered teacher-level questionnaire about a variety of topics, including 

instructional practices, classroom resources, views on teaching and the school, and teacher background. 
2-10 

Three additional questionnaires specifically about the focal child were also distributed for teachers in 

reading, math, and science. Each reading and math or science teacher received a teacher questionnaire in 

addition to at least one child-level questionnaire in reading, mathematics, or science. 

All students were assigned to have their reading teacher complete questionnaires. To reduce 

respondent burden for teachers, half of students were randomly assigned to have a mathematics teacher 

complete questionnaires, and half of students were randomly assigned to have a science teacher complete 

questionnaires. In some schools, the sampled children were taught reading, mathematics, and science by 

the same person in one classroom. In other schools, different teachers taught these subjects to the sampled 

children. During the fifth grade data collection, 53 percent of the students were taught in self-contained 

classrooms, 24 percent received departmentalized instruction, 18 percent were team taught, and 5 percent 

were pulled out of class for instruction. For more information about how questionnaires were distributed, 

see section 4.5.5. 

The reading teacher questionnaire had three different parts. The first part included questions 

from the Social Rating Scale (SRS) that collected data on five areas of children’s social skills. The second 

part had questions from the Academic Rating Scale (ARS) and gathered data on each sampled child’s 

skills in areas of language and literacy. The third part asked child-specific instructional information (for 

example, child’s grade, additional tutoring or services the child received), asked the teacher to tell how 

this child behaved and performed in language and literacy relative to the other children in the class, and 

asked about the teacher’s classroom and the characteristics of the students, instructional activities and 

curricular focus, and instructional practices in language arts. The mathematics teacher questionnaire 

included questions from the ARS gathering data on each sampled child’s skills in mathematics, asked 

child-specific specific instructional information (for example, child’s grade, additional tutoring or 

services the child received), asked the teacher to tell how this child behaved and performed in 

mathematics class relative to the other children in the class, and asked about the teacher’s classroom and 

the characteristics of the students, instructional activities and curricular focus, and instructional practices 

in mathematics. The science teacher questionnaire was similar to the mathematics teacher questionnaire 

with the questions focusing on science rather than mathematics. Teachers responded to two of these 

questionnaires for each sampled child. Therefore, data were gathered on each sampled child’s skills in the 

areas of language and literacy and mathematical thinking, or in the areas of language and literacy and 

science. For more information on data collection with teachers, see section 4.5.5. 
2-11 

In addition to the teacher questionnaires described above, the ECLS-K also included special 

education teacher questionnaires. These were similar to the ones given to special education teachers in 

previous rounds and had two parts, A and B. Part A of the special education teacher questionnaire was 

designed to collect information about the special education teacher’s professional background and 

experience. Part B asked about the special education services provided to the child and the nature of the 

child’s special education curriculum. Except for one change, the spring-fifth grade special education 

teacher questionnaires were identical to the ones used in spring-third grade. A question on the receipt of 

special education or related services due to an attention deficit/hyperactivity disorder (ADHD) was added 

to Part B in the spring-fifth grade questionnaire. 

2.6 Modifications to School Administrator Questionnaire 

The principal, administrator, or headmaster at the school attended by the sampled child was 

asked to complete the school administrator questionnaire in the spring of 2004. As in previous rounds of 

the study, this self-administered questionnaire was intended to gather information about the school, 

student body, teachers, school policies, and administrator characteristics. 

Two main changes were made to the questionnaire in spring-fifth grade. First, it included 

items that in third grade had been in a questionnaire called the school fact sheet (e.g., the grades taught in 

the school, school sector and focus, the length of the school year). Also, a new content area about student 

food consumption was added. The main purpose of these questions was to determine the availability at 

school of various foods, including those that are healthy and those that are high in fat, sodium, and/or 

added sugars. Questions were asked about whether students could purchase food or beverages from 

vending machines at the school or a school store, canteen, or snack bar. School administrators were also 

asked if the school offered a la carte lunch or breakfast items to students that were not sold as part of the 

National School Lunch or the School Breakfast Program. In addition, questions were asked about whether 

children could buy particular foods and beverages at school, such as milk, sweetened beverages (e.g., soft 

drinks), fruits and vegetables, candy, and salty snacks; where these foods could be obtained in the school 

(e.g., a school store, a vending machine); and how full the cafeteria was at peak meal times. Questions on 

the availability of foods not part of USDA meal programs and cafeteria crowding were taken from 

SHPPS. The sources for the other food consumption questions in the school administrator questionnaire 

are the same as those described in section 2.4 about the children’s food consumption questions. 
2-12 

This page is intentionally left blank. 
3-1 

3. SAMPLE DESIGN AND IMPLEMENTATION 

This chapter describes the sample design of the Early Childhood Longitudinal Study, 

Kindergarten Class of 1998–99 (ECLS-K), and how it was modified and implemented for each round of 

data collection. An overview of the sample design is given here and described in more detail in the 

following sections. 

The ECLS-K employed a multistage probability sample design to select a nationally 

representative sample of children attending kindergarten in 1998–99. In the base year the primary 

sampling units (PSUs) were geographic areas consisting of counties or groups of counties. The secondstage units were schools within sampled PSUs. The third- and final-stage units were students within 

schools. During the base year, data were collected in both the fall and the spring. 

Base year respondents were eligible for the first-grade data collection and nonrespondents 

were not eligible. A case was considered responding for the base year if there was a completed child 

assessment or parent interview in fall- or spring-kindergarten. A child with a disability who could not be 

assessed was also considered a base year respondent whether or not this child had a complete parent 

interview. Background characteristics such as sex, race/ethnicity, age, height, and weight are available for 

children with disabilities who could not be assessed. While all base year respondents were eligible for the 

spring-first grade data collection, fall-first grade was limited to a 30 percent subsample. The spring-first 

grade student sample was freshened to include current first-graders who had not been enrolled in 

kindergarten in 1998–99 and, therefore, had had no chance of being included in the ECLS-K base year 

kindergarten sample. For both fall- and spring-first grade, approximately 50 percent of sampled students 

who had transferred from their kindergarten schools were followed. 

The third-grade data collection included base year respondents and children sampled in first 

grade through a freshening operation in which the spring-first grade sample was freshened to include 

first-graders who had not been enrolled in kindergarten in 1998–99 and therefore had no chance of being 

included in the ECLS-K base year kindergarten sample. As in the first-grade data collection, where only a 

subsample of students who had transferred from their kindergarten schools was followed, subsampling of 

the movers was also used in third grade. In third grade, however, the subsampling rate applied to movers 

was slightly higher: children whose home language was non-English (also known as children belonging 

to the language minority group) and who had transferred for the first time between kindergarten or first 
3-2 

grade to third grade were followed with certainty. In other words, 100 percent of the children belonging to 

the language minority group who had not moved between kindergarten and first grade but had moved 

between first grade and third grade were followed into their new third-grade schools. Language minority 

children who had moved between kindergarten and first grade and were not subsampled for followup in 

first grade did not re-enter the third-grade sample; those who were subsampled for followup in first grade 

were followed with certainty into their third-grade schools if they had moved again between first grade 

and third grade. The higher subsampling rate allowed for the preservation of this group in the sample for 

analytic reasons. Children not in the language minority group continued to be subsampled for followup at 

a 50 percent rate if they had moved out of the original sample schools. 

In fifth grade, the sample that was fielded was reduced by excluding certain special groups 

of children from data collection, and by setting differential sampling rates for movers in different 

categories. Specifically, children in four groups were not fielded for the fifth-grade survey, irrespective of 

other subsampling procedures that were implemented. These were children who had become ineligible in 

an earlier round because they had died or moved out of the country, children who were subsampled out in 

previous rounds because they were movers, children whose parents emphatically refused to cooperate 

(hard refusals), and children eligible for the third-grade data collection for whom there are neither firstgrade nor third-grade data. Of the remaining children, those who had moved from their original schools 

during fifth grade or earlier were subsampled for followup. Children whose home language was not 

English (language minority) continued to be a special domain of analytic interest, and were subsampled at 

higher rates. Children were subsampled at different rates depending on the longitudinal data available for 

those children. 

The precision requirements and achieved sample sizes for the different waves of data 

collection are discussed in section 3.1. The base year, fall-first grade, spring-first grade, and spring-third 

samples are discussed in section 3.2, 3.3, 3.4 and 3.5, respectively. Sampling issues that were considered 

prior to the fifth-grade data collection are discussed in section 3.6. Section 3.6.3 includes a discussion of 

the characteristics of the fifth-grade sample and those of the children excluded from the fifth-grade data 

collection. 
3-3 

3.1 Precision Requirements and Achieved Sample Sizes 

The ECLS-K is a nationally representative longitudinal survey of children who attended 

kindergarten in 1998–1999, supplemented with children who were in first grade in spring 2000, but were 

not in kindergarten the previous year. Data on these children were collected from a variety of sources at 

two points in the base year (kindergarten in 1998–1999), two points in the 1999–2000 school year (as 

noted earlier, the fall collection was limited to a subsample of children) when most of the children were in 

first grade, in spring of 2002 when most of the children were in third grade, and again in spring of 2004 

when most of the children were in fifth grade. 

The overall design for the survey evolved over time. The initial design study recommended 

sampling 23,500 children in approximately 1,000 kindergarten programs sampled from 100 PSUs. The 

initial plans also called for sampling children in private schools at a higher rate than children in public 

schools, as well as sampling minorities (children of Black, Hispanic, or Asian or Pacific Island [API] race 

or ethnicity) at higher rates than nonminorities. The design study assumed that because of nonresponse 

and losses due to children moving, the final number of completed interviews at the end of the survey 

would be about 10,300. While the design study was useful in providing overall direction, the final 

framework for the sample design differed in many ways from its recommendations. 

The sample design implemented through the fifth grade in the ECLS-K is described in this 

chapter. The remainder of this section gives an overview of the sampling objectives and how the design 

was revised to accommodate changes in those objectives over the course of the study. Subsequent 

sections of the chapter give the details of the procedures used to implement the sample in the various 

rounds or waves of data collection, beginning with the base year in 1998–1999. 

Four precision requirements for the survey were identified and formed the basis for the base 

year sample design and plans for the followups in subsequent rounds. These requirements are the ability 

to do the following: 

 Measure a relative change of 20 percent in proportions across waves; 

 Measure a relative change of 5 percent in a mean assessment score across waves; 
3-4 

 Estimate a proportion for each wave with a coefficient of variation (CV) of 10 percent 

or less; and 

 Estimate a mean assessment score for each wave with a CV of 2.5 percent or less. 

The goals were interpreted as being objectives not only for all children, but for subgroups of 

analytic interest that include children attending public and private schools (Catholic, non-Catholic), and 

children from different race and ethnic groups (Hispanic, Black, Asian and Pacific Islander, all other 

races). After the spring-first grade data collection, language minority children were a newly identified 

subgroup of analytic interest for sample design purposes. A large number of assumptions had to be made 

to estimate sample sizes sufficient to meet the precision requirements. The key assumptions included 

projections of the losses due to nonresponse and attrition due to children moving, the design effects1

associated with the sample design, the element mean and standard deviations of the assessment scores, 

and the correlation of the statistics across waves. Since the ECLS-K is the first study of this population 

using this methodology, many of the assumptions had to be based on judgments without much supporting 

empirical data. 

The precision requirements that drive the sample design (those demanding the largest sample 

size) have to do with estimating changes over time and estimating the precision of estimates in the fifthgrade data collection. Based on assumptions described above, it was determined that a sample in fifth 

grade of about 10,000 children would be adequate to meet the precision requirements overall and for most 

subgroups. A sample of about 800 to 1,000 children in a subgroup would be achieved for most of the 

subgroups with an overall sample of 10,000 children and these would approximately meet the precision 

goals. For example, with a sample size of 10,000, the number of Hispanic and Black children would 

exceed 1,000, as shown in section 3.6.3. Children in private schools and APIs were the two subgroups 

that were expected to fall short of the goals if higher sampling rates were not applied. As noted in the 

following sections, sampling procedures were implemented to increase the sample size for these two 

groups. 

After the spring-first grade data collection was completed, the assumptions were reviewed 

and the ability of the sample to meet the survey goals was re-examined. At that time, language minority 

children were included as a subgroup of analytic interest. The evaluation showed that the sample sizes 

were adequate for most subgroups, but special efforts were needed to retain language minority children in 

subsequent rounds. Table 3-11 in section 3.5 shows the outcome of the spring-first grade data collection 

 

1

 When a clustered sample with unequal sampling weights is used, the estimates are less precise than those expected from a simple random 

sample and the ratio of the actual to simple random sampling variance is called a design effect. 
3-5 

by type of children. Since funding was made available to support these efforts, sampling procedures for 

retaining movers were modified. In the first-grade data collection, half of the movers were subsampled 

and included for followup, without taking any characteristics of the children into account. To increase the 

sample of language minority children, the sampling procedures were revised for the third-grade followup 

to retain as many of these children as possible. 

The evaluation also showed that the assumed design effects for assessment scores (reading, 

math and general knowledge) were larger than originally expected, ranging from 4.5 to 9.5. The larger 

than expected design effects for scores were first identified after the base year. The design effects for 

percentages, ranging from 1.6 to 6.9 for proportions greater than 30 percent, were close to those originally 

anticipated (3.8 on average).2

 The evaluation showed that the correlation over time of the scores was 

higher than expected. The higher correlation makes estimates of change in scores over time more precise. 

Consequently, the only precision objective that is substantially affected by the higher than expected 

design effects is for the mean assessment scores for fifth grade. This partially offsets the loss in precision 

due to the higher design effect. 

Table 3-1 tracks the ECLS-K sample from the base year through fifth grade. The table shows 

that the large initial sample of children has been reduced over time due to subsampling movers and 

nonresponse, as expected. While the initial assumptions that drove the sample design were not always 

accurate separately, the overall effect of the losses has been very close to what was expected. For 

example, in several rounds of the ECLS-K, the assumed moving rate was lower than the actual moving 

rate, but this was offset by higher completion rates. The overall number of eligible children at the end of 

the fifth-grade wave was more than 12,000 children, and the final sample size for the fifth-grade sample 

exceeded the 10,000 children in the initial projections. 

 

2

 See design effects for selected survey items in chapter 4 of the ECLS-K Base Year Public-Use Data Files and Electronic Code Book: User’s 

Manual (NCES 2001–029r) (Tourangeau, Burke, et al. 2004). 
3-6 

Table 3-1. ECLS-K sample size from the base year through fifth grade: School years 1998–99, 

1999–2000, 2001–02, and 2003–04 

Characteristic 

Fallkindergarten Springkindergarten 

Fall-

 first

 grade1

Springfirst 

grade 

Springthird 

grade 

Springfifth 

grade 

Initial sample 21,387 22,772 2 6,507 21,357 3 21,357 3 16,143 4

Fielded after subsampling movers † † 5,728 18,507 17,240 12,380 

Fielded after locating movers † 22,088 5,691 17,708 16,951 12,170 

Number of eligibles 21,356 21,941 5,652 17,652 16,829 12,129 

Child-complete5

 19,173 19,967 5,291 16,727 14,470 11,346 

Parent-complete6

 18,097 18,950 5,071 15,626 13,489 10,996 

Child- or Parent-complete 19,864 20,578 5,424 17,324 15,305 11,820 

Child- and Parent-complete 17,586 18,339 4,938 15,029 12,654 10,522 

† Not applicable. 

1 Only 30 percent of base year schools were included in the fall-first grade sample. 

2 Including 1,426 students from refusal converted schools and excluding 41 students in schools that cooperated in Fall-Kindergarten and refused 

in Spring-Kindergarten. 

3 Only students who have at least one of the four base year data points (fall-kindergarten assessment or parent data, or spring-kindergarten 

assessment or parent data, and the 165 students sampled in first grade through sample freshening. 4 Excluding students described in section 3.6.1. 

5 Child-complete if the child had assessment data or was not assessed due to a disability. 6 Parent-complete if the child had parent interview data. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), fall 1998, spring 1999, spring 2000, spring 2002, and spring 2004. 

The details on the sample sizes for subgroups at the end of the fifth grade are provided later 

in this chapter (see tables 3-20, 3-21, and 3-22). Those tabulations show that the number of fifth-grade 

respondents for all of the specific subgroups of interest exceeds 1,000, except for children in non-Catholic 

private schools and API children. For most of the key analytic groups the numbers of respondents are 

much larger than 1,000. For API children, the number of respondents is 970, which exceeds the minimum 

target of 800 and is very close to 1,000. The number of respondents in non-Catholic private schools is 

957. 

3.2 Base Year Sample 

In the base year, the ECLS-K selected a nationally representative sample of children 

attending kindergarten in 1998–99, using a dual-frame multistage probability sample design. Counties 

and groups of counties constituted the first-stage sampling units or PSUs, schools or kindergarten 

programs within PSUs were the second-stage units, and children were the third- and final-stage units. 
3-7 

3.2.1 Selecting the Area Sample 

The point of departure for the ECLS-K area sample frame development was an existing 

multipurpose frame of PSUs created, using 1990 county-level population data and 1988 per capita income 

data from the U.S. Department of Commerce, Bureau of Economic Analysis. This frame contained 1,404 

PSUs that were counties or groups of contiguous counties. PSUs did not cut across census regional3

boundaries, but were allowed to cross state boundaries. Each 1990 metropolitan statistical area (MSA)4

constituted a single PSU except where an MSA crossed census regions, and it was split into two PSUs. 

The minimum size of a PSU in the multipurpose frame was 15,000 persons. 

Since the focus of the ECLS-K is kindergarten students, the existing PSU frame was updated 

with 1994 population estimates of 5-year-olds by race/ethnicity, the most up-to-date estimates available 

from the U.S. Bureau of the Census at the time. The counts of 5-year-olds by race/ethnicity were used to 

revise PSU definitions relative to a different minimum PSU size and to construct a measure of size 

(MOS) that facilitated the oversampling of APIs. Each PSU in the frame that did not have at least 320 

5-year-olds was collapsed with an adjacent PSU. This minimum PSU size was developed based on 

assumptions concerning anticipated school response rates, the average number of schools that would be 

selected per PSU, and the target number of students to be sampled per school. After this collapsing, the 

final ECLS-K PSU frame contained 1,335 records. 

The MOS used for selecting PSUs took into account the amount of oversampling of APIs 

required to meet the ECLS-K precision goals. The weighted MOS was calculated as follows: 

MOS n n = 2.5× + API other

where 2.5 is the oversampling rate for APIs, and nAPI and nother are the counts of 5-year-old APIs and all 

others, respectively. The oversampling rate for APIs was calculated as the target number of completed 

API cases divided by the expected number of completed API cases without oversampling. In all, 100 

PSUs were selected for the ECLS-K. The 24 PSUs with the largest measures of size were designated as 

certainty selections or self-representing (SR)5

 and were set aside. They were included in the sample with 

certainty. Once the self-representing PSUs were removed, the remaining PSUs, called non-self-

 

3

 A census region is a geographic region defined by the U.S. Bureau of the Census. 4

 A metropolitan statistical area (MSA) is a geographic entity designated as one or more counties in a metropolitan area, except in New England, 

where MSA is defined in terms of county subdivisions. MSAs generally have under 1 million in population. 5

 A self-representing PSU is selected into the sample with certainty (i.e., with probability 1). 
3-8 

representing (non-SR)6

, were partitioned into 38 strata of roughly equal MOS. The frame of non-selfrepresenting PSUs was first sorted into eight superstrata by crossing the two MSA categories (MSA and 

non-MSA) and the four census region (Northeast, Midwest, South, and West). Within the four MSA 

superstrata, the variables used for further stratification were race/ethnicity (high concentration of API, 

Black, or Hispanic), size class (MOS ≥ 13,000 and MOS < 13,000), and 1988 per capita income range 

(shown in table 3-2, each range was defined so as to have roughly equal population in each of the stratum, 

where applicable) Within the four non-MSA superstrata, the stratification variables were race/ethnicity 

and per capita income. The term “superstrata” is used here to distinguish between the larger strata created 

by crossing MSA categories and census regions and the smaller strata defined by race/ethnicity, size class 

and per capita income. Table 3-2 describes how the 38 non-self-representing strata were created. 

Two PSUs were selected from each non-self-representing stratum using Durbin’s Method 

(Durbin 1967). This method selects two first-stage units per stratum without replacement, with probability 

proportional to size and a known joint probability of inclusion. The Durbin method was used because it 

has statistical properties that make it easier to compute variances. Table 3-3 summarizes the 

characteristics of the ECLS-K PSU sample. 

The Durbin method required two passes of the frame with a different selection probability at 

each pass to obtain the desired probabilities of inclusion and joint probabilities of inclusion. In the first 

pass, one PSU was selected in the stratum with probability p1

. In the second pass, the selected PSU was 

excluded and another PSU was selected with probability proportional to 

2

1 2

1 1

12 12 p

p p

⎡ ⎤ ⎢ + ⎥ ⎣ − − ⎦

where p1 = M1/M and p2 = M2/M, M1 is the MOS of the first unit selected, M2 the MOS of the second 

unit selected, and M the MOS of the stratum. 

The overall selection probability of non-self-representing unit i is 

2 , 1, 2 i

i

M p i M = = . 

 

6

 A non-self-representing PSU is selected into the sample with probability proportional to its measure of size (MOS). 
3-9 

Table 3-2. Stratum definitions for the 38 non-self-representing strata: School year 1998–99 

Per capita income range 

Stratum 

Metropolitan 

statistical area 

(MSA) status1

Census 

region2

Race/ethnicity 

(percentage range) 

PSU3

 measure 

of size (MOS) Low High 

1 MSA Northeast Any ≥ 13,000 $22,062 $25,424 

2 MSA Northeast Any ≥ 13,000 16,342 22,030 

3 MSA Northeast Any < 13,000 18,128 29,084 

4 MSA Northeast Any < 13,000 16,697 18,032 

5 MSA Northeast Any < 13,000 12,279 16,616 

6 MSA Midwest Any ≥ 13,000 17,277 18,150 

7 MSA Midwest Any ≥ 13,000 16,103 17,092 

8 MSA Midwest Any < 13,000 16,552 24,009 

9 MSA Midwest Any < 13,000 15,732 16,475 

10 MSA Midwest Any < 13,000 14,450 15,693 

11 MSA Midwest Any < 13,000 10,185 14,433 

12 MSA South Hispanic ≥ 30 Any Any Any 

13 MSA South Black ≥ 40 Any Any Any 

14 MSA South 26 ≤ Black < 40 Any 14,743 18,731 

15 MSA South 26 ≤ Black < 40 Any 10,892 14,573 

16 MSA South Black < 26 ≥ 13,000 16,435 16,601 

17 MSA South Black < 26 ≥ 13,000 14,586 16,337 

18 MSA South Black < 26 < 13,000 15,572 22,824 

19 MSA South Black < 26 < 13,000 14,194 15,432 

20 MSA South Black < 26 < 13,000 11,262 13,979 

See notes at end of table. 
3-10 

Table 3-2. Stratum definitions for the 38 non-self-representing strata: School year 1998–99—Continued 

Per capita income range 

Stratum 

Metropolitan 

statistical area 

(MSA) status1

Census 

region2

Race/ethnicity 

(percentage range) 

PSU3

 measure 

of size (MOS) Low High 

21 MSA West Asian/Pacific Islander ≥ 15 Any Any Any 

22 MSA West Asian/Pacific Islander ≥ 15 Any Any Any 

23 MSA West Hispanic ≥ 30 Any Any Any 

24 MSA West 12 ≤ Hispanic < 30 Any Any Any 

25 MSA West Hispanic < 12 Any 15,048 21,840 

26 MSA West Any Any 9,993 14,839 

27 Non-MSA Northeast Any Any Any Any 

28 Non-MSA Midwest Any Any 14,124 17,446 

29 Non-MSA Midwest Any Any 13,277 14,121 

30 Non-MSA Midwest Any Any 12,169 13,272 

31 Non-MSA Midwest Any Any 6,992 12,147 

32 Non-MSA South Black ≥ 42 Any Any Any 

33 Non-MSA South 25 ≤ Black < 42 Any Any Any 

34 Non-MSA South Any Any 12,727 20,059 

35 Non-MSA South Black < 25 Any 11,165 12,676 

36 Non-MSA South Any Any 6,018 11,142 

37 Non-MSA West Any Any 12,887 23,286 

38 Non-MSA West Any Any 6,959 12,884 

1 MSA is a geographic entity designated as one or more counties in a metropolitan area, except in New England, where MSA is defined in terms 

of county subdivisions. Non-MSA designates one or more counties not in a metropolitan area. MSA and non-MSA are as defined by the Bureau 

of the Census. 

2 A census region is a geographic region defined by the U.S. Bureau of the Census. 3 Primary sampling unit. 

NOTE: In this table, “Any” means any value of the column variable. For example, stratum 1 includes PSUs that have MSA status, are located in 

the Northeast region, with a MOS greater than or equal to 13,000 and per capita income ranging between $22,062 and $25,424, and can have any 

value of the race/ethnicity percentage. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), fall 1998 and spring 1999. 

The joint probability of inclusion of the first and second units is 

1,2 1 2

1 2 1

1 1 2 1

12 12 12

N

k

k k

p p p

p p p

π

=

⎡ ⎤ ⎛ ⎞ ⎛ ⎞

= + ÷+ ⎢ ⎥ ⎜ ⎟ ⎜ ⎟ ⎝ ⎠ −− − ⎣ ⎦ ⎝ ⎠ ∑ . 
3-11 

Table 3-3. Distribution of the ECLS-K primary sampling unit (PSU) sample by self-representing (SR) 

status, metropolitan statistical area (MSA) status, and census region: School year 1998–99 

 Census region1

SR status MSA status Total Northeast Midwest South West

 

Total 100 18 25 34 23

SR MSA 24 6 5 6 7

Non-SR MSA 52 10 12 18 12

Non-SR Non-MSA 24 2 8 10 4

1

 A census region is a geographic region defined by the U.S. Bureau of the Census. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), fall 1998 and spring 1999. 

3.2.2 Selecting the School Sample 

In the second stage of sampling, public and private schools offering kindergarten programs 

were selected. For each ECLS-K PSU, a frame of public and private schools offering kindergarten 

programs was constructed, using existing school universe files: the 1995-96 Common Core of Data 

(CCD) (U.S. Department of Education 1995-96) and the 1995-96 Private School Universe Survey (PSS) 

(Broughman and Colaciello 1998). The school frame was freshened in the spring of 1998 to include 

newly opened schools that were not included in the CCD and PSS and schools that were in the CCD and 

PSS but did not offer kindergarten, according to those sources. A school sample supplement was selected 

from the supplemental frame. 

3.2.2.1 School Frame Construction 

The 1995-96 CCD Public School Universe File was the primary source for the ECLS-K 

public school sampling frame. Most schools run by the U.S. Department of the Interior, Bureau of Indian 

Affairs (BIA) and the schools run by the U.S. Department of Defense (DOD) were not included on the 

1995-96 CCD. The 1995-96 Office of Indian Education Programs Education Directory (U.S. Department 

of the Interior, Bureau of Indian Affairs unpublished document) was consulted, in order to complete the 

list of BIA schools in the CCD file. For the DOD schools, a 1996 list of schools obtained directly from 

the DOD was used. The 1995-96 PSS Universe File was used as the primary source of the private school 

sampling frame. 
3-12 

The first step in frame construction involved subsetting the file to schools located in counties 

that constituted the ECLS-K PSU sample. Further subsetting retained only those schools that offered 

transitional kindergarten, kindergarten, or transitional first grade, or which were strictly ungraded, as 

indicated by the school’s grade span. 

The constructed ECLS-K school frame included 18,911 public-school records and 12,412 

private-school records. This frame constituted the original frame. The original frame was supplemented in 

the spring of 1998 to include schools that would be operational in fall 1998 but had not been included in 

the original frame. The procedures used to construct the supplemental or freshened frame are given later 

in this section. 

Table 3-4 gives the estimated number of schools offering kindergarten programs and the 

number of kindergarten students from the ECLS-K school frame. These are the numbers of schools and 

students in the sampled PSUs in the frame weighted by the inverse of the PSU selection probabilities. 

Table 3-4. Estimates of the number of kindergarten schools and students, by primary sampling unit 

(PSU) status: School year 1998–99 

 Estimated number of 

kindergarten schools 

Estimated number of 

kindergarten students 

 Total Public Private Total Public Private

 Total 73,095 50,084 23,011 4,089,781 3,521,040 568,741

Self-representing PSUs 19,721 11,283 8,438 1,277,419 1,059,535 217,884

Non-self-representing PSUs 53,374 38,801 14,573 2,812,362 2,461,505 350,857

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), fall 1998 and spring 1999. 

3.2.2.2 School Measure of Size 

Within each PSU, schools with fewer than a predetermined minimum number of 

kindergarten students were clustered together before sampling in order to obtain a sample that is closer to 

self-weighting. The minimum number of kindergartners was 24 for public schools and 12 for private 

schools. Schools were selected with probability proportional to size. As with the PSU sample, a weighted 

MOS was constructed taking into account the oversampling of APIs: 
3-13 

, , 2.5 ij API ij other ij SCHMOS n n = × +

where 2.5 is the oversampling rate for APIs, and nAPI,ij and nother,ij are the counts of API kindergarten 

students and all other kindergarten students, respectively, in school j of PSU i. 

3.2.2.3 School Allocation 

Schools were sampled at rates designed to result in an approximately self-weighting sample 

of students within public and private school strata. The target number of sampled schools per PSU was 

calculated separately for public schools and private schools, and for self-representing and non-selfrepresenting PSUs. The number of schools selected was the target number of schools adjusted upward by 

the estimated school response and eligibility rate. 

3.2.2.3.1 Public Schools 

The total MOS for public schools was partitioned into the self-representing and non-selfrepresenting strata. There are 100 PSUs in the ECLS-K sample, of which 24 are in the self-representing 

strata. The number of public schools selected from the self-representing strata was calculated as 

24

1

100

1

i i

i

SR

i i

i

w PSUMOS

n n

w PSUMOS

=

=

×

= ×

×

∑

∑

where n is the total number of public schools to be selected, wi is the weight of PSU i, and 

i ij

j

PSUMOS SCHMOS = ∑ . 

The value for n is 800/.85 = 941 where .85 is the expected eligibility and response rate for 

public schools. The supplement of public schools was expected to add relatively few schools to the frame 

and thus the 85 percent rate was not modified. The distribution of sampled schools was approximately 

291 for self-representing strata and 650 for non-self-representing strata. For self-representing and non-
3-14 

self-representing strata alike, the number of schools allocated to each PSU was proportional to the 

weighted MOS of the PSU (wi×PSUMOSi). 

In the ECLS-K public school frame, 4 percent of public schools had fewer than 24 

kindergarten students. These schools were combined with other schools in the same PSU to form clusters 

with at least 24 students prior to sampling. Schools with 24 students or more were not grouped, but were 

also referred to as clusters (of one school each). To sample approximately 941 public schools, around 915 

clusters (single schools or groups of schools) have to be selected. As a general rule, if a sampled school or 

cluster of schools had 24 or more students, 24 students were selected. However, for practical reasons, all 

students in the sampled school or cluster were selected if there were fewer than 27 students. More details 

on the clustering of schools are found in the next section. 

The number of clusters was allocated to each PSU proportionally to the weighted MOS of 

the PSU (wi×PSUMOSi). When the 915 clusters were allocated to PSUs, it was discovered that in 5 PSUs 

there were not enough clusters in the frame to select the required number of clusters. As a result, only 900 

clusters were selected. Table 3-5 shows the expected distributions of clusters, schools, and students. 

Table 3-5. Expected number of clusters, schools, and students—public schools: School year 1998–99 

Type of primary sampling 

unit (PSU) 

Number of 

clusters to 

select

Expected 

number of 

schools sampled

Expected 

number of 

students sampled 

Average 

number of 

students/school

 Total 900 944 21,643 23

Self-representing PSUs 283 285 6,792 24

Non-self-representing PSUs 617 659 14,851 23

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class 

of 1998–99 (ECLS-K), fall 1998 and spring 1999. 

3.2.2.3.2 Private Schools 

The procedure used to determine the allocation of the public schools was also used for 

allocating the private schools. The private school target samples are labeled nSR′ and nNSR′ for selfrepresenting and non-self-representing PSUs respectively, and n′ is the sum of nSR′ and nNSR′. The value 

of n′ is 200/.60=333, where .60 is the expected eligibility and response rate. The supplement to the frame 
3-15 

was expected to add some private schools with kindergarten programs. The 60 percent rate was used 

because of the uncertainties associated with the estimate of the eligibility and response rate for private 

schools. 

The percentage of schools with fewer than 24 kindergarten students was large for private 

schools. Approximately 56 percent of private schools offered a kindergarten program that had fewer than 

24 students, and 44 percent of these small schools have fewer than 12 students in their kindergarten 

program. Schools having fewer than 12 kindergarten students (according to the frame) were grouped into 

clusters of schools with at least 12 students in each cluster, following the clustering rules discussed in the 

next section. Schools with 12 students or more were not grouped. As a general rule, if a sampled school or 

cluster of schools had 24 or more students, 24 students were selected; if a sampled school or cluster had 

fewer than 24, all students were sampled. However, for practical reasons, all students in the sampled 

school or cluster were selected if there were fewer than 27 students. 

In order to sample approximately 333 private schools, 278 clusters were selected (single 

schools or groups of schools). Table 3-6 shows the expected distributions of clusters, schools, and 

students. 

The number of clusters was not allocated separately to each self-representing PSU, since 

sampling was done on the aggregated list of school clusters in the self-representing PSUs. This 

aggregated list of school clusters in the self-representing PSUs had been sorted prior to sampling by 

religious affiliation in order to have better control of the sample distribution by religious affiliation. For 

the non-self-representing PSUs, the sample was allocated to each PSU proportionally to the weighted 

MOS of the PSU (wi×PSUMOSi), with a minimum of one cluster per PSU imposed if the PSU was so 

small that it was not allocated any clusters. 
3-16 

Table 3-6. Expected number of clusters, schools, and students—private schools: School year 1998–99 

Type of primary sampling 

unit (PSU) 

Number of 

clusters to 

select

Expected 

number of 

schools sampled

Expected 

number of 

students sampled 

Average 

number of 

students/school

 Total 278 333 6,336 19

Self-representing PSUs 107 125 2,456 20

Non-self-representing PSUs 171 208 3,880 19

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), fall 1998 and spring 1999. 

3.2.2.4 Clustering of Small Schools 

As noted above, schools with fewer than 24 students (public) or 12 students (private) were 

clustered together in order to obtain a sample that was closer to self-weighting. For example, if a school 

with 12 students was not clustered, the students from that school would be sampled at about half the 

probability as students in larger schools. The goal of the clustering of small schools was to form school 

clusters with a small number of schools, each cluster having close to 24 students and including 

heterogeneous schools. This goal was set so that if a cluster was selected, it would not be necessary to 

recruit many small schools; furthermore, the heterogeneity of schools improves the reliability of the 

estimates. Heterogeneity was defined by school size for public schools, and by religious affiliation and 

school size for private schools. Within each PSU, schools with fewer than a predetermined minimum 

number of kindergarten students were separated from the frame and clustered together. A few exceptions 

to this general rule did occur and are discussed later. The procedures for clustering of schools are 

described below. 

3.2.2.4.1 Public Schools 

Public schools with fewer than 24 kindergarten students were clustered. Within each PSU, 

the list of small schools (i.e., schools with fewer than 24 kindergartners) was sorted in ascending order of 

kindergarten enrollment; it was then split in half, with the second half re-sorted in descending order. The 

two halves were then put together in an interleaving fashion. Beginning at the top of the list, clusters of 
3-17 

schools with at least 24 kindergarten students were formed. If the last cluster on the list still did not have 

the required 24 minimum, then it was put together with the next-to-last cluster on the list. 

This clustering scheme resulted in 18 clusters with 5 or more schools, which were 

considered problematic as far as fieldwork was concerned. The worst case was one cluster with 13 

schools and only 41 students. In order to minimize the number of clusters having 5 or more schools, each 

problematic cluster was broken into groups of 2 or 3 schools, and each group was combined with the 

smallest of the “large” schools having 25 or more kindergarten students. Since enrollment in schools with 

missing kindergarten enrollment was imputed to be equal to 24, grouping any of these imputed schools 

with another school was avoided, lest they turn out not to have kindergarten students. 

In addition to the 18 problematic clusters above, there were 12 PSUs with only 1 small 

school (with fewer than 24 kindergarten students) and there were 2 PSUs with only 2 small schools that, 

when grouped together, still had fewer than 24 kindergarten students. These small schools or groups of 

small schools were manually combined with the smallest school in another PSU (not one with only 1 or 2 

schools) having 25 or more students (see table 3-7). 

Table 3-7. Number of clusters and schools in the public school frame: School year 1998–99 

Number in cluster Number of clusters Number of schools

 Total 18,399 18,911

1 school 18,095 18,095

2 schools 153 306

3 schools 97 291

4 schools 51 204

5 schools 3 15

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), fall 1998 and spring 1999. 

3.2.2.4.2 Private Schools 

Private schools with fewer than 12 kindergarten students were clustered. Within each PSU, 

the list of private schools was first sorted by religious and nonreligious affiliation. If the number of 

religious schools and nonreligious schools in the PSU differed by no more than a factor of 3, the smaller 

of the two lists (religious or nonreligious) was sorted in descending order while the larger of the two lists 
3-18 

was sorted in ascending order of kindergarten enrollment. The two lists were then put together in an 

interleaving fashion, so that the records that were at the bottom of the longer list were records with larger 

kindergarten enrollment, and did not have to be grouped together. Beginning at the top of the entire list, 

clusters of schools of at least 12 kindergarten students were formed. If the last cluster on the list still did 

not have the required minimum size, it was put together with the next-to-last cluster on the list. 

If the number of religious schools and nonreligious schools in the PSU differed by a factor 

greater than 3, schools were not separated into religious and nonreligious lists. Instead, the entire list of 

schools was sorted in ascending order of kindergarten enrollment; it was then split in half, with the second 

half re-sorted in descending order. The two halves were then put together in an interleaving fashion. 

Clusters of schools were formed as above. 

There were 3 PSUs where the clustering of small schools as specified above did not work 

well. Two of the 3 PSUs had only 1 small school each and the third one had 2 small schools that, when 

grouped together, still had fewer than 12 kindergarten students. These small schools or groups of small 

schools were manually combined with other large schools in another PSU (table 3-8). 

Table 3-8. Number of clusters and schools in the private school frame: School year 1998–99 

Number in cluster Number of clusters Number of schools

 Total 9,955 12,412

1 school 7,640 7,640

2 schools 2,184 4,368

3 schools 121 363

4 schools 9 36

5 schools 1 5

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), fall 1998 and spring 1999. 

3.2.2.5 Implicit Stratification of Schools/Clusters of Schools 

Public schools with more than 24 kindergarten students and private schools with more than 

12 kindergarten students were not clustered. However, they are referred to as clusters (of one school each) 

for simplicity. 
3-19 

3.2.2.5.1 Public Schools 

Within each PSU, the clusters were sorted by the MOS and separated into three size classes 

of roughly equal size (high, medium, and low). Within each size class, clusters were sorted by the 

proportion of APIs in a serpentine manner (alternating the sort order from one size class to the next). 

3.2.2.5.2 Private Schools 

Within each PSU, each cluster was identified as religious, mixed, or nonreligious.7 The list 

of clusters was then sorted by these three categories. Within each category, the clusters were sorted in a 

serpentine manner by the MOS prior to selection. However, for the self-representing PSUs, all clusters 

were sorted as if they were from the same PSU, i.e., the aggregated list of clusters from the 24 selfrepresenting PSUs was sorted by religious affiliation (religious/mixed/nonreligious). This procedure 

provided better control of the sample distribution of religious/mixed/nonreligious clusters. Across nonself-representing PSUs, clusters were sorted by religious affiliation, and within each category of religious 

affiliation, by the MOS in a serpentine manner. 

3.2.2.6 School Selection 

Selection of the clusters of schools was systematic, with probability proportional to the 

MOS. Sampling of public schools was done independently within PSU (i.e., each PSU forms a separate 

sampling stratum) after the clusters of schools were sorted by MOS and proportion of API. Sampling of 

private schools was done separately for self-representing PSUs and for non-self-representing PSUs. All 

self-representing PSUs were placed in one sampling stratum and all non-self-representing PSUs were 

placed in a second stratum. In the self-representing stratum, sampling was done with one random start 

after sorting clusters of schools by religious affiliation and MOS. In the non-self-representing stratum, 

sampling was done with one random start after sorting clusters of schools by PSU, religious affiliation, 

and MOS. 

 

7

 A private school cluster is “religious” if all schools in the cluster are Catholic schools or non-Catholic religious schools; “nonreligious” if all 

schools in the clusters have no religious affiliation; “mixed” if it has a combination of schools with or without religious affiliation. 
3-20 

3.2.2.7 The ECLS-K Main School Sample 

A total of 1,280 schools were selected from the main school frame for the ECLS-K, of which 

934 were public and 346 were private schools. The characteristics of the school sample are presented in 

table 3-9. 

3.2.2.8 Supplemental School Sample 

As mentioned earlier, the public and private school frames were supplemented in the spring 

of 1998. The procedures for supplementing the frames were different for public schools, Catholic schools 

and non-Catholic private schools. These procedures are discussed below separately. 

3.2.2.8.1 Public Schools 

Each public school district having one or more schools sampled was sent a sampling framebased list of all schools offering kindergarten. Districts were asked whether any school that was expected 

to offer kindergarten in academic year 1998–1999 was missing from the list. For each school identified by 

the district, school name, address, telephone number, grade span, and kindergarten enrollment were 

obtained. Districts were also contacted that fell within the boundaries of the ECLS-K PSUs, but for which 

the CCD file listed no schools offering kindergarten, unless it was clear from their name that they were 

strictly secondary school districts (e.g., Middlebury Union High School District). The information 

obtained from the school districts was checked against the ECLS-K public school frame to confirm that 

these schools were truly new or newly eligible. Bona fide new schools were given a chance of being 

sampled. A new school’s chance of selection was conditioned on the school district’s probability of 

selection. Overall, 252 new public schools were identified. Of these, 19 were selected, using systematic 

sampling with probability proportional to size where the MOS was the same as it was for schools sampled 

from the main sample. Thus, a total of 953 public schools were included in the sample (934 + 19). 
3-21 

Table 3-9. Number of sample schools, by school characteristics: School year 1998–99 

Characteristic Total Public Private

 Total 1,280 934 346

Region 

 Northeast 238 166 72

 Midwest 297 215 82

 South 420 309 111

 West 325 244 81

Type of locale 

 Large central city 245 164 81

 Mid-size central city 252 176 76

 Urban fringe of large city 386 273 113

 Urban fringe of mid-size city 98 78 20

 Large town 32 25 7

 Small town 107 80 27

 Rural 160 138 22

Kindergarten enrollment

 < 25 210 55 155

 25 – 49 224 110 114

 50 – 99 467 400 67

 100 – 149 236 228 8

 150 – 199 88 86 2

 200 – 249 26 26 0

 250 – 299 15 15 0

 > 300 14 14 0

School affiliation 

 Public 934 934 †

 Catholic 117 † 117

 Non-Catholic, religious 143 † 143

 Nonreligious, private 86 † 86

National school lunch program1

 

 Low (<=25% eligible students) 284 284 † 

 Medium low (>25% and <=50%) 169 169 † 

 Medium high (>50% and <=75%) 122 122 † 

 High (>75%) 118 118 † 

 Unknown 241 241 † 

† Not applicable. 

1

 National school lunch program applies only to public schools and hence the counts of schools in the program do not add up to 1,280 schools. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), fall 1998 and spring 1999. 
3-22 

3.2.2.8.2 Private Schools 

The procedure for obtaining new school information from Catholic dioceses was exactly the 

same as for public schools. Since a diocese could cut across county or even state lines, each school 

identified by a diocese had to be associated with the correct county, and hence the correct PSU, before 

checking to see whether it was truly new. Since dioceses might cross PSU boundaries, a new Catholic 

school’s chance of being sampled had to be conditioned on the diocese’s probability of selection within 

the PSU where the new school was located. There were 126 new Catholic schools identified, and 6 were 

selected using systematic sampling with probability proportional to size. When combined with the main 

sample, the final Catholic school sample size was 123 (117 + 6). 

3.2.2.8.3 Non-Catholic Private Schools 

The search for non-Catholic private schools was considerably more complicated. Three 

classes of schools that had previously not been given a chance of selection from the PSS were 

reconsidered. Those were schools that had an unknown grade span because they had not responded to the 

1995–96 PSS, those that responded but did not report offering kindergarten, and those that appeared for 

the first time on the 1997–98 PSS file. Together these accounted for 2,544 potential new non-Catholic 

private schools. Beyond these additions from PSS, procedures similar to those used by the Bureau of the 

Census in the PSS area frame search were followed. These procedures included collecting lists of schools 

from different sources, matching them against the PSS list frame to remove duplicates, and further 

screening by telephone to verify new school status. The majority of new schools found by the Bureau of 

the Census for PSS came from telephone book yellow page listings. The yellow pages search was the 

main source of new non-Catholic private schools in the ECLS-K as well, yielding an additional 8,861 

possible new private schools. Since the number of kindergartners enrolled in these schools was unknown, 

a minimum kindergarten enrollment was assumed for sampling purposes (typically 24, unless the name 

was suggestive of day care, in which case 12 was assumed). 

The supplemental frame contained 11,405 private schools. A sample of 279 schools was 

selected, using systematic sampling with a probability proportional to these imputed enrollments. Each 

sampled school was contacted by telephone and screened to ascertain whether the school was public or 

private, whether it would be open in academic year 1998–1999; and whether it would offer kindergarten. 
3-23 

If the school met all of these conditions and was not Catholic, the school was eligible and released for 

data collection. 

A second supplemental procedure involved contacting local education agencies (LEAs) and 

local government offices for information on non-Catholic private schools. This procedure was done only 

in the smallest ECLS-K PSUs, on the theory that if these PSUs had coverage problems their large weights 

were likely to introduce a larger bias in the estimates. All LEAs within these PSUs were contacted by 

telephone. For each city/town within the PSU, a list of local government offices was compiled using the 

blue pages. Successive government offices were called within a city or town until one was found that 

could provide information on private schools. As with the yellow pages, new schools identified by LEAs 

and local government offices were unduplicated against the PSS file before being added to the new school 

frame. Since kindergarten enrollment was unknown, it was imputed as described in the previous 

paragraph and sampling was performed using systematic sampling with probability proportional to size. 

The LEA search resulted in the identification of 30 new private schools after unduplication, of which 14 

were sampled. The local government search yielded 19 new schools, of which 8 were sampled. Finally, 

three additional new private schools were reported by field staff based on personal knowledge. Of these, 

two schools were sampled. The same screening procedures to ascertain whether the school was public or 

private; whether it would be open in academic year 1998–1999; and whether it would offer kindergarten 

were then applied to these sampled schools. 

The total number of non-Catholic private schools that were sampled was 303. After the 

screening procedures were applied, only 109 of these schools were eligible. These 109 schools are 

referred to as the supplemental sample of non-Catholic private schools. 

The final ECLS-K school sample for the base year was 1,413 schools, including 953 public 

schools, 123 Catholic schools, and 337 non-Catholic private schools. Of these, 136 schools (72 percent 

private) were later found to be ineligible because they did not have any kindergarten programs; three 

schools participated in fall-kindergarten, but not in spring-kindergarten (1 public and 2 private); 259 

schools (38 percent private) refused to participate in both fall and spring; and 65 schools (42 percent 

private) refused to participate in the fall but were converted to cooperating schools in the spring during 

the spring refusal conversion. At the end of the base year, 1,014 schools were still participating in the 

ECLS-K. 
3-24 

3.2.3 Sampling Children, Parents, and Teachers Within Schools 

The goal of the student sample design was to obtain an approximately self-weighting sample 

of students to the extent possible while achieving the minimum required sample size for APIs (the only 

subgroup that needed to be oversampled to meet the study’s precision goals). Two independent sampling 

strata were formed within each school, one containing API students and the second all other students. 

Within each stratum, students were selected using equal probability systematic sampling, using a higher 

rate for the API stratum. In general, the target number of children sampled at any one school was 24. The 

actual sample size per school ranged from 1 to 28. If one twin was selected into the sample then both 

twins were included, raising the maximum number of children to sample from 24 to 28 in a small number 

of schools. Once the sampled children were identified, parent contact information was obtained from the 

school and was used to identify a parent or guardian for the parent interview. 

During the fall-kindergarten data collection, a census of kindergarten teachers was taken at 

each school. In spring-kindergarten, new teachers who had joined the schools and teachers in schools 

participating after the fall were added to the census of teachers. In the spring-first and spring-third grade 

data collections, the only teachers included were the teachers of the sampled children. For every data 

collection, each sampled child was linked to his or her teacher. A child could be linked to only one 

general education teacher. In cases where a child had more than one general education teacher, a 

‘primary’ teacher was identified for the child. In addition, special education teachers and service 

providers were linked to sample cases who received such services. As with the general education 

teachers, a child would be linked to only one special education teacher or service provider. Details on the 

linking of teachers to the children are found in chapter 4. 

3.3 Fall-First Grade Subsample 

The fall data collection consisted of a 30 percent sample of schools containing 

approximately 25 percent of the base year students eligible to participate in the second year. The goal of 

this subsample was to measure the extent of summer learning loss and the factors that contribute to such 

loss and to better disentangle school and home effects on children’s learning 
3-25 

3.3.1 PSU Sample 

A subsample of ECLS-K PSUs was selected for the fall-first grade data collection. All 24 of 

the self-representing PSUs were retained. Of the 76 non-self-representing PSUs, 38 were retained by 

sampling one PSU per stratum with equal probability. 

3.3.2 School Sample 

Base year schools in the 62 fall-first grade sampled PSUs were stratified by frame source 

(original public, original private, supplemental public, and supplemental private as described in 

section 3.2.2.8) and arranged in their original selection order. A 30 percent equal probability sample of 

schools was drawn in the 24 self-representing PSUs and a 60 percent sample of schools was drawn in the 

38 non-self-representing PSUs. In total 311 schools that had cooperated in either fall- or springkindergarten were selected. The characteristics of the base year cooperating schools selected for fall-first 

grade are presented in table 3-10. 

3.3.3 Child Sample 

Fall-first grade data collection consisted of the direct child assessment and the parent 

interview. Data collection was attempted for every eligible child found still attending the school in which 

he or she had been sampled during kindergarten. “Eligible” was defined as a base year respondent (i.e., a 

child who had either a fall- or spring-kindergarten child assessment or parent interview or was excluded 

from assessment because of a disability or because the child belonged in the language minority (not 

Spanish) group. Base year nonrespondents were not sampled and were handled by adjusting the weights 

(see section 7.2.1.2.1 for details of adjustment for base year nonresponse). 

Because of the additional burden of school recruiting, the cost of collecting data for a child 

who had transferred from the school in which he or she was originally sampled exceeds that for a child 

who stayed enrolled. To contain these costs, a random 50 percent of children were subsampled to be 

followed for fall-first grade data collection in the event that they had transferred. 
3-26 

Table 3-10. Characteristics of base year cooperating schools selected for fall-first grade: School year 

1999–2000 

Characteristic Total Public Private

 Total 311 228 83

Region 

Northeast 

Midwest 

South 

West 

57 

83 

99 

72

39 

59 

77 

53 

18 

24 

22 

19

Type of locale 

Large city 

Midsize city 

Urban fringe of large city 

Urban fringe of midsize city 

Large town 

Small town 

Rural 

62 

59 

86 

18 

15 

28 

43

42 

45 

61 

14 

12 

19 

35 

20 

14 

25 

4 

3 

9 

8

School affiliation 

Public 

Catholic 

Other religious 

Nonreligious, private 

228 

29 

33 

21

228 

† 

† 

†

†

29 

33 

21

School type 

Regular 

Ungraded 

No grade beyond kindergarten 

292 

1 

18

222 

1 

5 

70 

0 

13

† Not applicable. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), fall 1999 and spring 2000. 

Except for children who were repeating kindergarten, all base year children sampled in 

schools with a high grade of kindergarten are de facto movers. Since many of these movers might move 

en masse to the same first-grade school, steps were taken to follow these children at a higher rate. Using 

the information collected during spring-kindergarten, a list of destination schools was compiled for each 

such school. The destination school having the most movers was designated as primary, unless no such 

school had more than three movers. Children who moved en masse into a primary destination school in 

fall-first grade were treated as “nonmovers” and were not subsampled, that is, they continued to be 

followed and were part of the ECLS-K sample. All other movers were sampled at the rate of 50 percent. 
3-27 

As discussed above, a random 50 percent of children were subsampled to be followed if they 

moved out of the kindergarten school. Sampling was done with equal probability. Prior to sampling, 

children were stratified into groups of nonmovers, movers with information identifying their new schools, 

and movers without such identifying information. A flag was created for each child indicating whether the 

child had been sampled to be followed. 

3.4 Spring-First Grade Sample 

The ECLS-K spring-first grade data collection targeted all base year respondents and not just 

the fall-first grade subsample. Hence, the sample includes children who were assessed and whose parents 

were interviewed in fall- or spring-kindergarten, as well as the 70 children who could not be assessed in 

fall- or spring-kindergarten because of a disability or because they belonged in the language minority (not 

Spanish) group. In addition, the spring student sample was freshened to include current first-graders who 

had not been enrolled in kindergarten in 1998–99 and therefore had no chance of being included in the 

ECLS-K base year kindergarten sample. This group includes children who skipped kindergarten 

altogether in 1998–99, children who attended a kindergarten program outside of the U.S. in 1998–99, and 

children who were in first grade in 1998–99 and repeating it in 1999–2000. While all students still 

enrolled in their base year schools were recontacted, only a 50 percent subsample of base year sampled 

students who had transferred from their kindergarten school was followed for data collection. 

3.4.1 Subsampling Movers 

In spring-first grade all children in a random 50 percent subsample of base year schools were 

flagged to be followed for data collection if they transferred from their base year school. (This is in 

contrast to fall-first grade where a random 50 percent of children in each of the 30 percent of schools 

subsampled were flagged.) In order to maximize the amount of longitudinal data, care was taken during 

spring-first grade sampling to ensure that any child who had been flagged to be followed in fall-first grade 

would continue to be followed. 

In selecting the spring-first grade 50 percent subsample of schools where movers would be 

flagged for followup, the three primary strata were self-representing PSUs, non-self-representing PSUs 

that had been selected for fall-first grade, and non-self-representing PSUs that had not been selected for 
3-28 

fall-first grade. Within these major strata, schools were grouped by frame source (original public, original 

private, supplemental public, and supplemental private). Finally within each frame source, schools were 

stratified by whether the school participated in the base year study, and arranged in original selection 

order. Schools that had been part of the 30 percent fall-first grade sample were automatically retained. 

Then equal probability sampling methods were employed to augment the sample to the desired 50 percent 

of schools. The net result of these procedures was that every base year selected school had a 50 percent 

chance of having its ECLS-K movers followed during spring-first grade, and any mover who had been 

followed in fall-first grade would still be followed in spring-first grade. 

3.4.2 Sample Freshening 

As noted earlier, a sample freshening procedure was used to make it possible to produce 

estimates of all children enrolled in first grade in the spring of 2000. The spring-first grade student 

freshening used a half-open interval sampling procedure (Kish 1965). The procedure was implemented in 

the same 50 percent subsample of ECLS-K base year schools where movers were flagged for followup. 

Each of these schools was asked to prepare an alphabetic roster of students enrolled in first grade and the 

names of ECLS-K kindergarten-sampled children were identified on this list. Beginning with the name of 

the ECLS-K first kindergarten-sampled child, school records were checked to see whether the student 

directly below in the sorted list attended kindergarten in the United States in fall 1998. If not, (1) that 

child was considered to be part of the freshened sample and was linked to the base year sampled student 

(i.e., was assigned that student’s probability of selection), and (2) the record search procedure was 

repeated for the next listed child, and so forth. When the record search revealed that a child had been 

enrolled in kindergarten the previous year, that child was not considered part of the freshened sample and 

the procedure was resumed with the second base year ECLS-K sampled student name, and so on.8

Student freshening brought 165 first graders into the ECLS-K sample, which increased the weighted 

survey estimate of the number of first graders in the United States by about 2.6 percent. 

The student freshening procedure was not entirely free of bias. A first grader would have no 

chance of being in the ECLS-K first grade sample if he or she was enrolled in a school where neither the 

child nor any of his or her classmates had attended kindergarten in the United States in fall 1998. This 

would be a rare circumstance and is not thought to be an important source of bias. A more significant 

 

8

 The student roster was “circularized” (i.e., the first name on the roster was considered to follow the last name on the roster in the 

implementation of the procedure). 
3-29 

source of potential bias is nonresponse. One source of nonresponse inherent to the freshening plan was 

that the procedure only involved students who had not transferred from the school in which they had been 

sampled during the base year. Another source of nonresponse that also affected the freshening procedure 

were schools that refused to provide or could not provide the necessary information, such as alphabetic 

roster of students enrolled in first grade or whether students had attended kindergarten the previous year. 

The school freshening completion rate is slightly higher for public schools than for private schools. Of the 

494 schools eligible for freshening, 380 are public schools and 114 are private schools. Ninety four 

percent of the public schools and 93 percent of the private schools participated in the freshening process. 

3.5 Spring-Third Grade Sample 

The procedures used in spring-first grade to subsample movers reduced the loss in sample 

size and reduced data collection costs since movers cost considerably more to interview than nonmovers. 

These procedures were also used for the ECLS-K third-grade data collection with some modifications. 

One reason for modifying the procedures was that some children had already moved out of their original 

school, and some of the movers were sampled and some were not. In addition, there were concerns about 

special domains of interest and methods that might be used to increase the sample size for the children in 

these groups. Results from the first-grade collection were used to address these third-grade sample design 

issues. 

3.5.1 Estimates from Spring-First Grade 

Table 3-11 presents data on the outcome of the spring-first grade data collection activities, 

by subgroups of interest and by mover status. In this table and subsequent tables, school affiliation and 

type of locale are those of the original sample schools. Race/ethnicity and language characteristics of the 

children are from the ECLS-K base year data, which are available for all children. Data from first grade 

are only available for first-grade respondents. For children sampled in first grade, data are from springfirst grade. The table shows that overall 26 percent (5,477) of the children moved from the school they 

were sampled in, about 48 percent (2,620) of these movers were sampled, and the unweighted completion 

rate for movers was 83 percent (1,967 mover respondents). For nonmovers, the completion rate was 
3-30

Table 3-11. Spring-first grade data collection results by mover status: School year 1999–2000 

Mover status All sampled children Movers 

Subgroup1 Total2 Respond3 Not respond4 Ineligible

Unweighted 

completion 

rate5

Movers 

not 

sampled

Total 

movers

Percent 

moved Respond3

Not 

respond4 Ineligible

Percent 

sampled

Unweighted 

completion 

rate5 Resp

 All 21,331 17,324 899 251 95.1 2,857 5,477 25.7 1,967 403 250 47.8 83.0 15School affiliation 

Public 16,761 13,661 710 221 95.1 2,169 4,189 25.0 1,466 334 220 48.2 81.4 12Private 4,570 3,663 189 30 95.1 688 1,288 28.2 501 69 30 46.6 87.9 3

Catholic 2,354 2,031 66 12 96.9 245 433 18.4 150 26 12 43.4 85.2 1Non-Catholic 2,216 1,632 123 18 93.0 443 855 38.6 351 43 18 48.2 89.1 1

Type of locale 

Rural 2,509 2,227 86 14 96.3 182 428 17.1 194 39 13 57.5 83.3 2Non-Rural 18,822 15,097 813 237 94.9 2,675 5,049 26.8 1,773 364 237 47.0 83.0 13

Race/ethnicity 16

Hispanic 3,777 2,988 164 81 94.8 544 1,048 27.7 349 75 80 48.1 82.3 2Black 3,229 2,468 150 69 94.3 542 1,066 33.0 374 81 69 49.2 82.2 2Asian/Pacific  Islander 1,579 1,291 61 18 95.5 209 359 22.7 114 18 18 41.8 86.4 1Other 12,746 10,577 524 83 95.3 1,562 3,004 23.6 1,130 229 83 48.0 83.1 9

Race/ethnicity 2 6

Hispanic 3,698 2,926 160 81 94.8 531 1,023 27.7 340 72 80 48.1 82.5 2Black 3,229 2,468 150 69 94.3 542 1,066 33.0 374 81 69 49.2 82.2 2Asian/Pacific  Islander 1,867 1,537 69 20 95.7 241 429 23.0 144 24 20 43.8 85.7 1Other 12,537 10,393 520 81 95.2 1,543 2,959 23.6 1,109 226 81 47.9 83.1 9

Language minority 

Non-English 5,372 4,317 228 107 95.0 720 1,397 26.0 472 98 107 48.5 82.8 3English 15,959 13,007 671 144 95.1 2,137 4,080 25.6 1,495 305 143 47.6 83.1 11

1 Characteristics of the schools (school affiliation and type of locale) are from the original sample schools. 2 The total number of children excludes 68 children who responded in fall-kindergarten and became ineligible in spring-kindergarten, and includes 139 children sampled 3 A respondent is a child with assessment data or parent interview data, or a child who could not be assessed due to a disability. 4 Nonrespondents include those who did not participate fully and movers who could not be located. 5 The unweighted completion rate was computed as the number of respondents divided by the sum of respondents and nonrespondents. 6 Race/ethnicity 1 was the strict definition of API (RACE=5-Asian, or 6-Native Hawaiian or other Pacific Islander), while race/ethnicity 2 was the broader definiti

Hawaiian or other Pacific Islander or WKASIAN=1-Child is Asian, or WKPACISL=1-Child is Native Hawaiian or other Pacific Islander). Variables are from the ECLSSOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 kindergarten 

2000. 
3-31 

97 percent (15,357 nonmover respondents). A child was considered a respondent in these computations if 

either the child assessment or the parent interview was completed for spring-first grade or the child was 

not assessed due to a disability. The completion rate in this table was computed as the number of 

respondents divided by the sum of respondents and nonrespondents. Nonrespondents include those who 

did not participate fully and movers who could not be located. For first grade, 269 of the movers who 

were sampled for follow-up could not be located, or about 11 percent of all movers eligible for the first 

grade data collection. 

The mover rates show the types of variation that had been expected, with higher mover rates 

for Black and Hispanic children, for example. A total of 39 percent of the children in non-Catholic private 

schools moved to other schools when they advanced from kindergarten to first grade (855 movers as 

shown in table 3.11). Seventy-six percent of children who moved from kindergarten in private schools to 

first grade in public schools attended non-Catholic private schools in kindergarten, about three times the 

number of children who moved from Catholic to public schools. 

One of the concerns in using the kindergarten to first-grade mover rates to make estimates 

for future transitions was whether the mover rates for the 1-year time period between kindergarten and 

first grade were reasonable when applied to the transition between first and third grade. One might argue 

that a 2-year period should result in a higher mover rate than the 1-year rate. However, parents may be 

more reluctant to change the school for a child between first and third grade than between kindergarten 

and first grade. Kindergarten is also special for other reasons. For example, the availability of full- and 

part-day classes may be an important factor in the choice of the kindergarten. There are no other data 

sources that could be used to examine differential mover rates between years. As a result, the 1-year 

moving rates in table 3-11 were applied to the 2-year period between first and third grade after adding 

another 5 percent to the rates to account for the 2-year period. An exception was made for children who 

attended non-Catholic private schools in the base year and had the highest rates of moving among all the 

domains examined. This was assumed to be a special case for kindergarten and the average mover 

percentage was applied to these children for the third grade. 

The other main concern was whether the extremely high completion rate for nonmovers 

(97 percent) could be duplicated in future years. To be more conservative and to account for the fact that 

nonrespondents from earlier rounds (i.e., base year respondents were included in the third-grade sample 

whether or not they responded in first grade) were included in subsequent rounds of data collection, it was 

assumed that a 95 percent completion rate would be achieved for nonmovers in third grade. 
3-32 

3.5.2 Third-Grade Sample Design 

The basic plan for third grade was the plan implemented for first grade where only 50 

percent of the children who moved from the original sample schools were followed into their new 

schools. This plan was modified for third grade as described below. To be eligible for the third-grade 

sample, a child had to have been a base year respondent or sampled in first grade. Children who moved 

out of the country or died were excluded (i.e., ineligible). The following children were fielded for third 

grade: 

 All the children responding in the base year who remained in their original schools, 

where the original schools also included destination schools (described later). 

 All the children who moved from an original school in a previous wave of data 

collection and were retained in the subsample of movers for that wave. For example, 

if a child moved between kindergarten and first grade and was part of the 50 percent 

subsample that was followed, then the child would be retained for future rounds 

without subsampling as long as the child remained eligible. 

 A subsample of 50 percent of the children who moved from their original school at 

any time after the base year. For example, a child who moved between first grade and 

third grade would be subject to subsampling and had a 50 percent chance of being 

included in the third-grade followup. In alternatives discussed later, differential 

subsampling rates were introduced. 

To prevent an accumulation of nonresponse, the ECLS-K design does not use the approach 

of many longitudinal studies that exclude sampled units from future rounds if they did not respond in a 

particular wave. Instead, the basic plan was modified so that all eligible base-year respondents who were 

sampled in the first-grade followup would be eligible for the third-grade followup even if they did not 

respond in the first grade. Even though the participation rate for first-grade nonrespondents might be 

lower compared with first-grade respondents in the subsequent followups, the effort was an attempt to 

increase overall response rates by including first-grade nonrespondents in third grade. The approach is 

also consistent with the analytic use of the data for the ECLS-K, since many analyses may include less 

than complete wave responses. For example, a change in scores from kindergarten to third grade for 

subgroups is an important analytic objective, and it can be estimated without complete data at each wave. 

A second procedure that was part of the modification of the basic plan for the third-grade 

followup was an extension of a procedure that was used in the first-grade followup to deal with schools 

that ended with kindergarten (i.e., kindergarten was the highest grade offered). A school was called a 

destination school if at least 4 students from a school ending in kindergarten attended this school in first 

grade. For the third grade 28 original schools ended in second grade, and 3 of the destination schools 
3-33 

identified in first grade ended in second grade. In total, 3 percent of all eligible first graders in the 

ECLS-K sample attended schools ending in second grade. As was done for the first-grade sample, 

children in the destination schools were treated as nonmovers for the third-grade sample. As nonmovers, 

they were all followed into their new schools, resulting in a 2 percent increase of the third-grade sample 

size over that which would result if 50 percent of these children were subsampled out as movers. 

3.5.3 Expected Sample Size 

Table 3-12 gives the expected sample sizes of children in third grade by subgroups of 

interest and mover status for the basic plan. In this table, a respondent is defined as a child with either a 

complete child assessment or parent interview, or a child who could not be assessed because of a 

disability. This table shows that the expected number of children with completed child assessments or 

parent interviews in the third grade was 14,304 under the assumed mover rates (differential by subgroups 

as shown in the table), subsampling rate (47 percent instead of 50 percent to account for ineligibility in 

third grade), and completion rate (95 percent). The estimates for the selected groups of high interest are 

given in the rows below. The third from last column is the estimated design effect resulting from 

sampling movers and nonmovers differentially. It does not include any other factors such as clustering. 

The next to last column is an estimate of the number of new schools that would enter the ECLS-K 

sample. The last column is an estimate of the total number of schools that sampled children would be 

attending, assuming 1.5 movers attended the same school on average. 
3-34

Table 3-12. Expected sample size for selected subgroups for third grade, by mover status: School year 2001–02  Sampled from first grade Mover2 Completion rate2 Total responding (expected) Subgroup1 Total Movers Nonmovers Rate Sampled3 Movers Nonmovers

Sample of 

new movers Total Movers Nonmovers

Su

 All children 18,223 2,370 15,853 — — — — 2,284 14,304 3,860 10,444 School affiliation  Public 14,371 1,800 12,571 30 47 81 95 1,772 11,270 2,909 8,361  Private 3,852 570 3,282 33 47 88 95 512 3,034 951 2,083  Catholic 2,097 176 1,921 23 47 85 95 211 1,728 330 1,398  Non-Catholic 1,755 394 1,361 31 47 89 95 198 1,420 528 892 Type of locale  Rural 2,313 233 2,080 22 47 83 95 216 1,914 374 1,540  Non-rural 15,910 2,137 13,773 32 47 83 95 2,080 12,402 3,482 8,920 Race/ethnicity 15 Hispanic 3,152 424 2,728 33 47 82 95 420 2,438 695 1,743  Black 2,618 455 2,163 38 47 82 95 386 1,965 692 1,274  Asian/Pacific Islander 1,352 132 1,220 28 47 86 95 159 1,089 251 838  Other 11,101 1,359 9,742 29 47 83 95 1,308 8,829 2,218 6,611 

See notes at end of table. 
3-35

Table 3-12. Expected sample size for selected subgroups for third grade, by mover status: School year 2001–02—Continu Sampled from first grade Mover2 Completion rate2 Total responding (expected) Subgroup1 Total Movers Nonmovers Rate Sampled3 Movers Nonmovers

Sample of 

new movers Total Movers Nonmovers

Su

Race/ethnicity 25 Hispanic 3,086 412 2,674 33 47 83 95 411 2,389 679 1,711  Black 2,618 455 2,163 38 47 82 95 386 1,965 692 1,274  Asian/Pacific Islander 1,606 168 1,438 28 47 86 95 189 1,290 306 984  Other 10,913 1,335 9,578 29 47 83 95 1,288 8,675 2,179 6,497 Language minority  Non-English 4,545 570 3,975 31 47 83 95 579 3,557 952 2,605  English 13,678 1,800 11,878 31 47 83 95 1,706 10,747 2,912 7,835 

— Not available. † Not applicable. 1 Characteristics of the schools (school affiliation and type of locale) are from the original sample schools. 2 The mover rates and the completion rates for movers are those used in the computation of expected sample size, and are differential by subgroups. Since no “total” rat

they are not available. 

3 The sampling rate for movers is set at 47 percent (instead of 50 percent) to account for ineligibility of students in future rounds. 4 The design effects in this column are the results of sampling movers and nonmovers differentially. They do not include the effect of clustering. 5 The number of new third-grade schools is estimated as 1.5 schools per sampled new mover. 6 2,621 schools from kindergarten/first grade plus the new third-grade schools. 7 Race/ethnicity 1 was the strict definition of API (RACE=5-Asian or 6-Native Hawaiian or other Pacific Islander), while race/ethnicity 2 was the broader definit

Hawaiian or other Pacific Islander or WKASIAN=1-Child is Asian or WKPACISL=1-Child is Native Hawaiian or other Pacific Islander). Variables are from the ECLS-

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 kindergarten (E
3-36 

3.5.4 Protecting the Language Minority Children 

Special attention was paid to language minority and API children to ensure that the sample 

sizes would be large enough to support analytic goals in developing the sampling plans for the third 

grade. Children in the language minority group are children whose home language is non-English or who 

were screened using the Oral Language Development Scale (OLDS) prior to assessments during the base 

year (or first grade for freshened children).9

 Two classifications of APIs are shown in tables 3-11 and 3-

12. The first classification was identified using a strict definition of API, i.e., if the child was identified 

only as API by the composite race variable (RACE = 5-Asian or 6-Native Hawaiian or other Pacific 

Islander). The second classification was identified using a broader definition, i.e., if a child was identified 

only as API as in the strict definition (RACE = 5-Asian or 6-Native Hawaiian or other Pacific Islander) or 

if a child has positive answers to the API race identification variables (WKASIAN = 1-Child is Asian or 

WKPACISL = 1-Child is Native Hawaiian or other Pacific Islander). The variables RACE, WKASIAN 

and WKPACISL are in the base year data file. The broader definition of API yields a larger population of 

children. 

After reviewing the expected yields without oversampling, it was decided to increase only 

the sample size for children belonging to the language minority group. Beginning in third grade, these 

children would not be subsampled for followup if they moved from their original school. Instead, data 

collection would be attempted for all language minority children. Table 3-13 is analogous to table 3-12 

but is adjusted for this approach of retaining all movers in the language minority group (in practice the 

subsampling rates are shown as 95 percent because some children became ineligible). 

One consequence of protecting this subgroup is to increase the sample size and precision for 

the subgroup. The design effect due to subsampling is slightly lower under this plan because a smaller 

proportion of the movers were subsampled than under the basic plan (only the movers that were already 

subsampled in first grade are subsampled). Another consequence is that the number of schools that the 

sampled children attended increased. Because all language minority children were followed, table 3-13 

shows an expected increase of 395 schools in third grade (1,918 – 1,523 = 395). 

 

9

 Information about home language came from the parent interview and whether or not children were screened with the OLDS was based on 

information provided by their schools (see the Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K) Base Year PublicUse Data Files and Electronic Code Book: User’s Manual (NCES 2001-029r) (Tourangeau, Burke, et al. 2004). 
3-37 

Table 3-13. Expected sample size for third grade, by mover status, if standard subsampling rates were used and if languag

were preserved: School year 2001–02 

 Sampled from first grade Mover1 Completion rate1 Total responding (expected) Sampling approach Total Movers Nonmovers Rate Sampled2 Movers Nonmovers

Sample of 

new movers Total Movers Nonmovers

Su

Using standard  subsampling rates 18,223 2,370 15,853 — — — — 2,285 3,864 10,440 14,304 

Non-English 4,545 570 3,975 31 47 83 95 579 952 2,605 3,557 English 13,678 1,800 11,878 31 47 83 95 1,706 2,912 7,835 10,747 

Preserving the language  minority group 18,223 2,370 15,853 — — — — 2,877 4,354 10,440 14,794 

Non-English 4,545 570 3,975 31 95 83 95 1,171 1,442 2,605 4,047 English 13,678 1,800 11,878 31 47 83 95 1,706 2,912 7,835 10,747 

— Not available. † Not applicable. 1 The mover rates and the completion rates for movers are those used in the computation of expected sample size, and are differential by subgroups. Since no “total” rat

they are not available. 

2 The sampling rate for movers is set at 47 percent (instead of 50 percent) to account for ineligibility of students in future rounds. If the language minority group is preser3 The design effects in this column are the results of sampling movers and nonmovers differentially. They do not include the effect of clustering. 4 The number of new third-grade schools is estimated as 1.5 schools per sampled new mover. 5 2,621 schools from kindergarten/first grade plus the new third-grade schools. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spri
3-38 

3.5.5 Precision Requirements 

When the precision estimates were computed from the kindergarten sample at the end of the 

base year, higher than expected design effects for assessment scores were observed. The design effects for 

most other statistics, such as proportions of children with a particular characteristic, were moderate and 

within the range expected (1.6 to 6.9 for proportions greater than 30 percent for an average of 4.0). The 

design effects for assessment scores (4.5 to 9.5 for an average of 6.9) were investigated and found to be 

correct and unrelated to data collection artifacts. For example, interviewer effects were found to be 

negligible and did not bias assessment scores. The design effects for test scores were much larger than the 

average of 3.8 that was expected at the design stage. For all students, the design effects for math and 

reading scores were about 6.5, while for general knowledge the design effects were even larger, at 7.7. 

For design effects from the base year, see chapter 4 of the ECLS-K Base Year Public-Use Data Files and 

Electronic Code Book: User’s Manual (NCES 2001–029r) (Tourangeau, Burke, et al. 2004). These larger 

design effects are one component that affects the ability of the survey to meet the precision requirements 

as described in section 3.1. 

The spring-third grade estimates of design effects are similar to those in the earlier rounds 

and are larger than had been predicted prior to any data collection. The longitudinal estimates have design 

effects that are not as large as might be expected given the larger cross-sectional design effects. In fact, 

the correlations for mean test scores are as high as 0.8 to 0.9. The higher than expected correlations make 

estimates of changes in scores over time more precise, thus it is possible to meet the precision 

requirements for estimates of change with smaller sample sizes. Table 3-14 shows that the sample sizes 

for the key analytic subgroups (public, Catholic, non-Catholic, Hispanic, Black, Asian/Pacific Islander, 

other races together, and language minority) were expected to be at least 1,000.10 Samples of this size 

were expected to be sufficient for estimating most characteristics. For example, test scores were expected 

to have a coefficient of variation of about 3 percent with samples of 1,000. More details on estimates of 

design effects can be found in chapter 4 of the ECLS-K User’s Manual for the First Grade Public-Use 

Data Files and Electronic Code Book (NCES 2002-135) (Tourangeau et al. 2002) and the ECLS-K User’s 

Manual for the Third Grade Public-Use Data File and Electronic Code Book (NCES 2004-001) 

(Tourangeau, Brick, et al. 2004). 

 

10 The sample sizes for key analytic groups are the totals for each group, not for each cell in table 3-14. 
3-39 

Table 3-14. Characteristics of third-grade respondents—number of third-graders by subgroup: 

School year 2001–02 

Expected2

 Achieved 

Subgroup1

Total 

Language 

minority

Not 

language 

minority Total

Language 

minority 

Not 

language 

minority

 

 Total 14,794 4,047 10,747 15,305 4,041 11,264

School affiliation 

Public 11,643 3,356 8,287 12,070 3,374 8,696

Private 3,151 691 2,460 3,235 667 2,568

Catholic 1,625 373 1,252 1,817 383 1,434

Non-Catholic 1,526 318 1,208 1,418 284 1,134

 

Type of locale 

Rural 1,712 210 1,502 2,005 222 1,783

Non-rural 13,082 3,837 9,245 13,300 3,819 9,481

Race/ethnicity 13 

Hispanic 2,773 2,165 608 2,752 2,156 596

Black 2,188 128 2,060 2,007 118 1,889

Asian/Pacific Islander 1,165 962 203 1,174 947 227

Other 8,668 792 7,876 9,372 820 8,552

 

Race/ethnicity 23 

Hispanic 2,716 2,130 587 2,691 2,116 575

Black 2,188 128 2,060 2,007 118 1,889

Asian/Pacific Islander 1,369 1,057 312 1,404 1,056 348

Other 8,520 732 7,788 9,203 751 8,452 1

 Characteristics of the schools (school affiliation and type of locale) are from the original sample schools. 2

 The expected sample size was computed using assumed mover rates (differential by subgroups as shown table 3-12), a 47 percent subsampling 

rate (instead of 50 percent to account for ineligibility in third grade), and a 95 percent completion rate. 3

 Race/ethnicity 1 was the strict definition of API (RACE=5-Asian, or 6-Native Hawaiian or other Pacific Islander), while race/ethnicity 2 was the 

broader definition (RACE=5-Asian, or 6-Native Hawaiian or other Pacific Islander or WKASIAN=1-Child is Asian, or WKPACISL=1-Child is 

Native Hawaiian or other Pacific Islander). Variables are from the ECLS-K base year data file. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2002. 

3.5.6 Spring-Third Grade Sampling Outcome 

To summarize, the sample of children for spring-third grade consists of all children who 

were base year respondents and children who were brought into the sample in spring-first grade through 
3-40 

the sample freshening procedure. Sample freshening was not implemented in third grade; hence no new 

students entered the sample. 

While all students still enrolled in their base year schools were recontacted, slightly more 

than 50 percent of the base year sampled students who had transferred from their kindergarten school 

were followed for data collection. This subsample of students was the same 50 percent subsample of base 

year movers followed in spring-first grade, including the movers whose home language was not English 

(language minority students). Children who were followed in spring-first grade were retained in the 

sample (i.e., the mover followup still targeted the same 50 percent subsample of children in the base year 

schools). In addition, children whose home language was not English and who had moved between 

spring-first grade and spring-third grade were all retained rather than being subsampled at the 50 percent 

rate. If they had moved before first grade, they were not to be followed. This modification to the mover 

followup procedure provided a larger sample of children whose home language is not English for analytic 

purposes. The mover followup activities that originally targeted a 50 percent subsample of children in 

base year schools resulted in a 54 percent subsample with the addition of language minority children. 

Tables 3-14 (count) and 3-15 (percent) show the characteristics of the achieved third-grade 

sample compared with the expected third-grade sample. The total number of children in the language 

minority group is virtually the same as the expected number while the total number of children in the 

other group is about 5 percent larger than the expected number. In computing the expected sample size, 

the same mover rate was assumed for both groups of children. The third-grade sample shows that the nonlanguage minority children moved at a lower rate (42 percent) than the language minority children (44 

percent) resulting in a slightly larger sample of non-language minority children. The agreement between 

the expected and achieved sample sizes is rather remarkable given the numerous assumptions required. 

The actual percent distribution of third-graders within each subgroup is as expected with the exception of 

the Catholic and non-Catholic private schools where the percent of children in Catholic schools is higher 

than that of children in non-Catholic private schools. This may be due to the lower completion rate of 

children in non-Catholic private schools compared with children in Catholic private schools (93 percent 

and 97 percent, respectively). Elsewhere among the children in the language minority group, the 

difference between the expected distribution and the actual distribution is less than 1 percent. Elsewhere 

among the children not in the language minority group, the difference is less than 3 percent. 
3-41 

Table 3-15. Characteristics of third-grade respondents—percent distribution by subgroup: 

School year 2001–02 

 Expected2 Achieved2

Subgroup1 Total 

Language 

minority

Not 

language 

minority Total

Language 

minority 

Not 

language 

minority

 

 Total 100.0 100.0 100.0 100.0 100.0 100.0

School affiliation 

Public 78.7 82.9 77.1 78.9 83.5 77.2

Private 21.3 17.1 22.9 21.1 16.5 22.8

Catholic 51.6 54.0 50.9 56.2 57.4 55.8

Non-Catholic 48.4 46.0 49.1 43.8 42.6 44.2

 

Type of locale 

Rural 11.6 5.2 14.0 13.1 5.5 15.8

Non-rural 88.4 94.8 86.0 86.9 94.5 84.2

 

Race/ethnicity 13 

Hispanic 18.7 53.5 5.7 18.0 53.4 5.3

Black 14.8 3.2 19.2 13.1 2.9 16.8

Asian/Pacific Islander 7.9 23.8 1.9 7.7 23.4 2.0

Other 58.6 19.6 73.3 61.2 20.3 75.9

 

Race/ethnicity 23 

Hispanic 18.4 52.6 5.5 17.6 52.4 5.1

Black 14.8 3.2 19.2 13.1 2.9 16.8

Asian/Pacific Islander 9.3 26.1 2.9 9.2 26.1 3.1

Other 57.6 18.1 72.5 60.1 18.6 75.0 1 Characteristics of the schools (school affiliation and type of locale) are from the original sample schools. 2 The expected sample size was computed using assumed mover rates (differential by subgroups as shown table 3-12), a 47 percent subsampling 

rate (instead of 50 percent to account for ineligibility in third grade), and a 95 percent completion rate. 3 Race/ethnicity 1 was the strict definition of API (RACE=5-Asian, or 6-Native Hawaiian or other Pacific Islander), while race/ethnicity 2 was the 

broader definition (RACE=5-Asian, or 6-Native Hawaiian or other Pacific Islander, or WKASIAN=1-Child is Asian, or WKPACISL=1-Child is 

Native Hawaiian or other Pacific Islander)). Variables are from the ECLS-K base year data file. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2002. 
3-42 

Table 3-16 shows the third-grade data collection results by mover status (analogous to 

table 3-11). In this table, the total number of children is 21,357, which is larger than the total in table 3-11 

by 26 children; these are children sampled in first grade who did not have completed assessment data or 

parent interview in first grade (hence not included in table 3-11), but participated in the third-grade study 

(hence included in table 3-16). Overall the unweighted completion rate for third grade is 79 percent for 

movers and 95 percent for nonmovers, compared with the expected completion rate of 83 and 95 percent. 

The rate of base year respondents who moved out of their original sample schools is 42 percent 

(compared with the expected overall moving rate of 47 percent). The achieved sample size shown in table 

3-14 is a function of both the completion rate and the mover rate. Even though the actual completion rate 

for movers is lower than expected, the actual mover rate is also lower than expected. Fewer movers 

resulted in a larger sample size. Note that in all tables in this chapter a respondent is defined as a child 

with completed assessment data or completed parent interview data or a child who could not be assessed 

due to a disability, so that the completion rate calculated here is not the same as the completion rate in 

chapter 6 of this report or chapter 5 of the ECLS-K User’s Manual for the Third Grade Public-Use Data 

File and Electronic Code Book (NCES 2004-001) (Tourangeau, Brick, et al. 2004) which is instrumentspecific. 
3-43 

Table 3-16. Spring-third-grade data collection results by mover status: School year 2001–02  Spring-third grade response status Movers-sampled Subgroup1 Total2 Respond3

Not 

respond Ineligible

Not 

sampled

Unweighted 

completion 

rate Respond3

Not 

respond Ineligible 

Percent 

moved

Percent 

sampled

Unweighted 

completion 

rate Respond3

 All 21,357 15,305 1,524 364 4,164 90.9 3,583 931 361 42.3 53.9 79.4 11,722 School affiliation  Public 16,784 12,070 1,236 312 3,166 90.7 2,726 760 311 41.5 54.5 78.2 9,344  Private 4,573 3,235 288 52 998 91.8 857 171 50 45.4 51.9 83.4 2,378  Catholic 2.356 1,817 115 14 410 94.0 329 62 14 34.6 49.7 84.1 1,488  Non-Catholic 2,217 1,418 173 38 588 89.1 528 109 36 56.9 53.4 82.9 890 Type of locale  Rural 2,478 2,005 162 20 291 92.5 304 102 19 28.9 59.4 74.9 1,701  Non-rural 18,879 13,300 1,362 344 3,873 90.7 3,279 829 342 44.1 53.5 79.8 10,021 Race/ethnicity 14

 Hispanic 3,777 2,752 274 145 606 90.9 756 193 144 45.0 64.3 79.7 1,996  Black 3,229 2,007 267 81 874 88.3 604 186 81 54.0 49.9 76.5 1,403  Asian/Pacific Islander 

1,579 1,174 139 37 229 89.4 298 92 37 41.5 65.1

76.4 876 

 Other 12,772 9,372 844 101 2,455 91.7 1,925 460 99 38.7 50.3 80.7 7,447 Race/ethnicity 24

 Hispanic 3,698 2,691 271 144 592 90.9 739 191 143 45.0 64.4 79.5 1,952  Black 3,229 2,007 267 81 874 88.3 604 186 81 54.0 49.9 76.5 1,403  Asian/Pacific Islander 

1,867 1,404 151 40 272 90.3 351 102 40 41.0 64.4 77.5 1,053 

 Other 12,563 9,203 835 99 2,426 91.7 1,889 452 97 38.7 50.1 80.7 7,314 Language minority  Non-English 5,372 4,041 412 203 716 90.7 1,162 291 203 44.2 69.8 80.0 2,879  English 15,985 11,264 1,112 161 3,448 91.0 2,421 640 158 41.7 48.3 79.1 8,843 

1 Characteristics of the schools (school affiliation and type of locale) are from the original sample schools. 2 The total number of children excludes 68 children who responded in fall-kindergarten and became ineligible in spring-kindergarten, and includes 165 children sampled 3 A respondent is a child with assessment data or parent interview data, or a child who could not be assessed due to a disability. 4 Race/ethnicity 1 was the strict definition of API (RACE=5-Asian or 6-Native Hawaiian or other Pacific Islander), while race/ethnicity 2 was the broader definit

Hawaiian or other Pacific Islander or WKASIAN=1-Child is Asian or WKPACISL=1-Child is Native Hawaiian or other Pacific Islander). Variables are from the ECLS-

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spri
3-44 

3.6 Spring-Fifth Grade Sample 

For the fifth-grade data collection, different options for subsampling movers were explored 

to reduce the sample size in order to contain the cost of data collection. The original plan would use the 

same procedures for third grade to subsample and follow 50 percent of children who moved in fifth grade 

or earlier and retain all language minority children who had not been subsampled out before fifth grade. 

Three alternative plans were developed to decrease the sample sizes by reducing the subsampling rates. 

One of the alternatives was adopted as the final plan. The final subsampling rates maximize the amount of 

longitudinal data available for key analytic groups. 

A new feature of the fifth-grade sample is the subsampling of children for the administration 

of the mathematics or science questionnaires. While all children retained for the fifth-grade data 

collection had child-level questionnaires filled out by their reading teachers, half were subsampled to 

have child-level questionnaires filled out by their mathematics teachers and the other half had child-level 

questionnaires filled out by their science teachers. This affects only the computation of the combined 

child-parent-teacher weights as discussed in section 7.2.4. 

3.6.1 Options for Subsampling Movers 

All sampling options considered for fifth grade were based on the beginning sample of 

21,357 children: 21,192 base year respondents who were still eligible after the base year, and 165 children 

sampled in first grade as part of the freshening procedure (see section 3.4.2). The first decision regarding 

the fifth-grade sample was to exclude the following groups of children from the fifth-grade survey, 

irrespective of other subsampling procedures that might be implemented: (1) children who had become 

ineligible in an earlier round (because they had died or moved out of the country); (2) children who were 

subsampled out in previous rounds because they had moved out of the original schools and were not 

subsampled to be followed; (3) children whose parents emphatically refused to cooperate (hard refusals) 

in any of the data collection rounds since spring-kindergarten; and (4) children eligible for the third-grade 

sample for whom there are neither first-grade nor third-grade data (i.e., no direct assessment data and no 

parent interview data from first grade and third grade). The children who met any of these conditions 

were not eligible for sampling in the fifth grade for any of the sampling options considered. In total, 5,214 

children were excluded from the fifth-grade survey; they are distributed as shown in table 3-17. 
3-45 

Table 3-17. Number of children eligible after the base year but excluded from the fifth-grade data 

collection: School year 2003–04 

Characteristics1

Beginning 

sample size 

after the base 

year 

Total 

number of 

children 

excluded 

Mover 

subsampled out 

in first or third 

grade2

Ineligible 

in first or 

third grade 

Hard 

refusal 

Eligible for third 

grade sample, 

with no first or 

third grade data 

 

 Total 21,357 5,214 4,117 122 571 404 

School affiliation 

 Public 16,771 4,000 3,129 98 433 340 

 Private 4,570 1,198 988 23 132 55 

 Catholic 2,354 485 405 7 52 21 

 Non-Catholic 2,216 713 583 16 80 34 

 Unknown 16 16 0 1 6 9 

 

Type of locale 

Rural 2,480 381 288 5 51 37 

Non-rural 18,733 4,824 3,829 113 518 364 

Unknown 144 9 0 4 2 3 

Race/ethnicity 

 Hispanic 3,782 811 584 47 82 98 

 Black 3,229 1,061 867 12 88 94 

 Asian/Pacific Islander 1,580 313 225 20 46 22 

 Other 12,678 2,995 2,430 41 343 181 

 Unknown 88 34 11 2 12 9 

 

Language minority 

 Not English 5,372 1,000 684 84 124 108 

 English 15,985 4,214 3,433 38 447 296 

1

 Characteristics are from the most recent data available for the child (e.g., if a child was not subsampled in third grade and had data from first 

grade, then the characteristics of the child come from first grade). 2

 These are statistical movers, not operation movers as discussed in chapter 4. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

In the original plan, 50 percent of children who moved in fifth grade or earlier would be 

subsampled to be followed, and all language minority children who had not been subsampled out before 

fifth grade would be retained. This plan protects the language minority sample, as had been done in third 

grade. 

In the first alternative plan, language minority movers would be subsampled for followup 

depending on the amount of data that they had from previous rounds. If they had both spring-first grade 

and spring-third grade data, then 50 percent would be subsampled and followed if they had only one data 

point after the base year, 25 percent would be subsampled and followed. Similarly, 25 percent of other 

movers would be subsampled and followed if they had both first- and third-grade data, and 12.5 percent 

would be subsampled and followed if they had only one data point besides base year data. 
3-46 

In the second alternative plan, only children with complete longitudinal data would be 

fielded, i.e., base year respondents who had first-grade and third-grade data. Of these children, 50 percent 

of language minority children who moved in fifth grade (or earlier) would be subsampled for followup, 

and 25 percent of other movers (in fifth grade or earlier) would be subsampled for followup. Children 

who were sampled in first grade through the sample freshening procedure would not be retained in the 

sample. 

The third and last option, adopted for the fifth-grade study, called for using rates that are 

approximately equal to those given below for subsampling base year respondents who are movers in fifth 

grade (or earlier): 

 33 percent for non-language minority (LM) movers with full longitudinal data; 

 25 percent for non-LM movers with third-grade but not first-grade data; 

 15 percent for non-LM movers with first-grade but not third-grade data; 

 75 percent for LM movers with full longitudinal data; 

 50 percent for LM movers with third-grade but not first-grade data; and 

 25 percent for LM movers with first-grade but not third-grade data. 

For subsampling freshened children (i.e., children sampled in first grade) who are movers in 

fifth grade (or earlier) the rates proposed were the following: 

 33 percent for non-LM movers with full longitudinal data; 

 15 percent for non-LM movers with third-grade but not first-grade data; 

 15 percent for non-LM movers with first-grade but not third-grade data; 

 75 percent for LM movers with full longitudinal data; 

 25 percent for LM movers with third-grade but not first-grade data; and 

 25 percent for LM movers with first-grade but not third-grade data. 

Table 3-18 shows the expected fifth-grade sample size separately for language minority 

children and by mover status for the different subsampling plans, the estimated design effect due to the 

subsampling of base year children, the effective sample size, and the expected number of children with a 
3-47 

completed fifth-grade assessment, assuming that 90 percent of children would be assessed successfully. 

The completion rate of 90 percent took into account children who had moved and whether they would be 

located. According to the third-grade collection, the unweighted completion rate for child assessment was 

95 percent for nonmovers and 63 percent for movers, with an overall unweighted completion rate of 86 

percent. Since fewer movers would be included in fifth grade compared with third grade, the assumption 

was for a slightly higher completion rate for the child assessment. 

Table 3-18. ECLS-K options for subsampling movers for fifth grade: School year 2003–04 

Characteristic Sample size

Design 

effect

Effective 

sample size

Expected number of 

children assessed1

Original plan 

Total 14,135 1.13 12,509 12,722

Non-language minority: Nonmover 4,907 — 4,342 4,416

Non-language minority: Mover 4,856 — 4,297 4,371

Language minority: Nonmover 1,595 — 1,412 1,436

Language minority: Mover 2,777 — 2,458 2,499

Option 1 

Total 11,336 1.30 8,720 10,202

Non-language minority: Nonmover 4,908 — 3,775 4,417

Non-language minority: Mover 3,219 — 2,476 2,897

Language minority: Nonmover 1,595 — 1,227 1,435

Language minority: Mover 1,614 — 1,242 1,453

Option 2 

Total 10,308 1.26 8,181 9,277

Non-language minority: Nonmover 4,767 — 3,783 4,290

Non-language minority: Mover 3,067 — 2,434 2,760

Language minority: Nonmover 768 — 610 691

Language minority: Mover 1,706 — 1,354 1,536

Option 3 

Total 12,635 1.24 10,190 11,372

Non-language minority: Nonmover 4,908 — 3,958 4,417

Non-language minority: Mover 3,784 — 3,052 3,406

Language minority: Nonmover 1,595 — 1,286 1,436

Language minority: Mover 2,348 — 1,894 2,113

— Not available. 

1

 The expected number of children assessed is computed using an assumed completion rate of 90 percent. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 kindergarten (ECLS-K), spring 2004. 
3-48 

3.6.2 Expected Sample Size 

Table 3-19 shows the expected sample sizes of children in fifth grade by subgroups of 

interest based on the third mover subsampling option adopted for fifth grade. The format of this table is 

different from table 3-12 for third grade. While the mover subsampling rate for third grade was constant, 

the subsampling rates vary according to the different groups described earlier. It is also difficult to 

estimate the number of schools in the fifth-grade sample because schools that were in the sample prior to 

fifth grade may drop out because of the reduction in the sample size and new schools may enter due to 

new movers. 

Table 3-19. Expected sample size for selected subgroups for fifth grade, by mover status: School year 

2003–04 

 Beginning fifth-grade sample After subsampling movers Total responding 

Subgroup1 Total Movers Nonmovers Total Movers Nonmovers Total Movers Nonmovers 

 

 All children 16,143 4,320 11,823 12,635 6,132 6,503 11,372 5,519 5,852 

School affiliation 

 Public 12,771 3,340 9,431 10,009 4,821 5,187 9,008 4,339 4,668 

 Private 3,372 980 2,392 2,627 1,311 1,316 2,364 1,180 1,184 

 Catholic 1,869 373 1,496 1,434 611 823 1,291 550 741 

 Non-Catholic 1,503 607 896 1,193 700 493 1,073 630 444 

 

Type of locale 

 Rural 2,099 388 1,711 1,558 617 941 1,402 555 847 

 Non-rural 14,044 3,932 10,112 11,077 5,515 5,562 9,969 4,964 5,005 

Race/ethnicity 

 Hispanic 2,971 958 2,013 2,579 1,472 1,107 2,321 1,325 996 

 Black 2,168 757 1,411 1,658 881 776 1,492 793 698 

 Asian/Pacific Islander 1,267 382 885 1,104 618 487 994 556 438 

 Other 9,737 2,223 7,514 7,294 3,161 4,133 6,564 2,845 3,719 

 

Language minority 

 Non-English 4,372 1,472 2,900 3,943 2,348 1,595 3,549 2,113 1,436 

 English 11,771 2,848 8,923 8,692 3,785 4,908 7,823 3,406 4,417 

1

 Characteristics of the schools (school affiliation and type of locale) are from the original sample schools. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 kindergarten (ECLS-K), spring 2004 

3.6.3 Sample Outcome and Precision Requirements 

The fifth-grade sample of 16,143 excludes the base year respondents identified in section 

3.6.1 who were not subject to data collection in the fifth grade, i.e., children who had become ineligible in 
3-49 

an earlier round (because they had died or moved out of the country); children who were subsampled out 

in previous rounds because they had moved out of the original schools and were not subsampled to be 

followed; children whose parents emphatically refused to cooperate (hard refusals) in any of the data 

collection rounds since spring-kindergarten; and children eligible for the third-grade data collection for 

whom there are neither first-grade nor third-grade data (i.e., no direct assessment data and no parent 

interview data from first grade and third grade). As discussed in section 3.1, a sample in fifth grade of 

about 10,000 children would be adequate to meet the precision requirements overall and for most 

subgroups. A sample of about 800 to 1,000 children in a subgroup would be achieved for most of the 

subgroups with an overall sample of 10,000 children and these would approximately meet the precision 

goals described in section 3.1. 

Table 3-20 (count) and table 3-21 (percent) show the characteristics of the achieved fifthgrade sample compared with the expected fifth-grade sample. The numbers of fifth-grade respondents for 

all of the identified subgroups of interest exceed 1,000, except for children in non-Catholic private 

schools and API children. For most of the key analytic groups the numbers of respondents are much 

larger than 1,000. For API children, the number of respondents is 970, which exceeds the minimum target 

of 800 and is very close to 1,000. The number of respondents in non-Catholic private schools is 957. The 

sample of 11,820 respondents attended 2,008 public schools and 356 private schools. 

Two thirds of the public schools attended by the fifth-grade sample (1,355) are transfer 

schools, and almost half of the private schools (166) are transfer schools. The large number of transfer 

schools corresponds to the heavy movement of the ECLS-K children between schools. 
3-50 

Table 3-20. Characteristics of fifth-grade respondents—number of fifth-graders by subgroup: School 

year 2003–04 

 Expected2

 Achieved 

Subgroup1 Total 

Language 

minority

Not 

language 

minority Total

Language 

minority 

Not 

language 

minority

 

 Total 11,372 3,549 7,823 11,820 3,405 8,415

School affiliation 

 Public 9,008 2,969 6,039 9,412 2,845 6,567

 Private 2,364 580 1,784 2,408 560 1,848

 Catholic 1,291 325 966 1,451 330 1,121

 Non-Catholic 1,073 255 818 957 230 727

 

Type of locale 

 Rural 1,402 198 1,204 1,659 199 1,460

 Non-rural 9,969 3,350 6,619 10,161 3,206 6,955

 

Race/ethnicity 

 Hispanic 2,321 1,894 427 2,244 1,821 423

 Black 1,492 108 1,384 1,352 91 1,261

 Asian/Pacific Islander 994 836 158 970 799 171

 Other 6,564 710 5,854 7,254 694 6,560 1

 Characteristics of the schools (school affiliation and type of locale) are from the original sample schools. 2

 The expected sample size was computed using assumed mover rates, differential subsampling rates, and a 90 percent completion rate. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

Table 3-21 is analogous to table 3-20 but shows the percent distribution instead of counts. 

The achieved sample is about 4 percent larger than the expected sample, with the language minority 

sample about 4 percent smaller than the expected sample. As in third grade, language minority children 

moved at a higher rate (48 percent) than non-language minority children (37 percent) resulting in the 

larger sample of non-language minority children. 

As shown in table 3-21, the agreement between the expected and achieved fifth-grade 

samples is as seen in third grade. Children in non-Catholic private schools responded at a lower rate than 

children in Catholic schools, causing the achieved sample size for children in non-Catholic schools to be 

lower than expected. For all other characteristics, the difference between expected and achieved sample 

sizes is less than 1 percent for language minority children and around 3 percent or less for non-language 

minority children. 
3-51 

Table 3-21. Characteristics of fifth-grade respondents—percent distribution by subgroup: School year 

2003–04 

 Expected2

 Achieved 

Subgroup1 Total 

Language 

minority 

Not 

language 

minority Total

Language 

minority 

Not 

language 

minority

 

 Total 100.0 100.0 100.0 100.0 100.0 100.0

School affiliation 

 Public 79.2 83.7 77.2 79.6 83.6 78.0

 Private 20.8 16.3 22.8 20.4 16.4 22.0

 Catholic 54.6 56.0 54.1 60.3 58.9 60.7

 Non-Catholic 45.4 44.0 45.9 39.7 41.1 39.3

 

Type of locale 

 Rural 12.3 5.6 15.4 14.0 5.8 17.3

 Non-rural 87.7 94.4 84.6 86.0 94.2 82.7

 

Race/ethnicity 

 Hispanic 20.4 53.4 5.5 19.0 53.5 5.0

 Black 13.1 3.0 17.7 11.4 2.7 15.0

 Asian/Pacific Islander 8.7 23.6 2.0 8.2 23.5 2.0

 Other 57.7 20.0 74.8 61.4 20.4 78.0 1

 Characteristics of the schools (school affiliation and type of locale) are from the original sample schools. 2

 The expected sample size was computed using assumed mover rates, differential subsampling rates, and a 90 percent completion rate. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

Table 3-22 shows the details on the number of sampled students by their response and mover 

status for subgroups at the end of the fifth grade. Of the 16,143 students, 40 percent were movers and 42 

percent of the movers were subsampled for followup in fifth grade. Overall, the unweighted completion 

rate for fifth grade is 96 percent with 85 percent of movers responding and 99 percent of nonmovers 

responding. A respondent is defined as a child with completed assessment (or excluded from assessment 

due to a disability) or completed parent interview data, so that the completion rate here is not the same as 

the instrument-specific completion rate in chapter 6. 

Based on the achieved sample, the sampling and data collection procedures developed in the 

initial stages and modified throughout the course of the study did produce samples that met or exceeded 

requirements for the vast majority of key analytic groups. However, the introduction of the more intensive 

subsampling of children who moved to achieve the desired cost savings did result in some increases in the 

design effects for the estimates, as was expected. See chapter 4 of the ECLS-K Combined User’s Manual 
3-52 

for the ECLS-K Fifth-Grade Public-Use Data Files and Electronic Codebook (NCES 2006–032) 

(Tourangeau et al. forthcoming) for a discussion of design effects. 
3-53 

Table 3-22. Spring-fifth-grade data collection results by mover status: School year 2003–04  Spring-fifth grade response rates Movers-sampled Subgroup1 Total2 Respond2

Not 

respond Ineligible

Not 

sampled

Unweighted 

completion 

rate Respond3

Not 

respond Ineligible 

Percent 

moved

Percent 

sampled

Unweighted 

completion 

rate 

 All 16,143 11,820 519 41 3,763 95.8 2,272 396 41 40.1 41.9 85.2 

School affiliation 

Public 12,771 9,412 433 39 2,887 95.6 1,758 332 39 39.3 42.4 84.1 Private 3,372 2,408 86 2 876 96.6 514 64 2 43.2 39.8 88.9 

Catholic 1,869 1,451 40 0 378 97.3 234 26 0 34.1 40.8 90.0 Non-Catholic 1,503 957 46 2 498 95.4 280 38 2 54.4 39.1 88.1 

Type of locale 

 Rural 2,099 1,659 44 2 394 97.4 157 36 2 28.1 33.1 81.3  Non-rural 14,044 10,161 475 39 3,369 95.5 2,115 360 39 41.9 42.7 85.5 

Race/ethnicity 1 

 Hispanic 2,971 2,244 136 14 577 94.3 658 127 14 46.3 58.1 83.8  Black 2,168 1,352 69 4 743 95.1 325 58 4 52.1 34.2 84.9  Asian/Pacific Islander 1,267 970 74 10 213 92.9 293 62 10 45.6 63.1 82.5  Other 9,737 7,254 240 13 2,230 96.8 996 149 13 34.8 34.2 87.0 

Language minority 

 Non-English 4,372 3,405 242 33 692 93.4 1,140 222 33 47.7 66.8 83.7  English 11,771 8,415 277 8 3,071 96.8 1,132 174 8 37.3 30.0 86.7 

1 Characteristics of the schools (school affiliation and type of locale) are from the original sample schools. 2 A respondent is a child with assessment data or parent interview data, or a child who could not be assessed due to a disability. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spri
3-54 

This page is intentionally left blank. 
4-1 

4. DATA COLLECTION METHODS 

The following sections discuss the data collection procedures in the fifth-grade data 

collection phase of the Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K). 

Section 4.1 gives an overview of the data collection methods. Detailed information is provided on roles 

and responsibilities in the study (section 4.2), study training procedures (section 4.3), fall preassessment 

school contacts (section 4.4), spring-fifth grade data collection (section 4.5), and data collection quality 

control procedures (section 4.6). 

4.1 Overview of Data Collection Methods 

The ECLS-K fifth-grade data collection was conducted in the fall and spring of the 2003–04 

school year. Fall data collection included contacting sampled schools to schedule appointments to conduct 

the child assessments in the spring of the school year, verify the parent consent procedures, link children 

to their teachers, identify children who had withdrawn from the school, and obtain locating information 

about their new schools. Spring data collection instruments included the administration of direct child 

assessments and parent interviews and the collection of teacher and school questionnaires, student record 

abstracts, and facilities checklists. The activities to locate children and gain cooperation of the schools 

into which they transferred began in the fall and continued during the spring data collection. The content 

and timeline of the fifth–grade data collection is shown in exhibit 4-1. 

The mode of data collection was computer-assisted personal interviewing (CAPI) for the 

child assessments; telephone and in-person computer-assisted interviewing (CAI) was the mode of data 

collection for the parent interview; and self-administered questionnaires were used to gather information 

from teachers, school administrators, and student records. Field staff completed the facilities checklist. 
4-2 

Exhibit 4-1. Timeline of fifth-grade data collection: School year 2003–04 

Fifth grade 

2003 2004 

Fall Winter Spring 

 

Advance school 

contact 

 Advance school 

contact 

Child assessments 

conducted 

 

Tracing sampled 

households 

 Tracing sampled 

households 

Parent interviews 

conducted 

 

 Teacher 

information 

collected 

 

 School and school 

administrator data, 

facilities checklist 

collected 

 

 Student records data 

collected 

 

 Tracing children 

who moved 

 

 Activities to gain transfer 

school cooperation 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

4.2 Roles and Responsibilities in the ECLS-K Study 

4.2.1 School’s Role 

During school recruitment, the schools were asked to designate a staff member to be the 

school coordinator to assist the ECLS-K staff with all school arrangements. 

Since the child assessments were administered at the schools, schools needed to provide 

appropriate space for conducting the assessments. 
4-3 

4.2.2 School Coordinator’s Role 

A school coordinator was designated by the principal to facilitate the ECLS-K activities in 

the school. The school coordinator played a significant role in the smooth functioning and successful 

completion of the ECLS-K child assessments in each cooperating school. He or she knew the personality

of the school, the most opportune times to schedule the assessments, the available locations where the 

one-on-one assessments could be conducted, and the best way to notify students, their parents, and their 

teachers of the assessment. 

The coordinator was asked to assist the ECLS-K in four ways: 

 Notify selected students, their teachers, and their parents of the study; 

 Arrange for suitable space for the assessment activities; 

 Provide information on sampled children, such as their grade and teachers’ names; 

and 

 Distribute teacher and school questionnaires. 

4.2.3 Supervisor’s Role 

There were a total of 81 supervisors during the fifth-grade data collection who oversaw field 

staff conducting both the parent interviews and child assessments. The supervisors’ responsibilities were 

as follows: 

 Contact each school assigned to them to 

1. Arrange for space to conduct the assessment; 

2. Schedule the spring assessment in original schools; 

3. Discuss parental contact information and consent procedures; 

4. Link children to teachers and domains; and 

5. Identify children who had withdrawn from the school; 

 Follow up and track receipt of parental consent forms, as necessary; 

 Update the Field Management System (FMS) regularly and report to their field 

manager; 
4-4 

 Transmit updated FMS data to the home office; 

 Pick up e-mail regularly; and 

 Return all materials at the end of the field period.

4.2.4 Assessor’s Role 

A team of three to four assessors worked with each supervisor in a work area. The primary 

responsibilities of the ECLS-K assessors were to conduct the computer-assisted one-on-one child 

assessments and parent interviews. There were a total of 258 assessors, with 16 having conducted the 

parent telephone interview only (these 16 assessors are referred to as interviewers rather than assessors in 

the remainder of this chapter). The remaining 242 conducted both the parent interview and the child 

assessments. 

In addition to these responsibilities, some assessors were asked by their supervisor to assist 

with various other activities that took place in the school. These included, but were not limited to, 

preparing parental consent forms (if required), collecting teacher questionnaires, and assisting with 

various other recordkeeping tasks. 

4.2.5 Field Manager’s Role 

Six experienced regional field managers were assigned to oversee the work of the 81 

supervisors. The field managers held weekly telephone conference calls with each supervisor assigned to 

them. If a supervisor had an immediate problem, he or she was encouraged to call the field manager at 

any time. 

Depending on the stage of the field period, the telephone conference calls between 

supervisors and field managers reviewed those activities that were in the planning stage, in progress, or in 

the process of being completed. These discussions included the following topics: 

 Status of telephone contacts with original and transfer schools; 

 Status of assessments scheduled in original schools by work area; 
4-5 

 Status of parent consent and followup in original and transfer schools; 

 Status of linking children to teachers; 

 Status of children who were withdrawn from the school; 

 Any refusal cases; 

 Receipt of all school materials; and 

 Overall and individual costs in the work area. 

Exhibit 4-2 presents the fifth-grade field organization and the number of staff in each 

position. 

Exhibit 4-2. Fifth-grade field organization 

 Field Managers 

6 

 

 Field Supervisors 

81 

 

Assessors 

204 

Lead Assessors 

27 

Troubleshooters 

11 

Interviewers 

16 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

4.3 Field Staff Training 

Several in-person training sessions were conducted to prepare staff for the fifth-grade data 

collection. In the fall of 2003, supervisors were trained to contact original schools and recruit transfer 

schools. In the spring of 2004, three training sessions were held: one to train trainers, one for field 

supervisors, and one for assessors. All training sessions were conducted using scripted training manuals 

to ensure that all trainees received the same information. Training sessions consisted of interactive 

lectures, scripted role plays, interactive exercises, and self-administered exercises. Interactive lectures 

were lectures with discussion and questions occurring periodically during the lecture. Scripted role plays 

usually consisted of pairs of trainees each pretending to be, for example, the assessor and the child or the 
4-6 

interviewer and the parent. Such role plays gave trainees a chance to become more familiar with their 

materials and duties. Interactive exercises were group exercises led by the trainer in which all trainees 

participated. Self-administered exercises were, as the name suggests, completed by trainees working 

independently. Because of the complexity of the ECLS-K, trainees were required to become familiar with 

the functionality of their laptop computers and with the programs installed on them. Trainees were also 

required to become familiar with the different child assessment materials. See chapter 2 of the ECLS-K 

Psychometric Report for the Third Grade (NCES 2005–062) (Pollack et al. 2005) for a description of the 

child assessment materials. The following sections discuss the fall and spring trainings. 

4.3.1 Advance Contact and Recruitment Training 

During the fall 2003, advance contact was made with the schools in order to remind them 

about the study and to collect information that would be helpful in the spring 2004 national data 

collection. The advance effort not only reduced the burden on supervisors in the spring but also 

reacquainted schools with the study’s procedures and gave supervisors a chance to encourage the schools’ 

participation. The major fall tasks were to contact schools to set appointments for the child assessments in 

the spring, to verify the parent consent procedures, to link children to teachers and domains, and to 

identify children who had withdrawn from the school and obtain locating information about their new 

schools. Field supervisors were trained for 3 days in September 2003 to contact original sampled schools 

and transfer schools to set up the data collection in the spring. A total of 39 field supervisors and 2 field 

managers completed advance contact and recruitment training. Topics included an overview of study 

activities to date, verifying parent consent procedures, identifying and locating children who moved from 

the schools they attended in the first grade, identifying the teachers of ECLS-K children and linking them 

to those children, and exercises on scheduling schools efficiently within a work area. 

As in the third-grade training, advance contact and recruitment training was conducted using 

the automated FMS. The FMS is a database that is used throughout the data collection period to enter 

information about the sampled children, parents, teachers, and school and to monitor production on all 

data collection activities. The FMS contains information essential to conducting and monitoring the 

progress of the data collection. For example, it contains the names and addresses of each school, the 

principal’s name and telephone number, the name and telephone number of the school coordinator, the 

first and last day of classes, the school hours, and the names of the sampled children in the school. For 

children, it contains information on their names and their parents’ names, whether they have any 
4-7 

accommodations or need to use a hearing aid or glasses, as well as other information. Information of the 

same sort is also collected about the teachers and parents. For example, the FMS contains the name, 

address, and telephone number of each parent. For each parent, the name of the sampled child (or 

children) is also listed. For teachers, the FMS contains their names, schools, and the sampled children in 

their classrooms. It also indicates whether the teacher was a regular or a special education teacher, and 

what subject the regular teacher taught and in how many classrooms. To monitor production, the FMS 

contains case disposition codes and case assignment information. During training presentations, the field 

supervisors entered information into the FMS, thus acquiring hands-on experience with the FMS and all 

field procedures prior to beginning data collection. The field supervisors also completed role-play 

exercises that involved entering information into the FMS. 

4.3.2 Spring-Fifth Grade Training 

Field supervisors, interviewers, and assessors were trained for the spring-fifth grade data 

collection in two sessions in February 2004. The first session was trainers’ training and certification. The 

second and largest training involved the training of the supervisors and assessors. Before the February inperson training session, supervisors and assessors completed 8 hours of home study training on the study 

design, field procedures, and computer keyboard skills. Staff conducting only the parent interviews did 

not have a pre-session home study to complete. 

4.3.2.1 Trainers’ Training and Certification 

The purpose of trainers’ training was to (1) introduce lead and co-trainers to the training 

materials; (2) evaluate the flow, language, exercises, and time allotment of the training sessions; and 

(3) certify trainers on the child assessment. Some, but not all, co-trainers and runners (staff who assisted 

trainees who experienced difficulties with the CAI application and helped with the management and 

distribution of training materials) were also certified on the child assessment. Experienced trainers with 

in-depth knowledge of the ECLS-K conducted the trainer training sessions. Not only had these trainers 

developed the CAI specs, but they had also worked with expert consultants to develop the child 

assessments and the assessment materials. In addition, they conducted nine assessments on nonsampled 

children in October 2003 in order to become certified on the assessments of the ECLS-K prior to the 

launch of the national data collection (see section 4.3.2.2). 
4-8 

Approximately 16 lead trainers, 16 co-trainers, and 16 runners were trained at trainers’ 

training in a single room. As noted earlier, experienced trainers conducted the parent interview and the 

child assessment training sessions. In addition, a data display person responsible for running the 

electronic data display and two runners assisted in the training. Trainers’ training was conducted for 4 

days in January 2004 in Rockville, Maryland. The trainers’ training agenda covered many of the same 

topics that were presented during the assessment and parent interview training for the national data 

collection, as well as some additional topics on the trainers’ tasks and responsibilities. Exhibit 4-3 is the 

trainers’ training agenda. In addition, trainers were certified on the child assessment following the 

procedures described in section 4.3.2.2. The only difference between trainer certification and field staff 

certification was that trainers were certified on an entire child assessment and the field staff were certified 

on approximately half of a child assessment. 

Exhibit 4-3. Spring-fifth grade Trainer Training Agenda: School year 2003–04 

Day Time Session # Topic Type of session Trainer On Computer?

1 1/28/2004 

 9:00-9:30 1 Study Introduction Interactive Lecture Lead Trainer No 

 9:30-10:00 2 Parent Interview 

Introduction 

Interactive Lecture Lead Trainer Yes 

 BREAK 

 10:15-12:15 3 Parent Interactive #1 Interactive Role Play Lead Trainer Yes 

 12:15-1:30 LUNCH 

 1:30-2:30 5 Parent Interact #2 

(with Twins) 

Interactive Role Play Co-Trainer Yes 

 2:30-2:45 BREAK 

 2:45-3:45 5 Parent Interact #2 

(with Twins) (con’t) 

Interactive Role Play Co-Trainer Yes 

 3:45-5:00 8 Child Assessment and 

Certification 

Overview 

Interactive Lecture Lead Trainer No 

2 1/29/2004 

 9:00-10:00 9 Introduction and Selfdescription 

Questionnaire 

Interactive Lecture Lead Trainer Yes 

 10:00-10:15 10 Reading—Routing 

and Low Form 

Interactive Lecture Lead Trainer Yes 

See note at end of exhibit. 
4-9 

Exhibit 4-3. Spring-fifth grade Trainer Training Agenda: School year 2003–04—Continued 

Day Time Session # Topic Type of session Trainer On Computer?

 10:15-10:30 BREAK 

 10:30-11:00 10 Reading—Routing 

and Low Form (con’t) 

Interactive Lecture Lead Trainer Yes 

 11:00-11:15 11 Reading—Routing 

and Low Form 

(practice/exercises) 

Individual Practice Yes 

 11:15-12:00 12 Reading—Routing 

and Middle Form 

Interactive Lecture Co-Trainer Yes 

 12:00-1:00 LUNCH 

 1:00-1:15 13 Reading—Routing 

and Middle Form 

(practice/exercises) 

Individual Practice Yes 

 1:45-2:00 15 Reading—High Form 

(practice/exercises) 

Individual Practice Yes 

 2:00-2:45 16 Math—Routing and 

Low Form 

Interactive Lecture Lead Trainer Yes 

 2:45-3:00 17 Math—Routing and 

Low Form 

(practice/exercises) 

Individual Practice Yes 

 3:00-3:15 BREAK 

 3:15-4:00 18 Math—Routing and 

Middle Form 

Interactive Lecture Co-Trainer Yes 

 4:00-4:15 19 Math—Routing and 

Middle Form 

(practice/exercises) 

Individual Practice Yes 

 4:15-4:45 20 Math—High Form Interactive Lecture Lead Trainer Yes 

 4:45-5:00 21 Math—High Form 

(practice/exercises) 

Individual Practice Yes 

3 1/30/2004 

 9:00-9:45 22 Science—Routing and 

Low Form 

Interactive Lecture Lead Trainer Yes 

 9:45-10:00 23 Science—Routing and 

Low Form 

(practice/exercises) 

Individual Practice Lead Trainer Yes 

 10:00-10:15 BREAK 

 10:15-11:00 24 Science—Routing and 

Middle Form 

Interactive Lecture Co-Trainer Yes 

 11:00-11:15 25 Science—Routing and 

Middle Form 

(practice/exercises) 

Individual Practice Lead Trainer Yes 

 11:15-11:45 26 Science—High Form Interactive Lecture Lead Trainer Yes 

See note at end of exhibit. 
4-10 

Exhibit 4-3. Spring-fifth grade Trainer Training Agenda: School year 2003–04—Continued 

Day Time Session # Topic Type of session Trainer On Computer?

 11:45-12:00 27 Science—High Form 

(Practice/exercises) 

Individual Practice 

Yes 

 12:00-1:00 LUNCH 

 1:00-1:30 28 Food Consumption 

Questionnaire 

Interactive Lecture Co-Trainer 

Yes 

 1:30-2:00 29 Height and Weight 

Lecture 

 Co-Trainer 

 2:00-2:30 30 Height and Weight 

Practice 

Individual Practice 

Yes 

 2:30-3:00 31 Closing Interactive Interactive Lecture Lead Trainer Yes 

 3:00-3:15 BREAK 

 3:15-4:45 32 Full Child Interactive Interactive Lecture Lead Trainer Yes 

 4:45-5:15 33 Maximizing the 

Child’s Performance 

Interactive Lecture Co-Trainer 

No 

Day Time Session # Topic Type of session Trainer On Computer?

4 1/31/2004 

 9:00-10:00 Training 

responsibilities 

Discussion No 

 10:00-12:00 Wave 1: Child 

respondents 

 

No 

 12:00-1:00 LUNCH No 

 1:00-3:00 Wave 2: Child 

respondents 

 

No 

 3:00-5:00 Wave 3: Child 

respondents 

 

No 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

Field Supervisor Training. Field supervisors had 3 days of training before the assessor 

training in Los Angeles, California. New supervisors for spring data collection attended the first and 

second day of training and all supervisors, including those who conducted fall advance school contacts, 

attended the third day. Field supervisors were also trained to use the FMS, and the field supervisors 

entered information into the FMS during training presentations. Eighty-one field supervisors completed 

training. The topics covered in the field supervisor training session included reviewing materials from the 

fall school contact and recruitment, role playing to practice contacting school coordinators; identifying 

and locating children who moved from their third-grade schools, identifying the regular and special 

education teachers of ECLS-K children and linking them to those children and domains, distributing and 

following up on teacher and school administrator questionnaires, completing the facilities checklist, and 

conducting quality control observations. Exhibit 4-4 presents the field supervisor training agenda. 
4-11 

Exhibit 4-4. Spring-fifth grade field supervisor training agenda: School year 2003–04 

ECLS-K Round 5 Training 

National Supervisor Training 

Day Time Session # Topic Type of session 

1 2/13/05 

 8:30-9:00 1 Study Overview Interactive Lecture 

 9:00-9:45 2 School Contacts and Materials Interactive Lecture 

 9:45-10:15 3 

Advance School Contact 

Activities Interactive Lecture 

 10:15-10:30 BREAK 

 10:30-12:00 4 Call to the School Coordinator Interactive Lecture 

 12:00-1:00 LUNCH 

 1:00-1:45 5 School Coordinator Exercises Interactive Exercises 

 1:45-2:30 6 School Coordinator Role Plays Dyad Role Plays 

 2:30-3:15 7 

Additional Preassessment 

Activities Interactive Lecture 

 3:15-3:30 BREAK 

 3:30-4:30 8 Identifying Transfer Children Interactive Lecture 

 4:30-5:30 9 

Identifying Transfer Children’ 

Exercises Interactive Exercises 

2 2/14/04 

 8:30-9:30 10 

Assessment Day 

Responsibilities Interactive Lecture 

 9:30-10:30 11 School/Teacher Questionnaires Interactive Lecture 

 10:30-10:45 BREAK 

 10:45-11:15 12 

School/Teacher Questionnaires 

Exercises Interactive Exercises 

 11:15-12:00 13 

Strategies for Obtaining 

Completed School/Teacher 

Questionnaires Interactive Lecture 

 12:00-1:00 LUNCH 

 1:00-1:45 14 

Strategies for Obtaining 

Completed School/Teacher 

Questionnaires Role Plays Dyad Role Plays 

 1:45-3:30 15 Refielding Transfer Children Interactive Lecture 

 3:30-3:45 BREAK 

 3:45-4:30 16 

Refielding Transfer Children’ 

Exercises Interactive Exercises 

 4:30-5:30 17 Solo Practice on the FMS Self-administered Exercises

See note at end of exhibit. 
4-12 

Exhibit 4-4. Spring-fifth grade field supervisor training agenda: School year 2003–04—Continued 

National Supervisor Training 

Day Time Session # Topic Type of session 

3 2/15/04 

 8:30-9:30 18 Case Management Interactive Lecture 

 9:30-10:30 19 Parent Interview Field Issues Interactive Lecture 

 10:30-10:45 BREAK 

 10:45-11:15 20 Refusal Conversion Workshop Interactive Lecture 

 11:15-12:00 21 Refusal Conversion Role Plays Dyad Role Plays 

 12:00-1:00 LUNCH 

 1:00-1:45 22 Quality Control Interactive Lecture 

 1:45-2:30 23 Quality Control Exercises Interactive Exercises 

 2:30-3:00 24 Assigning Disposition Codes Interactive Lecture 

 3:00-3:15 BREAK 

 3:15-4:00 25 

Assigning Disposition Codes 

Exercises Interactive Exercises 

 4:00-5:30 26 Administrative Issues Interactive Lecture 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

Assessor Training. The assessor training sessions were conducted in Los Angeles, 

California. Assessor training lasted for 5 days; field supervisors were also trained to perform all assessor 

activities; bilingual assessors were trained for half a day to conduct the parent interview in Spanish. Two 

hundred forty-two assessors, 16 parent interviewers, and 81 field supervisors completed training. 

Assessor training included an overview of study activities to date, interactive lectures on the direct child 

assessments and the parent interview, role-play scripts to practice parent interviews and direct child 

assessments, direct child assessment precertification exercises on each form of the direct child domain 

assessments, techniques for parent refusal avoidance, and strategies for building rapport with children. A 

major goal of the assessor training was to train field staff in the proper procedures to conduct the direct 

child assessments. The sessions provided trainees with practical experience with all the direct child 

assessment materials and procedures and the CAI programs before data collection. Trainees practiced 

entering information into the CAI system on laptop computers during training sessions on conducting the 

direct child assessments and parent interview role-play scripts. Exhibit 4-5 presents the assessor training 

agenda. 
4-13 

Exhibit 4-5. Spring-fifth grade assessor training: School year 2003–04 

Day Time Session # Topic Type of session Trainer 

1 2/17/2004 

 8:30-9:00 1 Study Introduction Interactive Lecture Lead Trainer  9:00-9:30 2 Parent Interview Introduction Interactive Lecture Lead Trainer  9:30-11:45 3 Parent Interactive #1 Interactive Role Play Lead Trainer  10:00-10:15 BREAK  11:45-1:00 LUNCH (includes Welcome)  1:00-2:00 4 Parent Dyad #1 Dyad Role Play Lead Trainer  2:00-3:00 5 Parent Interact #2 (with Twins) Interactive Role Play Co-Trainer  3:00-3:15 BREAK  3:15-5:30 6 Parent Dyad #2 Dyad Role Plays Lead Trainer 

See note at end of exhibit. 
4-14 

Exhibit 4-5. Spring-fifth grade assessor training: School year 2003–04—Continued 

Day Time Session # Topic Type of session Trainer 

2 2/18/2004 

 8:30-9:45 7 Final evaluation role play Dyad Role Plays Lead Trainer  9:45-10:00 BREAK Lead Trainer  10:00-11:15 8 

Child Assessment and Certification Overview Interactive Lecture Lead Trainer 

 11:15-12:15 9 

Introduction and Selfdescription Questionnaire Interactive Lecture Lead Trainer 

 12:15-1:15 LUNCH  1:15-2:00 10 

Reading—Routing and Low Form Interactive Lecture Lead Trainer 

 2:00-2:45 11 

Reading—Routing and Low Form (practice/exercises) Individual Practice 

 2:45-3:30 12 

Reading—Routing and Middle Form Interactive Lecture Co-Trainer 

 3:00-3:15 BREAK  3:30-4:15 13 

Reading—Routing and Middle Form (practice/exercises) Individual Practice 

 4:15-4:45 14 Reading—High Form Interactive Lecture Lead Trainer 

 4:45-5:30 15 

Reading—High Form (practice/exercises) Individual Practice 

See note at end of exhibit. 
4-15 

Exhibit 4-5. Spring-fifth grade assessor training: School year 2003–04—Continued 

Day Time Session # Topic Type of session Trainer 

3 2/19/2004 

 8:30-9:15 16 

Math—Routing and Low Form Interactive Lecture Lead Trainer 

 9:15-10:00 17 

Math—Routing and Low Form (practice/exercises) Individual Practice 

 10:00-10:15 BREAK  10:15-11:00 18 

Math—Routing and Middle Form Interactive Lecture Co-Trainer 

 11:00-11:45 19 

Math—Routing and Middle Form (practice/exercises) Individual Practice 

 11:45-12:15 20 Math—High Form Interactive Lecture Lead Trainer  12:15-1:15 LUNCH  1:15-2:30 21 

Math—High Form (practice/exercises) Individual Practice 

 2:30-3:15 22 

Science—Routing and Low Form Interactive Lecture Lead Trainer 

 3:15-3:30 BREAK  3:30-4:15 23 

Science—Routing and Low Form (practice/exercises) Individual Practice Lead Trainer 

 4:15-4:45 24 

Science—Routing and Middle Form Interactive Lecture Co-Trainer 

 4:45-5:30 25 

Science—Routing and Middle Form (practice/exercises) Individual Practice Lead Trainer 

See note at end of exhibit. 
4-16 

Exhibit 4-5. Spring-fifth grade assessor training: School year 2003–04—Continued 

Day Time Session # Topic Type of session Trainer 

4 2/20/2004 

 8:30-9:00 26 Science—High Form Interactive Lecture Lead Trainer  9:00-9:45 27 

Science—High Form (Practice/exercises) Individual Practice 

 9:45-10:15 28 

Food Consumption Questionnaire Interactive Lecture Co-Trainer 

 10:15-10:30 BREAK  10:30-11:00 29 Height and Weight Lecture Co-Trainer  11:00-11:30 Height and Weight Practice Individual Practice  11:30-12:15 31 Closing Interactive Interactive Lecture Lead Trainer  12:15-1:15 LUNCH  1:15-3:00 32 Full Child Interactive Interactive Lecture Lead Trainer  3:00-3:15 BREAK  3:15-3:45 33 

Maximizing the Child’s Performance Interactive Lecture Co-Trainer 

 3:45-5:30 34/35 Full Child Role Plays 1 & 2 Dyad Role Plays See note at end of exhibit. 
4-17 

Exhibit 4-5. Spring-fifth grade assessor training: School year 2003–04—Continued 

Day Time Session # Topic Type of session Trainer 

5 2/21/2004 

 8:30-8:45 36 Transmission Interactive Lecture Lead Trainer  8:45-9:45 37 Full Child Practice Individual Practice  9:45-10:00 BREAK  10:00-11:15 W1 Wave 1 Live Respondents Observation  CAPILAB/Review AssignmentsWorkshop  11:30-12:45 W2 Wave 2 Live Respondents Observation  CAPILAB/Review AssignmentsWorkshop  Live Assessment Debriefing Discussion  11:30-2:30 LUNCH  1:00-2:15 W3 Wave 3 Live Respondents Observation  CAPILAB/Review AssignmentsWorkshop  Live Assessment Debriefing Discussion  2:30-3:45 W4 Wave 4 Live Respondents Observation  CAPILAB/Review AssignmentsWorkshop  Live Assessment Debriefing Discussion 

See note at end of exhibit. 
4-18 

Exhibit 4-5. Spring-fifth grade assessor training: School year 2003–04—Continued 

Day Time Session # Topic Type of session Trainer 

6 2/22/2004 SPANISH TRAINING 

 8:30-9:30 38 Gaining Cooperation Interactive Role Play Lead Trainer  9:30-11:00 39 Parent Interactive #1 Interactive Role Play Lead Trainer  10:15-10:30 BREAK  11:00-12:00 40 Parent Dyad #1 Dyad Role Play  12:00-1:00 LUNCH  1:00-2:00 40 Parent Dyad #2 Dyad Role Play Lead Trainer 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 
4-19 

4.3.2.2 Certification of the Child Assessors 

Assessors and field supervisors conducted child assessments. Training to administer the 

assessment battery included exercises that took into account the different ways children may answer 

questions. Ambiguous answers were included in both training interactives and role plays to provide 

practice in coding a variety of responses. Particular attention was paid to the question-by-question 

specifications (QxQs) for constructed response/open-ended questions so that a clear understanding of 

coding guidelines was established. These guidelines were reviewed frequently during training so that 

trainees were prepared for answers that were not phrased exactly as found in the scoring rubrics. 

Training on the specific subdomains was implemented in four parts. The first two parts 

involved precertification activities and the second two parts involved certification activities. The 

precertification activities involved an interactive lecture in which the trainer reviewed the specific items 

within the subdomain (i.e., reading, mathematics, and science test forms) using the QxQs and a 

standardized training script and individual practice, which addressed both item administration skills and 

accurate coding of responses. The certification activities involved written certification exercises and a live 

certification session during which each trainee conducted a portion of the assessment with an actual child 

respondent. 

The first part of the precertification training for each subdomain was an interactive lecture 

using the QxQs, which highlighted the constructed response/open-ended assessment items. Although 

conceivably children could produce a variety of answers for any item, there are only 54 questions (25 

percent of the items) in the entire assessment that were open-ended items and would be expected to have 

the most variation. A constructed response/open-ended question is a short answer question that is scored 

against a specific criterion or a scoring rubric (scoring guide). These types of questions require the 

assessor to interpret and use multiple criteria in evaluating the responses. The assessor compared a child’s 

response to a scoring rubric in order to determine if the child’s response was correct or incorrect. Most of 

the open-ended assessment items were in the reading and science domains. 

The second part of the precertification training involved independent review and individual 

practice in administering the assessment domain. Once an interactive lecture was completed, the trainees 

practiced administering the subdomain that had just been presented. The practice was not scripted. 

However, trainees were instructed to concentrate on practicing item administration skills, such as reading 

verbatim from the laptop computer screen and gesturing. Trainers observed trainees during the individual 
4-20 

practice, recorded the results of the individual practice in the Trainee Evaluation Form and gave each 

trainee specific feedback on performance. The trainer reviewed the item administration skills with the 

trainee and provided additional instruction to trainees requiring additional practice. 

The third part of the training on each subdomain involved written certification exercises. 

These exercises concentrated on the open-ended questions and were designed to evaluate the trainees’ 

ability to accurately select the appropriate response codes. Each exercise presented a constructed 

response/open-ended question with a variety of responses (some from the scoring rubric as well as some 

variations on those), and the assessor was required to code each response as either correct or incorrect. 

Training staff scored these exercises and feedback was given to the trainee on coding accuracy. Most 

trainees passed the written exercises on their first attempt. Less than a quarter of the trainees (77 trainees 

or 24 percent) did not pass at least one element of the reading certification exercises on the first attempt. 

The mathematics and science certification exercises were considerably easier for trainees; only two 

trainees were required to repeat any mathematics exercise and 55 trainees (17 percent) were required to 

repeat the science yellow certification exercise. This variability was due to the complexity of the fifthgrade reading scoring rubrics and the unfamiliarity of the exercises themselves (reading exercises were 

distributed first, with mathematics and science exercises on later days). The trainees who did not pass the 

written certification exercises completed specified remedial training steps prescribed by training staff. All 

of the trainees who had to re-take the exercises after the remedial evening session achieved a passing 

score. No field staff were released because of failure on the certification exercises. 

Training staff implemented a variety of remedial training steps for individual trainees and 

trainees collectively. The QxQs and interactive lectures were reviewed as necessary and the trainee(s) 

practiced the subdomain test again. The help laboratory was made available after the daily training 

session to provide additional instruction and help to trainees before the live portion of the child 

assessment certification. 

The culmination of the child assessments training was administering the cognitive 

assessment battery to children. This allowed the trainees to experience what it is like in the field and to 

put into practice all their new skills. Training staff who were already certified on the assessment observed 

trainees as they administered parts (e.g., a routing test and a level test) of the assessment to fifth gradeaged children. Feedback on trainee performance was given using the Assessment Certification Form. 

Supervisors and assessors were certified on the child assessment by administering approximately half of a 

cognitive assessment to a child while being observed by certified evaluators. As evaluators observed the 
4-21 

assessment, they completed the appropriate sections of the Assessment Certification Form. This form had 

both the trainee and the evaluator’s names recorded on the cover as well as the date the evaluation was 

conducted. 

The Assessment Certification Form had two sections: section 1 (exhibit 4-6) rated the trainee 

on key skill areas, such as building rapport, using neutral praise, responding to behaviors presented by the 

child, appropriate pacing, and avoiding coaching. The evaluator marked each skill area that the trainee did 

not demonstrate appropriately. 

Exhibit 4-6. Section 1 of the Assessment Certification Form: School year 2003–04 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

Section 2 listed specific questions from each routing and subdomain form. The instructions 

for completing section 2 are shown in exhibit 4-7. For each of the listed questions observed, the evaluator 

recorded both the child’s response and noted if the trainee did not demonstrate the specified required 

administration skills for that question. The required administration skills included reading questions 

verbatim, using appropriate probes, and using appropriate hand motions (gesturing). For each question on 

which the evaluator observed that the trainee did not demonstrate the required administration skill(s), he 

or she checked a box, indicating which skill was not performed. 
4-22 

Exhibit 4-7. Instructions for Section 2 of the Assessment Certification Form: School year 2003–04 

SECTION 2: Specific Assessment Activities 

Supervisor/Evaluator: Code the items as the assessor administers the assessment. 

Code the child’s response as the item is administered. 

If the item requires probing, check the box if the assessor does not use the appropriate probe. 

Check the box in the “Verbatim” column if the assessor does not read the item exactly as worded on the 

screen. 

Check the box in the “Gesturing” column if the assessor does not use appropriate hand motions. 

For each validation code item, check the box in the “Validation” column if the response coded by the 

assessor is not what you have coded. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

As mentioned earlier, during the practice child assessment, the evaluator simultaneously 

coded the child’s response to the open-ended assessment items listed in the Assessment Certification 

Form. These open-ended items were flagged in the CAI program for quality control review and a screen 

showing how the trainee coded each of those answers to the questions was accessed at the end of the 

assessment for review with the evaluator as shown in exhibit 4-8. 
4-23 

Exhibit 4-8. Quality review screen: School year 2003–04 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

Exhibit 4-9 presents an example of a question from the Assessment Certification Form, 

which required the evaluator to evaluate the required assessment administration skills: reading verbatim, 

using appropriate gesturing, using the appropriate probe, and accurately coding the child’s response to 

this open-ended question. The evaluator checked the boxes only if the assessor did not demonstrate the 

correct behavior. For all questions observed by the evaluator and listed in the Assessment Certification 

Form, the evaluator recorded if the assessor did not read verbatim or gesture appropriately. Evaluators 

only marked a box when the assessor did not use the appropriate probe for those questions that displayed 

probes (a box labeled “Assessor Used Inappropriate Probe?” in the Assessment Certification Form). Also, 

the evaluators only marked the box if their response code did not match the assessors’ for open-ended 

questions (a box labeled “Validation Code Did Not Match” was presented in the Assessment Certification 

Form). 
4-24 

Exhibit 4-9. Sample question from Assessment Certification Form: School year 2003–04 

DID NOT 

READ 

QUESTION 

VERBATIM 

GESTURED 

INAPPROPRIATELY 

VALIDATION 

CODE DID 

NOT MATCH

R5.245 What do you think is the difference between the 

storyteller’s bedroom and the sister’s bedroom?** 

 IF STUDENT SAYS, “ A mess” OR “A lot of junk”, 

PROMPT: “How is it different?” 

 

CORRECT ........................................................... 1

STORY TELLER’S BEDROOM IS: 

(BIGGER MESS/MORE 

MESS/MESSIER) 

(MORE JUNK/JUNKIER) 

(IS A MORE DISGUSTING SPOT) 

CORRECT ANSWER MUST INVOLVE 

A COMPARISON 

INCORRECT........................................................ 2

(STORY TELLER’S BEDROOM IS 

CLEAN/AS MESSY AS SISTER’S BEDROOM)

DETAILS ABOUT THE SISTER’S 

BEDROOM 

 EXAMPLES: 

(PIZZA UNDER THE DESK) 

(SOCKS ON THE LIGHT) 

(MESS/ALSO A MESS) 

(JUNK/ A LOT OF JUNK) 

REFUSED ............................................................ 777

DON’T KNOW...................................................... 999

ASSESSOR USED INAPPROPRIATE PROBE? 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

At the end of the “live” child assessment, after the child had been escorted from the room, 

the evaluator and the trainee reviewed each trainee’s overall performance on the half of the cognitive 

assessment that he or she conducted. After discussing the ratings in section 1, the trainee accessed the 

quality control screen. The trainee and evaluator reviewed their codes for each open-ended question asked 

in section 2 of the Assessment Certification Form. 

The evaluator then scored the certification assessment using the scoring form shown in 

exhibit 4-10. The evaluator first counted the number of boxes checked in section 1: Rapport Building of 

the Assessment Certification Form and recorded that number in the appropriate row of Form A of the 
4-25 

Certification Scoring Form. The evaluator then counted the number of check marks for each section of the 

assessment that was observed and recorded those numbers in the appropriate boxes of Form A. Exhibit 4-

11 presents an example of a completed Form A. In this example, the evaluator observed one trainee who 

administered Reading Routing, Reading Yellow and Math Routing. The evaluator recorded zero (0) check 

marks for section I, zero (0) check marks for Reading Routing, one (1) check mark for Reading Yellow, 

and zero (0) check marks for Math Routing. The total check marks recorded was one (1). 

The evaluator then calculated the number of possible points for the sections of the 

assessment observed using the Total Possible Points Chart displayed in exhibit 4-12. This chart was 

necessary for the assessment certification because each trainee only conducted half of the cognitive 

assessment with a child respondent; this chart helped the evaluator use the correct denominator for 

calculating the score. For each section of the assessment observed, the evaluator circled the last question 

recorded in the Assessment Certification Form and wrote the number of possible points in the appropriate 

box in the second section of the scoring form (Form B) (see exhibit 4-13). Continuing with the example 

started earlier, the evaluator observed all of Reading Routing (6 possible points), all of Reading Yellow 

(21 possible points), and all of Math Routing (8 possible points). In addition, section 1 contributes 6 

possible points. Based on these sections of the assessment, the total number of possible points for this 

trainee is 41. 

Finally, the evaluator determined the proportion correct by using the Proportion Correct 

Chart displayed in exhibit 4-14. Continuing the example, the evaluator found the column on the 

Proportion Correct Chart that displayed the total possible points for the trainee based on the portions of 

the assessment observed (41) and the row on the Proportion Correct Chart that displayed the total check 

marks recorded for the trainee (1), and recorded the proportion from the corresponding box on the 

Proportion Correct Chart in the box under Step 6 (exhibit 4-15) on the scoring form (.98). Once the 

evaluator completed the scoring, the trainee was rated as Certified, Remedial Action, or Failed. 
4-26 

Exhibit 4-10. Certification Scoring Form: School year 2003–04 

Certification Scoring 

Step 1: Record the number of check marks from Section 1: Rapport Building and Section 2: Specific 

Assessment Activities in the appropriate boxes of Form A. 

Step 2: Sum each row and record total in Row Totals column of Form A. 

Step 3: Sum Row Totals column (including number of check marks from Section 1) and record in Total 

check marks box of Form A. 

Form A: Checks Marks Recorded in Sections 1 and 2

Assessment Routing Form Red Form Yellow Form Blue Form Row Totals 

Reading 

Mathematics 

Science 

Number of check marks from Section 1: Rapport Building: 

Total Check Marks: 

Step 4: Using the Total Possible Points Chart, circle the last question in the assessment certification 

form for each routing and sub-domain that the trainee completed. Put the number to the right of 

the item in the appropriate box of Form B. For example, the Trainee completed Reading 

Routing and through item R5.245 of Reading Red, the total number of possible points is 20 (6 

for Routing and 14 for Red). Record 6 in the Reading row under Routing Form and record 14 

under Red Form and 20 in the Row Totals column of Form B. 

Step 5: Sum Row Totals column (including Possible Points for Section 1) and record in Total Possible 

Points box of Form B.

Form B: Total Possible Points

Assessment Routing Form Red Form Yellow Form Blue Form Row Totals 

Reading 

Mathematics 

Science 

Possible Points for Section 1: Rapport Building: 6 

Total Possible Points: 

Step 6: Use Proportion Correct Chart to determine the proportion correct and write that proportion in 

this box: 

Result: If Proportion correct is greater than or equal to .85, Assessor is Certified. 

If Proportion correct is between .70 and .85, Assessor requires Remedial Action (Complete 

Remedial Task Chart below) 

 If Proportion correct is less than .70, Assessor Failed. 

See note at end of exhibit. 
4-27 

Exhibit 4-10. Certification Scoring Form: School year 2003–04—Continued 

Remedial Task Chart

Circle the number of the task(s) needing improvement. Discuss remedial action with the assessor. 

TASK # REMEDIAL ACTION 

APPROPRIATE PACE 1 PRACTICE/REVIEW JOB AIDS 

AVOID COACHING 2 REVIEW THE JOB AIDS 

READ VERBATIM 3 PRACTICE READING FROM THE SCREEN 

GESTURING APPROPRIATELY 4 PRACTICE GESTURING 

USE APPROPRIATE PROBES 5 REVIEW THE JOB AIDS/Q-BY-Qs 

VALIDATION CODE DID NOT MATCH 6 REVIEW THE Q-BY-Qs 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

Exhibit 4-11. Example of Form A for Trainee 1: School year 2003–04 

Form A: Checks Marks Recorded in Sections 1 and 2 

Assessment Routing Form Red Form Yellow Form Blue Form Row Totals 

Reading 0 1 1 

Mathematics 0 0 

Science 

Number of check marks from section 1: Rapport Building: 0 

Total Check Marks: 1 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 

Class of 1998–99 (ECLS-K), spring 2004. 
4-28 

Exhibit 4-12. Total Possible Points Chart: School year 2003–04 

Total Possible Points Chart 

Reading

Routing Red Yellow Blue 

Item # Total 

Points 

Item # Total 

Points 

Item # Total 

Points 

Item # Total 

Points 

R5.055 3 R5.185 4 R5.270 4 R5.405 3 

R5.060 6 R5.215 7 R5.285 7 R5.415 6 

R5.235 10 R5.305 10 R5.460 8 

R5.245 14 R5.335 12 R5.470 11 

R5.250 17 R5.340 15 R5.480 14 

R5.260 20 R5.365 18 R5.530 17 

R5.375 21 R5.535 20 

 

Mathematics

Routing Red Yellow Blue 

Item # Total 

Points 

Item # Total 

Points 

Item # Total 

Points 

Item # Total 

Points 

M5.045 3 M5.125 3 M5.185 2 M5.275 4 

M5.070 5 M5.135 5 M5.205 6 M5.285 6 

M5.085 8 M5.155 7 M5.210 9 M5.325 9 

M5.160 9 M5.235 13 M5.335 12 

M5.170 11 M5.245 15 M5.340 16 

M5.265 18 M5.360 19 

 M5.365 23 

Science

Routing Red Yellow Blue 

Item # Total 

Points 

Item # Total 

Points 

Item # Total 

Points 

Item # Total 

Points 

S5.015 3 S5.125 3 S5.205 2 S5.285 4 

S5.020 6 S5.140 6 S5.215 5 S5.290 6 

S5.040 9 S5.160 8 S5.225 8 S5.295 10 

S5.085 11 S5.170 11 S5.230 11 S5.300 12 

S5.180 14 S5.235 13 S5.310 15 

S5.250 15 S5.325 18 

 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 

Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 
4-29 

Exhibit 4-13. Example of Form B: School year 2003–04 

Form B: Total Possible Points 

Assessment Routing Form Red Form Yellow Form Blue Form Row Totals 

Reading 6 21 28 

Mathematics 7 7 

Science 

Possible Points for section 1: Rapport Building: 6 

Total Possible Points: 41 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 

Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 
4-30 

Exhibit 4-14. Proportion Correct Chart: School year 2003–04 

69 70 72 73 74 76 77 78 79 80 81 82 83 84 85 86 87 

1 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 2 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 3 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.97 0.97 4 0.94 0.94 0.94 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 5 0.93 0.93 0.93 0.93 0.93 0.93 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 6 0.91 0.91 0.92 0.92 0.92 0.92 0.92 0.92 0.92 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 7 0.90 0.90 0.90 0.90 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.92 0.92 0.92 0.92 0.92 8 0.88 0.89 0.89 0.89 0.89 0.89 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.91 0.91 0.91 9 0.87 0.87 0.88 0.88 0.88 0.88 0.88 0.88 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.90 0.90 

10 0.86 0.86 0.86 0.86 0.86 0.87 0.87 0.87 0.87 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.89 11 0.84 0.84 0.85 0.85 0.85 0.86 0.86 0.86 0.86 0.86 0.86 0.87 0.87 0.87 0.87 0.87 0.87 12 0.83 0.83 0.83 0.84 0.84 0.84 0.84 0.85 0.85 0.85 0.85 0.85 0.86 0.86 0.86 0.86 0.86 13 0.81 0.81 0.82 0.82 0.82 0.83 0.83 0.83 0.84 0.84 0.84 0.84 0.84 0.85 0.85 0.85 0.85 14 0.80 0.80 0.81 0.81 0.81 0.82 0.82 0.82 0.82 0.83 0.83 0.83 0.83 0.83 0.84 0.84 0.84 15 0.78 0.79 0.79 0.79 0.80 0.80 0.81 0.81 0.81 0.81 0.81 0.82 0.82 0.82 0.82 0.83 0.83 16 0.77 0.77 0.78 0.78 0.78 0.79 0.79 0.79 0.80 0.80 0.80 0.80 0.81 0.81 0.81 0.81 0.82 17 0.75 0.76 0.76 0.77 0.77 0.78 0.78 0.78 0.78 0.79 0.79 0.79 0.80 0.80 0.80 0.80 0.80 18 0.74 0.74 0.75 0.75 0.76 0.76 0.77 0.77 0.77 0.78 0.78 0.78 0.78 0.79 0.79 0.79 0.79 19 0.72 0.73 0.74 0.74 0.74 0.75 0.75 0.76 0.76 0.76 0.77 0.77 0.77 0.77 0.78 0.78 0.78 20 0.71 0.71 0.72 0.73 0.73 0.74 0.74 0.74 0.75 0.75 0.75 0.76 0.76 0.76 0.76 0.77 0.77 21 0.70 0.70 0.71 0.71 0.72 0.72 0.73 0.73 0.73 0.74 0.74 0.74 0.75 0.75 0.75 0.76 0.76 22 0.68 0.69 0.69 0.70 0.70 0.71 0.71 0.72 0.72 0.73 0.73 0.73 0.73 0.74 0.74 0.74 0.75 23 0.67 0.67 0.68 0.68 0.69 0.70 0.70 0.71 0.71 0.71 0.72 0.72 0.72 0.73 0.73 0.73 0.74 24 0.65 0.66 0.67 0.67 0.68 0.68 0.69 0.69 0.70 0.70 0.70 0.71 0.71 0.71 0.72 0.72 0.72 25 0.64 0.64 0.65 0.66 0.66 0.67 0.68 0.68 0.68 0.69 0.69 0.70 0.70 0.70 0.71 0.71 0.71 26 0.62 0.63 0.64 0.64 0.65 0.66 0.66 0.67 0.67 0.68 0.68 0.68 0.69 0.69 0.69 0.70 0.70 27 0.61 0.61 0.63 0.63 0.64 0.64 0.65 0.65 0.66 0.66 0.67 0.67 0.67 0.68 0.68 0.69 0.69 28 0.59 0.60 0.61 0.62 0.62 0.63 0.64 0.64 0.65 0.65 0.65 0.66 0.66 0.67 0.67 0.67 0.68 29 0.58 0.59 0.60 0.60 0.61 0.62 0.62 0.63 0.63 0.64 0.64 0.65 0.65 0.65 0.66 0.66 0.67 30 0.57 0.57 0.58 0.59 0.59 0.61 0.61 0.62 0.62 0.63 0.63 0.63 0.64 0.64 0.65 0.65 0.66 

NOTE: Proportion correct = (Total Possible Points – Number of Check Marks)/Total Possible Points. EXAMPLE: If an assessor received 3 check marks out of a proportion correct is .88. This Assessor would be certified. SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spri
4-31 

Exhibit 4-15. Example of Proportion Correct Chart: School year 2003–04 

Step 6: Use Proportion Correct Chart to determine the proportion correct and write that proportion in 

this box: 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

Table 4-1 presents the results of the training certification. There were 242 assessors and 81 

field supervisors for a total of 323 trainees who were certified; 16 assessors were only trained to complete 

parent interviews. Trainees scoring 85 percent or above were certified as qualified to administer the child 

assessments. Trainees who scored between 70 and 84 percent were required to complete remedial training 

and have an additional certification conducted before beginning assessments. 

Table 4-1. Results of child assessment certification, fifth grade: School year 2003–04 

Trainees Number Percent

Total 323 100

Score on certification form 

85 percent or above 320 99.0

70–84 percent 3 1.0

Below 70 percent 0 0

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

The majority of the trainees (99.0 percent) scored above 85 percent on the certification form, 

with only 1.0 percent (n=3) scoring between 70 and 84 percent. None of the trainees failed to meet the 

70 percent threshold on the Assessment Certification Form. All trainees who needed remedial training 

were certified qualified to administer the child assessments after they conducted a second assessment on a 

fifth grade-aged child who was not part of the ECLS-K sample. 

The three trainees who required remedial action were assigned remedial training, exercises, 

and practice assessments to perform on an age-appropriate child in their area. Their supervisor observed 

the practice assessment and certified them. The scoring form specifies the type of remedial training 

activities that were assigned based on the type of problems observed by the evaluator (see exhibit 4-16). 

Once the remedial training activities were completed, the field supervisor or field manager was 

responsible for conducting an additional certification assessment with the trainee with a fifth grade-aged 

child not in the ECLS-K sample prior to being able to start work. 

.98 
4-32 

Exhibit 4-16. Certification Scoring Form: School year 2003–04 

Remedial Task Chart

Circle the number of the task(s) needing improvement. Discuss remedial action with the assessor. 

TASK # REMEDIAL ACTION 

APPROPRIATE PACE 1 PRACTICE/REVIEW JOB AIDS 

AVOID COACHING 2 REVIEW THE JOB AIDS 

READ VERBATIM 3 PRACTICE READING FROM THE SCREEN 

GESTURING APPROPRIATELY 4 PRACTICE GESTURING 

USE APPROPRIATE PROBES 5 REVIEW THE JOB AIDS/Q-BY-Qs 

VALIDATION CODE DID NOT MATCH 6 REVIEW THE Q-BY-Qs 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

4.3.2.3 Parent Interview Training and Certification 

Training assessors to conduct the parent interview also included interactive lectures and role 

plays that were designed not only to review the intent of the questions but also to demonstrate the 

different ways that parents may answer questions. Ambiguous answers were included in both training 

interactive lectures and role-play scripts to provide practice in probing and handling a variety of 

responses. Parent interview QxQs were carefully reviewed throughout training so that 

assessors/interviewers would be prepared for a variety of responses from parents. The culmination of 

parent interviewer training was a final certification role play designed to test all the protocols and 

techniques reviewed during training. The final role play was conducted using a scripted parent interview 

and the final parent role play evaluation form. The final role-play script was designed to test the trainee’s 

understanding of the interview content, proper interviewing techniques, including probing, and accurate 

response recording in CATI. Trainees were paired up and each completed half of a parent interview as the 

interviewer. As they conducted the interview as the respondent, trainees completed the evaluation form 

that consisted of a checklist to evaluate key areas, such as contacting and selecting the respondent, asking 

the questions verbatim, probing properly, and following the correct question path. Two points were given 

for each item performed correctly. Trainees had to score at least 30 out of a possible 36 points to pass. 

Table 4-2 presents the results of the parent certification. 
4-33 

Table 4-2. Results of parent interview certification, fifth grade: School year 2003–04 

Trainees Number Percent

Total 308 100

Score on certification form 

85 percent or above 294 95.5

70–84 percent 14 4.5

Below 70 percent 0 0

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

All trainees who expected to conduct parent interviews completed the parent certification 

procedures. The majority of trainees (95.5 percent or 294 trainees) were successfully certified on the 

parent interview. Parent certification scores were not reported for sixteen trainees. The 14 trainees who 

did not pass the parent certification at training worked with their supervisors to improve the areas in 

which they lacked skills. These staff completed another parent certification role play with their 

supervisor. All 14 staff were certified on the parent interview after the remedial training. 

4.4 Fall Preassessment School Contact 

Beginning in September 2003, all participating ECLS-K schools (i.e., schools that 

participated in previous rounds of data collection) were contacted by telephone to prepare for the spring 

data collection. When children were identified as having transferred to another school, the child’s new 

school (and district, if necessary) was recruited. As noted in section 4.3.1, the advance contact 

served several purposes. It reminded schools about the study and reacquainted them with the study’s 

procedures; it provided supervisors the opportunity to persuade the schools to participate; and it allowed 

the collection of information necessary for the spring 2004 national data collection. There were four 

primary tasks to be accomplished during the fall contact. These were to schedule appointments to conduct 

the child assessments in the spring; to verify parent consent procedures; to identify the children’s 

teachers; and to identify children who had withdrawn from the school and obtain locating information 

about their new schools. The fall contact activities are described below. 
4-34 

4.4.1 Advance Mailings 

In September 2003, an advance package was mailed via Federal Express to all participating 

ECLS-K schools, i.e., schools that participated in third grade, asking them to prepare for the 

preassessment contact telephone call. The schools were asked to identify a school staff coordinator to 

serve as a liaison with the study (in original sampled schools, this person was usually the coordinator 

from the previous rounds of data collection). A package containing study materials was sent to the 

schools. The package contained the following materials and was customized by school type—original or 

transfer schools: 

Original school advance package: 

 A letter printed on ECLS-K letterhead reminding school staff about the study, 

describing the fifth-grade data collection, and alerting the school coordinator of the 

advance contact in the fall; 

 A School Summary Sheet (original schools) —a two-page document providing a brief 

review of the study to date and the fifth-grade data collection activities; 

 A Study Findings Sheet (original schools)—a summary of findings about children 

from the previous rounds of data collection; and 

 An ECLS-K Study Children Form and Instructions—a listing of all the sampled 

children and instructions for completing the form with specific information such as 

the children’s continued attendance at the school, their grade, their teachers’ names, 

classroom numbers, receipt of special education services, and receipt of assessment 

accommodations/exclusions before the preassessment call. 

Transfer schools: 

 A letter printed on ECLS-K letterhead introducing school staff to the study, describing 

the fifth-grade data collection, and alerting the school coordinator of the advance 

contact in the fall; 

 A School Summary Sheet (transfer schools)—a two-page document providing a brief 

overview of the study and the fifth-grade data collection activities; 

 A Study Findings Sheet—a summary of findings about children from the previous 

rounds of data collection; and 

 An ECLS-K Study Children Form and Instructions—a listing of all the sampled 

children and instructions for completing the form with specific information such as 

the children’s continued attendance at the school, their grade, their teachers’ names, 
4-35 

classroom numbers, receipt of special education services, and receipt of assessment 

accommodations/exclusions before the preassessment call. 

4.4.2 Fall Preassessment School Coordinator Contact 

The preassessment contacts were made by telephone from September through December 

2003. The preassessment school contacts were successful in meeting all four tasks described above. 

Contacting original sampled schools to set up the spring assessment and identifying children who 

withdrew from their spring-third grade school and moved into their fifth-grade transfer school, enabled 

the identification of schools that were ineligible for fifth-grade data collection. Schools were determined 

to be ineligible for fifth-grade data collection if no ECLS-K sampled children were currently enrolled. 

Original sampled schools became ineligible because fourth grade was the highest grade in the school or 

because the school had closed, that is, was no longer operational. More transfer schools were determined 

to be ineligible as children transferred out of them into other schools. 

During the preassessment contact, the field supervisor contacted the school coordinator to 

collect some basic information about the school and some detailed information about each ECLS-K 

sampled child. The field supervisor used the School Information Form to collect basic information about 

the school, such as school start and end dates, vacation and holiday schedules, and parking directions. The 

form was also used to determine if the school was a year-round school, taught fifth grade, or required new 

parent consent, and to obtain information on class organization. The supervisor used the Child Work Grid 

(exhibit 4-17) to collect basic information about the child such as his or her grade, the name and 

classroom number of the child’s primary teacher to link the child to a teacher, and whether the child had 

an Individualized Education Program (IEP) or its equivalent. In addition, in original sample schools, the 

assessment date was scheduled; assessment dates for new transfer schools were scheduled in the spring. 

Reviewing parent consent with the school. Because parental consent was obtained in the 

base year and re-obtained in the third-grade year, field supervisors did not raise parental consent with the 

school coordinator unless the school district required it. However, if the school was a transfer school then 

the supervisor did ask the school coordinator whether it was necessary to obtain parental consent. If the 

schools required current consent forms or changed the type of consent that was required (e.g., from 

implicit to explicit), parent letters and consent forms were either mailed to the school for distribution to 

parents or sent directly to parents by Westat depending on the schools’ preference. Parent cover letters 
4-36 

Exhibit 4-17. Child Work Grid, spring-fifth grade: School year 2003–04 See note at end of exhibit. 
4-37 

Exhibit 4-17. Child Work Grid, spring-fifth grade: School year 2003–04—Continued 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (EC
4-38 

and consent forms were available in English and Spanish. Overall, 16 percent of all schools contacted 

required an updated parental consent. Of the schools requiring updated parental consent, approximately 

50 percent required explicit consent from the parents. 

Collecting information about ECLS-K sampled children. Field supervisors primarily used 

the Child Work Grid to collect information from the school coordinator about the ECLS-K sampled 

children. The school coordinator was asked to complete the school version of the form to expedite the 

call. Field supervisors collected information from the school coordinators about the ECLS-K sampled 

children still enrolled in the school, including the child’s current grade, the name and classroom for the 

child’s regular teacher in order to link the child to a teacher, and whether or not the child had an 

Individualized Education Program (IEP) (see exhibit 4-18). If the child had an IEP, then the name and 

classroom of the child’s special education teacher was noted, along with whether the child required any 

accommodations to participate in the direct cognitive assessment. Through this process, 1,088 children 

were identified with IEPs and 793 special education teachers were linked to these children. The 

accommodations to the fifth-grade direct cognitive assessment were the same as those for the third-grade 

direct cognitive assessment (see section 4.5.2). Field supervisors contacted the teachers of the ECLS-K 

children as necessary for any of this information. Field supervisors also identified the respondent for the 

student record abstract for each child. 

If a child was identified as having transferred out of the school, the field supervisor asked the 

school coordinator to provide the names, addresses, and telephone numbers of these transfer schools Of 

those children who transferred, only a subset was followed to their new school (see section 3.6.1 in 

chapter 3 for more detail on how mover children were subsampled). If the new school belonged to a 

district that was new to the study, the district was mailed a letter with the new school name. A field 

supervisor contacted the district by telephone and recruited the district into the study before any contact 

was made with the school. If the district was already cooperating, the district was notified by mail and the 

new school was contacted and recruited directly. Field supervisors also verified with the school that no 

child who had previously transferred had returned to the school. 
4-39 

Exhibit 4-18. Individualized Education Program (IEP) process, spring fifth-grade: School year 2003–04 

WHO WILL BE ASSESSED?

DOES CHILD HAVE IEP OR IFSP ON FILE?

Yes

Does IEP specify that child cannot 

participate in standardized 

assessments?

No

Use Standard Assessment 

Procedures

Yes

Exclude child from 

assessment

No

Does child require an 

assessment accommodation?

Yes

Is the required assessment 

accommodation offered by 

ECLS-K?

Link child to a regular

and special education

teacher

No

Use standard assessment 

procedures

Yes

Assessment conducted with 

provision of accommodation.

 Is the accommodation an IEP?

No

Exclude child from 

assessment

Yes

Link child to a regular and 

special education

teacher

No

Child is linked to a regular 

teacher only

ACCOMMODATIONS OFFERED BY ECLS-K ACCOMMODATIONS NOT OFFERED BY 

ECLS-K

SETTINGS

SCHEDULING/TIMING

HEALTH CARE AIDES (PERSONAL 

ATTENDANTS)

ASSISTIVE DEVICES

IEP

BRAILLE ADMINISTRATION

ENLARGED PRINT

SIGN LANGUAGE ADMINISTRATION

Link child to a

 regular and special 

education teacher

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 
4-40 

Contacting families of homeschooled children. As part of the fall preassessment contact, 

children in the ECLS-K sample who were homeschooled in previous rounds were identified. The status of 

home-schooled children who were identified in rounds 1 through 4 was verified with their parents and 

updated as necessary. In addition, during the preassesment contact some schools identified homeschooled 

children. Their status was also verified with their parents during data collection. Parents of these children 

were contacted by telephone in September through December 2003 to determine if the child was still 

homeschooled or had enrolled in a school. If the child had enrolled in a school, the new school was 

contacted and recruited into the study. Parents of children who were still schooled at home were notified 

about the next round of data collection in the spring. 

Identifying the key child in classrooms with multiple study children. In grade 5, the 

design of the child-level teacher questionnaire was changed to include collecting data about the child’s 

reading class and mathematics or science class. In previous rounds, children were taught primarily in selfcontained classrooms and teachers only reported classroom level information once for the classroom. Due 

to the grade 5 design change, the teacher-child links were restructured to include the domain (reading, 

mathematics, or science) and a classroom identifier (e.g., room number and the time the class meets). A 

unique link number was created for each teacher/child/domain/class combination. In order to reduce data 

collection burden for teachers who were linked to multiple sample children in the same class, a “Key 

Domain Child” was identified for each separate subject and class that each teacher taught. The teachers 

were asked to report classroom level information only once in the questionnaire for the key domain child 

and child-level information for all sampled children in their class. Field supervisors collected the teacherchild-domain-classroom link information about each child and entered the information into the FMS. The 

information was used to generate the hardcopy teacher questionnaires (see section 5.4.3 for more 

information). 

4.4.3 Fall Preassessment Contact Results 

The goals for the fall preassessment contact with schools were the following: (1) set 

appointments for the spring assessment in original sample schools, (2) identify schools that ended at 

fourth grade to determine the school to which the sample children transferred, (3) identify children who 

changed schools since third grade, (4) link children to teachers and domains (i.e., reading, mathematics, 

science) for the advance school and teacher questionnaire mailings, and (5) contact as many transfer 

schools as possible within the field period to ascertain whether the child was still there and recruit the 
4-41 

school into the study. It was not expected that every transfer school identified within the fall contact could 

be contacted within the fall field period because of the numbers of children that were expected to move. It 

was also expected that additional schools would be contacted during the spring round because children 

were expected to continue to move between fall and spring of the school year. 

Approximately 30 percent of the ECLS-K sampled children transferred to other schools 

between the spring of third grade and beginning of fifth grade (i.e., fall 2003). At the start of the fall 2003 

field period, there were 1,016 transfer schools identified prior to the fall contact. An additional 744 

transfer schools were identified during the fall contact field period (September–December) for a total of 

1,760 transfer schools at the end of the field period. All of the transfer schools identified during the fall 

were contacted at least once during the field period to ascertain if the child was still enrolled. During the 

fall preassessment contact, 4,039 children were identified as movers and processed in preparation for the 

spring data collection. Table 4-3 presents the status of these 4,039 movers after the fall preassessment 

contact. 

Table 4-3. Status of movers identified during fall preassessment contact: School year 2003–04 

Status Total Percent 

 Total 4,039 100 

Fielded for assessment in spring 2,381 58.9 

 Assess in school 2,261 55.9 

 Assess in home 120 2.9 

Unlocatable 279 6.9 

 Moved to nonsampled PSU 128 3.2 

 Moved outside of U.S. 27 <1 

 Subsample not followed 1,220 30.0 

 End of field period 4 <1 

NOTE: Movers who had a status of “Fielded for assessment in spring” were treated like the nonmover cases during the spring data collection. 

Those with a status of “Unlocatable” could not be located during the fall preassessment contact; additional attempts to locate them continued in 

spring data collection. Cases identified as “Moved to nonsampled PSU” or “Moved to outside of U.S.” were not fielded for the spring data 

collection because they had moved out of the designated data collection area. Cases with a status of “Subsample not followed” were not fielded in 

the spring because the sampling plan called for collecting data from only a subsample of movers (see chapter 3, section 3.6.1). These cases were 

not in the subsample to be followed. Only 4 cases with a status of “End of field period” were not contacted in the fall because the schools were 

either new to the school sample frame with limited information available to contact them or in school districts new to the sample that required an 

additional contact at the district level before the schools could be contacted for spring data collection. These cases were closed out and rolled over 

for contact in the spring. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 
4-42 

The fall preassessment contact protocol was completed for 100 percent of the original 

ECLS-K schools and 97 percent of the transfer schools (both those identified before and during the fall 

field period) within the fall field period. Twenty percent of schools were identified as out of scope, since 

they did not contain any sampled students (9 percent of original sampled schools and 25 percent of 

transfer schools). All of the schools that children transferred to as a result of the school ending at fourth 

grade, closing, or merging with another school were identified within the field period. Tables 4-4 through 

4-6 present the production reports for the fall preassessment contact for original sample and transfer 

schools. 

Table 4-4. Fall preassessment contact productivity report, by week: School 

year 2003–04 

Week Week ending Total schools completed Original Transfer

 

1 9/10/03 9 6 3

2 9/17/03 103 78 25

3 9/24/03 284 211 73

4 10/11/03 511 360 151

5 10/8/03 766 510 256

6 10/15/03 1001 650 351

7 10/22/03 1233 737 496

8 10/29/03 1463 800 663

9 11/5/03 1637 842 795

10 11/12/03 1826 867 959

11 11/19/03 2047 908 1139

12 11/26/03 2171 927 1244

13 12/3/03 2240 933 1307

14 12/10/03 2333 940 1393

15 12/17/03 2392 949 1443

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood 

Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 
4-43 

Table 4-5. Fall preassessment contact production report—original schools: School year 2003–04 Week # 

Week 

ending 

Response 

rate 

(percent) 

Total 

number

Out of 

scope In scope

No 

action Pending Complete Refusal

1 9/10/03 0.6 967 3 964 948 9 6 0 2 9/11/03 8.1 993 31 962 635 247 78 1 3 9/24/03 22.2 1,004 55 949 403 331 211 2 4 10/0/03 38.0 1,016 69 947 218 634 360 3 5 10/8/03 54.2 1,026 85 941 99 328 510 2 6 10/15/03 67.8 1,048 90 958 46 256 650 3 7 10/22/03 77.4 1,044 92 952 22 187 737 3 8 10/29/03 83.9 1,048 94 954 13 135 800 3 9 11/5/03 88.2 1,050 95 955 8 99 842 3 10 11/12/03 90.7 1,051 95 956 5 78 867 3 11 11/19/03 94.9 1,051 95 957 2 41 908 3 12 11/26/03 96.8 1,052 94 958 2 22 927 2 13 12/3/03 97.4 1,052 94 958 2 16 933 1 14 12/10/03 98.1 1,052 94 958 0 9 940 0 15 12/17/03 99.1 1,052 94 958 0 0 949 0 SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), 
4-44 

Table 4-6. Fall preassessment contact production report—transfer schools, by week: School year 2003–04 Week # 

Week 

ending 

Response 

rate 

(percent) 

Total 

number

Out of 

scope In scope No action Pending Complete Refusal

1 9/10/03 0.3 1,166 2 1,164 1,160 1 3 0 2 9/11/03 2.1 1,223 25 1,198 1,133 40 25 0 3 9/24/03 5.9 1,293 57 1,236 1,096 65 73 1 4 10/0/03 11.5 1,407 92 1,315 1,071 91 1,511 1 5 10/8/03 18.5 1,537 155 1,382 997 127 256 1 6 10/15/03 25.3 1,606 218 1,388 873 156 351 4 7 10/22/03 3409 1,708 288 1,420 710 201 496 7 8 10/29/03 46.5 1,785 359 1,426 451 299 663 8 9 11/5/03 54.8 1,847 396 1,451 346 292 795 10 10 11/12/03 65.7 1,898 438 1,460 214 266 959 11 11 11/19/03 78.1 1,922 463 1,459 106 191 1,139 12 12 11/26/03 84.2 1,957 480 1,477 70 142 1,244 6 13 12/3/03 88.4 1,972 494 1,478 41 108 1,307 4 14 12/10/03 94.1 1,988 508 1,480 19 42 1,393 3 15 12/17/03 97.4 1,997 516 1,481 0 0 1,443 0 SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), 
4-45 

4.4.4 Tracing Activities during Fifth-Grade Data Collection 

In June 2003 an attempt was made to obtain a current address for all eligible households 

using Telematch, a company that searches for current address information through its many databases. 

Telematch provided updated information, either a new or partial address, a new or partial telephone 

number, or both, for over half the households. For the remaining households, Telematch provided no new 

information beyond what was already contained Westat’s tracing and locating files. Table 4-7 presents the 

results of the spring Telematch search. 

Table 4-7. Tracing results for spring 2003 

Result Number Percent

Total households searched 13,390 100.0

Households located and entered 

into tracing database 

7,641 57.4

No new information provided 5,749 42.9

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

In order to ensure that as many of the sampled children as possible were contacted in the 

spring, locating efforts were conducted from June through early August 2003. Staff in Westat’s 

Telephone Research Center (TRC) traced the 829 children who could not be located during previous 

rounds of data collection. TRC staff used the Internet, telephone directories, and other means to locate 

these children and their households. When children and/or households were found, the new school and 

contacting information was entered into the tracing system database for fielding in the spring. Table 4-8 

presents the results of the summer tracing effort. 

Table 4-8. Results of the Telephone Research Center’s locating efforts for cases with a status of 

unlocatable at the beginning of fifth grade: School year 2003–04 

Result Number Percent

 Total cases worked 827 100.0

Located and entered into database 305 36.8

Unlocatable 519 62.6

Final refusal 5 .06

NOTE: “Unlocatable” means that the children and their households could not be found using the available tracing and locating strategies; “final 

refusal” means that the child’s family indicated that they did not want to participate. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 
4-46 

In mid-September 2003, all eligible households were mailed a letter asking the head of the 

household to record their current address and current school information for the sampled child on an 

enclosed postcard. In mid-October staff in Westat’s Telephone Research Center (TRC) began calling 

households that had not returned the postcard. By the end of December 2003, over 75 percent of the 

households had responded. Table 4-9 presents the results of the fall tracing contacts. 

Table 4-9. Results of the fall household tracing contact: School year 2003–04 

Result Total Percent 

Total households contacted 13,379 100.0 

New locating information provided 2,457 18.4 

No new locating information 

provided 

7,612 56.9 

Refused 83 0.6 

No response to contact 3,227 24.1 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 

Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 

4.5 Spring-Fifth Grade Data Collection 

All children who were assessed during the base year or for whom a parent interview was 

completed in the base year were eligible to be assessed in the spring-fifth grade data collection, with four 

exceptions: They were (1) children who became ineligible in an earlier round (because they died or 

moved out of the country), (2) children who were subsampled out in previous rounds because they moved 

out of the original schools and were not subsampled to be followed, (3) children whose parents 

emphatically refused to cooperate (hard refusals) in any of the data collection rounds since springkindergarten, and (4) children in the fifth-grade sample for whom there were neither first-grade nor thirdgrade data. Eligibility for the study was not dependent on the child’s current grade, that is, children were 

eligible whether they were promoted to fifth grade or were retained in fourth grade. In spring-fifth grade, 

the children attended 2,008 public schools and 356 private schools. Two-thirds of the public schools 

(1,355) and almost half of the private schools (166) were transfer schools.

As in previous rounds of data collection, the field staff were organized into work areas, each 

with a data collection team consisting of one field supervisor and two or more assessors. The data 

collection teams were responsible for all data collection activities in their work areas; they conducted the 
4-47 

direct child assessments and the parent interviews, collected all school and teacher questionnaire and 

completed checklists. The majority of field staff members in fifth grade were continuing from previous 

rounds of data collection; a few new staff were hired in areas where no experienced ECLS-K staff lived. 

4.5.1 Spring Preassessment Activities 

Based on the information collected in the fall of 2003, packets of hard-copy teacher and 

school administrator questionnaires were assembled and mailed to schools in January and February 2004, 

along with letters confirming the scheduled visits to the school. Teachers and school administrators were 

asked to complete the questionnaires for pickup by the field supervisor or lead assessor on assessment 

day. Letters were also mailed to parents at this time reminding them of the spring-fifth grade data 

collection activities. 

Most preassessment contact with schools was by telephone starting in March 2004. Field 

supervisors contacted the school to confirm the assessment date and the receipt of the hard-copy 

questionnaires and to arrange for space to conduct the assessments. This follow-up call to the schools was 

essentially to confirm the logistics for the assessments.

4.5.2 Conducting the Direct Child Assessments 

The direct child assessments were conducted from March through June 2004, the same time 

of year as in prior spring data collections. The school coordinator set the assessment date with a 

supervisor based on the school’s schedule. Over three-quarters of the assessments were completed by the 

end of April, with 20.5 percent completed in May and 1.3 percent completed in June. There was no 

evidence that certain types of children or schools were assessed early or late in the field period. 

Table 4-10 presents the weekly completion rates for the child assessments. In year-round schools, 

assessment teams made multiple visits to the school, visiting when each track was in session to assess the 

sampled children. There were 501 (2.3 percent) sampled children in year-round schools. 
4-48 

Table 4-10. Weekly completion of the child assessments: School year 2003–04 

Week # 

Week 

starting

Completed

 assessments1

Cumulative 

completed 

assessments 

Cumulative

Percent 

of total

1 2/25/04 55 55 0.5

2 3/3/04 666 721 6.3

3 3/10/04 1,543 2,264 19.9

4 3/17/04 1,080 3,344 29.4

5 3/24/04 1,112 4,456 39.2

6 3/31/04 1,074 5,530 48.6

7 4/7/04 981 6,511 57.3

8 4/14/04 666 7,177 63.1

9 4/21/04 904 8,081 71.1

10 4/28/04 804 8,885 78.2

11 5/6/04 814 9,699 85.3

12 5/13/04 704 10,403 91.5

13 5/20/04 541 10,944 96.3

14 5/27/04 279 11,223 98.7

15 6/2/04 82 11,305 99.4

16 6/9/04 33 11,338 99.7

17 6/16/04 22 11,360 99.9

18 6/23/04 8 11,368 100.0

1

 Based on field production reports. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

The direct child assessments were usually conducted in a school classroom or library. Before 

conducting the assessments, field supervisors and assessors set up the room for the assessments. They 

followed procedures for meeting children that were agreed upon during the preassessment contact with 

the school. Each child was signed out of his or her classroom prior to the assessment and signed back into 

the classroom upon the conclusion of the assessment. During the scheduling of schools in the fall, an 

attempt was made to conduct the direct child assessments at about the same point in time from the 

beginning of school year and the end of the year to increase the chances that exposure to instruction was 

about the same for all children. The fifth-grade direct child assessments averaged 97 minutes in length. 

Table 4-11 displays the number of completed child assessments for each round of data 

collection, including spring-fifth grade. All of the assessments in spring-fifth grade were completed in 

English. The majority of the assessments (83.0 percent) were completed in original schools, and 

interestingly, the number of assessments in transfer schools decreased in spring-fifth grade. Only 17 
4-49 

Table 4-11. Completed child assessments by round of data collection: School years 1998–99, 1999–2000, 2001–02, and 

 Fallkindergarten Spring- kindergarten Fall- first grade Spring- first grade Spring- third grade  Number Percent Number Percent Number Percent Number Percent Number Perc

Child assessments completed 

Assessed 

19,147 100 19,987 100 5,297 100 16,622 100 14,502 1

In English, no accommodation1 17,019 88.9 18,342 91.8 4,848 91.5 15,460 93.0 13,565 9

In Spanish 1,008 5.3 724 3.6 176 3.3 286 1.7 † 

In other language 410 2.1 229 1.1 33 0.6 37 0.2 † With accommodation1 515 2.7 579 2.9 195 3.7 761 4.6 814 

Excluded 88 0.5 70 0.4 28 0.5 47 0.3 74 Partial complete 107 0.6 43 0.2 17 0.3 31 0.2 49 

Child assessments completed 19,147 100 19,987 100 5,297 100 16,622 100 14,502 1

Original sampled school 19,147 100 19,463 97.4 4,867 91.9 14,830 89.2 10,820 7Transfer school 0 0 524 2.6 430 8.1 1,792 10.8 3,682 2

†Not applicable.

1The term accommodation in this table is the field operational definition of accommodation, which includes the wearing of glasses and hearing aids. These types of ai

ensure that every child had the same chance at a successful assessment. With this information, assessors could prompt a child (e.g., to get her glasses before being assess

NOTE: This table reflects final production numbers prior to statistical adjustment. This table does not include children who were subsampled out in fall- and spring-first section 5.5.4.) These numbers should not be used to estimate student mobility. SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), 

spring 2000, spring 2002, and spring 2004. 
4-50 

percent of the sample was assessed in transfer schools in spring-fifth grade. Assessments of 

homeschooled children, or children in nonparticipating transfer schools, were held at the parent’s home or 

another location, such as a library, of the parent’s choosing. 

Less than 1 percent of participating children in fifth grade required accommodations or were 

excluded from the direct child assessments. Children were excluded from the direct assessments if they 

had a disability, e.g., blindness, or deafness, that could not be accommodated by the ECLS-K direct 

assessments, or their Individualized Education Program prevented their participation in assessments or 

required an accommodation not offered by the ECLS-K assessments. Accommodations offered by the 

ECLS-K assessments were as follows: alternative setting, scheduling, or timing; health care aide present; 

or the use of a personal assistive device. Table 4-12 presents the number of children excluded from or 

requiring an accommodation to the direct child assessment procedures in the spring of fifth grade. 

Table 4-12. Number of children excluded from or accommodated in the spring-fifth grade assessments: 

School year 2003–04 

Category Number of children 

Exclusions 

Excluded for disability 63

Accommodation1

 

Alternative setting accommodation 50

Scheduling/timing accommodation 64

Health care aide present 12

Personal assistive device 9 1 The term accommodation in this table includes only those accommodations offered during the assessment such as an alternative setting. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

4.5.3 Conducting the Parent Interview 

Parent interview procedures mirrored those of previous rounds of data collection. The parent 

interview was administered, primarily by telephone interview using CAI, between February and June 

2004. Slightly over 50 percent of the parent interviews were completed in February and March, 43 

percent were completed in April and May, and 6 percent were completed in June. The parent interview 

averaged 43 minutes. Table 4-13 presents the weekly completion of parent interviews. 
4-51 

Table 4-13. Weekly completion of parent interviews, spring-fifth grade: School year 2003–04 

Week 

Week 

starting

Completed 

 interviews1

Cumulative 

completed 

interviews 

Cumulative 

percent 

of total

1 2/25/04 46 46 0.4

2 3/3/04 675 721 6.6

3 3/10/04 1,218 1,939 17.7

4 3/17/04 1,286 3,225 29.5

5 3/24/04 1,201 4,426 40.5

6 3/31/04 1,083 5,509 50.4

7 4/7/04 917 6,426 58.7

8 4/14/04 737 7,163 65.5

9 4/21/04 724 7,887 72.1

10 4/28/04 660 8,547 78.1

11 5/6/04 460 9,007 82.3

12 5/13/04 456 9,463 8605

13 5/20/04 464 9,927 90.7

14 5/27/04 326 10,253 93.7

15 6/2/04 215 10,468 95.7

16 6/9/04 190 10,658 97.4

17 6/16/04 147 10,805 98.8

18 6/23/04 91 10,896 99.6

19 6/30/04 44 10,940 100.0

1 Based on field production reports from the parent interview. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

As in previous rounds of data collection, the parent interview was conducted in person if the 

respondent did not have a telephone. Table 4-14 contains the number of parent interviews per round, 

including spring-fifth grade. In fifth grade, only 2.7 percent of all completed parent interviews were 

conducted in person; 8.1 percent of all completed parent interviews were conducted in a language other 

than English; 95.1 percent of the latter were conducted in Spanish. 

In spring-fifth grade data, 10 percent of the parent cases were classified as final nonresponse. 

As in most field studies, the primary reasons for final nonresponse were parents who could not be located 

and parents who refused to complete the interview; 32.4 percent of the nonresponse parent cases were 

parents who could not be located, 43.9 percent were refusals and 23.8 percent were other nonresponse 

(e.g., language barrier). 
4-52 

Table 4-14. Number and percent of completed parent interviews by data collection mode, language, and wave of data colle

1998–99, 1999–2000, 2001–02, and 2003–04 

 Fallkindergarten Spring- kindergarten Fall- first grade Spring- first grade Spring- third grade  Number Percent Number Percent Number Percent Number Percent Number Percent N

 Parent interviews 

 completed 

17,997 100 18,907 100 5,073 100 15,576 100 13,504 100 

In person 618 3.4 619 3.3 211 4.2 456 2.9 319 2.4 By phone 17,379 196.6 18,288 96.7 4,862 95.8 15,120 97.1 13,185 97.6  Parent interviews 

 completed 

17,997 100 18,907 100 5,073 100 15,576 100 13,504 100 

English 17,379 96.6 17,482 92.5 4,717 93.0 14,319 91.9 12,416 91.9 Spanish 618 3.4 1,321 7.0 351 6.9 1,071 6.9 932 6.9 Other language 0 0 81 0.4 0 0 75 0.5 41 0.3 Partial complete 0 0 23 0.1 5 0.1 111 0.7 115 0.9 NOTE: This table completes final production numbers prior to statistical adjustment. SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), 

fall 1999, spring 2000, spring 2002, and spring 2004. 
4-53 

4.5.4 Conducting Data Collection on Children Who Withdrew From Their Previous Round 

School 

While contacting schools, field supervisors asked school coordinators to identify children 

who had withdrawn from the school since the spring of third grade. School staff were asked whether they 

knew the name and address of the school to which the child transferred, as well as any new information 

about the child’s household address. For the children who had moved from their spring-third grade school 

and were not part of the sample to be followed, information was collected only from the school personnel 

and not from parents. For children who had withdrawn from their spring-third grade school and were 

identified to be followed (i.e., were part of the sample of movers), supervisors also consulted parents and 

other contacts for information on the children’s new school. This information was entered into the FMS 

and processed at Westat for data collection. 

Table 4-15 presents the status of the children who were identified as movers in fifth grade; 

12,717 children were identified as having transferred from the school in which they were enrolled during 

the spring of base year, first grade, or third grade. The movers described in this table are defined as 

“operations movers” rather than “statistical movers” since cooperation must be secured from the transfer 

schools in order for data collection to proceed (see section 3.6.1 for a discussion of statistical movers.). 

There are two flags that describe movers. One flag is for statistical purposes and the other for operations 

purposes. The definitions for the two flags necessarily differ. For example, children who move into a 

destination school are flagged as a mover for operations purposes, since cooperation must be secured 

from the school to follow the child there, but the statistical flag is not set since these children are not 

movers in the statistical definition. Table 4-15 describes total movers across all rounds, which is the only 

way they were tracked; the chapter 3 tables describe the grade 5 movers from a statistical perspective. Of 

the 12,717 mover children in spring-fifth grade, 4,187 (32.9 percent) were in scope (i.e., children selected 

to be followed) and followed. The remaining 8,530 mover children were out of scope and were not 

followed; no child assessment or parent interview was conducted for these children. 
4-54 

Table 4-15. Number of children who moved in spring-fifth grade by completion category: School year 

2003–04 

Spring-fifth grade Category Number of children Percent

Total movers1 12,717 100.0 

Out-of-scope2

 8,530 67.1 

Did not follow3

 7,880 92.4 

Moved to outside of U.S.3

 153 1.8 

Deceased3

 7 <1 

Excluded from spring-fifth grade4

 490 5.7 

In-scope and followed2

 4,187 32.9 

Completed assessment5

 3,299 78.8 

Unlocatable5

 281 6.7 

Nonsampled primary sample unit5

 395 9.4 

Assessment refused5

 149 3.6 

Not assessed/absent5

 63 1.5 1

 The movers described in this table are defined as “operations movers” rather than “statistical movers” since cooperation must be secured from 

the transfer schools in order for data collection to proceed. 2

 Percent based on total movers. 

3 Percent based on out-of-scope children. 

4

 In fifth grade, four groups of children were excluded, irrespective of other subsampling procedures that were implemented. They are (1) 

children who became ineligible in an earlier round (because they died or moved out of the country); (2) children who were subsampled out in 

previous rounds because they moved out of the original schools and were not subsampled to be followed; (3) children whose parents emphatically 

refused to cooperate (hard refusals) in any of the data collections rounds since spring-kindergarten; and (4) children in the third-grade sample for 

whom there are neither first-grade nor third-grade data. 5 Percent based on in-scope children. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

Parent interviews were attempted for all in-scope children. However, different school and 

assessment data collection strategies were followed for children who moved, depending on where they 

moved and the cooperation status of their new school. School and assessment data collection was 

attempted for children who moved and were flagged as “follow” in spring-fifth grade in the following 

ways: 

 Data collected for children moving into cooperating base year sampled schools 

included the child assessments in the school, school administrator questionnaire, 

regular and/or special education teacher questionnaires, facilities checklist, and 

student record abstract forms. 

 Data collected for children moving into nonsampled schools in base year cooperating 

districts included the child assessments in the school, school administrator 

questionnaires, regular and/or special education teacher questionnaires, and student 

record abstract forms, if school permission was obtained. If school permission was not 
4-55 

obtained, the assessments were conducted in the home and no school or teacher data 

were collected. 

 For children moving into transfer schools that refused, schools in sampled districts 

that refused, or originally sampled schools that were ineligible when sampled because 

they did not have kindergarten classes, the direct child assessments were conducted in 

the home. No school or teacher data were collected. 

 For children moving into schools in nonsampled districts or dioceses: 

- If the school was within the primary sampling unit (PSU), data collected 

included the child assessments in the school, school administrator 

questionnaire, regular and/or special education teacher questionnaires, facilities 

checklist, and student record abstract forms, if school permission was obtained. 

If school permission was not obtained, the assessments were conducted in the 

home and no school or teacher data were collected. 

- If the school was outside a sampled PSU, no child, school, or teacher data were 

collected. 

 For children who were not enrolled in school in the spring (including children who 

were home schooled), data collected included the child assessments in the home if the 

child was in the sampled PSU. If the child was outside the sampled PSU, no child 

assessment or school or teacher data were collected. 

As previously mentioned, 4,039 movers were identified during the fall preassessment 

contact. During spring-fifth grade, an additional 366 movers were identified. The identification of 92 

percent of the fifth-grade movers during the fall contact accomplished two important goals: (1) allowed 

additional time to trace movers; and (2) reduced the burden of tracing a large number of movers during 

the spring-fifth grade data collection. Table 4-16 presents the final status of the children who were 

identified as movers in fifth grade; a total of 12,717 children were identified as having transferred from 

the school in which they were enrolled when they were sampled in kindergarten. Of the 12,717 children 

identified as movers in spring-fifth grade, 4,187 children were selected to be followed and were followed 

(32.9 percent of total movers). The remaining 8,530 mover children were out-of-scope for this round of 

data collection because they moved out of the country, were deceased, excluded from fifth grade because 

they had no first- or third-grade data, or were subsampled out; no child assessments or parent interviews 

were conducted for these children. 
4-56 

Table 4-16. Number of children who moved by completion category: School years 1998–99, 1999–2000, 2001–02, an Spring-kindergarten Spring-first grade Spring-third grade Category 

Number 

of 

children Percent

Number 

of 

children Percent

 Number 

of 

children Percent

Num

chil

Total movers1 1,568 100 5,759 100 9,889 100 12

Out of scope2 16 1 2,893 50.2 4,221 42.7 8

Did not follow3 0 0 2,848 98.4 4,102 97.2 7

Moved to outside of U.S.3 16 1 44 1.5 117 2.7 

Deceased3 0 0 1 <.1 2 <.1 

Excluded from fifth grade †† †† † † 

In-scope and followed2 1,552 98.9 2,866 49.8 5,668 57.3 4

Completed assessment3 510 32.8 1,792 62.5 3,682 65.0 3

Unlocatable3 501 32.3 271 9.5 607 10.7 

Nonsampled primary sample unit3 282 18.2 454 15.8 871 15.4 

Nonresponse 259 16.7 349 12.2 508 8.9 

† Not applicable. 1 The movers described in this table are defined as “operations movers” rather than “statistical movers” since cooperation must be secured from the transfer schools 

proceed. 

2 Percent based on total movers. 3 Percent based on out-of-scope children. 4 In fifth grade, four groups of children were excluded, irrespective of other subsampling procedures that were implemented. They are (1) children who became 

(because they died or moved out of the country); (2) children who were subsampled out in previous rounds because they moved out of the original schools and 

followed; (3) children whose parents emphatically refused to cooperate (hard refusals) in any of the data collections rounds since spring-kindergarten; and (4) childfor whom there are neither first-grade nor third-grade data. 

5 Percent based on in-scope children. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), 
4-57 

Of the children who moved in fifth grade and were selected to be followed, 66.9 percent 

moved into nonsampled schools, 5.3 percent moved into a school outside the PSU and 10.4 percent of the 

movers could not be located. Table 4-17 presents the fifth-grade movers by school and district status. 

Table 4-17. Children moving in fifth grade: School year 2003–04 

Category Number Percent

 Total movers eligible and followed 3,036 100

Moved into cooperating schools 459 15.1

Moved into nonsampled schools 2,030 66.9

Moved into schools/districts that refused 27 <1

Home schooled 41 1.4

Moved outside the primary sampling unit 162 5.3

Moved—could not be located 317 10.4

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

4.5.5 Teacher and School Data Collection 

Data were collected from school administrators, regular classroom teachers, and special 

education teachers from February through June 2004. The school and teacher questionnaires were mailed 

to the school coordinators in January and February 2004. This schedule allowed additional time for these 

respondents to complete and return the instruments to Westat. 

During the fall advance school contact, it became apparent that many fifth-grade teachers 

would be asked to report about many sampled children and multiple domains (e.g., reading and math or 

science) due to the modification to the fifth-grade teacher questionnaires (see section 2.5 for a detailed 

discussion). In fifth grade, there was still considerable clustering of students in classrooms and teachers 

would be repeating classroom level information across many questionnaires. Table 4-18 presents the 

number of child-level questionnaires and percent of teachers asked to complete them based on fall 

linkages of children to teachers and domains as well as the percent of teachers completing that number of 

questionnaires during spring data collection. 
4-58 

Table 4-18. Number of child-level questionnaires and percent of teachers: School year 2003–04 

Number of child-level questionnaires to 

complete 

Percent of teachers 

expected from fall school 

contact

Percent of teachers 

completing questionnaires 

from spring-fifth grade

1 to 4 questionnaires 63.0 75.0 

5 to 10 questionnaires 26.0 18.0 

11 to 15 questionnaires 6.0 4.0 

16 to 46 questionnaires 4.0 2.0 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 

Class of 1998–99 (ECLS-K), spring 2004. 

A procedure to reduce burden on teachers was developed to identify a “Key Child” for each 

domain and class. Teachers were asked to complete all items in the reading, mathematics, and science 

questionnaires for the designated key child, designated by a blue dot on the questionnaire cover, for the 

appropriate domains/classes. They were asked to complete only the child-level items, and not the 

classroom-level items, for the remaining children in that domain/class. 

Using the child-teacher-domain-class linkage information collected in the fall, customized 

packages of questionnaires were assembled for each child’s reading and mathematics or science teachers 

and/or special education teachers. The customized teacher questionnaire materials included the following: 

 A Teacher Questionnaire cover letter; 

 Teacher Questionnaire; 

 A Reading Instruction Sheet attached to the child-level reading questionnaires for 

each separate reading class; 

 A Math Instruction Sheet attached to the child-level math questionnaires for each 

separate math class; 

 A Science Instruction Sheet attached to the child-level science questionnaires for each 

separate science class; and, 

 A Special Education Instruction Sheet attached to the child-level special education 

questionnaires (if appropriate).

The Teacher Questionnaire Cover letter was a much simplified document from previous 

rounds. It laid out the teacher tasks in succinct language on one page and clearly described the 

questionnaires, including the questionnaire cover colors, to help the teacher easily identify each part of 

the package. A checklist of instructions, an easy step-by-step guide, was included to aid the teacher in 
4-59 

completing the questionnaires and returning them to the school coordinator. Domain instruction sheets 

(Reading, Mathematics, Science, and Special Education) included all the information the teacher required 

to correctly identify the class/domain. Teachers only received instruction sheets required for the sampled 

children in the domains they taught. All questionnaires were labeled with names and ID numbers. Teacher 

questionnaires were labeled with teacher name and ID number. Child questionnaires were labeled with: 

teacher name and ID number, child name and ID number, domain, reading, mathematics, or science; 

classroom; and class time. The key domain child questionnaires were labeled with a blue dot. Child-level 

questionnaires were assembled in bundles by domain and class identifier (e.g., reading, 9:15 am, 

mathematics, Room A) for each teacher. All materials and questionnaires for each teacher were placed in 

an envelope and the envelope labeled with the teacher name. 

Teacher and school administrator packets were bundled together by school and mailed to the 

school coordinator for distribution. If the school and/or teacher and school administrator were not 

identified in the fall advance contact, then the supervisor gathered the relevant information during the 

preassessment call in the spring and mailed the packets at that time. During their visits to the schools, 

field supervisors also completed a facilities checklist for each sampled school. 

Field supervisors began prompting for the return of questionnaires when they contacted 

schools to confirm the assessment schedule. During the field period, field supervisors followed up with 

school administrators and teachers by telephone and with visits to the schools to prompt for the return of 

the questionnaires. Field supervisors returned completed questionnaires to Westat. 

In April 2004, the field supervisors were instructed to conduct intensive followup for 

missing school administrator questionnaires for schools with high minority student populations to reduce 

potential bias. There were 334 such schools identified. As a result of the followup efforts by the field 

supervisors, 324 of the 334 schools (97 percent) completed and returned the school administrator 

questionnaires. 

4.5.5.1 Hard-Copy Data Retrieval 

Retrieval Procedure. Data retrieval involved collecting missing items for some 

questionnaires that were otherwise complete. Based on analyses of the success of the third-grade data 

retrieval, critical items were determined only for the school administrator questionnaire and the Key 
4-60 

Domain child questionnaires for each class. The approach to retrieval for the school administrator 

questionnaire and child-level teacher questionnaires called for scanning and retrieval in the school. School 

coordinators, administrators, and teachers were informed in their letters that the supervisors would collect 

completed questionnaires on assessment day. No mailer for returning the questionnaires to the home 

office was provided to the schools. Respondents were asked to complete the instruments, seal them in 

envelopes, and give them to the school coordinator. Supervisors scanned the school administrator 

questionnaire and Key Child domain questionnaires on-site, and conducted retrieval as necessary. Exhibit 

4-19 presents the critical items by questionnaire. Each of the teacher questionnaire items proposed for 

retrieval were completed in the Key Child instruments only, since these are class-level items. 

Exhibit 4-19. Critical items by questionnaire type: School year 2003–04 

Instrument Item Question Number 

School administrator (SAQ) Grade level 

School type/control 

Race/ethnicity 

School lunch program 

eligibility/participation 

Question 4 

Questions 5, 6, 7 

Question 8 

Question 14 

Key Domain child-level reading 

teacher questionnaires 

Grades taught 

Race/ethnicity 

Sex 

Gifted/talented, LEP, Disability, Absent 

Question 15 

Question 16 

Question 17 

Question 18 

Key Domain child -level 

mathematics teacher 

questionnaires 

Grades taught 

Race/ethnicity 

Sex 

Gifted/talented, LEP, Disability, Absent 

Question 5 

Question 6 

Question 7 

Question 8 

Key Domain child -level science 

teacher questionnaires 

Grades taught 

Race/ethnicity 

Sex 

Gifted/talented, LEP, Disability, Absent 

Question 4 

Question 5 

Question 6 

Question 7 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

Data Retrieval Attempts. Field supervisors attempted to retrieve missing critical items and 

missing questionnaires in the schools in their assignments. They scheduled their retrieval efforts for the 

day the assessments were scheduled and attempted to find the respondents in person. Otherwise, they 

attempted these retrieval attempts by telephone. Field supervisors recorded any changes to missing critical 

items in blue pencil in the questionnaire. If the respondent did not know the answer, they recorded “DK” 

by the item; if the respondent refused to answer, they recorded “RF” by the item. Field supervisors 
4-61 

recorded the status of the questionnaire as one of the following: 

 Questionnaire complete with no missing critical items: no data retrieval required. 

 Questionnaire is missing one or more critical items: data retrieval required; one or 

more critical items collected. 

 Questionnaire is missing one or more critical items: data retrieval required; no critical 

items were collected. 

 Questionnaire refused: unit nonresponse. 

Table 4-19 presents the results of attempts to retrieve critical items. 

4.5.6 Incentives in the ECLS-K 

In order to gain respondent cooperation and ensure participation throughout the various data 

collection phases of the ECLS-K study, various incentives were offered. The type of incentive, monetary 

or nonmonetary, depended on whether the respondent was a sampled child, parent, teacher, or school. 

Exhibit 4-20 delineates the types of incentives used in the ECLS-K. 

Child Incentives. Children were given a small token at the end of the assessment to thank 

them for their cooperation in completing the assessment. In the spring-fifth grade, they were given a green 

lanyard with the ECLS-K sun log and study name. In addition, each month Westat mailed birthday cards 

to children whose birthdays fell within that month. Children were sent birthday cards throughout the 

calendar year, not just during the school year. By mailing these cards, children were not only thanked 

again for their participation, but parents were also reminded about the study. These periodic reminders are 

important in a longitudinal study, in which respondents may become apathetic toward the study during 

later rounds. Not only do these reminders encourage respondent participation, but they help the home 

office update addresses of families that have moved. 
4-62 

Table 4-19. Total number and percent of the final results of critical item retrieval: School year 2003–04 

Cases needing no data retrieval 

Data retrieval 

complete Data retrieval refused 

Questionnaire type 

Total 

expected1 Number Percent2 Number Percent2 Number Percent3 

School administrator 2,442 607 24.9 472 19.3 1,045 42.8 Reading teacher questionnaire 12,308 9,737 79.1 625 5.1 517 4.2 Mathematics teacher questionnaire 5,751 4,743 82.5 322 5.6 317 5.5 Science teacher questionnaire 5,811 4,867 83.8 273 4.7 305 5.2 

1 This column represents the number of questionnaires expected from school and teacher respondents based on schools and teachers linked to sampled children. 2 Percent based on number expected. 3 Percent based on number complete, needing data retrieval. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spri
4-63 

Exhibit 4-20. Types of incentives used in the ECLS-K: School year 2003–04 

Respondent Incentive 

Child  ECLS-K lanyard 

 Birthday card 

Parent  Newsletters 

 ECLS-K magnet with notepad 

Teacher  $20 mailed with questionnaire package 

 $7 for each completed child- level questionnaire 

School—original  $50 mailed with school administrator questionnaire package 

 $150 for cooperating schools 

 $7 for each completed student records abstract 

School—transfer  $50 mailed with school administrator questionnaire package 

 $7 for each completed student records abstract 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

Parent Incentives. In the fall of 2003, a newsletter about the study was published and 

mailed to parents. The newsletter served to update respondents on the initial findings from the first grade 

year and inform them about the future rounds of data collection. Parent involvement with homework, 

teacher views on homework and the average time spent on homework were a few of the topics discussed. 

Not only did the newsletter update parents on the findings of the study and highlight its importance, but it 

also was an incentive for future rounds of participation. Respondents were able to see the results of their 

participation in the study. 

Parents received an incentive for participating in the parent interview. At the end of the field 

period, thank-you letters were generated for every respondent who completed a parent interview. The 

letters, along with an ECLS-K magnet with attached notepad, were mailed to the respondents. Letters 

translated into Spanish were mailed to those respondents who completed the interview in Spanish. 

Teacher Incentives. In order to maximize response rates, all teachers were mailed a $20 

check with the questionnaire package. Teachers were also reimbursed $7 for each child for whom they 

completed a questionnaire, regardless of the number of questionnaires. Special education teachers were 

reimbursed in the same manner. 

School Incentives. Schools were also paid a monetary incentive for participating in the 

ECLS-K. Because school staff are often very busy and may not be aware of the benefits of cooperating, 

the cooperating original sample schools were remunerated $200 for participating, of this $50 was mailed 

with the school administrator questionnaire package and $150 was mailed once the school assessment 
4-64 

work was begun. Transfer schools received the $50 with the school administrator questionnaire package. 

School staff who completed the student records abstract received $7 for each questionnaire completed. 

All checks to schools, teachers, and school staff completing the SRAs were mailed weekly 

during the field period and were sent with thank-you letters. The checks to teachers and to staff 

completing the SRA were made out to the individual who completed the forms. 

4.6 Data Collection Quality Control 

The ECLS-K data are used by researchers to study children’s school experience and its 

relation to student outcomes, and by educators and policymakers to inform policy decisions. It is 

important that the information used by these groups is based on sound research practice and that 

considerable attention be paid to identifying potential sources of error, quantifying this error, and 

designing techniques to either reduce the error or minimize its impact on survey estimates. The work 

carried out in support of the ECLS-K includes a variety of activities that are directed toward ensuring that 

the data are of high quality. 

4.6.1 Maintaining Reliability on the Child Assessment 

To ensure that assessors maintained the standard that they achieved at training, assessors 

were observed by their supervisor in the field at two different points in time. The first observation was to 

be conducted by the end of March and the second observation by the end of April. The supervisor 

completed the Assessment Observation Form, which rated the assessor on key areas of the assessment 

protocol. In the Assessment Observation Form, the supervisor simultaneously coded with the assessor 

those open-ended assessment items that required judgment by the assessor to determine whether the 

child’s answer was correct. At the end of the assessment period, after the child was escorted from the 

room, the supervisor and the assessor reviewed the assessor’s overall performance. The two also 

compared the way that they each handled the open-ended questions. If there were large discrepancies, 

they reviewed the QxQs for these items carefully. 
4-65 

4.6.1.1 Assessment Observation Form 

The Assessment Observation Form had the names of the assessor and the supervisor, the 

case ID observed, the observation number, as well as the date the observation was conducted. The form 

had two sections: section 1 (shown in exhibit 4-21) was used by supervisors to rate the assessor on key 

overall skill areas, such as building rapport, using neutral praise, responding to behaviors presented by the 

child, pacing appropriately, and coaching. In section 1 the supervisor checked “No” for each skill area 

that the assessor did not demonstrate appropriately. 

Exhibit 4-21. Section 1 of the Assessment Certification Form: School year 2003–04 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

Section 2 listed specific questions from each routing and subdomain (e.g., mathematics) 

form. The instructions for completing section 2 are shown in exhibit 4-22. For each of the listed questions 

observed, the supervisor recorded both the child’s response and if the assessor did not demonstrate the 

specified required administration skills for that question. The required administration skills included 

reading questions verbatim, using appropriate probes, and using appropriate hand motions. For each 

question in which the supervisor observed that the assessor did not demonstrate the required 

administration skill(s), he or she checked a box, indicating which skill was not performed. 
4-66 

Exhibit 4-22. Instructions for Section 2 of the Assessment Observation Form: School year 2003–02 

SECTION 2: Specific Assessment Activities 

Supervisor/Evaluator: Code the items as the assessor administers the assessment. 

Code the child’s response as the item is administered. 

If the item requires probing, check the box if the assessor does not use the appropriate probe. 

Check the box in the “Verbatim” column if the assessor does not read the item exactly as worded on the 

screen. 

Check the box in the “Gesturing” column if the assessor does not use appropriate hand motions. 

For each validation code item, check the box in the “Validation” column if the response coded by the 

assessor is not what you have coded. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

As described earlier, during the observed assessment, the supervisor coded the child’s 

response to those open-ended assessment items that required judgment on the part of the assessor. These 

open-ended items were flagged in the CAI program for quality control review, and a screen showing how 

the assessor coded each of the questions was accessed at the end of the assessment for review with the 

supervisor. The supervisor and assessor compared their codes to these open-ended questions. For each 

item where the coding did not match, the supervisor recorded this discrepancy by checking the 

corresponding box for that question in the Assessment Observation Form. 

Exhibit 4-23 shows an example of a question from the Assessment Observation Form, which 

required the supervisor to evaluate the required assessment administration skills: reading verbatim, using 

appropriate gesturing, using appropriate probe, and accurately coding the child’s response to this openended question. The supervisor was instructed to check the boxes only if the assessor did not demonstrate 

the correct behavior. For all questions that are observed by the supervisor and listed in the Assessment 

Observation Form, the supervisor recorded if the assessor did not read verbatim or gesture appropriately. 

Supervisors only recorded if assessors did not use the appropriate probe for those questions that display 

probes (a box labeled “Assessor Used Inappropriate Probe?” was presented in the Assessment 

Observation Form). Also, the supervisors only recorded if their response coding did not match the 

assessors’ for those questions that are open-ended (a box labeled “Validation Code Did Not Match” was 

presented in the Assessment Observation Form). 
4-67 

Exhibit 4-23. Sample question from Assessment Certification Form: School year 2003–04 

 DID NOT READ 

QUESTION 

VERBATIM 

GESTURED 

INAPPROPRIATELY 

VALIDATION 

CODE DID 

NOT MATCH

R5.245 What do you think is the difference between the 

storyteller’s bedroom and the sister’s bedroom?** 

 IF STUDENT SAYS, “ A mess” OR “A lot of junk”, 

PROMPT: “How is it different?” 

 

CORRECT .................................................... 1

STORY TELLER’S BEDROOM IS: 

(BIGGER MESS/MORE 

MESS/MESSIER) 

(MORE JUNK/JUNKIER) 

(IS A MORE DISGUSTING 

SPOT) 

CORRECT ANSWER MUST 

INVOLVE A COMPARISON 

INCORRECT................................................. 2

(STORY TELLER’S BEDROOM IS 

CLEAN/AS MESSY AS SISTER’S BEDR

DETAILS ABOUT THE SISTER’S 

BEDROOM 

 EXAMPLES: 

(PIZZA UNDER THE DESK) 

(SOCKS ON THE LIGHT) 

(MESS/ALSO A MESS) 

(JUNK/ A LOT OF JUNK) 

REFUSED ................................................. 777

DON’T KNOW........................................... 999

ASSESSOR USED INAPPROPRIATE PROBE? SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

At the end of the child assessment, after the child was escorted from the room, the supervisor 

and the assessor reviewed the assessor’s overall performance. After discussing the ratings on section 1, 

the assessor accessed the quality control screen. The assessor and supervisor reviewed their codes for 

each open-ended question asked in section 2 of the Assessment Observation Form. 

The supervisor then scored the assessment observation using the scoring form shown in 

exhibit 4-24. In the first part of the form (Form A), the supervisor counted the number of check marks 

recorded in section 1: Rapport Building and recorded that number in the appropriate row. The supervisor 

then recorded the number of check marks for each section of the assessment that was observed and 

entered those numbers in the appropriate box. 
4-68 

Exhibit 4-24. Observation Scoring Form: School year 2003–04 

Observation Scoring 

Step 1: Record the number of check marks from Section 1: Rapport Building and Section 2: Specific 

Assessment Activities in the appropriate boxes of Form A. 

Step 2: Sum each row and record total in Row Totals column of Form A. 

Step 3: Sum Row Totals column (including number of check marks from Section 1) and record in Total 

check marks box of Form A. 

Form A: Checks Marks Recorded in Sections 1 and 2

Assessment Routing Form Red Form Yellow Form Blue Form Row Totals 

Reading 

Mathematics 

Science 

Number of check marks from Section 1: Rapport Building: 

Total Check Marks: 

Step 4: Using Form B, circle the number in the form used for each assessment. For example, if the Red 

Form was used for reading, circle the number 6 in the Routing Row and the number 20 in the 

Reading row under Red Form. Record each circled number in the Row Totals column of Form 

B. 

Step 5: Sum Row Totals column (including Possible Points for Section 1) and record in Total Possible 

Points box of Form B.

Form B: Total Possible Points

Assessment Routing Form Red Form Yellow Form Blue Form Row Totals 

Reading 6 20 21 20 

Mathematics 8 11 18 23 

Science 11 14 15 18 

Possible Points for Section 1: Rapport Building: 6 

Total Possible Points: 

Step 6: Use Proportion Correct Chart to determine the proportion correct and write that proportion in 

this box: 

Result: If Proportion correct is greater than or equal to .85, Assessor is Certified. 

If Proportion correct is between .70 and .85, Assessor requires Remedial Action (Complete 

Remedial Task Chart below) 

 If Proportion correct is less than .70, Assessor Failed. 

See note at end of exhibit. 
4-69 

Exhibit 4-24. Observation Scoring Form: School year 2003–04—Continued 

Remedial Task Chart

Circle the number of the task(s) needing improvement. Discuss remedial action with the assessor. 

TASK # REMEDIAL ACTION 

APPROPRIATE PACE 1 PRACTICE/REVIEW JOB AIDS 

AVOID COACHING 2 REVIEW THE JOB AIDS 

READ VERBATIM 3 PRACTICE READING FROM THE SCREEN 

GESTURING APPROPRIATELY 4 PRACTICE GESTURING 

USE APPROPRIATE PROBES 5 REVIEW THE JOB AIDS/Q-BY-Qs 

VALIDATION CODE DID NOT MATCH 6 REVIEW THE Q-BY-Qs 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

Exhibit 4-25 shows an example of a completed Form A. In this example, the supervisor 

recorded one (1) check mark for section 1, zero (0) check marks for Reading Routing, zero (0) check 

marks for Reading Blue, one (1) check mark for Math Routing, zero (0) check marks for Math Yellow, 

zero (0) check marks for Science Routing, and one (1) check mark for Science Red. The total check 

marks recorded was three (3). 

Exhibit 4-25. Example of Form A: School year 2003–04 

Form A: Checks Marks Recorded in Sections 1 and 2 

Assessment Routing Form Red Form Yellow Form Blue Form Row Totals 

Reading 0 0 0 

Mathematics 1 0 1 

Science 0 1 1 

Number of check marks from section 1: Rapport Building: 1 

Total Check Marks: 3 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 

Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 

The supervisor then calculated the number of possible points for the sections of the 

assessment observed using the second section of the scoring form (Form B) (exhibit 4-26). Continuing 

with the example started earlier, the supervisor observed all of Reading Blue (20 possible points), Math 

Yellow (18 possible points), and Science Red (14 possible points). In addition, section 1 and all the 

Routing Forms contribute 31 possible points. Based on the sections of the assessment that the supervisor 

observed the total number of possible points for this assessor was 83. 
4-70 

Exhibit 4-26. Example of Form B: School year 2003–04 

Form B: Total Possible Points

Assessment Routing Form Red Form Yellow Form Blue Form Row Totals 

Reading 6 20 21 20 

Mathematics 8 11 18 23 

Science 11 14 15 18 

Possible Points for Section 1: Rapport Building: 6 

Total Possible Points: 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 

Class of 1998–99 (ECLS-K), spring 2004. 

Finally, the supervisor determined the proportion correct by using the Proportion Correct 

Chart displayed in exhibit 4-27. Continuing the example, the supervisor found the column on the 

Proportion Correct Chart displaying the total possible points for the assessor based on the portions of the 

assessment observed (74) and the row on the Proportion Correct Chart displaying the total check marks 

the supervisor recorded for the trainee (3), and recorded the proportion from the corresponding box on the 

Proportion Correct Chart in the box under step 6 (exhibit 4-28) on the scoring form (.96). 

Once the supervisor completed the scoring, the assessor was rated as Passed, Remedial 

Action, or Failed. 

The field supervisors recorded their observations on the form and then reviewed the form 

with the assessor. The most frequent problems observed were not reading the items verbatim and 

inappropriate gesturing. Feedback was provided to the assessors on the strengths and weaknesses of their 

performance and, when necessary, remedial training was provided in areas of weakness. Table 4-20 

presents the result of the observations. As table 4-20 shows, all assessors received scores of 85 percent or 

better while being observed. 
4-71 

Exhibit 4-27. Proportion Correct Chart: School year 2003–04 

69 70 72 73 74 76 77 78 79 80 81 82 83 84 85 86 87 88 

1 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 2 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.97 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 3 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.96 0.97 0.97 0.97 4 0.94 0.94 0.94 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 0.95 5 0.93 0.93 0.93 0.93 0.93 0.93 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 0.94 6 0.91 0.91 0.92 0.92 0.92 0.92 0.92 0.92 0.92 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 0.93 7 0.90 0.90 0.90 0.90 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.92 0.92 0.92 0.92 0.92 0.92 8 0.88 0.89 0.89 0.89 0.89 0.89 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.90 0.91 0.91 0.91 0.91 9 0.87 0.87 0.88 0.88 0.88 0.88 0.88 0.88 0.89 0.89 0.89 0.89 0.89 0.89 0.89 0.90 0.90 0.90 

10 0.86 0.86 0.86 0.86 0.86 0.87 0.87 0.87 0.87 0.88 0.88 0.88 0.88 0.88 0.88 0.88 0.89 0.89 11 0.84 0.84 0.85 0.85 0.85 0.86 0.86 0.86 0.86 0.86 0.86 0.87 0.87 0.87 0.87 0.87 0.87 0.88 12 0.83 0.83 0.83 0.84 0.84 0.84 0.84 0.85 0.85 0.85 0.85 0.85 0.86 0.86 0.86 0.86 0.86 0.86 13 0.81 0.81 0.82 0.82 0.82 0.83 0.83 0.83 0.84 0.84 0.84 0.84 0.84 0.85 0.85 0.85 0.85 0.85 14 0.80 0.80 0.81 0.81 0.81 0.82 0.82 0.82 0.82 0.83 0.83 0.83 0.83 0.83 0.84 0.84 0.84 0.84 15 0.78 0.79 0.79 0.79 0.80 0.80 0.81 0.81 0.81 0.81 0.81 0.82 0.82 0.82 0.82 0.83 0.83 0.83 16 0.77 0.77 0.78 0.78 0.78 0.79 0.79 0.79 0.80 0.80 0.80 0.80 0.81 0.81 0.81 0.81 0.82 0.82 17 0.75 0.76 0.76 0.77 0.77 0.78 0.78 0.78 0.78 0.79 0.79 0.79 0.80 0.80 0.80 0.80 0.80 0.81 18 0.74 0.74 0.75 0.75 0.76 0.76 0.77 0.77 0.77 0.78 0.78 0.78 0.78 0.79 0.79 0.79 0.79 0.80 19 0.72 0.73 0.74 0.74 0.74 0.75 0.75 0.76 0.76 0.76 0.77 0.77 0.77 0.77 0.78 0.78 0.78 0.78 20 0.71 0.71 0.72 0.73 0.73 0.74 0.74 0.74 0.75 0.75 0.75 0.76 0.76 0.76 0.76 0.77 0.77 0.77 21 0.70 0.70 0.71 0.71 0.72 0.72 0.73 0.73 0.73 0.74 0.74 0.74 0.75 0.75 0.75 0.76 0.76 0.76 22 0.68 0.69 0.69 0.70 0.70 0.71 0.71 0.72 0.72 0.73 0.73 0.73 0.73 0.74 0.74 0.74 0.75 0.75 23 0.67 0.67 0.68 0.68 0.69 0.70 0.70 0.71 0.71 0.71 0.72 0.72 0.72 0.73 0.73 0.73 0.74 0.74 24 0.65 0.66 0.67 0.67 0.68 0.68 0.69 0.69 0.70 0.70 0.70 0.71 0.71 0.71 0.72 0.72 0.72 0.73 25 0.64 0.64 0.65 0.66 0.66 0.67 0.68 0.68 0.68 0.69 0.69 0.70 0.70 0.70 0.71 0.71 0.71 0.72 26 0.62 0.63 0.64 0.64 0.65 0.66 0.66 0.67 0.67 0.68 0.68 0.68 0.69 0.69 0.69 0.70 0.70 0.70 27 0.61 0.61 0.63 0.63 0.64 0.64 0.65 0.65 0.66 0.66 0.67 0.67 0.67 0.68 0.68 0.69 0.69 0.69 28 0.59 0.60 0.61 0.62 0.62 0.63 0.64 0.64 0.65 0.65 0.65 0.66 0.66 0.67 0.67 0.67 0.68 0.68 29 0.58 0.59 0.60 0.60 0.61 0.62 0.62 0.63 0.63 0.64 0.64 0.65 0.65 0.65 0.66 0.66 0.67 0.67 30 0.57 0.57 0.58 0.59 0.59 0.61 0.61 0.62 0.62 0.63 0.63 0.63 0.64 0.64 0.65 0.65 0.66 0.66 

NOTE: Proportion Correct = (Total Possible Points – Number of Check Marks)/Total Possible Points EXAMPLE: If an Assessor received 8 check marks out of a total of 86 possible points, the proportion correct is 0.91. This Assessor passed. SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spring 200
4-72 

Exhibit 4-28. Example of Proportion Correct Chart: School year 2003–04 

Step 6: Use Proportion Correct Chart to determine the proportion correct and write that proportion in 

this box: .96 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

Table 4-20. Results of the child assessment observations, fifth-grade data collection: School year 

2003–04 

Number of observations1

 Score on certification form

Total: 466 

465 85 percent or above

1 70-84 percent

0 Below 70 percent 1 Two hundred and forty-two assessors were to be observed after training; assessors completing only parent interviews were not observed. 12 

assessors had only one observation; 227 assessors had two observations. Only 1 assessor failed to pass the first observation and was released from 

the project by mutual consent. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

4.6.1.2 Assessor Interrater Reliability 

As part of the child assessment observations described in section 4.6.1, field supervisors 

completed an assessment certification form for each observation they conducted. An important element of 

this form was the “validation items.” With the exception of the reading routing test, all of the assessments 

included at least one item that both the observer and the assessor scored. These items had open-ended 

responses that called for interpretation on the part of the assessor to determine whether a child’s response 

was correct. By comparing the extent to which assessors and observers agreed on scoring these validation 

items, a measure of interrater reliability was obtained. Interrater reliability provided a measure of the 

accuracy of the assessor’s scoring compared with the standard, the observer’s. 

Table 4-21 contains the results of these comparisons. As can be seen, overall interrater 

reliability was very high throughout all the forms. It was highest for mathematics (98 percent or better 

depending upon the form) and lowest for reading, with the reading yellow level (the medium reading 

level) showing the lowest percent agreement (95.7 percent). The reading yellow level path received a 

.98 
4-73 

relatively large number of observations (232) and also contained a relatively large number of validation 

items (5) compared with some of the other paths. Thus, there was greater opportunity for disagreement on 

this path compared with the others. The science blue level (the high science level) also had a relatively 

higher opportunity for disagreement (212 observations and 4 validation codes) and it, too, exhibited a 

somewhat lower interrater reliability (96.7 percent) compared with some of the other paths. The 

reliability, however, even on these more difficult paths, was high and demonstrated that the assessors 

accurately coded open-ended items. 

Table 4-21. Interrater reliability on child assessment validation items: School year 2003–04 

Category 

Number of

Observations

Number of

Validation items

Percent agreement:

Assessors and observers1

 

Reading 458 13 96.5

 Routing † 0 †

 Red 103 3 97.1

 Yellow 232 5 95.7

 Blue 123 5 97.4

 

Mathematics 461 10 99

 Routing 461 2 99.7

 Red 162 1 100

 Yellow 161 3 98.8

 Blue 138 4 98.4

 

Science 463 14 96.8

 Routing 460 3 99.1

 Red 53 4 98.1

 Yellow 198 3 96.6

 Blue 212 4 96.7

† Not applicable. 1 Percent agreement was calculated as follows: number of validation items observed in which observer agreed with the assessor divided by 

number of validation items observed. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 
4-74 

4.6.2 Validation of Parent Interviews 

Approximately 10 percent of the respondents who completed parent interviews were called 

back by a field supervisor (i.e., validated). The first parent interview completed by an assessor was always 

validated. Over the course of the field period, a running count of an assessor’s completed parent 

interviews was maintained, and each tenth completed parent interview was selected for validation, thus 

ensuring that 10 percent of each assessor’s cases were selected for validation. The parent validation was 

approximately 5 minutes long and was conducted by telephone. No in-person cases were validated. 

Field supervisors used a standardized parent validation script, the Parent Interview 

Validation Form, to make validation calls to parents. The validation script included verification of the 

child’s name, date of birth, and sex, and 7 questions from the parent interview. Parent Validation 

Interview Forms were generated throughout the field period. By the end of June, nearly 10 percent 

(1,023) of the total completed parent interviews had been validated. Validation results were as follows: 

(1) “No changes” meaning responses to the original interview and the validation interview were identical; 

(2) “Minor changes” meaning there was a minor discrepancy (e.g., the ZIP code was different) between 

the responses to the original interview and the validation interview; and (3) “Major changes” meaning 

there was a discrepancy between the response to the original interview and the response in the validation 

interview (e.g., bed times varied between the two contacts). Table 4-22 presents the results of the parent 

validations. 

4.6.3 Validations of School Visits 

To ensure that assessments proceeded smoothly, a validation call was completed with the 

school principal in at least two of each supervisor’s assigned original schools in the spring-fifth grade data 

collection. Field managers conducted the school validations. No validation calls were made to transfer 

schools because the impact of the survey was so minimal due to most having only one child enrolled. 
4-75 

Table 4-22. Results of parent interview validations: School year 2003–04 

March April May Ju

Parent validation status Number Percent Number Percent Number Percent Number  Completed parent interviews 5,218 100.0 8,468 100.1 10,204 100.0 10,930 Validation cases generated 603 11.6 931 11.0 1,140 11.2 1,213  Validation cases receipted 258 42.8 632 67.9 864 75.8 1,028  No changes 228 88.4 586 92.7 793 91.8 943  Minor changes 8 3.1 14 2.2 22 2.5 24  Major changes 22 8.5 32 5.1 49 5.7 61  Other (specify) 0 0.0 0 0.0 0 0.0 0  Cases pending 345 57.2 299 32.1 276 24.2 185 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spri
4-76 

The first original school that each team completed was called to ascertain how well the 

preassessment and assessment activities went. If the feedback from the school was positive, the fifth 

school that each team completed was called. If any problems were indicated in the first validation call, 

immediate action was taken with the field supervisor. The validation feedback was discussed with the 

supervisor and remedial action was taken, including telephone training on areas needing improvement or 

in-person observation of the supervisor’s next school, if necessary; no remedial action was warranted 

based on the school validation calls. 

Field managers used a standardized telephone script, the School Validation Script, to call the 

school principals. The script covered the following topics: 

 An overall rating of how the assessments went; 

 Feedback the principal had received about the study from children and teachers; 

 Suggestions for improving procedures and making it easier for a school to participate; 

and 

 General comments and suggestions. 

Field managers called schools throughout the field period as scheduled assessments were 

completed. By the end of the field period in June, field managers had called 162 original sample schools 

(100 percent of the schools to be validated) to ascertain whether the conduct of the child assessments in 

the school was satisfactory; one school administrator could not be reached before the end of the school 

year. Table 4-23 presents the results of the school validation calls. 

Table 4-23. Results of school validation calls: School year 2003–04 

March April May June 

Number Percent Number Percent Number Percent Number Percent 

 Total validation cases generated 162 100.0 162 100.0 162 100.0 162 100.0 

Validation cases completed 28 17.3 109 67.3 156 96.3 162 100.0 

 Overall rating 

 Very satisfactory 28 100.0 106 97.2 146 93.6 152 93.8

 Satisfactory 2 7.1 3 2.8 10 6.4 10 6.2

 Unsatisfactory 0 0.0 0 0.0 0 0.0 0 0.0

 Very unsatisfactory 0 0.0 0 0.0 0 0.0 0 0.0 

 Validation cases not completed 134 82.7 53 32.7 6 3.7 1 0.6

NOTE: Detail may not sum to totals due to rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 
5-1 

5. DATA PREPARATION AND EDITING 

As described in chapter 4, two types of data collection instruments were used for the Early 

Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K) data collection in the springfifth grade: computer-assisted instruments and self-administered paper forms (hard copy). The data 

preparation approach differed with the mode of data collection. The direct child assessments and parent 

interview were conducted using computer-assisted interviewing (CAI) techniques. Editing specifications 

were built into the computer programs used by assessors to collect these data. The teacher, school 

administrator, and student records abstract forms were self-administered. When forms and questionnaires 

were received at Westat, coders reviewed them to ensure data readability for transfer into an electronic 

format and for the completion of critical items. The visual review included changing (upcoding) any 

“Other, specify” responses that actually fit within the available response categories of the question. For 

example, if the parent said “skiing” in response to the question on the types of exercise or physical 

activity the child participated in, this answer was upcoded to the existing category, “individual sports.” 

There were some items for which upcoding was conducted after the data were keyed due to the large 

volume of “Other” responses. Once they finished this review, the coders sent the instruments to data entry 

to be manually transferred to an electronic format and reviewed for range and logic consistency. The 

following sections describe the data preparation activities for both modes of data collection in more detail. 

5.1 Coding and Editing Specifications for Computer-Assisted Interviews (CAI) 

The very nature of designing a computer-assisted interview requires decisions about edit 

specifications to be made at the development stage. Both acceptable ranges and logical consistency 

checks were preprogrammed into the electronic questionnaires. The next few sections describe the coding 

and editing of the data that were conducted during and after the CAI parent interview. 

5.1.1 Range Specifications 

Within the CAI parent interview instruments, respondent answers were subjected to both 

“hard” and “soft” range edits during the interviewing process. A “soft range” is one that represents the 

reasonable expected range of values but does not include all possible values. Responses outside the soft 
5-2 

range were confirmed with the respondent and entered a second time. For example, the number of hours 

each week a child attended a day care center on a regular basis had a soft range of 1 to 50. A value outside 

this range could be entered and confirmed as correct by the interviewer as long as it was within the hard 

range of values (1 to 70). 

“Hard ranges” have a finite set of parameters for the values that can be entered into the 

computer, for example, “0–5 times” for the number of times the child, in the previous 5 days, ate a 

breakfast that was not provided by the school. Out-of-range values for closed-ended questions were not 

accepted. If the respondent insisted that a response outside the hard range was correct, the interviewer 

could enter the response in a comments data file. Data preparation and project staff reviewed these 

comments. Out-of-range values were accepted and entered into the data file if the comments supported 

the response. 

The child assessments did not employ hard and soft ranges. Children’s answers were 

recorded verbatim. 

5.1.2 Logical Consistency Checks (Logical Edits) 

Logical consistency checks, or logical edits, examined the relationship between responses to 

ensure that they did not conflict with one another or that the response to one item did not make the 

response to another item unlikely. For example, in the household roster, one could not be recorded as a 

mother and male. A male mother or female father (or any other gender/relationship discrepancy) would 

cause the case to fail the parent edits and the case would be examined by hand. We certainly identified 

some same-sex partnership households this way, usually by looking at the combination of relationship, 

sex, and first name. Such households would be left as is. Only cases that were deemed in error were 

corrected. When a logical error such as this occurred during an interviewing session, the interviewer was 

presented with a message requesting verification of the last response and a resolution of the discrepancy. 

In some instances, if the verified response still resulted in a logical error, the interviewer recorded the 

problem either in a comment field or on a problem report. Consistency checks were not applicable to the 

child assessments. 
5-3 

5.1.3 Coding 

Additional coding was required for some of the items collected in the CAI instruments by 

data preparation and project staff after an interview was completed. These items included “Other, specify” 

text responses, occupation, and race/ethnicity. Interviewers entered verbatim responses to these items. 

Data preparation staff were trained to code these data using coding manuals designed by Westat and the 

National Center for Education Statistics (NCES) to support the coding process. This section describes the 

coding activities for the CAI instruments. 

Review of “Other, Specify” Items. There were twenty-five “Other, specify” open-ended 

responses in the parent interview. All of these items were reviewed to determine if they should be coded 

into one of the existing response categories. During data collection, when a respondent selected an 

“other” response in the parent interview, the interviewer entered the text into a “specify” overlay that 

appeared on the screen. The data preparation staff reviewed these text “specify” responses and, where 

appropriate, coded them into one of the existing response categories. If a response did not fit into one of 

the existing categories, it remained in “other.” If there were numerous responses that were essentially the 

same, approximately one hundred, then a new code was added to the item. Three new codes were added 

to parent interview items during fifth-grade data processing as presented in table 5-1. 

Table 5-1. “Other, specify” codes added during fifth grade: School year 2003–04 

Item 

“Other, specify” item 

descriptions New code 

EMQ.070 Job Find Effort Read Want Ads/Internet 

CHQ.365 Emotional Diagnosis No Problem 

CHQ.337 Behavior Diagnosis No Problem 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood 

Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 

The parent “Other, specify” coding system was revised from previous rounds of data 

collection and in production testing in April 2004. The revisions consisted of adding new “Other, specify” 

items that had not been part of the previous rounds; six items in the parent interview included an “Other, 

specify” code and were added to the system (see table 5-2 for new items). A total of 3,601 “Other, 

specify” text strings were processed through the parent “Other, specify” coding system. All possible 

upcodes were applied to the 10,901 cases that had at least one “Other, specify” text string. As noted 

above, whenever appropriate, responses were upcoded to existing categories. There were no “Other, 
5-4 

specify” items in the child assessments. Table 5-3 presents the number of text strings for each “Other, 

specify” item including the new ones added in fifth grade. 

Table 5-2. “Other, specify” items added to the coding system: School year 2003–04 

New item Description of “Other, specify” item description 

COQ.005 Country of biological mother’s birth 

COQ.020 Country of biological father’s birth 

CHQ.547 Reason child no longer participates in special education services 

CHQ.751 Other medication name 

CHQ.781 Main reason for receiving family therapy 

CHQ.791 Type of other person family saw 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 

Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 

Table 5-3. Number of text strings by “Other, specify” item, fifth-grade parent interview: 

School year 2003–04 

Item “Other, specify” item description Number of strings 

Total 3,601 

CHQ.337 Behavior Diagnosis 68 

COQ020 Birth Place DAD 13 

COQ005 Birth Place MOM 13 

NRQ.261 Child Support Agreement 82 

CCQ.393 Center Pay Unit 10 

EMQ.080 Doing Last Week 261 

CHQ.365 Emotional Diagnosis 160 

CHQ.726 Exercise Types 616 

FSQ240 FSQ Birth Place 4 

CMQ.690 Interview Language 39 

EMQ.070 Job Find Effort 124 

CHQ.060 Mental Diagnosis 159 

FSQ.181 Nonrelative 129 

CCQ.225 Nonrelative Pay Unit 1 

WPQ.130 Other, specify Stamps 7 

WPQ.106 Other, specify TANF 7 

CHQ750 Prescription Medicine 80 

FSQ.198 Race 131 

CHQ 781 Main Reason for Receiving Family Therapy 41 

FSQ.015 Reason Left Household 530 

CHQ547 Reason No Service 117 

CCQ.128 Relative Pay Unit 3 

CHQ791 Type of Therapist Seen 11 

CHQ.120 What Act Diagnosis 54 

PIQ.015 Why Call School 941 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal 

Study, Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 
5-5 

Parent Occupation Coding. As in previous data collections, occupations were coded using 

the Manual for Coding Industries and Occupations (U.S. Department of Education 1999). This coding 

manual was created for the Adult Education Survey of the National Household Education Surveys 

Program (NHES) and used an aggregated version of industry and occupation codes. The industry and 

occupation codes used by NHES were originally developed for the 1989–90 National Postsecondary 

Student Aid Study (NPSAS) and contained one to four digits. Analysis of the NPSAS categories revealed 

that some categories had very small numbers of cases and similar categories had similar participation 

rates, suggesting that the separate codes could be collapsed without significant loss of information. The 

NHES industry and occupation code categories use a two-digit code, the highest level of aggregation, to 

have sufficient numbers of cases to support analysis without collapsing categories. There are 13 industry 

codes and 22 occupation codes in the NHES coding scheme. If an industry or occupation could not be 

coded using this manual, the Index of Industries and Occupations—1980 (U.S. Department of Commerce 

1982) and Standard Occupational Classification Manual—1980 (U.S. Department of Commerce 1980) 

were used. Both of these manuals use an expanded coding system and at the same time are directly related 

to the much more condensed NHES coding scheme. These manuals were used for reference in cases 

where the NHES manual did not adequately cover a particular situation. Exhibit 5-1 describes the 

aggregated categories that were used for coding occupation in the ECLS-K. 

Exhibit 5-1. Aggregated occupation coding categories in the ECLS-K: School years 1998–99, 2001–02, 

and 2003–04 

1. Executive, 

Administrative, and 

Managerial 

Occupations 

This category includes senior-level and middle management occupations 

and occupations that directly support management. Senior-level managers 

are persons concerned with policymaking, planning, staffing, directing, 

and/or controlling activities. Middle managers include persons who plan, 

organize, or direct and/or control activities at the operational level. 

Workers in this category are not directly concerned with the fabrication of 

products or with the provision of services. Other officials and 

administrators include consultants, library directors, custom house builders, 

and location managers. Legislators are also included in this category. 

2. Engineers, Surveyors, 

and Architects 

The category includes occupations concerned with applying principles of 

architecture and engineering in the design and construction of buildings, 

equipment and processing systems, highways and roads, and land utilization. 

3. Natural Scientists and 

Mathematicians 

This category includes those engaged primarily in the application of 

scientific principles to research and development. Natural scientists are 

those in the physical sciences (e.g., chemistry, physics) and the life 

sciences (e.g., biology, agriculture, medicine). In addition, this category 

includes those in computer science, mathematics (including statistics), and 

operations research. 

See note at end of exhibit. 
5-6 

Exhibit 5-1. Aggregated occupation coding categories in the ECLS-K: School years 1998–99, 2001–02, 

and 2003–04—Continued 

4. Social Scientists, 

Social Workers, 

Religious Workers, 

and Lawyers 

This category includes occupations concerned with the social needs of 

people and in basic and applied research in the social sciences. 

5. Teachers: College, 

University, and Other 

Postsecondary 

Institution; Counselors, 

Librarians, and 

Archivists 

This category includes those who teach at higher education institutions and 

at other postsecondary (after high school) institutions, such as vocational 

institutes. In addition, vocational and educational counselors, librarians, 

and archivists are included here. 

6. Teachers, except 

Postsecondary 

Institution 

This category includes prekindergarten and kindergarten teachers, 

elementary and secondary teachers, special education teachers, 

instructional coordinators, and adult education teachers (outside 

postsecondary). 

7. Physicians, Dentists, 

and Veterinarians 

This category includes health care professionals who diagnose and treat 

patients. In addition to physicians, dentists, and veterinarians, this category 

includes optometrists, podiatrists, and other diagnosing and treating 

professionals, such as optometrists, podiatrists, chiropractors, 

hypnotherapists, and acupuncturists. 

8. Registered Nurses, 

Pharmacists, 

Dieticians, Therapists, 

and Physician’s 

Assistants 

This category includes occupations concerned with the maintenance of 

health, the prevention of illness and the care of the ill through the provision 

and supervision of nursing care; compounding drugs, planning food service 

or nutritional programs; providing assistance to physicians; and the 

provision of therapy and treatment as directed by physicians 

9. Writers, Artists, 

Entertainers, and 

Athletes 

This category includes occupations concerned with creating and executing 

artistic works in a personally interpreted manner by painting, sculpturing, 

drawing, engraving, etching, and other methods; creating designs for 

products and interior decorations; designing and illustrating books, 

magazines, and other publications; writing; still, motion picture and 

television photography/filming; producing, directing, staging, acting, 

dancing, singing in entertainment; and participating in sports and athletics 

as a competitor or player and administering and directing athletic 

programs. 

10. Health Technologists 

and Technicians 

This category includes occupations concerned with providing technical 

assistance in the provision of health care. For example, clinical laboratory 

technologists and technicians, dental hygienists, radiologic technicians, 

licensed practical nurses (LPNs), and other health technologists are 

included here 

11. Technologists and 

Technicians, except 

Health 

This category includes those providing technical assistance in engineering 

and scientific research, development, testing, and related activities, as well 

as operating and programming technical equipment and systems. 

12. Marketing and Sales 

Occupations 

This category includes occupations involving selling goods or services, 

purchasing commodities and property for resale, and conducting wholesale 

or retail business. 

See note at end of exhibit. 
5-7 

Exhibit 5-1. Aggregated occupation coding categories in the ECLS-K: School years 1998–99, 2001–02, 

and 2003–04—Continued 

13. Administrative 

Support Occupations, 

including Clerks 

This category includes occupations involving preparing, transcribing, 

transferring, systematizing, and preserving written communications and 

records; collecting accounts; gathering and distributing information; 

operating office machines and data processing equipment; operating 

switchboards; distributing mail and messages; and other support and 

clerical duties such as bank teller, data entry keyer, etc 

14. Service Occupations The category includes occupations providing personal and protective 

services to individuals, and current maintenance and cleaning for building 

and residences. Some examples include food service, health service (e.g., 

aides or assistants), cleaning services other than household, and personal 

services. 

15. Agricultural, Forestry, 

and Fishing 

Occupations 

This category is concerned with the production, propagation 

(breeding/growing), gathering, and catching of animals, animal products, 

and plant products (timber, crop, and ornamental); the provision of services 

associated with agricultural production; and game farms, fisheries, and 

wildlife conservation. “Other agricultural and related occupations” include 

occupations concerned with the production and propagation of animals, 

animal products, plants, and products (crops and ornamental). 

16. Mechanics and 

Repairers 

Mechanics and repairers are persons who do adjustment, maintenance, part 

replacement, and repair of tools, equipment, and machines. Installation 

may be included if installation is usually done in conjunction with other 

duties of the repairers. 

17. Construction and 

Extractive 

Occupations 

This category includes occupations that normally are performed at a 

specific site, which will change over time, in contrast to production 

workers, where the work is usually at a fixed location. Construction 

workers include those in overall construction, brickmasons, stonemasons, 

carpenters, electricians, drywall installers, paperhangers and painters, etc. 

Extractive occupations include oil well drillers, mining machine operators, 

and so on. 

18. Precision Production 

Occupations 

Precision production includes occupations concerned with performing 

production tasks that require a high degree of precision or attainment of 

rigid specification and operating plants or large systems. Examples are tool 

and die makers, pattern and model makers, machinists, jewelers, engravers, 

and so on. Also included are some food-related occupations including 

butchers and bakers. Plant and system operators include water and sewage, 

gas, power, chemical, petroleum, and other plant or system operators. 

19. Production Working 

Occupations 

This category includes occupations concerned with setting up, operating, 

and tending of machines and hand production work usually in a factory or 

other fixed place of business. 

20. Transportation and 

Material Moving 

Occupations 

This category includes occupations concerned with operating and 

controlling equipment used to facilitate the movement of people or 

materials and the supervising of those workers. 

See note at end of exhibit. 
5-8 

Exhibit 5-1. Aggregated occupation coding categories in the ECLS-K: School years 1998–99, 2001–02, 

and 2003–04—Continued 

21. Handlers, Equipment 

Cleaners, Helpers, and 

Laborers 

This category includes occupations that involve helping other workers and 

performing routine nonmachine tasks. A wide variety of helpers, handlers, 

etc., are included in this category. Examples include construction laborers, 

freight, stock, and material movers, garage and service station related 

occupations, parking lot attendants, and vehicles washers and equipment 

cleaners. 

22. Unemployed, Retired, 

Disabled, or 

Unclassified Workers 

This category includes persons who are unemployed, have retired from the 

work force, or are disabled. It also includes unclassified occupations that 

do not fit in the categories above (e.g., occupations that are strictly 

military, such as “tank crew member” and “infantryman”). 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

Occupation coding began with an autocoding procedure using a computer string match 

program developed for the NHES. The program searched the responses for strings of text for each 

record/case and assigned an appropriate code. A little over a third of the cases were autocoded (39 

percent). Cases that could not be coded using the autocoding system were coded manually using a 

customized coding utility program designed for coding occupations. The customized coding utility 

program brought up each case for coders to assign the most appropriate codes. In addition to the text 

strings, other information, such as main duties, highest level of education, and name of the employer, was 

available for the coders. The coders used this information to ensure that the occupation code assigned to 

each case was appropriate. 

After the cases were coded (either manually or via autocoding), they were reviewed and 

verified. One hundred percent of the cases were verified. Verification of coding is an important tool for 

ensuring quality control and extending coder training. As a verification step, a second coder 

independently assigned codes (i.e., double-blind coding) to industry and occupation cases that had been 

initially coded either by the autocoding system or manually by a coder. A coding supervisor adjudicated 

disagreements between the initial code and the verification code. In the early stages, 100 percent of each 

coder’s work was reviewed. Once the coder’s error rate had dropped to 1 percent or less, 10 percent of the 

coder’s work was reviewed. Fifteen percent of the cases that were autocoded required adjudication 

because the verifier disagreed with the autocoding. Thirty percent of the manually coded cases required 

adjudication because the manual coder and the verifier disagreed. 
5-9 

Table 5-4 summarizes the results of the coding and verification process for occupation 

coding. In the table, manually coded indicates that occupation was initially coded by a coder as opposed 

to using the autocoding system. Discrepancies are the count of disagreements between the autocoder and 

the verifier or between the manual coder and the verifier: the discrepant cases required adjudication. The 

percentage of times in which the coding supervisor disagreed with the coder’s (or the autocoding 

system’s) initial coding is referred to as the coder error rate. The percentage of times in which the coding 

supervisor disagreed with the verifier’s coding is referred to as the verifier error rate. The denominator 

used in calculating these error rates is the number of cases verified. The error rate for manually coded 

cases was higher for coders (20.8 percent) than for verifiers (12 percent). The autocoded cases had a 

lower error rates for both coders (10.8 percent) and verifiers (5.5 percent) compared with the manually 

coded rates. 

Table 5-4. Number and percent of occupations coded, by coding status: School year 2003–04 

Coding Status Number coded Percent

 Total 12,691 100.0

Coded 12,691 100.0

 Autocoded 4,920 38.8

 Manually coded 7,771 61.2

Verified 12,691 100.0

 Verified from autocoding 4920 38.8

 Verified from manual coding 7,771 61.2

Discrepancies between coding and verifying 3,015 100.0

 Adjudicated from autocoding 720 23.9

 Adjudicated from manual coding 2,295 76.1

Autocoding system or manual coder wrong 2,158 100.0

Autocoded 521 24.1

Manually coded 1,637 75.9

Verifier wrong 1,056 100.0

Autocoded 238 22.5

Manually coded 818 77.5

NOTE: Occupation was collected for up to two key persons in spring-fifth grade. It was only collected for persons who had a different job 

from spring-third grade or who had not been employed in the previous round, but were employed in spring-fifth grade. For everyone else, 

the spring-third grade data were carried forward. The numbers in this table represent occupational text strings that were coded into 

appropriate occupation categories and applied to the appropriate key person (e.g., mother or father). 1 Total discrepancies do not equal the sum of coder wrong and verifier wrong because sometimes both coder and verifier were deemed to 

be wrong. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 

Class of 1998–99 (ECLS-K), spring 2004. 
5-10 

5.1.4 Editing the Household Roster in the Parent Interview 

The parent data edit system was modified from spring-third grade in April and May 2004 to 

reflect changes to the fifth-grade parent interview. The parent interview data were edited in two batches 

as the interviews were completed (see table 5-5). This was done to make the process more efficient. The 

first batch consisted of all cases received from the beginning of the round through May 20, 2004. The 

second batch consisted of cases completed from May 21 through the end of data collection. 

Table 5-5. Data editing of the parent interview household roster, fifth-grade data collection: School year 

2003–04 

Batch 

number Data 

extraction date 

Number of 

households 

extracted

Percent of 

total households

Number of 

households 

failing edits 

Percent of 

households 

extracted

Total † 10,895 100 435 3.2

1 5/20/04 6,004 55.1 190 3.1

2 6/3/04 4,897 44.9 168 3.4

† Not applicable. 

NOTE: The data in this table are household-level data, not case-level data (that is, not child-level). Detail may not sum to totals due to rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

The first step in the editing process was to extract the household roster data and run the data 

edits. The second step was to apply the programmatic updates to the cases failing the edits to correct any 

errors programmatically. The third step was for an expert reviewer to manually review the cases, conduct 

as-needed discussions with NCES for resolution, and resolve and correct data errors. Six cases were lost 

from the data file after the expert review and discussion with NCES because the children had no data and 

were removed from the data file. 

Several tests were run on the household roster to look for missing or inaccurate information. 

There were essentially three general types of roster tests performed to determine which cases required 

editing. First, the relationship of an individual to the focal child was compared to the individual’s listed 

age and sex. Problems found were corrected on the basis of data from prior data collections wherever 

possible. Second, households with more than one mother or more than one father were scrutinized for 

errors. While it was possible to have more than one mother in a household, such cases warranted closer 

inspection. Corrections were made whenever clear errors and a clear resolution existed. Last, the 

relationship of an individual to both the focal child and the respondent was examined, as there were cases 
5-11 

in which the relationship of an individual to the focal child conflicted with his status as the spouse/partner 

of the reference person. For example, in a household containing a child’s grandparents but not his or her 

parents, the grandmother may be designated the “mother” figure, and the grandfather thus becomes the 

“father” (for the purposes of some questions in the interview) by virtue of his marriage to the 

grandmother. These cases were examined but left unchanged. Both the original—and correct 

(grandfather)—relationship data and the new “parent-figure” designation (father) that had been 

constructed were kept. 

Updates to the household roster were required when one or more of the edit checks described 

above failed. The data for the case would be inspected, resolved, and, if necessary, corrected. There were 

435 households, with 446 children linked to them, completed a parent interview that required editing in 

fifth grade (identified by P6EDIT=1). There were 76 cases in which the interviewer noted in the Family 

Structure section (FSQ) that they had entered a person incorrectly in the household roster. These cases 

can be identified by the flag “P6ERRFLG” on the final data files. These “error” cases may or may not 

have also been edit cases, depending on whether or not the error triggered one of the edit checks. 

5.2 Coding and Editing Specifications for Hard-Copy Questionnaires 

5.2.1 Receipt Control 

In order to monitor the more than 40,000 documents that were to be received in the fifthgrade year, the project-specific receipt and document control system developed in the base year was used, 

with some modifications. The electronic receipt and document control system was initially loaded or 

filled with identifying information, such as identification numbers for schools, teachers, and children; the 

identification numbers linking teachers, children, and domain; and the questionnaires that were expected 

from each school and teacher for each cooperating school in the sample. As data were collected in the 

field, field supervisors completed transmittal forms for each school to indicate which questionnaires were 

being mailed to the home office. 

Once data collection started, receipt control clerks reviewed the questionnaires sent in from 

the field for accuracy and completeness. The identification number on each form was matched against the 

identification numbers in the tracking system to verify that the appropriate number of forms for each 

school was returned. Questionnaires that matched identification numbers in the tracking system were 
5-12 

receipted in the system and assigned a batch number. All receipted questionnaires matched identification 

numbers in the tracking system. Each questionnaire type had a different batch number so that 

questionnaires of the same type would be processed together. Processing data in batches was more 

efficient because it allowed staff to process in volume one type of questionnaire rather than switching 

from one type to another. Questionnaires with the same batch numbers were then compiled into batches 

of 25 instruments. The batch sheets with all the questionnaire identification numbers were printed from 

the system; the questionnaires in the batches were verified against the batch sheets. Verified batches were 

then sent for data entry. Once the batches had completed data entry, that is, all the questionnaires in a 

given batch had been keyed, the batches, accompanied by the electronic data, were returned to the data 

preparation department. The electronic data were loaded into the editing system and edited. At each point 

in the process, a flag was set in the receipt and document control system, which indicated the status of the 

instrument and the batch to which it was assigned. These statuses were the following: 

 Batches created; 

 Batches printed; 

 Batches verified; 

 Batches sent to keypunch; 

 Batches logged from keypunch; 

 Cases loaded to BES (editing system); 

 Cases cleaned (in edit system); and 

 Cases pending (in edit system). 

5.2.2 Scan Edit Procedures 

Critical items were identified for the school administrator questionnaire and the child-level 

reading, mathematics, and science teacher questionnaires. Prior to mailing the school administrator or 

child-level teacher questionnaires to Westat, the field supervisors reviewed them to ensure that critical 

items had been completed. If the critical items were missing, field supervisors attempted to retrieve them 

and recorded the outcome, completed or refused, in green pencil in the questionnaire. Exhibit 5-2 presents 

the critical items for these questionnaires. 
5-13 

Exhibit 5-2. Grade 5 critical items for questionnaires: School year 2003–04 

Instrument Item Question number 

School administrator (SAQ) Grade level 

School type/control 

Race/ethnicity 

School lunch program eligibility/participation 

Question 4 

Questions 5, 6, 7 

Question 8 

Question 14 

 

Child-level reading teacher 

questionnaires 

Grades taught 

Race/ethnicity 

Sex 

Gifted/talented, Limited English Proficient (LEP), 

Disability, Absent 

Question 15 

Question 16 

Question 17 

Question 18 

Child-level mathematics teacher 

questionnaires 

Grades taught 

Race/ethnicity 

Sex 

Gifted/talented, LEP, Disability, Absent 

Question 5 

Question 6 

Question 7 

Question 8 

 

Child-level science teacher 

questionnaires 

Grades taught 

Race/ethnicity 

Sex 

Gifted/talented, LEP, Disability, Absent 

Question 4 

Question 5 

Question 6 

Question 7 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class 

of 1998–99 (ECLS-K), spring 2004. 

Prior to receipting returned questionnaires, trained clerks scanned each instrument for 

completeness and assigned a status code of “Complete, No Data Retrieval,” “Complete, Data Retrieval 

Required, Complete,” or “Complete, Data Retrieval Required, Refused” based on the results of field 

supervisors efforts. Questionnaires were then logged into the receipt and document control system and 

batched for data entry. Once questionnaires were logged in, the data were first keypunched into electronic 

format and than coded and edited. Questionnaires that contained no data due to refusal by the respondents 

were logged into the receipt and document control system as “refusal.” Table 5-6 presents data on the 

number of questionnaires receipted by week. 
5-14 

Table 5-6. Number of questionnaires receipted by week, fifth-grade data collection: School year 

2003–04 

Week Date 

Number 

receipted

Cumulative number 

receipted

Percent of total 

receipted

 

 Total 45,090 45,090 100.0 

1 03/08/2004 1,158 1,158 2.6 

2 03/15/2004 1,904 3,062 6.8 

3 03/22/2004 2,768 5,830 12.9 

4 03/29/2004 3,904 9,734 21.6 

5 04/05/2004 5,387 15,121 33.5 

6 04/12/2004 2,591 17,712 39.3 

7 04/19/2004 2,575 20,287 45.0 

8 04/26/2004 2,888 23,175 51.4 

9 05/03/2004 2,801 25,985 57.6 

10 05/10/2004 2,941 28,926 64.2 

11 05/17/2004 2,365 31,291 69.4 

12 05/24/2004 2,284 33,575 74.5 

13 05/31/2004 1,868 35,443 78.6 

14 06/07/2004 2,868 38,311 85.0 

15 06/14/2004 1,368 39,679 88.0 

16 06/21/2004 3,074 42,753 94.8 

17 06/28/2004 1,357 44,110 97.8 

18 07/05/2004 882 44,992 99.8 

19 07/12/2004 98 45,090 100.0 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 

Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 

The following sections describe the data entry, coding, and editing processes for hard-copy 

questionnaires. 

5.2.3 Data Entry 

Data entry consisted of two steps: (1) entering the data and (2) verifying that the data had 

been entered accurately. Westat data entry staff keyed the forms in each batch. A set of keying rules was 

established for each questionnaire and followed by the data entry staff. To verify the accuracy of the data 
5-15 

entry, more senior data entry operators then rekeyed 100 percent of the data. The results of the two data 

entry sessions were compared and differences identified. When differences were found, the hard-copy 

form was pulled and examined to determine what corrections, if any, had to be made to the keyed data. 

These corrections were rekeyed, resulting in an accuracy rate exceeding 99 percent. The verified batches 

were then transmitted electronically to Westat’s computer system for data editing. 

5.2.4 Coding 

The hard-copy questionnaires required a quick visual review of particular questions in each 

questionnaire, coding of race/ethnicity for teachers, and review of “Other, specify” text responses. The 

quick visual review was to ensure that the questionnaire values accurately reflected existing categories, 

were complete and consistent across variables, and that the numbers were converted to the appropriate 

unit of measurement prior to converting data to an electronic format. Once the hard-copy questionnaires 

had been visually reviewed, they were coded. The coding staff was trained on the procedures and had 

manuals to support the coding process. Senior coders verified coding. The verification rate was set at 100 

percent for each coder until accuracy of less than 1 percent error rate was established. After that point, 

work was reviewed at a rate of 10 percent. 

The “Other, specify” text responses were reviewed by the data editing staff and, where 

appropriate, upcoded into one of the existing response categories. The specify responses that remained 

after upcoding were reviewed to evaluate whether the addition of any new response categories would be 

appropriate. There was no need for the addition of new response categories in fifth grade. 

5.3 Data Editing Management Process 

The management of the data editing process involved the creation of several data files, 

including the Collection Database, Holding Database, Editing Database, and Delivery Database. 

Exhibit 5-3 provides a diagram of the process described below. 
5-16

Exhibit 5-3. ECLS-K hard-copy data editing, fifth-grade data collection: School year 2003–04 SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spri

Collection

Holding

From Field

Editing

Delivery

Copy as is. Set Collection Status to CollectionNew

Copy to Holding and set Holding status to New. Set Collection status to CollectionCopied. Apply code-allthat-apply recoding and yes/no recoding. Extract Other Specify text. Convert all blanks to -9, and all don't knows and refused to -8 and -7 as appropriate. Enforce skips (apply checkrules)..

Copy DirtyEdit and DirtyNew to Edit, set Holding status to InEdit and set Edit status to WaitingForEdit.Generate error reports. 

Other assorted edits and transpose issues. Apply other specify upcodes. Holding status would be set to InDelivery. 

Move cases with status WasEdited or KeepAsIs to Holding. Set Holding status to CheckEdit.

* Go to step 3.

1

4

6

7

 Error Files

Check New and CheckEdit cases, set Holding status as follows: - if case is DIRTY - if status was New, set status to DirtyNew.  - if status was CheckEdit, set  status to DirtyEdit.- if case is CLEAN - if status was New, set status to CleanNew. - if status was CheckEdit, set status to CleanEdit.

3

Editor will look at/analchange the cases that WaitingForEdit. After the editor is doneeditor will assign a staWasEdited or KeepAscase. KeepAsIs is assif there are rule violatiowill not be changed.

2

5
5-17 

5.3.1 Collection Database 

This database contained the keyed records for hard-copy questionnaires. One Collection 

Database was created for each instrument and as additional data were keyed, the cases were added to the 

database. The Collection Databases were Blaise databases. The ASCII file resulting from the key entry 

process was converted to Blaise data in the Collection Database so that they could be merged with the 

parent interview data and undergo additional data review (see section 5.4) 

Records in the Collection Databases were assigned status codes reflecting their current 

status. All new records were assigned a status of CollectionNew. When cases were copied to the Holding 

Database, the status was updated to CollectionCopied. The data in the Collection Database were retained 

in their original form; that is, they were not modified based upon later steps. 

5.3.2 Holding Database 

Data were copied from the Collection Database to the Holding Database for the editing 

process. The Holding Database for each instrument was also a Blaise database. The copied cases were 

assigned a status code of New. Cases that had already been involved in a prior editing cycle and had been 

returned to the Holding Database were assigned a status of CheckEdit or KeepAsIs. 

As the data were copied from the Collection Database to the Holding Database, a number of 

processes were run. Code-all-that-apply (COTA) recoding and “yes/no” recoding were applied. COTA 

recoding involved changing the multiple-response values of 0/1, 0/2, 0/3, etc., to a series of yes/no (1/2) 

responses. Yes/no recoding provided a means to resolve questions left unanswered in a series of yes/no 

items. If all marked answers were “Yes,” then the unanswered items were converted to “No.” However, if 

any item was “No,” “Don’t know,” or “Refused,” all unanswered items were converted to –9 (Not 

ascertained). All blanks were converted to –9 (Not ascertained) and don’t know and refused responses 

were converted to –7 and –8 as appropriate. It was at this stage that skip patterns were enforced using the 

Blaise CheckRules function and legitimate skips were assigned the standard code of –1. 

Edit programs (range and logical checks) were run against all cases contained in the Holding 

Database. As the editing process continued, the Holding Database contained both new cases copied from 

the Collection Database and edited cases returned from the Editing Database (see section 5.3.3). Each 
5-18 

case was assigned a status code that reflected its current status. For cases that were new to the Holding 

Database, the CheckRules function assigned one of two codes. The status CleanNew was assigned to new 

cases that contained no edit (range or logical) errors. The status DirtyNew was assigned to new cases that 

failed one or more edit checks. Those cases that had undergone edit updating were also subjected to edit 

checks to identify any errors that remained or were inadvertently introduced during edit updating. The 

CheckRules function assigned the status CleanEdit to cases with no remaining errors. The status DirtyEdit

was assigned to cases returned from edit updating that had remaining or new errors. Those cases that were 

assigned a status of KeepAsIs in a previous editing round were considered clean. 

Cases that were found to have edit errors (DirtyNew and DirtyEdit) were copied to the 

Editing Database for review and updating. At that time, their status in the Holding Database was set to 

InEdit. A face sheet was generated for each case with editing errors, giving the batch number, case ID, 

and edit rules that had been violated. 

5.3.3 Editing Database 

Cases in the Holding Database that failed edit checks were copied to the Editing Database 

for the correction of errors. As cases were copied to the Editing Database, they were assigned a status of 

WaitingForEdit in the Editing Database. Editing staff worked from face sheets produced during the edit 

checks conducted on the Holding Database to retrieve and correct case records. Using the batch number 

and case ID number, editors retrieved and reviewed hard-copy instruments as necessary to resolve editing 

errors. 

Once the editor had reviewed and updated each case as necessary, he or she assigned one of 

two outcome codes. The status code of WasEdited was assigned when all edit errors had been corrected. 

A status of KeepAsIs was assigned when the editor’s review indicated that data that violated an edit check 

should be retained, for example, when the hard-copy instrument indicated that an out-of-range value was 

correct. 

Cases with the statuses of WasEdited and KeepAsIs were moved back to the Holding 

Database. Cases that had a status of WasEdited in the Editing Database were assigned the status 

CheckEdit in the Holding Database. The edit rules were applied to these cases to ensure that they were 

clean. As noted earlier, cases assigned a status of KeepAsIs in the editing process were considered clean. 
5-19 

5.3.4 Delivery Database 

The main purpose of the Delivery Database was to store the instrument data at the school, 

teacher, or child level in a “rectangular” format consistent with downstream activities in preparation for 

data delivery. Cases for which editing and coding activities were completed were copied from the 

Holding Database to the Delivery Database. These were cases with status codes of CleanNew, CleanEdit, 

or KeepAsIs. When the data were copied to the Delivery Database, the “Other, specify” upcodes and 

parent interview occupation codes were applied. See exhibit 5-4 for a summary of the status codes 

assigned for data management databases. 

Exhibit 5-4. Status codes assigned for data management databases, fifth-grade data collection: School 

year 2003–04 

Database Status codes assigned 

Collection CollectionNew: New cases 

CollectionCopied: Copied to Holding Database 

 

Holding New: New case copied from the Collection Database 

CheckEdit: Cases returned from the Editing Database to be subjected to edit checks again 

DirtyNew: New cases that fail edit check(s) 

DirtyEdit: Cases returned to from the Editing Database that have been edited and have 

failed edit check(s) 

CleanNew: New cases with no errors 

CleanEdit: Edited cases with no errors 

InEdit: Cases copied to the Editing Database 

KeepAsIs: Cases returned from the Editing Database that have been reviewed in the 

Editing Database and are not to be subjected to edits again 

 

Editing WaitingForEdit: Cases with errors waiting to be edited 

WasEdited: Cases for which editing is completed 

KeepAsIs: Edited cases for which the editor has determined that edit errors (e.g., out of 

range responses) should be retained as is 

 

Delivery InDelivery: Cases ready for delivery 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 
5-20 

5.4 Data Editing 

The data editing process consisted of running range edits for soft and hard ranges, running 

consistency edits in Blaise, and then manually reviewing frequencies of the results. 

5.4.1 Range Specifications 

Hard-copy range specifications set the parameters for high and low acceptable values for a 

question. Where values were printed on the forms, these were used as the range parameters. For openended questions, such as, “Counting this school year, how many years have you taught in your current 

school including part-time teaching?” high and low ranges were established as acceptable values. Data 

frequencies were run on the range of values to identify any errors. Values outside the range were 

identified as errors and were printed for a data editor to review. Cases with range errors were identified, 

and the original response was updated. In some cases, range violations were retained in the data because 

the value was checked and found to be the value reported by the teacher or school. These were marked as 

KeepAsIs cases. Data frequencies were then rerun and reviewed. This iterative process was repeated until 

no further range errors were found. 

5.4.2 Consistency Checks (Logical Edits) 

By programming logical edits between variables, consistency between variables not involved 

in a skip pattern was confirmed. For example, in the school administrator questionnaire, the number of 

children eligible for free breakfast could not exceed the total number of children enrolled in the school. 

These logical edits were run on the whole database after all data entry and range edits were complete. The 

logical edits were run separately for each form. All batches of data were combined into one large data file, 

and data frequencies were produced. The frequencies were reviewed to ensure the data remained logically 

consistent within the form. When an inconsistency was found, the case was identified and the 

inconsistency was printed for an editor to review. The original value was corrected (or checked and “kept 

as is” if the data item was confirmed, and the case was again run through the consistency edits. Once the 

case passed the consistency edits, it was returned to the main data set. The frequencies were then rerun 

and reviewed. This was an iterative process; it was repeated until no further inconsistencies were found. 

Table 5-7 shows hard-copy questionnaire data preparation production. More than 87 percent of all 
5-21 

questionnaires passed all the edits. The student record abstract accounted for 83 percent of all cases 

failing edits and, for all but a handful of cases, the data remained as it was reported. 

Table 5-7. Hard-copy editing progress report, fifth-grade data collection: School year 2003–04 

Instruments edited 

Instrument type 

Number of 

instruments 

Number of 

instruments 

clean after the 

first cleaning 

pass

Number of 

instruments 

edited 

Number of 

instruments clean 

from edit 

Number of 

instruments left 

as is from edit

Total 43,518 37,908 5,610 357 5,253

Percent 87.1 12.9 0.82 12.1

Facilities checklist 2,183 2,183 0 0 0

School administrator 

questionnaire 2,123 1,826 297 88 209

Special education A 722 712 10 6 4

Special education B 981 888 93 33 60

Student records 

abstract 10,122 5,461 4,661 3 4,658

Teacher 

questionnaire 5,683 5,530 153 91 62

Reading 

questionnaire 10,878 10,655 223 109 114

Math questionnaire 5,381 5,299 82 18 64

Science questionnaire 5,445 5,354 91 9 82

NOTE: The total in this table does not match the total receipted shown in table 5-6 because of refusals and other unusable 

questionnaires.

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

5.4.3 Teacher Responses to Key Child Items 

In fifth grade, teachers of sampled children were asked to respond to child-level 

questionnaires for the reading, mathematics, and science domains. In many cases, teachers had more than 

one sampled child in a class. The items in the child-level questionnaire that collected information about 

classroom characteristics were redundant under these circumstances. The key child approach was 

designed to minimize the burden on the teachers by designating one questionnaire in which the classroom 

characteristics items were to be completed. See section 4.4.2 for a description of the key child design and 

procedures. 
5-22 

Once the child-level questionnaires were keyed and loaded into the editing system, a review 

was conducted to identify cases in which teachers reported classroom characteristics on a different 

questionnaire than the one designated as the key child instrument for the given class. This process 

involved three steps: the review of missing data for classroom characteristics items within each domain 

(reading, mathematics, and science) for key child records, a detailed review of all data records in classes 

with multiple children and missing values for selected classroom characteristics items, and the updating 

of appropriate records. 

In the first step, data records for key children in all classrooms with more than one sampled 

child were selected. Frequency distributions of the classroom items were examined for the level of 

missing data within each domain. All classroom characteristics items were included in this review. The 

results of this initial review indicated that missingness was largely confined to the items concerning the 

race and sex composition of the classroom. 

In the second step, all returned instruments were selected for classrooms with multiple 

children that had missing data for the race and sex composition items. These cases were reviewed to 

ascertain whether the teacher had mistakenly reported the classroom characteristics items on a 

questionnaire other than that designated for the key child. 

In the third step, update specifications were prepared, directing data preparation staff to 

apply the classroom characteristics data to the key child record for the classroom. Updates were made to 

10 reading records, 5 mathematics records, and 3 science records as a result of this review. 

A review was also conducted to identify classrooms with multiple sampled children for 

which no key child instrument was returned. There were 5 such cases for reading, 7 such cases for 

mathematics, and 3 such cases for science. Another child for whom an instrument was returned was 

designated as the key child in these classrooms. 

5.4.4 Frequency and Cross-Tabulation Review 

As a final review, frequencies and cross-tabulations were run to determine consistency and 

accuracy within the various forms. If discrepancies could not be explained, no changes were made to the 

data. For example, in teacher questionnaire A, an item asking about languages other than English spoken 
5-23 

in the classroom included a response option of “No language other than English.” If a respondent circled 

that response, but also answered (in subsequent items) that other languages besides English were spoken 

in the classroom, then the response was left as recorded by the respondent because the discrepancy could 

not be resolved. 

5.5 Creation of the Socioeconomic Status (SES) Variable 

Socioeconomic status (SES) was computed at the household level using data for the set of 

parents who completed the parent interview in fifth grade. The SES variable reflects the socioeconomic 

status of the household at the time of data collection for fifth grade (spring 2004). The components used 

to create the SES were as follows: 

1. Father/male guardian’s education; 

2. Mother/female guardian’s education; 

3. Father/male guardian’s occupation; 

4. Mother/female guardian’s occupation; and 

5. Household income. 

Occupation was recoded to reflect the average of the 1989 General Social Survey (GSS) 

prestige score. This was computed as the average of the corresponding prestige scores for the 1980 

Census occupational categories covered by the ECLS-K occupation. Table 5-8 provides details of the 

recode of the occupation values to the 1989 GSS prestige score values 

Not all parents completed the parent interview; among those who did, not all responded to 

every question. Therefore, there were missing values for some of the components of the SES composite 

variable. For a description of how data for the components of the SES were collected, see section 7.5.2.7 

of the Combined User’s Manual for the ECLS-K Fifth-Grade Data Files and Electronic Codebooks

(NCES 2006–032) (Tourangeau et al. forthcoming). Table 5-9 shows that only a small percentage of 

values for the education and occupation variables were missing; a much larger proportion of households 

had missing values for the detailed income range. The total number of households in the third-grade data 

file (identified by the parents in the households) is 10,895, of which 77.8 percent are households with 

both parents present. Of the households with single parents, 10.6 percent are headed by single fathers. 
5-24 

Table 5-9 also includes labor force status that was used to determine whether missing occupation data 

would be imputed. Individuals were defined as “in the labor force” if they were working at a paid job, on 

vacation from a paid job, or looking for a job. Occupation was imputed only for parents in the labor force. 

A description of the levels of the SES components can be found in table 5-10. 

Table 5-8. Recode of occupation values to 1989 GSS prestige scores, spring-fifth grade: School 

year 2003–04 

ECLS-K occupation code 1980 Census 

occupation code 

1989 GSS

prestige score

01: Executive, administration, managerial 003-037 53.50

02: Engineer, surveyor, architect 043-059 64.89

03: Natural scientist, mathematician 063-083 62.87

04: Social scientist, social worker, lawyer 166,168-179 59.00

05: University teacher, postsecondary counselor, 

librarian 

113-149,163-165 72.10

06: Teacher, except postsecondary 153-159 63.43

07: Physician, dentist, veterinarian 084-089,167 77.50

08: Registered nurse, pharmacist 095-106 61.56

09: Writer, artist, entertainer, athlete 183-199 52.54

10: Health technologist/technician 203-208 57.83

11: Other technologist/technician 213-235 48.69

12: Marketing/sales occupation 243-285 35.78

13: Administrative support 303-389 38.18

14: Service occupation 403-427,433-469 34.95

15: Agriculture, forestry, fishing occupation 473-499 35.63

16: Mechanics/repairs 503-549 39.18

17: Construction/Extractive occupation 553-599 39.20

18: Precision production occupation 613-699 37.67

19: Other production occupation 703-799 33.42

20: Transportation, material moving 803-859 35.92

21: Handler, cleaner, helper, laborer 863-889 29.60

22: Unemployed/retired Missing

 No occupation Missing

 Cannot be coded Missing

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 

Class of 1998–99 (ECLS-K), spring 2004. 
5-25 

Table 5-9. Parents or households with missing value of the SES components, springfifth grade: School year 2003–04 

SES component Number missing Percent missing1

Education 

 Father 160 1.8 

 Mother 159 1.5 

Labor force status 

 Father 88 1.0 

 Mother 106 1.0 

Occupation2

 

 Father 166 1.9 

 Mother 100 0.9 

Detailed income range3

 

 Total 883 8.1 

 Broad income range4

 known 472 4.3 

 Broad income range unknown 411 3.8 

1 Percent missing is over the number of records that should have nonmissing value of the imputed variable. Note that not 

applicable cases are not included in denominator. 

2 These numbers include parents with missing labor force status. 

3 Detailed income range: $5,000 or less, $5,001-$10,000, $10,001-$15,000, $15,001-$20,000, $20,001-$25,000, $25,001-$30,000, 

$30,001-$35,000, $35,001-$40,000, $40,001-$50,000, $50,001-$75,000, $75,001-$100,000, $100,001-$200,000, more than 

$200,000. 

4 Broad income range: $25,000 or less, $25,001 or more. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 

Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 
5-26 

Table 5-10. Components of the SES variable, spring-fifth grade: School year 2003–04 

Component Level Description 

Education level of 

mother/father/legal guardian 

1 

2 

3 

4 

5 

6 

7 

8 

9 

8th grade or below 

9th to 12th grade 

High school diploma or equivalent 

Vocational/technical program 

Some college 

Bachelor’s degree 

Graduate or professional school, no degree 

Master’s degree 

Doctorate or professional degree 

Occupation of 

mother/father/legal guardian 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

97 

98 

Executive, administration, managerial 

Engineer, surveyor, architect 

Natural scientist, mathematician 

Social scientist, social worker, lawyer 

University teacher, postsecondary counselor, librarian 

Teacher, except postsecondary 

Physician, dentist, veterinarian 

Registered nurse, pharmacist 

Writer, artist, entertainer, athlete 

Health technologist/technician 

Other technologist/technician 

Marketing/sales occupation 

Administrative support 

Service occupation 

Agriculture, forestry, fishing occupation 

Mechanics/repairs 

Construction/extractive occupation 

Precision production occupation 

Other production occupation 

Transportation, material moving 

Handler, cleaner, helper, laborer 

Unemployed/retired 

No occupation 

Cannot be coded 

Income broad range 1 

2 

$25,000 or less 

More than $25,000 

Income detailed range1 2,500 

7,500 

12,500 

17,500 

22,500 

27,500 

32,500 

37,500 

45,000 

62,500 

87,500 

150,000 

300,000 

$5,000 or less 

$5,001 to $10,000 

$10,001 to $15,000 

$15,001 to $20,000 

$20,001 to $25,000 

$25,001 to $30,000 

$30,001 to $35,000 

$35,001 to $40,000 

$40,001 to $50,000 

$50,001 to $75,000 

$75,001 to $100,000 

$100,001 to $200,000 

More than $200,000 1 The midpoints of the detailed ranges are the levels used for this variable. For the top income range (more than $200,000), the 

level was computed based on the midpoint for the parents reporting in this range in the first-grade year. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 

Kindergarten Class of 1998–99 (ECLS-K), spring 2004.
5-27 

A two-stage procedure was used to impute missing values for each component of the SES 

composite variable. First, if a parent had completed an interview in the kindergarten, first-grade, or thirdgrade year, missing values for the fifth-grade education, occupation, and detailed income range were 

filled in with values from the previous years. The rationale for this approach was that the best source of 

data for an individual or a household was the data from a previous year. 

This first imputation stage was implemented as follows: 

1. Education level was brought forward from the most recent round of data collection 

prior to fifth grade, but only if the fifth-grade parent was the same person for whom 

the education information was reported in that previous round. 

2. Labor force status (whether the parent was in the labor force) was brought forward 

from the most recent round of data collection prior to fifth grade, but only if the fifthgrade parent was the same person who reported the labor force status information 

from that previous round. Even though labor force status is not a direct component of 

the SES, it cannot be missing since it was used to determine how to impute missing 

occupation information. 

3. Occupation was brought forward from the most recent round of data collection prior 

to fifth grade, but only if the fifth-grade parent was in the labor force (i.e., he or she 

was working at a paid job, on vacation from a paid job, or looking for a job) and was 

the same person who reported the occupation information from that previous round. 

4. Detailed income category was brought forward from the most recent round of data 

collection prior to fifth grade. 

Second, data still missing after this initial step were imputed using a hot deck methodology. 

In hot deck imputation, the value reported by a respondent for a particular item was assigned or “donated” 

to a “similar” person who failed to respond to that question. Auxiliary information known for both donors 

and nonrespondents was used to form groups of persons having similar characteristics. These groups of 

similar respondents and nonrespondents are called “imputation cells.” The imputed value for a case with a 

missing value is taken from a randomly selected donor among the respondents within the cell. 

Imputation cells were defined by respondent characteristics that were the best predictors of 

the variables to be imputed. These relationships had been determined previously by CHAID (Chi-squared 

Automatic Interaction Detector) analyses of the base year data, as shown in table 5-11. Missing values for 

the education, occupation, and detailed income range variables were imputed by the hot deck method for 

all households. Hot deck imputation was done in a sequential order, separately, by type of household 
5-28 

Table 5-11. Demographic characteristics used in creating imputation cells, spring-fifth 

grade: School year 2003–04 

Characteristic Level Description 

Census region 1 

2 

3 

4 

Northeast 

Midwest 

South 

West 

School affiliation 1 

2 

3 

4 

Private: Catholic 

 Private: Religious, non-Catholic 

Private: Nonreligious 

Public: Regular, Department of 

Defense, Bureau of Indian 

Affairs 

School locale 1 

2 

3 

4 

5 

6 

7 

8 

Large city 

Midsize city 

Urban fringe, large city 

Urban fringe, midsize city 

Large town 

Small town 

Rural, outside MSA 

Rural, inside MSA 

Household type 1 

2 

3 

Female single parent 

Male single parent 

Both parents present 

Race/ethnicity 1 

2 

3 

4 

5 

6 

7 

White 

Black 

Hispanic 

Asian 

Pacific Islander 

American or Alaskan Native 

More than one race 

See note at end of table. 
5-29 

Table 5-11. Demographic characteristics to be used in creating imputation cells, springfifth grade: School year 2003–04—Continued 

Characteristic Level Description 

Mother’s age, 

father’s age 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26 

27 

28 

16-17 years 

18–19 years 

20–21 years 

22–23 years 

24–25 years 

26–27 years 

28–29 years 

30–31 years 

32–33 years 

34–35 years 

36–37 years 

38–39 years 

40–41 years 

42–43 years 

44–45 years 

46–47 years 

48–49 years 

50–51 years 

52–53 years 

54–55 years 

56–57 years 

58–59 years 

60–61 years 

62–63 years 

64–65 years 

66–67 years 

68–69 years 

70 or more years 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal 

Study, Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 

(female single parent, male single parent, and both parents present). For households with both parents 

present, the mother’s and father’s variables were imputed separately. Imputed as well as reported values 

were used to define imputation cells; missing values for donor characteristics were treated as a separate 

category. No imputed value was used as a donor. No donor was used more than once. 

Occupation imputation involved two steps. First, the labor force status of the parent was 

imputed (i.e., whether the parent was employed). Then the parent’s occupation was imputed only for 

those parents whose status was identified as employed either through the parent interview or the first 
5-30 

imputation step. The detailed income range was imputed in two steps: first for cases where the broad 

income range was known, and second for cases where it was unknown. 

For households where both parents were present, the order of hot deck imputation was as 

follows: 

1. Mother’s education; 

2. Father’s education; 

3. Mother’s labor force status; 

4. Father’s labor force status; 

5. Mother’s occupation; 

6. Father’s occupation; 

7. Detailed income range, where the broad income range was known; and 

8. Detailed income range, where the broad income range was unknown. 

Following imputation, all values of the SES components were nonmissing including the 

value −1 for “not applicable.” Examples of not applicable cases are children with no father in the 

household, in which case the father’s education and occupation are not applicable. 

Table 5-12 summarizes the imputation results. Tables 5-13 to 5-19 summarize the 

distribution of the records before and after imputation. The percentage columns may not always add to 

100 percent due to rounding. 
5-31 

Table 5-12. Imputation of SES components by filling in with values from previous rounds or by hot 

deck, spring-fifth grade: School year 2003–04 

Filled in with values Imputed by 

 Number from previous round hot deck 

SES component missing n Percent n Percent

Education 

 Father 160 97 60.6 63 39.4

 Mother 159 108 67.9 51 32.1

 

Labor force status 

 Father in labor force status 79 66 83.5 13 16.5

 Father not in labor force status 9 7 77.8 2 22.2

 Mother in labor force status 75 68 90.7 7 9.3

 Mother not in labor force status 31 26 83.9 5 16.1

 

Occupation1

 

 Father 166 16 9.6 150 90.4

 Mother 100 18 18.0 82 82.0

 

Detailed income range 

 Total 883 786 89.0 11.0

 Broad income range known 472 388 82.2 84 17.8

 Broad income range unknown 411 398 96.8 13 3.2

1

 Occupation was not imputed if parent was not in labor force (whether labor force was filled in with data from previous round or imputed 

by hot deck). 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 

Class of 1998–99 (ECLS-K), spring 2004. 
5-32 

Table 5-13. Mother’s education, before and after imputation, spring-fifth grade: School 

year 2003–04 

Mother’s Before imputation Imputed After imputation 

education level Count Percent Count Percent Count Percent 

 Total 10,895 100.0 159 100.0 10,895 100.0 

1 447 4.1 21 13.2 468 4.3 

2 633 5.8 23 14.5 656 6.0 

3 2,622 24.1 44 27.7 2,666 24.5 

4 558 5.1 10 6.3 568 5.2 

5 3,124 28.7 37 23.3 3,161 29.0 

6 1,915 17.6 17 10.7 1,932 17.7 

7 285 2.6 1 0.6 286 2.6 

8 662 6.1 3 1.9 665 6.1 

9 233 2.1 3 1.9 236 2.2 

Not applicable 257 2.4 0 0.0 257 2.4 

Missing 159 1.5 † † 0 0.0 

† Not applicable. 

NOTE: Percentages may not sum to 100 due to rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 

Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 

Table 5-14. Father’s education, before and after imputation, spring-fifth grade: School year 

2003–04 

Father’s Before imputation Imputed After imputation 

education level Count Percent Count Percent Count Percent 

 Total 10,895 100.0 160 100.0 10,895 100.0 

1 375 3.4 16 10.0 391 3.6 

2 595 5.5 24 15.0 619 5.7 

3 2,253 20.7 40 25.0 2,293 21.1 

4 472 4.3 9 5.6 481 4.4 

5 1,955 17.9 34 21.3 1,989 18.3 

6 1,556 14.3 25 15.6 1,581 14.5 

7 215 2.0 3 1.9 218 2.0 

8 664 6.1 5 3.1 669 6.1 

9 492 4.5 4 2.5 496 4.6 

Not applicable 2,158 19.8 0 0.0 2,158 19.8 

Missing 160 1.5 † † 0 0.0 

† Not applicable. 

NOTE: Percentages may not sum to 100 due to rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 

Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 
5-33 

Table 5-15. Mother’s labor force status, before and after imputation, spring-fifth grade: 

School year 2003–04 

Mother’s labor Before imputation Imputed After imputation 

force status Count Percent Count Percent Count Percent 

 Total 10,895 100.0 106 100.0 10895 100.0 

1 8,202 75.3 75 70.8 8,277 76.0 

2 2,330 21.4 31 29.3 2,361 21.7 

Not applicable 257 2.4 0 0.0 257 2.4 

Missing 106 1.0 † † 0 0.0 

† Not applicable. 

NOTE: Percentages may not sum to 100 due to rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 

Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 

Table 5-16. Father’s labor force status, before and after imputation, spring-fifth grade: 

School year 2003–04 

Father’s labor Before imputation Imputed After imputation 

force status Count Percent Count Percent Count Percent 

 Total 10,895 100.0 88 100.0 10,895 100.0 

1 8,254 75.8 79 89.8 8,333 76.5 

2 395 3.6 9 10.2 404 3.7 

Not applicable 2,158 19.8 0 0.0 2,158 19.8 

Missing 88 0.8 † † 0 0.0 

† Not applicable. 

NOTE: Percentages may not sum to 100 due to rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 

Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 
5-34 

Table 5-17. Mother’s occupation, before and after imputation, spring-fifth grade: School year 

2003–04 

 Before imputation Imputed After imputation 

Mother’s occupation Count Percent Count Percent Count Percent

 Total 10,895 100.0 100 100.0 10,895 100.0

Not in labor force 2,361 21.7 † † 2,361 21.7

1 951 8.7 6 6.0 957 8.8

2 44 0.4 0 0.0 44 0.4

3 65 0.6 1 1.0 66 0.6

4 184 1.7 0 0.0 184 1.7

5 72 0.7 0 0.0 72 0.7

6 632 5.8 4 4.0 636 5.8

7 70 0.6 0 0.0 70 0.6

8 454 4.2 2 2.0 456 4.2

9 138 1.3 0 0.0 138 1.3

10 195 1.8 1 1.0 196 1.8

11 130 1.2 2 2.0 132 1.2

12 730 6.7 7 7.0 737 6.8

13 2,137 19.6 28 28.0 2,165 19.9

14 1,580 14.5 34 34.0 1,614 14.8

15 74 0.7 1 1.0 75 0.7

16 13 0.1 0 0.0 13 0.1

17 23 0.2 0 0.0 23 0.2

18 50 0.5 1 1.0 51 0.5

19 430 4.0 8 8.0 438 4.0

20 80 0.7 1 1.0 81 0.7

21 109 1.0 3 3.0 112 1.0

22 16 0.2 1 1.0 17 0.2

Not applicable 257 2.4 0 0.0 257 2.4

Missing 100 0.9 † † 0 0.0

† Not applicable. 

NOTE: Percentages may not sum to 100 due to rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 

Class of 1998–99 (ECLS-K), spring 2004. 
5-35 

Table 5-18. Father’s occupation, before and after imputation, spring-fifth grade: School year 

2003–04 

 Before imputation Imputed After imputation 

Father’s occupation Count Percent Count Percent Count Percent

 Total 10,895 100.0 166 100.0 10,895 100.0

Not in labor force 404 3.7 † † 404 3.7

1 1,453 13.3 25 15.1 1,478 13.6

2 303 2.8 3 1.8 306 2.8

3 131 1.2 2 1.2 133 1.2

4 198 1.8 1 0.6 199 1.8

5 62 0.6 0 0.0 62 0.6

6 147 1.4 0 0.0 147 1.4

7 174 1.6 1 0.6 175 1.6

8 60 0.6 1 0.6 61 0.6

9 100 0.9 1 0.6 101 0.9

10 48 0.4 1 0.6 49 0.5

11 260 2.4 5 3.0 265 2.4

12 670 6.2 8 4.8 678 6.2

13 414 3.8 5 3.0 419 3.9

14 669 6.1 19 11.5 688 6.3

15 316 2.9 8 4.8 324 3.0

16 665 6.1 11 6.6 676 6.2

17 895 8.2 30 18.1 925 8.5

18 133 1.2 3 1.8 136 1.3

19 677 6.2 14 8.4 691 6.3

20 562 5.2 15 9.0 577 5.3

21 202 1.9 13 7.8 215 2.0

22 28 0.3 0 0.0 28 0.3

Not applicable 2,158 19.8 0 0.0 2,158 19.8

Missing 166 1.5 † † 0 0.0

† Not applicable. 

NOTE: Percentages may not sum to 100 due to rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal 

Study, Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 
5-36 

Table 5-19. Household income, before and after imputation, spring-fifth grade: School year 2003–04 

 Before imputation Imputed After imputation 

Income category Count Percent Count Percent Count Percent

 Total 10,895 100.0 883 100.0 10,895 100.0

$5,000 or less 196 1.8 53 6.0 249 2.3

$5,001-$10,000 265 2.4 59 6.7 324 3.0

$10,001-$15,000 484 4.4 87 9.9 571 5.2

$15,001-$20,000 540 5.0 74 8.4 614 5.6

$20,001-$25,000 602 5.5 87 9.9 689 6.3

$25,001-$30,000 700 6.4 62 7.0 762 7.0

$30,001-$35,000 575 5.3 49 5.6 624 5.7

$35,001-$40,000 675 6.2 60 6.8 735 6.8

$40,001-$50,000 966 8.9 91 10.3 1,057 9.7

$50,001-$75,000 1,863 17.1 109 12.3 1,972 18.1

$75,001-$100,000 1,448 13.3 80 9.1 1,528 14.0

$100,001-$200,000 1,297 11.9 59 6.7 1,356 12.5

More than $200,000 401 3.7 13 1.5 414 3.8

Missing 883 8.1 † † 0 0.0

† Not applicable. 

NOTE: Percentages may not sum to 100 due to rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 

Class of 1998–99 (ECLS-K), spring 2004. 

The values of each SES component were then normalized so that the component has a mean 

of 0 and a standard deviation of 1. This is also known as the z-score. For the h-th SES component, a zscore zhi for the i-th household was computed as 

( ) , se x

x x

z

w

hi w

hi

− =

where 

hi x is the value of the h-th SES component for the i-th household; 

wi is the base weight for the i-th record; 

xw is the weighted mean of hi x ; and 

( ) w se x is the standard error of xw . 

Note that where h is household income, hi x is the natural log of the midpoint of the detailed 

income range. The log of the detailed income range midpoint does not vary widely within the levels of the 

detailed income range, so the midpoint was a reasonable choice. It was used only for the purpose of 

computing the SES composite and was not retained in the data file. 
5-37 

Thus, each component was converted to a z-score with mean of 0 and a standard deviation of 

one. The SES variable for the i-th household was computed as 

, m

z

SES

i

m

h

hi

i

i

∑

= = 1

where mi is the number of nonmissing SES components for the i-th household. 

Table 5-20 shows the distribution of the SES values. As described, the SES composite is the 

average of up to five measures, each of which was standardized to have a mean of 0 and a standard 

deviation of 1, hence the negative values. Note that for households with only one parent present, not all 

the components were defined. In these cases, the SES was computed averaging the available components. 

In the fifth-grade data file, the continuous SES variable is W5SESL. 

A categorical SES variable (W5SESQ5) was created that contains the quintile for the value 

of the composite SES for the child. Quintile 1 represents the lowest SES category and quintile 5 

represents the highest SES category. The quintiles were computed at the child level using the fifth-grade 

child-level parent weights. Unweighted frequencies for this variable are given in table 5-21. 
5-38 

Table 5-20. Distribution of socioeconomic status (SES) values, 

spring-fifth grade: School year 2003–04 

Characteristic SES

N 10,895 

Mean -0.01 

Standard deviation 0.81 

Minimum -2.48 

Maximum 2.54 

Range 5.02 

1st percentile -1.87 

5th percentile -1.29 

10th percentile -1.04 

25th percentile -0.68 

50th percentile -0.21 

75th percentile 0.42 

90th percentile 0.98 

95th percentile 1.30 

99th percentile 1.90 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early 

Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 

Table 5-21. Unweighted frequencies of the socioeconomic status 

(SES) variable, spring-fifth grade: School year 

2003–04 

SES quintile Frequency Percent

 Total 10,895 100.0

1st quintile 1,783 16.4

2nd quintile 2,000 18.4

3rd quintile 2,083 19.1

4th quintile 2,411 22.1

5th quintile 2,618 24.0

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early 

Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 
5-39 

5.6 Imputation of the School Lunch Composites 

The school lunch composites were computed at the school level for the set of public schools 

that have at least one child or parent respondent (i.e., the child has nonzero child weight, C6CW0, or 

nonzero child-level parent weight, C6PW0) in spring-fifth grade. There are two school lunch composites 

as follows: (1) Percent of children eligible for free school lunch; and, (2) Percent of children eligible for 

reduced-price lunch. For a description of how the data were collected and how the composites were 

computed, see section 7.5.4.6 of the Combined User’s Manual for the ECLS-K Fifth-Grade Data Files 

and Electronic Codebooks (NCES 2006–032) (Tourangeau et al. forthcoming). 

Not all schools completed the school administrator questionnaire, and among those who did, 

not all responded to all three questions needed to compute the school lunch composites. Therefore, there 

were missing values for some of the components of the school lunch composite variables. Prior to fifth 

grade, if the source variables have missing value, then the composites were filled in with values computed 

using the most recent Common Core of Data (CCD) where available, or left missing. In fifth grade, the 

composites were computed as in the past, but if they had missing values, they were imputed. The source 

variables, however, were not imputed. Table 5-22 shows the level of missing data for the school lunch 

composite variables among the 2,008 public schools that have 9,328 child or parent respondents in the 

fifth grade of the ECLS-K. 

Table 5-22. Public schools and child-parent respondents with missing value of the school lunch 

composites, spring-fifth grade: School year 2003–04 

 Number missing Percent missing 

School lunch composite Schools

Child-parent 

respondents Schools 

Child-parent 

respondents

Free lunch 691 2,545 34.4 27.3

Reduced-price lunch 712 2,601 35.5 27.9

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class 

of 1998–99 (ECLS-K), spring 2004. 

Similar to the components for the SES composite, a two-stage procedure was used to impute 

missing values for each school lunch composite variable. First, if a school had nonmissing value of the 

school lunch composite in the kindergarten, first-grade, or third-grade year, missing values for the springfifth grade school lunch composites were filled in with values from the previous years. The rationale for 

this approach was that the best source of data for a school was the data from a previous year. 
5-40 

Second, data still missing after this initial step were imputed using a hot deck methodology. 

In hot deck imputation, the values computed for a school for both composites were assigned or “donated” 

to a “similar” school that had missing values of the composites. Imputation cells were created using the 

Title I status of the school (hard boundary) and the school latitude and longitude (soft boundaries). The 

imputed value for a school with a missing value was taken from the nearest neighbor defined by latitude 

and longitude within the hard-boundary cell (Title I status). The Title I status is a derived variable using 

the data on whether the school received Title I funds (S6TT1) and whether Title I funds were targeted or 

used school-wide (S6TT1TA), both collected in the school administrator questionnaire. If these two 

variables have missing values for fifth grade, then the most recent available data (from third grade or first 

grade or kindergarten) were used. If these data were missing from the school administrator questionnaire 

for all rounds, then the information from the most recent CCD (2002–03) was used. 

The resolution of cases having missing data is shown for each school lunch composite in 

table 5-23 (for schools) and table 5-24 (for child-parent respondents). Schools that were imputed by hot 

deck are generally transfer schools with few sample children in those schools. This is reflected in tables 

5-23 and 5-24 where the percent of children with hot-deck values of the school composites is much 

smaller than the percent of schools with hot-deck values of the school composites. 

Since children were designated as eligible for either free lunch or reduced-price lunch but 

not for both services, the two school lunch composites should sum to no more than 100 percent. A very 

small number of schools (less than 2 percent) had imputed values of the two school lunch composites 

summing to more than 100 percent. These values came from two sources: (1) from values reported by the 

school in another year, or (2) from the hot-deck imputation. The reporting error has been present in all 

rounds of the ECLS-K, and the decision was to keep the reported values in the data file. If the erroneous 

values came from the hot-deck imputation, then they were corrected so that the two school lunch 

composites do not add to more than 100 percent. Correction was made by capping the hot-decked values 

of the two lunch composite variables. This was done by multiplying each value by 100 divided by the 

sum of the two variables before capping. This way, both values were reduced by the same amount so they 

sum to 100 percent. 

Tables 5-25 to 5-28 show the characteristics of the school lunch composites before and after 

imputation, at the school level and at the child level. 
5-41 

Table 5-23. Imputation of school lunch composites at the school level, spring-fifth grade: School year 

2003–04 

 Number 

Values from 

previous round 

 Imputed by 

Hot deck 

School lunch composite missing n Percent n Percent

Free lunch 691 256 37.0 435 63.0

Reduced-price lunch 712 265 37.2 447 63.8

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class 

of 1998–99 (ECLS-K), spring 2004. 

Table 5-24. Imputation of school lunch composites at the child level, spring-fifth grade: School year 

2003–04 

 Number 

Values from 

previous round 

 Imputed by 

Hot deck 

School lunch composite missing n Percent n Percent 

Free lunch 2,545 1,777 69.8 768 30.2

Reduced-price lunch 2,601 1,819 69.9 782 30.1

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class 

of 1998–99 (ECLS-K), spring 2004. 
5-42 

Table 5-25. Free lunch composite at the school level, before and after imputation, spring-fifth grade: 

School year 2003–04 

Characteristic Before imputation Imputed After imputation 

All cases 2,008 691 2,008 

Number of missing 691 0 0

Mean (not including missing) 38.75 41.22 39.60 

Standard deviation 28.29 29.79 28.83 

Minimum 0.00 0.00 0.00 

Maximum 95.00 95.00 95.00 

Range 95.00 95.00 95.00 

1st percentile 0.00 0.00 0.00 

5th percentile 2.33 0.00 2.06 

10th percentile 5.43 1.52 4.82 

25th percentile 14.10 3.47 14.12 

50th percentile 32.66 14.20 33.84 

75th percentile 60.41 36.94 62.52 

90th percentile 83.39 85.71 84.12 

95th percentile 95.00 95.00 93.75 

99th percentile 95.00 95.00 95.00 

NOTE: These statistics are from the restricted-use data file where some data have been masked according to the disclosure risk avoidance 

procedures. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class 

of 1998–99 (ECLS-K), spring 2004. 
5-43 

Table 5-26. Free lunch composite at the child level, before and after imputation, spring-fifth grade: 

School year 2003–04 

Characteristic Before imputation Imputed After imputation 

All cases 9,328 2,545 9,328 

Number of missing 2,545 0 0

Mean (not including missing) 37.12 44.54 39.15 

Standard deviation 26.74 32.13 28.50 

Minimum 0.00 0.00 0.00 

Maximum 95.00 95.00 95.00 

Range 95.00 95.00 95.00 

1st percentile 0.00 0.00 0.00 

5th percentile 2.17 1.95 2.08 

10th percentile 5.48 2.94 5.00 

25th percentile 15.43 15.26 15.26 

50th percentile 31.13 39.35 33.25 

75th percentile 53.57 74.07 60.41 

90th percentile 80.62 94.06 84.98 

95th percentile 90.38 95.00 94.42 

99th percentile 95.00 95.00 95.00 

NOTE: These statistics are from the restricted-use data file where some data have been masked according to the disclosure risk avoidance 

procedures. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class 

of 1998–99 (ECLS-K), spring 2004. 
5-44 

Table 5-27. Reduced-price lunch composite at the school level, before and after imputation, springfifth grade: School year 2003–04 

Characteristic Before imputation Imputed After imputation 

All cases 2,008 712 2,008 

Number of missing 712 0 0

Mean (not including missing) 9.68 9.37 9.57 

Standard deviation 10.75 9.12 10.36 

Minimum 0.00 0.00 0.00 

Maximum 100.00 100.00 100.00 

Range 100.00 100.00 100.00 

1st percentile 0.00 0.00 0.00 

5th percentile 0.35 0.00 0.00 

10th percentile 1.52 0.67 1.09 

25th percentile 4.69 4.05 4.49 

50th percentile 8.19 8.24 8.22 

75th percentile 12.09 12.00 12.03 

90th percentile 16.50 16.67 16.59 

95th percentile 20.00 20.26 20.12 

99th percentile 91.37 44.78 50.00 

NOTE: These statistics are from the restricted-use data file where some data have been masked according to the disclosure risk avoidance 

procedures. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class 

of 1998–99 (ECLS-K), spring 2004. 
5-45 

Table 5-28. Reduced-price lunch composite at the child level, before and after imputation, spring-fifth 

grade: School year 2003–04 

Characteristic Before imputation Imputed After imputation 

All cases 9,328 2,601 9,328 

Number of missing 2,601 0 0

Mean (not including missing) 9.06 9.05 9.06 

Standard deviation 8.13 11.38 9.15 

Minimum 0.00 0.00 0.00 

Maximum 100.00 100.00 100.00 

Range 100.00 100.00 100.00 

1st percentile 0.00 0.00 0.00 

5th percentile 0.26 0.00 0.00 

10th percentile 1.53 0.00 0.96 

25th percentile 4.85 3.02 4.36 

50th percentile 8.16 7.71 8.00 

75th percentile 11.84 11.72 11.80 

90th percentile 15.53 16.07 15.60 

95th percentile 18.36 19.53 18.53 

99th percentile 31.25 73.68 39.57 

NOTE: These statistics are from the restricted-use data file where some data have been masked according to the disclosure risk avoidance 

procedures. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 

Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 
5-46 

This page intentionally left blank. 
6-1 

6. RESPONSE RATES 

This chapter describes the computation of unit completion rates for the spring-fifth grade 

data collection of the ECLS-K, and unit overall response rates for the base year respondents. Weighted 

and unweighted unit completion rates are presented for three groups of children: (1) children sampled in 

kindergarten, (2) children sampled in first grade through the freshening procedure, and (3) both groups 

combined. Completion rates for the fifth-grade data collection were computed with the same procedures 

used for spring-first grade and spring-third grade to allow for comparisons of completion rates for the 

three years of data collection following the base year. Item response rates for selected items from the 

ECLS-K fifth-grade instruments are also presented. 

For spring-first grade and spring-third grade, the sample of children is the same: base-year 

respondents (i.e., children who had either a fall- or spring-kindergarten child assessment or parent 

interview) and children sampled in spring-first grade as part of sample freshening as described in section 

3.4.2. For spring-fifth grade, the sample of children was reduced to exclude base-year respondents who 

belonged in the following special groups as described in section 3.6: (1) children who became ineligible 

in an earlier round (because they died or moved out of the country), (2) children who were subsampled 

out in previous rounds because they moved out of the original schools and were not subsampled to be 

followed, (3) children whose parents emphatically refused to cooperate (hard refusals) in any of the data 

collection rounds since spring-kindergarten, and (4) children eligible for the third-grade sample for whom 

there are neither first-grade nor third-grade data. Among the 21,357 children who were eligible for the 

study after the base year, 16,143 were part of the fifth-grade data collection. 

6.1 Definition of Response and Completion Rates 

Response rates and completion rates are two ways to describe the outcomes of data 

collection activities. A response rate is the ratio of the number of units with completed interviews (for 

example, the units could be children, parents, schools or teachers) to the number of units sampled and 

eligible for the interview. The response rate indicates the percentage of possible interviews completed, 

taking all survey stages into account. On the other hand, the completion rate measures the percentage of 

interviews completed for a specific stage of the survey. For example, in the base year of the ECLS-K 

children were identified for assessment in a two-stage process. The first stage involved the recruitment of 
6-2 

schools to participate in the study. Preassessment visits were made to schools that agreed to participate. 

During the preassessment visit, field supervisors met with the participating school’s school coordinator to 

enumerate and sample the kindergartners. Assessments were then conducted for the sampled children whose 

parents consented. If the school refused to participate in the study, no children were sampled for assessment. 

Under this design, the completion rate for the child assessment is the percentage of sampled children who 

completed the assessment. The response rate is the product of the school participation or cooperation rate 

and the child assessment completion rate. 

Response and completion rates can be either unweighted or weighted. The unweighted rate, 

computed using the raw number of cases, provides a useful description of the success of the operational 

aspects of the survey. The weighted rate, computed by summing the weights (usually the reciprocals of 

the probability of selecting the units) for both the numerator and denominator, gives a better description 

of the success of the survey with respect to the population sampled since the weights allow for inference 

of the sample data (including response status) to the population. Both rates are usually not very different 

unless the probabilities of selection and the response rates in the categories with different selection 

probabilities vary considerably. 

For example, the weighted completion rate of the ECLS-K child assessment (CA) is 

computed as 

∈

∈ ∪

=

∑

∑

CA

CA CA

i

i ER

CA

i

i ER ENR

W

r

W

where Wi is the weight (inverse of the probability of selection of the child) for child i, and ERCA denotes 

eligible child assessment respondent and ENRCA eligible child assessment nonrespondent. To compute the 

unweighted rates, Wi is set to 1 for each child. 

The response rate of the child assessment can be computed as 

RCA S CA = r r ×

where rS is the school cooperation rate and rCA is the child assessment completion rate. 
6-3 

After the base year, only completion rates were computed for the different ECLS-K 

instruments, since the response rates of the schools where the children were sampled remained the same 

for each subsequent round. Data users can compute the fifth-grade response rate for each ECLS-K 

instrument by multiplying the school response rate from the base year and the fifth-grade completion rate 

for each instrument. 

Both unweighted and weighted rates are presented in the tables of completion rates in this 

chapter. While unweighted rates are useful for evaluating sample performance as mentioned earlier, only 

weighted rates are discussed in the text. 

6.2 Completion Rates 

For the ECLS-K fifth-grade data collection, there were 11 survey instruments: child 

assessment; parent interview; school administrator questionnaire; facilities checklist; student records 

abstract, teacher-level questionnaire, subject-specific child-level questionnaires (reading, mathematics, 

and science); and special education teacher questionnaire part A and part B. The mathematics teacher 

questionnaire was completed for about half of the children in the fifth-grade sample and the science 

teacher questionnaire was completed for the other half, so that each child would have data from no more 

than 10 instruments. Except for the child assessment and the parent interview, all other instruments were 

paper-and-pencil instruments. 

For each instrument, completion rates were computed separately for children who were 

sampled as part of the kindergarten cohort in the base year and for children who were sampled in first 

grade through the student sample freshening procedure. While the completion rate for children sampled in 

the base year has only one component (to account for nonresponse attrition during data collection), the 

completion rate for children sampled in first grade has two components (to account for nonresponse 

attrition during the freshening procedure and during data collection). Section 6.2.2 describes in detail the 

two components of the completion rates for the freshened children. The two sets of rates were combined 

to obtain the completion rates for all children in fifth grade. Movers who were sampled to be followed but 

could not be located are considered nonrespondents and included in the denominator of the computation 

of completion rates. 
6-4 

6.2.1 Children Sampled in Kindergarten 

For the ECLS-K, a completion rate is a response rate conditioned on the results of an earlier 

stage of data collection. In the case of the ECLS-K fifth-grade data collection, the condition is that 

children who were sampled in kindergarten were base year respondents since only base year respondents 

were eligible for subsequent data collection efforts. Children sampled in first grade were exempt from this 

condition in the computation of completion rates. They are discussed in section 6.2.2. 

For each instrument, the unweighted completion rate is the proportion of base year 

respondents with completed data for the instrument to the base-year respondents who remain eligible to 

have the fifth-grade instrument administered. Base-year respondents who were subsampled out because 

they moved from their base- year original sample schools and base-year respondents who died or moved 

out of the country were not included in the denominator. For the weighted completion rates, the weight 

used is the product of the school base weight, the within-school child weight, and the factor that was used 

to adjust for movers between base year and fifth grade who were subsampled out for data collection. For a 

description of these weights and adjustment factor, see chapter 7. 

Tables 6-1 to 6-4 present weighted and unweighted child-level completion rates for springfifth-grade data collection, broken out by school characteristics.1

 These rates pertain to children who were 

sampled as part of the kindergarten cohort in the base year. In general, completion rates for fifth grade are 

higher than they had been for the third grade. This is due to the exclusion of hard-to-field cases from the 

fifth-grade collection. Hard-to-field cases are the hard-refusal cases and cases that were nonrespondents in 

both first and third grades as described in section 3.6. If these cases had not been excluded from the fifth 

grade, they would most likely be nonrespondents and would bring down the completion rates. 

Table 6-1 shows that the completion rates for the child assessment are quite high and 

uniform across school characteristics. Excluding the “unknown” category, the rates range from 93.1 

percent in non-Catholic private schools to 99.7 percent in schools in large town. Similarly, the completion 

rates for the parent interviews were uniform across school characteristics ranging from 87.2 percent for 

children in schools with total enrollment of 750 or more, and in schools where 50 to 89 percent of 

 

1

 Children in schools with unknown characteristics are movers who could not be located (and considered as nonrespondents in the completion 

rates). Their weights are large because of the mover adjustment where movers who were followed carry the weight of movers who were 

subsampled-out for follow-up. The categories of school affiliation in the tables in this chapter do not match categories of school affiliation in the 

tables in chapter 3. This is to allow users to compare completion rates in fifth grade with those in previous years. 
6-5 

Table 6-1. Number of completed child-level cases and child-level completion rates for the child assessment and parent in

sampled in the base year, by school characteristics: School year 2003–04 

Child assessment Parent interview 

Completion rates Completion 

School characteristics1 Completes2 Weighted Unweighted Completes3 Weighted U All schools 11,260 84.7 93.6 10,913 89.1 School affiliation  Public 9,187 96.2 97.7 8,518 89.9  Private 2,049 95.2 97.7 1,977 92.0  Catholic 1,313 97.1 98.3 1,260 94.0  Other private 736 93.1 96.7 717 89.5  Unknown 24 5.8 4.6 418 82.5 Type of locale  Large city 1,863 97.2 97.6 1,697 89.6  Mid-size city 1,863 97.6 98.3 1,733 90.8  Urban fringe of large city 3,286 94.8 96.6 3,085 88.6  Urban fringe of mid-size city 767 93.5 96.8 725 90.3  Large town 283 99.7 99.6 270 91.9  Small town 825 94.1 98.8 781 94.3  Rural – outside MSA 1,286 97.2 97.9 1,205 88.1  Rural – inside MSA 922 97.7 99.4 871 93.4  Unknown 165 17.0 24.6 546 83.4 School size (total enrollment)  1 to 299 2,359 96.7 98.2 2,255 93.4  300 to 499 3,703 96.5 98.0 3,453 89.9  500 to 749 3,167 97.1 97.6 2,945 90.8  750 or more 1,963 94.2 97.0 1,798 87.2  Unknown 68 9.7 11.7 462 82.1 

See notes at end of table. 
6-6 

Table 6-1. Number of completed child-level cases and child-level completion rates for the child assessment and parent in

sampled in the base year, by school characteristics: School year 2003–04—Continued 

Child assessment Parent interview 

Completion rates Completio

School characteristics1 Completes2 Weighted Unweighted Completes3 Weighted Percent non-White enrolled  0 – 10 3,509 97.9 98.3 3,352 93.2  11 – 49 3,705 94.5 97.2 3,522 90.5  50 – 89 1,956 96.5 97.4 1,768 87.2  90 – 100 1,997 97.0 98.0 1,786 88.2  Unknown 93 11.3 15.4 485 82.4 Region  Northeast 2,080 95.5 96.8 1,956 90.5  Midwest 2,957 97.9 98.8 2,803 93.0  South 3,614 95.1 97.5 3,334 87.6  West 2,585 96.5 97.3 2,402 91.2  Unknown 24 5.8 4.6 418 82.5 1 School characteristics are for schools attended by children in the ECLS-K fifth-grade sample and are based on ECLS-K survey data, not data from the sampling frame. 2 Reading, mathematics, or science assessments was scorable, or child was disabled and could not be assessed. 3 Family structure portion of parent interview was completed. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spri
6-7 

children belong to the minority groups, to 94.3 percent for children in small towns (excluding the 

“unknown” category). The “unknown” category includes children who were unlocatable as their 

whereabouts were unknown and those children who had moved into a nonsampled county. The category 

“unknown” also includes 35 children who were homeschooled and thus had no information concerning 

schools. 

The “unknown” category aside, both the child assessment and the parent interview 

completion rates increased between third grade and fifth grade for all school characteristics. The 

completion rates by mover status are discussed later, but the rates of completing all the instruments are 

much lower for children who moved than for those who did not move. 

Table 6-2 shows that the overall weighted completion rate is 77.1 percent for the school 

administrator questionnaire, and 78.8 percent for the facilities checklist. The rate for school administrator 

questionnaires is 11 percentage points higher than the corresponding rate in third grade. The rate for 

facilities checklist is only about 2 percent higher. Excluding the “unknown” category, the completion 

rates for the school administrator questionnaire range from 87.4 percent for schools with 750 or more 

students to 100 percent for those in large towns. Rates for the facilities checklist range from 90.3 percent 

for schools in the urban fringe of mid-size cities to 100 percent for schools in large towns. It is worth 

noting that the completion rates for the school administrator questionnaire are lower for schools with 

higher percentages of minorities, a phenomenon also observed in previous rounds for the school 

administrator questionnaire. However, this disparity decreased considerably after the base year, reflecting 

the success of increased data collection efforts targeted toward these schools. 
6-8 

Table 6-2. Number of completed child-level cases and child-level completion rates for the school administrator questionn

checklist for children sampled in the base year, by school characteristics: School year 2003–04 

School administrator questionnaire Facilities checklis

Completion rates Completio

School characteristics1 Completes2 Weighted Unweighted Completes2 Weighted  All schools 10,937 77.1 89.6 11,154 78.8 School affiliation  Public 8,884 90.7 94.5 9,084 92.7  Private 2,053 93.1 97.9 2,070 95.7  Catholic 1,323 96.8 99.0 1,328 98.1  Other private 730 88.8 95.9 742 92.8 Type of locale  Large city 1,775 92.2 93.0 1,862 96.4  Mid-size city 1,825 93.6 96.3 1,844 94.9  Urban fringe of large city 3,173 87.7 93.2 3,240 90.4  Urban fringe of mid-size city 762 90.9 96.2 761 90.3  Large town 284 100.0 100.0 284 100.0  Small town 823 91.2 98.6 823 91.1  Rural – outside MSA 1,274 92.0 97.0 1,292 93.6  Rural – inside MSA 904 93.9 97.4 911 94.6  Unknown 117 8.2 13.9 137 9.4 School size (total enrollment)  1 to 299 2,360 95.1 98.2 2,368 95.5  300 to 499 3,662 92.0 96.9 3,684 92.6  500 to 749 3,071 92.3 94.6 3,136 94.3  750 or more 1,844 87.4 91.2 1,939 91.9  Unknown 0 0.0 0.0 27 1.7 See notes at end of table. 
6-9 

Table 6-2. Number of completed child-level cases and child-level completion rates for the school administrator questionn

checklist for children sampled in the base year, by school characteristics: School year 2003–04—Continued 

School administrator questionnaire Facilities checklist 

Completion rates Completio

School characteristics1 Completes2 Weighted Unweighted Completes2 Weighted Percent non-White enrolled  0 – 10 3,502 95.0 98.1 3,515 95.5  11 – 49 3,633 89.8 95.4 3,674 91.1  50 – 89 1,864 90.4 92.8 1,920 92.9  90 – 100 1,916 92.4 94.1 1,996 96.2  Unknown 22 0.9 2.8 49 2.5 Region  Northeast 2,008 87.7 93.5 2,048 90.4  Midwest 2,920 95.5 97.5 2,938 95.9  South 3,547 90.7 95.7 3,598 92.1  West 2,462 89.1 92.7 2,570 93.9 1 School characteristics are for schools attended by children in the ECLS-K third-grade sample and are based on ECLS-K survey data, not data from the sampling frame. 2 A completed questionnaire was defined as one that was not left completely blank. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spri
6-10 

Table 6-3 shows that the rates for the student records abstract are the lowest of all the 

instruments, as they always were in previous years of the ECLS-K. For fifth grade, this rate is about 70 

percent compared with 67 percent in third grade. The “unknown” category aside, the completion rates of 

the student records abstract range from 71.8 percent in the northeast region to 93.7 percent for children in 

large towns. 

All four of the teacher questionnaires were completed at an overall rate of 78 to 80 percent 

(tables 6-3 and 6-4), much higher than the 62 to 63 percent range achieved in third grade. Excluding the 

“unknown” category, the completion rates for the teacher-level questionnaire in table 6-3 are uniform 

across school characteristics, ranging from 86.9 percent for schools in the northeast and schools with 750 

or more students to 99.7 percent for schools in large towns. The same uniform rates are found for the 

subject-specific child-level teacher questionnaires in table 6-4: 85.7 to 99.7 percent for reading, 81.7 to 

99.4 percent for mathematics, and 86.8 to 100 percent for science. These rates are higher than in any 

previous years of the ECLS-K, a change most likely attributable to the higher incentives employed in fifth 

grade. For a discussion of the incentives used in the ECLS-K, see section 4.5.6. 

As noted above, the rate at which the survey instruments were completed varies markedly by 

mover status and, within movers, by whether the child was located and followed. As shown in table 6-5, 

the completion rate for the child assessment was 98.2 percent for children still enrolled in their base year 

school. For movers it dropped by about 6 points to 91.9 percent for those who were located and followed, 

and for those not located or followed due to a move to a non-ECLS-K PSU, it was zero. The parent 

interview completion rates varied from 91.6 percent for nonmovers to 87.1 percent for movers who were 

located and followed for the purposes of the child assessment, to 85.7 percent for movers who could 

either not be located or were not followed for the purposes of the child assessment. Even though children 

who had moved to a non-ECLS-K PSU were not administered the child assessment, a parent interview 

was conducted by telephone wherever possible, leading to the 86 percent response rate for this category. 
6-11 

Table 6-3. Number of completed child-level cases and child-level completion rates for the student records abstract and te

questionnaire for children sampled in the base year, by school characteristics: School year 2003–04 

Student records abstract Teacher-level questionn

Completion rates Completion 

School characteristics1 Completes2 Weighted Completes2 Weighted U All schools 10,015 69.9 82.1 10,872 79.9 School affiliation  Public 8,177 82.3 86.9 8,849 90.2  Private 1,838 84.6 87.6 2,023 93.6  Catholic 1,209 90.3 90.5 1,313 97.0  Other private 629 77.9 82.7 710 89.4 Type of locale  Large city 1,585 78.9 83.1 1,758 89.8  Mid-size city 1,688 85.3 89.1 1,819 94.1  Urban fringe of large city 2,778 76.8 81.6 3,131 87.4  Urban fringe of mid-size city 698 82.1 88.1 751 90.2  Large town 272 93.7 95.8 283 99.7  Small town 758 85.7 90.8 819 91.1  Rural – outside MSA 1,227 89.3 93.4 1,273 91.0  Rural – inside MSA 884 89.4 95.3 903 93.4  Unknown 125 8.8 14.9 135 12.0 School size (total enrollment)  1 to 299 2,114 85.2 88.0 2,340 94.8  300 to 499 3,385 84.1 89.6 3,628 91.3  500 to 749 2,808 83.3 86.5 3,039 91.4  750 or more 1,687 78.6 83.4 1,844 86.9  Unknown 21 1.3 2.8 21 1.7 See note at end of table. 
6-12 

Table 6-3. Number of completed child-level cases and child-level completion rates for the student records abstract and te

questionnaire for children sampled in the base year, by school characteristics: School year 2003–04—Continu

Student records abstract Teacher-level question

Completion rates Completio

School characteristics1 Completes2 Weighted Unweighted Completes3 Weighted Percent non-White enrolled  0 – 10 3,274 89.0 91.7 3,465 93.9  11 – 49 3,303 81.4 86.7 3,608 89.2  50 – 89 1,686 80.2 84.0 1,854 90.5  90 – 100 1,712 80.2 84.0 1,902 91.4  Unknown 40 2.1 5.2 43 2.8 Region  Northeast 1,635 71.8 76.1 1,970 86.9  Midwest 2,788 90.4 93.1 2,913 94.7  South 3,359 83.4 90.7 3,531 90.1  West 2,233 81.3 84.1 2,458 90.0 1 School characteristics are for schools attended by children in the ECLS-K third-grade sample and are based on ECLS-K survey data, not data from the sampling frame. 2 Reading, mathematics, or science assessments was scorable, or child was disabled and could not be assessed. 3 Family structure portion of parent interview was completed. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spri
6-13 

Table 6-4. Number of completed child-level cases and child-level completion rates for the child-level teacher questionnai

the base year, by school characteristics: School year 2003–04 

Child-level 

reading teacher questionnaire 

Child-level 

mathematics teacher questionnaire 

sc

Completion rates Completion rates 

School characteristics1 Completes2 Weighted Unweighted Completes2 Weighted Unweighted Comple

All schools 10,793 79.3 90.0 5,339 78.1 89.3 5,

School affiliation  Public 8,780 89.5 93.3 4,357 87.9 92.6 4, Private 2,013 92.9 96.0 982 93.6 96.0 1,

Catholic 1,306 96.7 97.8 635 96.5 97.2 

Other private 707 88.5 92.9 347 90.2 93.8 

Type of locale  Large city 1,753 90.2 91.9 853 90.3 91.8  Mid-size city 1,811 93.6 95.6 893 90.3 94.3  Urban fringe of large city 3,111 86.5 91.4 1,547 84.4 90.4 1, Urban fringe of mid-size city 741 88.3 93.6 366 87.6 93.1  Large town 283 99.7 99.6 140 99.4 99.3  Small town 808 90.3 96.8 383 85.0 96.5  Rural – outside MSA 1,258 90.1 95.7 634 92.8 95.5  Rural – inside MSA 893 91.8 96.2 456 90.5 95.6  Unknown 135 12.0 21.3 67 13.1 20.9 School size (total enrollment)  1 to 299 2,323 93.8 96.7 1,144 94.2 96.6 1, 300 to 499 3,605 91.0 95.4 1,830 90.7 95.8 1, 500 to 749 3,014 90.6 92.9 1,462 88.8 91.7 1, 750 or more 1,830 85.7 90.5 892 81.7 88.0  Unknown 21 1.7 3.9 11 2.2 4.0 

See notes at end of table. 
6-14 

Table 6-4. Number of completed child-level cases and child-level completion rates for the child-level teacher questionnai

the base year, by school characteristics: School year 2003–04—Continued 

Child-level 

reading teacher questionnaire 

Child-level 

mathematics teacher questionnaire 

sc

Completion rates Completion rates 

School characteristics1 Completes2 Weighted Unweighted Completes2 Weighted Unweighted CompletePercent non-White enrolled  0 – 10 3,444 93.7 96.5 1,695 93.2 96.3 1,7 11 – 49 3,584 88.1 94.1 1,791 87.2 93.8 1,7 50 – 89 1,823 89.1 90.8 919 85.8 89.7 8 90 – 100 1,899 91.3 93.2 911 90.4 91.7 9 Unknown 43 2.8 7.6 23 3.3 7.9 Region  Northeast 1,956 86.7 91.1 967 84.6 90.6 9 Midwest 2,909 94.4 97.2 1,423 94.5 96.9 1,4 South 3,498 89.0 94.4 1,747 87.5 93.6 1,7 West 2,430 89.0 91.5 1,202 87.4 90.5 1,21 School characteristics are for schools attended by children in the ECL-K third-grade sample and are based on ECLS-K survey data, not data from the sampling frame. 2 A completed questionnaire was defined as one that was not left completely blank. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spri
6-15 

Table 6-5. Number of completed child-level cases and child-level completion rates for the child assessment and parent 

interview for children sampled in the base year, by child’s mover status: School year 2003–04 

 Child assessment Parent interview 

 Completion rates Completion rates 

Mover status1 Completes2 Weighted Unweighted Completes3 Weighted Unweighted All children 11,260 84.7 93.6 10,913 89.1 90.7Mover status  Mover 1,814 71.8 74.7 2,094 86.8 86.2

Located, followed 1,814 91.9 92.1 1,704 87.1 86.5Other4 0 0.0 0.0 390 85.7 85.0

 Nonmover 9,446 98.2 98.4 8,819 91.6 91.9

1 This is the mover status used in weighting, which does not consider children who moved into identified destination schools as movers. A destination school is a 

school that received at least four students from the school where they had just completed the highest grade. 

2 Reading, mathematics, or science assessments was scorable, or child was disabled and could not be assessed. 3 Family structure portion of parent interview was completed. 4 This category includes movers who could not be located, and movers who moved into nonsampled PSUs. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), 

spring 2004. 
6-16 

The school administrator questionnaire completion rate is 15 points lower for movers, even 

when the children were located and followed; for the facilities checklist, it is 14 points lower (table 6-6). 

There are several reasons for this difference: located movers were not always assessed in schools; new 

schools in which movers enrolled had a lower level of commitment to the ECLS-K and often refused to 

complete the school administrator questionnaire; and some of these schools were contacted too late in the 

school year for them to consider completing it. The completion rate for nonmovers was 97.1 percent for 

the school administrator questionnaire and 98.8 percent for the facilities checklist. For located and 

followed movers it was 82.4 and 84.8 percent for the school administrator questionnaire and for the 

facilities checklist, respectively. The rates for the student records abstract are 90.1 percent for nonmovers 

and 72.3 percent for movers who were located and followed (table 6-7). 

The teacher-level questionnaire completion rate, as shown in table 6-7, is about 14 points 

lower for movers who could be located and followed (82.2 percent) than for nonmovers (96.5 percent). 

Movers who could not be located were all nonrespondents for this instrument, pulling the overall 

completion rate for movers downward to 63.9 percent. Table 6-8 shows the completion rates for all three 

child-level teacher questionnaires. These rates are between 95 and 96 percent for nonmovers, and between 

80 and 82 percent for movers who were located and followed. Children who could not be located were all 

nonsrespondents for the child-level teacher instruments. The reasons for lower completion rates from 

teachers if the child moved are similar to the reasons that affected the school administrator questionnaire 

and facilities checklist completion rates for movers. 
6-17 

Table 6-6. Number of completed child-level cases and child-level completion rates for the school administrator 

questionnaire and facilities checklist for children sampled in the base year, by child’s mover status: School ye2003–04 

School administrator questionnaire Facilities checklist 

 Completion rates Completion rates 

Mover status1 Completes2 Weighted Unweighted Completes2 Weighted Unweighted All children 10,937 77.1 89.6 11,154 78.8 91.4Mover status  Mover 1,589 59.2 61.1 1,647 61.0 63.3 Located, followed 1,589 82.4 82.1 1,647 84.8 85.1 Other3 0 0.0 0.0 0 0.0 0.0

 Nonmover 9,348 97.1 97.4 9,507 98.8 99.0

1 This is the mover status used in weighting, which does not consider children who moved into identified destination schools as movers. A destination school is a 

school that received at least four students from the school where they had just completed the highest grade. 

2 A completed questionnaire was defined as one that was not left completely blank. 3 This category includes movers who could not be located, and movers who moved into nonsampled PSUs. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 
6-18 

Table 6-7. Number of completed child-level cases and child-level completion rates for the student records abstract and 

teacher-level questionnaire for children sampled in the base year, by child’s mover status: School year 2003–0

Student records abstract Teacher-level questionnaire 

 Completion rates Completion rates 

Mover status1 Completes2 Weighted Unweighted Completes2 Weighted Unweighted All children 10,015 69.9 82.1 10,872 79.9 90.6Mover status  Mover 1,393 52.0 53.6 1,587 63.9 66.3 Located, followed 1,393 72.3 72.0 1,587 82.2 82.0 Other3 0 0.0 0.0 0 0.0 0.0

 Nonmover 8,622 90.1 89.8 9,285 96.5 96.7

1 This is the mover status used in weighting, which does not consider children who moved into identified destination schools as movers. A destination school 

is a school that received at least four students from the school where they had just completed the highest grade. 

2 A completed questionnaire was defined as one that was not left completely blank. 3 This category includes movers who could not be located, and movers who moved into nonsampled PSUs. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), 

spring 2004. 
6-19 

Table 6-8. Number of completed child-level cases and child-level completion rates for the child-level teacher questionnai

base year, by child’s mover status: School year 2003–04 

Child-level 

reading teacher questionnaire 

Child-level 

mathematics teacher questionnaire 

science 

 Completion rates Completion rates 

Mover status1 Completes2 Weighted Unweighted Completes2 Weighted Unweighted Completes2  All children 10,793 79.3 90.0 5,339 78.1 89.3 5,405 Mover status  Mover 1,568 63.3 65.5 818 62.2 65.2 734  Located, followed 1,568 81.4 81.0 818 79.7 80.4 734  Other3 0 0.0 0.0 0 0.0 0.0 0 

 Nonmover 9,225 95.8 96.1 4,521 95.3 95.7 4,671 

1 This is the mover status used in weighting, which does not consider children who moved into identified destination schools as movers. A destination school is a s

students from the school where they had just completed the highest grade. 

2 A completed questionnaire was defined as one that was not completely left blank. 3 This category includes movers who could not be located, and movers into nonsampled PSUs. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spri
6-20 

Tables 6-9 to 6-12 present child-level weighted and unweighted completion rates for the 

spring-fifth grade data collection for children who were sampled as part of the kindergarten cohort in the 

base year, this time broken out by child characteristics. When the “unknown” categories are not included, 

the differences in completion rates by sex and by year of birth are inconsequential, but for race and 

ethnicity they are more substantial. Table 6-9 shows that the child assessment completion rate was highest 

for Asians (87.6 percent) and lowest for American Indians or Alaska Natives (78.3 percent). For the 

parent interview it is the opposite; the rate was highest for American Indians or Alaska Natives (95.2 

percent) and lowest for Asian children (82.8 percent). 

Table 6-10 shows that, excluding the “unknown” categories, the highest completion rates for 

the school administrator questionnaire and for the facilities checklist are for Pacific Islanders (85.7 

percent and 86.7 percent, respectively), and the lowest completion rates are for American Indians or 

Alaska Natives (65.8 percent and 71.1 percent, respectively). Table 6-11 shows that the completion rate 

for the student records abstract is highest for children with “other” race (72.8 percent) and the lowest is 

for Black (63.9), excluding the “unknown” categories. 

For the teacher-level questionnaire (table 6-11), the highest rate is for Pacific Islanders (84 

percent) and the lowest rate is for American Indians and Alaska Natives (77 percent), excluding the 

“unknown” categories. For the child-level reading teacher questionnaire (table 6-12), the highest rate is 

for Asians (82 percent) and the lowest rate is for American Indians or Alaska Natives (76 percent). For 

the child-level mathematics teacher questionnaire, the highest rate is for Hispanic and for children whose 

race/ethnicity is not among the listed (80 percent) and the lowest rate is for Black and American Indians 

or Alaska Natives (76 percent). For the child-level science teacher questionnaire, the highest rate is for 

Asians (86 percent) and the lowest rate is for Blacks and for children whose race/ethnicity is not among 

the listed (76 percent). 

In addition to the child assessment, parent interview, school administrator questionnaire, 

facilities checklist, student records abstract, and teacher questionnaires, whose completion rates have been 

summarized in the preceding paragraphs, data were also collected in fifth grade from the special 

education teachers for children who had special education teachers. Table 6-13 presents counts of 

completes and weighted and unweighted completion rates at the overall student level for the special 

education questionnaires A and B. The number of special education teacher questionnaires is small but 
6-21 

Table 6-9. Number of completed child-level cases and child-level completion rates for the child assessment and parent in

children sampled in the base year, by child characteristics: School year 2003–04 

Child assessment Parent interview 

Completion rates Complet

Child characteristics1 Completes2 Weighted Unweighted Completes3 Weighted  All children 11,260 84.7 93.6 10,913 89.1 Sex4

 Male 5,675 84.1 93.3 5,525 89.9  Female 5,585 85.4 94.1 5,388 88.5 Race/ethnicity  White, non-Hispanic 6,466 84.8 94.4 6,394 91.3  Black, non-Hispanic 1,273 83.3 93.3 1,151 83.4  Hispanic 2,093 85.9 92.4 2,036 89.5  Asian 788 87.6 92.5 707 82.8  Pacific Islander 144 85.4 92.3 136 87.7  American Indian or Alaska Native 210 78.3 92.1 222 95.2  Other 272 86.3 93.8 253 84.5  Unknown 14 55.3 50.0 14 58.0 Year of birth  1992 3,307 83.9 93.6 3,211 88.7  1993 7,896 85.1 93.8 7,646 89.4  Other/unknown 57 71.0 76.0 56 76.0 

1 Based on ECLS-K survey data and not on data from the sampling frame. 2 Reading, mathematics, or science assessments was scorable, or child was disabled and could not be assessed. 3 Family structure portion of parent interview was completed. 4 There is a small number of children whose sex is unknown and who did not have completed child assessment and parent interview. The completion rates for these child

are not included in the table. SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spri
6-22 

Table 6-10. Number of completed child-level cases and child-level completion rates for the school administrator question

checklist for children sampled in the base year, by child characteristics: School year 2003–04 

School administrator questionnaire Facilities check

Completion rates Comple

Child characteristics1 Completes2 Weighted Unweighted Completes2 Weighted  All children 10,937 77.1 89.6 11,154 78.8 Sex3

 Male 5,517 77.1 89.5 5,621 78.6  Female 5,420 77.2 90.0 5,533 79.2 Race/ethnicity  White, non-Hispanic 6,356 78.8 92.6 6,413 80.1  Black, non-Hispanic 1,256 74.5 89.9 1,266 75.4  Hispanic 1,957 75.4 83.5 2,061 79.0  Asian 752 77.9 85.1 776 80.9  Pacific Islander 145 85.7 92.4 146 86.7  American Indian or Alaska Native 191 65.8 82.3 208 71.1  Other 266 77.7 90.8 270 79.1  Unknown 14 50.1 46.7 14 50.1 Year of birth  1992 3,223 77.0 90.0 3,275 78.7  1993 7,660 77.2 89.7 7,825 79.1  Other/unknown 54 64.0 70.1 54 59.5 

1 Based on ECLS-K survey data and not on data from the sampling frame. 2 A completed questionnaire was defined as one that was not left completely blank. 3 The completion of the school-level instruments does not depend on whether the child has completed assessment or parent interview data. Hence, while all children with 

or parent interview data have a known value of sex; there are children with completed school-level data whose sex is unknown. SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spri
6-23 

Table 6-11. Number of completed child-level cases and child-level completion rates for the student records abstract and t

questionnaire for children sampled in the base year, by child characteristics: School year 2003–04 

Student records abstract Teacher-level questi

Completion rates Comple

Child characteristics1 Completes2 Weighted Unweighted Completes2 Weighted  All children 10,015 69.9 82.1 10,872 79.9 Sex3

 Male 5,034 69.6 81.6 5,482 79.3  Female 4,981 70.4 82.7 5,390 80.7 Race/ethnicity  White, non-Hispanic 5,878 72.6 85.6 6,300 80.6  Black, non-Hispanic 1,101 63.9 78.8 1,238 77.1  Hispanic 1,769 68.4 75.5 1,962 80.2  Asian 692 70.7 78.3 751 82.6  Pacific Islander 125 69.6 79.6 138 84.0  American Indian or Alaska Native 197 66.0 84.9 206 77.0  Other 242 72.8 82.6 263 81.9  Unknown 11 30.5 36.7 14 55.3 Year of birth  1992 2,996 71.5 83.7 3,213 80.1  1993 6,976 69.4 81.7 7,610 80.1  Other/unknown 43 57.4 55.8 49 54.4 

1 Based on ECLS-K survey data and not on data from the sampling frame. 2 A completed questionnaire was defined as one that was not left completely blank. 3 The completion of the school-level instruments does not depend on whether the child has completed assessment or parent interview data. Hence, while all ch

assessment and parent interview data have a known value of sex; there is one child with completed facilities checklist data whose sex is unknown. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), sp
6-24 

Table 6-12. Number of completed child-level cases and child-level completion rates for the child-level teacher questionn

the base year, by child characteristics: School year 2003–04 

Child-level 

reading teacher questionnaire 

Child-level 

mathematics teacher questionnaire 

science 

 Completion rates Completion rates 

Child characteristics1 Completes2 Weighted Unweighted Completes2 Weighted Unweighted Completes2  All children 10,793 79.3 90.0 5,339 78.1 89.3 5,405 Sex3

 Male 5,436 78.7 89.7 2,670 78.0 88.9 2,739  Female 5,357 80.0 90.5 2,669 78.4 89.9 2,666 Race/ethnicity  White, non-Hispanic 6,255 80.0 91.7 3,111 78.6 91.4 3,147  Black, non-Hispanic 1,231 76.7 90.2 591 75.7 88.5 629  Hispanic 1,952 79.5 86.4 959 79.6 85.2 963  Asian 743 81.8 87.3 362 77.8 85.8 378  Pacific Islander 132 79.1 84.6 75 77.5 85.2 59  American Indian or Alaska  Native 204 76.4 90.7 109 75.6 91.6 93  Other 262 81.4 91.0 127 80.2 89.4 129  Unknown 14 55.3 50.0 5 47.5 35.7 7 Year of birth  1992 3,186 79.1 90.4 1,565 77.9 90.1 1,618  1993 7,559 79.6 90.0 3,751 78.4 89.1 3,762  Other/unknown 48 53.5 64.0 23 55.0 63.9 25 

1 Based on ECLS-K survey data and not on data from the sampling frame. 2 A completed questionnaire was defined as one that was not left completely blank. 3 There is a small number of children whose sex is unknown and who did not have completed teacher questionnaire data. The completion rates for these children, being z

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spri
6-25 

Table 6-13. Number of completed instruments and child-level completion rates for the special education 

teacher questionnaires for children sampled in the base year: School year 2003–04 

 Completion rates 

Category Completes Weighted Unweighted

Special education part A1 960 92.2 93.8

Special education part B1 967 93.7 94.4 1 A completed instrument was defined as one that was not left completely blank. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

their completion rates are higher, 92.2 percent for part A, which captures teacher information, and 93.7 

percent for part B, which relates to children who receive individualized special education services. These 

rates are not broken down by school and child characteristics because of the small sample sizes. 

6.2.2 Children Sampled in First Grade 

In spring-first grade, the student sample was freshened to include first graders who had no 

chance of selection in the base year because they had not attended kindergarten in the United States or 

were in first grade in the fall of 1998. For a detailed description of the freshening procedure see chapter 3, 

section 3.4.2. This same group of children was followed into spring-fifth grade, unless they belonged in 

the excluded groups. Nonresponse in the freshened student sample could occur at two stages: during the 

procedure for sampling schools for freshening and identifying children to be used as freshening links in 

spring-first grade (first component) and then during data collection from the freshened children in springthird grade (second component). 

The first component of the completion rate is the proportion of children sampled from the 

base year and subsampled for freshening for which the study was able to do freshening. The numerator 

includes all children available for freshening (i.e., those who did not move and in schools that cooperated 

with the freshening in first grade); the denominator includes children sampled for freshening (excluding 

movers not subsampled). For the weighted first component of the completion rate, both numerator and 

denominator were weighted by the product of the school base weight, the within-school child weight, and 

the freshening adjustment factor. The school base weight and the within-school child level weight reflect 

the multi-stage sampling of the ECLS-K design, while the freshening adjustment factor is necessary 

because schools were subsampled for freshening in first grade as described in section 3.4.2. These 

weights and adjustment factor are discussed in more detail in chapter 7. The first component alone can 
6-26 

further be decomposed into two sources: attrition due to the refusal of entire schools to implement the 

freshening procedure (the school term), and attrition due to ECLS-K sampled children moving to other 

schools (the child term). To contain costs, children who transferred from schools targeted for freshening 

were not used as links to identify freshened children, even when they were otherwise followed for data 

collection. These movers were considered freshening nonrespondents in the child term. 

The second component is the proportion of freshened children with completed data for the 

instrument from within the population brought into the sample by freshening. The weight for this 

component is the product of the school base weight, the within-school child weight, the school freshening 

subsample adjustment factor and the fifth-grade mover subsampling adjustment factor. 

The final completion rate is the product of the two components. For example, the final 

completion rate for the child assessment is computed as follows: 

1 2

1 2

∈ ∈

∈ ∈∪

= ×

∑ ∑

∑ ∑

CA

CA CA

i i

i A i ER

CA

i i

i B i ER ENR

W W

r

W W

where A is the set of children who could be freshened from (as described above), B is the set of children 

sampled for freshening, W1i is the weight for child i for the first component as described above, W2i is the 

weight for child i for the second component as described above, ERCA denotes eligible child assessment 

respondent, and ENRCA eligible child assessment nonrespondent. To compute the unweighted rates, W1i

and W2u are set to 1 for each child. 

Table 6-14 presents weighted and unweighted completion rates for freshened children. The 

two components of the completion rates are presented separately in table 6-14. The overall completion 

rates (i.e., the third set of rates in the table) are the products of the two components. The first component 

is separated into a school term and a child term as described earlier. For this component, the completion 

rate is defined as the freshening completion rates, as opposed to the survey instrument completion rates 

found in the second component. The weighted freshening completion rate for children in schools targeted 

for freshening (the school term) is 67.2 percent. As part of the freshening process, schools were asked to 

prepare an alphabetic roster of students enrolled in first grade. These schools were also requested to 

identify which students did not attend kindergarten the previous year. Schools did not participate in the 
6-27 

Table 6-14. Number of completed child-level cases and child-level completion rates for children 

sampled in first grade: School year 2003–04 

 Completion rates1

Category Completes Weighted Unweighted

First component (first-grade sample freshening) 7,032 66.0 79.1

 School term2

 7,089 67.2 79.7

 Child term3

 7,135 98.2 99.2

Second component (fifth-grade data collection)4

 

 Child assessment5

 86 78.6 86.0

 Parent interview6

 83 81.9 83.0

 School administrator questionnaire7

 86 74.4 83.5

 Facilities check list7

 89 77.8 86.4

 Student records abstract7

 75 67.0 72.8

 Teacher-level questionnaire7

 87 81.2 87.9

 Reading teacher questionnaire (child level)7

 84 79.8 84.8

 Mathematics teacher questionnaire 

 (child level)7

41

81.0 83.7

 Science teacher questionnaire (child level)7

 40 75.2 80.0

 Special education part A7

 15 100.0 100.0

 Special education part B7

 14 93.4 93.3

Overall completion rates 

 Child assessment5

 86 51.9 68.0

 Parent interview6

 83 54.0 65.6

 School administrator questionnaire7

 86 49.1 66.0

 Facilities check list7

 89 51.3 68.3

 Student records abstract7

 75 44.2 57.6

 Teacher-level questionnaire7

 87 53.6 69.5

 Reading teacher questionnaire (child level)7

 84 52.7 67.0

 Mathematics teacher questionnaire 

 (child level)7

41

53.5 66.2

 Science teacher questionnaire (child level)7 40 49.6 63.2

 Special education part A7

 15 66.0 79.1

 Special education part B7

 14 61.6 73.8 1 In the first component, this is the completion rate for freshening. In the second component, this is the completion rate for the survey instruments. 

The product of the two components is the overall completion rate for the survey instruments. 

2 The freshening completes and completion rates for children in schools targeted for freshening. 3 The freshening completes and completion rates for children in schools that agreed to the freshening procedure. 4 The number of children sampled via the freshening procedure was 165 but not all were eligible for the fifth-grade data collection. Some became 

ineligible by the time of the fifth-grade data collection, and some were movers not subsampled for followup. The denominator of the completion 

rate also varies according to the instrument. For example, children who were homeschooled were not eligible to have school and teacher 

questionnaires. The denominator for each instrument can be obtained by dividing the number of completes by the unweighted completion rate. 5 Reading, mathematics, or science assessment was scorable, or child was disabled and could not be assessed. 6 Family structure portion of parent interview was completed. 

7 A completed questionnaire was defined as one that was not completely left blank. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 
6-28 

freshening process because they either refused or were unable to provide the requested information. 

Within the schools that agreed to freshen, the freshening completion rate is 98.2 percent, the slight loss 

due to children who transferred to other schools (the child term). Multiplying these two terms together 

gives a first component completion rate of 66.0 percent. Note that the first component rate for spring-fifth 

grade is not identical to the first component rate for spring-first grade and spring-third grade because of 

the exclusion of children in special groups as explained in section 3.6. 

The second component varies by survey instrument. The rates for the paper-and-pencil 

instruments range from 67.0 percent for the student records abstract to 100.0 percent for the special 

education questionnaire part A. The rate for the child assessment at 78.6 percent is about 6 points lower 

than for the kindergarten sample, and the parent interview, at 81.9 percent, is about 7 points lower. The 

rates for the school instruments and the student records abstract are also lower than for the kindergarten 

sample, but by a smaller amount. The rates for some of the teacher instruments are higher than for the 

kindergarten sample. The larger difference in the parent interview rates, as compared with the difference 

in the rates for the child assessment, is due to the different number of followups given the two groups of 

parents. For parents of children sampled in first grade, the fifth-grade data collection was the second 

followup, while most parents of children sampled in kindergarten had at least four followups (springkindergarten, spring-first grade, spring-third grade, and spring-fifth grade). The number of followups does 

not affect the completion rate of the child assessment in the same way since the child assessment was 

done in the school for most children and hence could be perceived as part of school activities. The final 

completion rate for each instrument is the product of the two components. Because of the low rates at the 

first stage, these range from a high of 53.6 percent for the teacher-level questionnaire to a low of 44.2 

percent for the student records abstract. 

6.2.3 Spring-Fifth Grade Completion Rates for All Children 

To compute the fifth-grade completion rate for the combined set of children sampled in the 

base year and children sampled in first grade, the fifth-grade completion rate for each group is weighted 

by the proportion of all children in that group, and the two weighted fifth-grade completion rates were
6-29 

added together. For example, the weighted fifth-grade completion rate for the child assessment (CA) was 

computed as 

1

, ,1

1 1

∈ ∈

∈∪ ∈∪

=× + ×

∑ ∑

∑ ∑

i i

i BY i ST

CA CA BY CA ST

i i

i BY ST i BY ST

W W

rr r

W W

where BY denotes base year, 1ST denotes first grade, rCA,BY is the child assessment completion rate for 

children sampled in the base year, rCA,1ST is the child assessment completion rate for children sampled in 

first grade, and Wi is the final weight (C6CW0 for the child assessment) for child i. 

To get the weighted fifth-grade completion rate for the child assessment (which is 84.7 

percent for children sampled in the base year and 51.9 percent for children sampled in first grade), the 

weighted proportion of children who were sampled in the base year was 0.9762; the weighted proportion 

of children who were sampled in first grade was 0.0238. The fifth-grade weighted completion rate for the 

child assessment was 0.847×0.9762+0.519×0.0238=0.839, or 83.9 percent. 

Table 6-15 presents final spring-fifth grade completion rates for children sampled in 

kindergarten, children sampled in first grade, and all children combined. Because children sampled in first 

grade represent such a small fraction of the total population of children, their inclusion in the computation 

of the completion rate brings down the rates for all children by less than one percent relative to the rates 

for children sampled in kindergarten, even though the completion rates for children sampled in first grade 

are much lower than the kindergarten rates. The spring-fifth grade overall completion rates for the child 

assessment and the parent interview are 83.9 percent and 88.3 percent, respectively. These rates are 

higher than in third grade by about 4 percentage points for the child assessment and by about 11 

percentage points for the parent interview. In all tables, the unweighted completion rates are almost 

always higher than the weighted completion rates, by as much as 23 percent at the overall level. Where 

there is a large difference, it is due to movers who have larger weights and higher nonresponse rates than 

nonmovers. The weights of the movers were increased to account for the subsampling of movers. They 

also responded at a much lower rate than nonmovers, as shown earlier in tables 6-5 to 6-8. This difference 

is larger than in previous years because movers in fifth grade have much larger weights than in previous 

years (many more movers were not included in fifth grade, necessitating larger adjustment factors). Note 

that the unweighted completion rates follow the traditional ECLS-K pattern, that is, rates for the child 
6-30 

Table 6-15. Number of completed child-level cases and child-level completion rates, for children sampled in kindergarten 

survey instruments: School year 2003–04 

Children sampled in kindergarten Children sampled in first grade All c

 Completion rates Completion rates C

Survey instrument Completes Weighted Unweighted Completes Weighted Unweighted Completes WeigChild assessment1 11,260 84.7 93.6 86 51.9 68.0 11,346 

Parent interview2 10,913 89.1 90.7 83 54.0 65.6 10,996 

School administrator 

questionnaire3 10,937 77.1 89.6 86 49.1 66.0 11,023 

Facilities check list3 11,154 78.8 91.4 89 51.3 68.3 11,243 

Student records abstract3 10,015 69.9 82.1 75 44.2 57.6 10,090 

Teacher-level questionnaire3 10,872 79.9 90.6 87 53.6 69.5 10,959 

Reading teacher 

questionnaire (child level)3 10,793 79.3 90.0 84 52.7 67.0 10,877 

Mathematics teacher 

questionnaire (child level)3 5,339 78.1 89.3 

41 

53.5 66.2 5,380 

Science teacher 

questionnaire (child level)3 5,405 79.5 89.9 40 49.6 63.2 5,445 

Special education part A3 960 92.2 93.8 15 66.0 79.1 975 

Special education part B3 967 93.7 94.4 14 61.6 73.8 981 

1 Reading, mathematics, or science assessment was scorable, or child was disabled and could not be assessed. 2 Family structure portion of parent interview was completed. 3 A completed questionnaire was defined as one that was not left completely blank. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spri
6-31 

assessment are higher than rates for the parent interview (93.4 percent for the child assessment and 90.5 

percent for the parent interview). This is again due to movers with large weights and to the fact that there 

are more parent-responding movers than child-responding movers. Thus, the weighted completion rates 

are higher for the parent interview than for the child assessment. 

Table 6-16 shows the completion rates for the child assessment, the parent interviews, and 

the school and teacher instruments for children who have nonzero child weights (C6CW0>0). These are 

children whose spring-fifth grade reading, mathematics, or science assessments were scorable, or children 

who could not be assessed because of disabilities. These conditioned completion rates are useful to 

analysts who want to assess the relationship between the different instruments in term of participation. 

The completion rates from the different instruments are dependent in that if data from one instrument are 

missing (e.g., parent instrument) it is likely that data from other instruments are also missing. (e.g., school 

administrator questionnaire). The conditioned completion rate for the child assessment is by definition 

100 percent. The rate slightly less than 100 percent, shown when children sampled in kindergarten are 

combined with children sampled in first grade, is due to the school freshening nonresponse for children 

sampled in first grade. 

When the completion rates are conditioned on the presence of the child weight, they are at 

least 13 points higher than the unconditional completion rates for all instruments but the parent interview 

and the special education questionnaires. For these last two instruments, the difference between the 

number of completes for the conditional and unconditional rates is very small; hence the conditional rates 

are not affected as much as for the other instruments. For the parent interview, the unconditional rate is 

fairly high for the reason explained earlier; that is, movers in fifth grade have much larger weights than in 

previous years and there are more parent-responding movers than child-responding movers (the weighted 

completion rates are higher for the parent interview than for the child assessment). This results in the 

smaller difference between conditional and unconditional of about 4 percent. For all the other 

instruments, the conditional completion rates are higher by 13.6 points for the child-level science teacher 

questionnaire and as high as 17.2 points for the facilities checklist. 
6-32 

Table 6-16. Number of completed child-level cases and child-level completion rates, for children with scorable reading, m

assessment or children not assessed due to disabilities, by survey instruments: School year 2003–04 

Children sampled in kindergarten Children sampled in first grade All c

 Completion rates Completion rates C

Survey instrument Completes Weighted Unweighted Completes Weighted Unweighted Completes WeigChild assessment1 11,260 100.0 100.0 86 78.3 86.8 11,346 

Parent interview2 10,445 92.4 92.8 77 71.5 77.6 10,522 

School administrator 

questionnaire3 10,794 93.4 96.1 83 74.7 83.7 10,877 

Facilities check list3 11,015 95.7 98.0 86 78.3 86.8 11,101 

Student records abstract3 9,986 85.6 88.9 74 68.3 74.6 10,060 

Teacher-level questionnaire3 10,799 93.4 96.1 84 77.3 84.8 10,883 

Reading teacher 

questionnaire (child level)3 10,774 93.3 95.9 84 77.3 84.8 10,858 

Mathematics teacher questionnaire 

 (child level)3 5,331 92.7 95.5 

41 

76.2 82.7 5,372 

Science teacher 

questionnaire (child level)3 5,394 92.8 95.4 

40 

74.9 80.7 5,434 

Special education part A3 947 93.0 94.1 14 78.3 86.8 961 

Special education part B3 959 95.1 95.3 14 78.3 86.8 973 

1 Reading, mathematics, or science assessment was scorable, or child was disabled and could not be assessed. 2 Family structure portion of parent interview was completed. 3 A completed questionnaire was defined as one that was not left completely blank. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spri
6-33 

As explained in section 3.6, four groups of children were excluded from the fifth-grade data 

collection. These are (1) children who became ineligible in an earlier round (because they had died or 

moved out of the country), (2) children who were subsampled out in previous rounds because they had 

moved out of the original schools and were not subsampled to be followed, (3) children whose parents 

emphatically refused to cooperate (hard refusals) in any of the data collection rounds since springkindergarten, and (4) children eligible for the third-grade sample for whom there are neither first-grade 

nor third-grade data. Table 6-17 shows the completion rates for all instruments had children in the last 

two exclusion groups been counted as nonrespondents. These are children who would have been eligible 

for the fifth-grade collection but past experience showed that they would most likely be nonrespondents. 

When compared to table 6-15, the completion rates for all instruments in table 6-17 are lower, as 

expected, but only by about 2 percent, with the smallest difference for the student records abstract and 

largest difference for the parent interview. Note that the rates for mathematics and science teacher appear 

to be unchanged. Recall that only about half of the children had a mathematics teacher questionnaire 

filled out for them and the other half a science teacher questionnaire. Since the mathematics/science 

sampling flags were not assigned to children not included in the sample, we were not able to compute a 

correct completion rate for these two instruments for these children. But the pattern of completion rates 

would be the same as for the other instruments and we would expect a drop of about 2 percent for the 

mathematics and science teacher questionnaire. 

6.3 Unit Nonresponse Bias Analysis 

Among all the instruments administered in the fifth-grade data collection, the student records 

abstract (SRA) is the only instrument with an unweighted unit completion rate lower than 85 percent. The 

unweighted completion rate for this instrument is 82.1 percent for children sampled in kindergarten, 57.6 

percent for children sampled in first grade, for an overall unweighted response rate of 81.9 percent (see 

table 6-15). When these rates were computed for children with scorable reading, mathematics or science 

assessment or children not assessed due to disabilities, the unweighted completion rate is much higher 

(88.8 percent overall as shown in table 6-16). 

An analysis was undertaken for the SRA to examine the potential nonresponse bias that may 

exist for this instrument. Methods for nonresponse bias analysis in this situation include (1) comparisons 

of respondents and nonrespondents using available sample frame, (2) multivariate analysis to identify the 

characteristics of cases most likely to respond, and (3) comparison of respondents to known population 

characteristics from external sources. 
6-34 

Table 6-17. Number of completed child-level cases and child-level completion rates, if excluded children were fielded, b

School year 2003–04 

Children sampled in kindergarten Children sampled in first grade All c

 Completion rates Completion rates C

Survey instrument Completes Weighted Unweighted Completes Weighted Unweighted Completes WeigChild assessment1 11,260 82.7 86.7 86 47.6 57.7 11,346 

Parent interview2 10,913 87.0 84.0 83 49.7 55.8 10,996 

School administrator 

questionnaire3 10,937 75.3 83.3 86 45.2 56.3 11,023 

Facilities check list3 11,154 77.0 84.9 89 47.3 58.2 11,243 

Student records abstract3 10,015 68.3 76.3 75 40.7 49.1 10,090 

Teacher-level questionnaire3 10,872 78.0 84.1 87 49.2 58.9 10,959 

Reading teacher 

questionnaire (child level)3 10,793 77.4 83.5 84 48.3 56.8 10,877 

Mathematics teacher questionnaire 

 (child level)3 5,339 78.1 89.3 

41 

52.1 65.2 5,380 

Science teacher 

questionnaire (child level)3 5,405 79.5 89.9 40 48.3 62.3 5,445 

Special education part A3 960 92.2 93.8 15 64.3 77.9 975 

Special education part B3 967 93.7 94.4 14 60.0 72.7 981 

1 Reading, mathematics, or science assessment was scorable, or child was disabled and could not be assessed. 2 Family structure portion of parent interview was completed. 3 A completed questionnaire was defined as one that was not left completely blank. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spri
6-35

There are 45 items in the SRA and all but seven are suppressed in the public-release data 

file. The suppressed items are dates when the child entered and left school, whether school kept 

attendance record, presence of IEP records, and items on learning disabilities. The seven items that 

are not suppressed are items on absence and tardiness. Because there are no school frames that report 

this type of data and there are no external sources that include these data, the nonresponse bias 

analysis is limited to analyzing the relationship between the SRA response status and response 

indicators such as key school characteristics. The Chi-square Automatic Interaction Detector 

(CHAID) was used to look at the relationship on variables with known values for both respondents 

and nonrespondents. CHAID is a classification algorithm that uses chi-square tests to divide the 

sample into subgroups that are related to whether the unit responds. 

The analysis in CHAID begins by dividing the sample into two or more groups based on 

the categories of the best predictor of response. Each of these groups is divided into smaller 

subgroups based on the available predictors at each level. The splitting process continues until no 

statistically significant predictor remains. The CHAID software displays the final subgroups in the 

form of a tree diagram whose branches correspond to the groups, showing all potential response 

predictors. The resulting classification tree reveals the response cells, as defined by combination of 

variables, which identify cells with the lowest response rate. In other words, CHAID divides the 

sample into cells so that the response rate within cells is as constant as possible, while the response 

rate between cells is as different as possible. 

The response indicators used in CHAID are school characteristics that are known for the 

respondents and nonrespondents to the SRA: census region, school affiliation, type of locale, total 

enrollment, and percent non-White enrolled. Exhibit 6-1 shows the CHAID tree when the SRA 

response status is analyzed together with these response indicators. Each box represents a group of 

SRAs with specified characteristics, the weighted completion rate and the number of fifth graders 

with completed SRA (i.e., SRA respondents). The first branch of the tree shows that census region is 

the first indicator that was used to divide the SRAs into four groups with response patterns that are 

very different from each other (with significance level of 0.05). The response rates in the four groups 

are 71.5 in the Northeast, 89.7 percent in the Midwest, 84.4 percent in the South, and 80.7 percent in 

the West. In the next branch of the tree, SRAs in the Northeast could be further split into four groups 

with very different response pattern by percent of non-White enrolled in the school, while those in the 

Midwest were subdivided into two groups with differential response patterns based on the type of 
6-36

locale. Thus, the tree continues and ends after the sample was divided into 49 cells with varying 

response patterns. 

All variables used in the analysis show up in the tree as significant predictors of response 

patterns. Some are more prevalent than others. For example, school affiliation is not a significant 

indicator of response in the South while the total school enrollment is a significant predictor of 

response for all instances in the South. School affiliation is a significant response indicator in two 

groups of SRAs in the Midwest (SRAs from schools in large and mid-size cities/suburb of large and 

mid-size cities/large towns where the percent of non-White enrolled is less than 50 percent, and where 

the total school enrollment is less than 750; and, SRAs from schools in small towns and rural areas 

where the percent of non-White enrolled is less than 50 percent), and in the West (where schools have 

11 or more but less than 50 percent of non-White enrolled in large and mid-size cities, small towns 

and rural areas, and where schools have 50 or more but less than 90 percent of non-White enrolled). 

School affiliation is significant in only one group in the Northeast (where schools have 10 percent or 

less non-White enrolled). The other variables (total school enrollment, percent non-White enrolled, 

and type of locale) are all good predictors of response. The lower completion rates are in the 

Northeast, in schools that are nonreligious private and in schools with high enrollment 

Potential nonresponse bias that may exist is likely to have been lessened by the 

weighting adjustment procedures described in chapter 7. School affiliation, type of locale and region 

were used as raking dimensions in the last step of the weighting procedures for all weights including 

the child-level weight C6CW0, the most appropriate weight to use in the analysis of the SRA data. 

6.4 Item Response Rates 

In the ECLS-K, as in most surveys, the responses to some data items are not obtained for 

all interviews. There are numerous reasons for item nonresponse. Some respondents do not know the 

answer for the item or do not wish to respond for other reasons. Some item nonresponse arises when 

an interview is interrupted and not continued later, leaving items at the end of the interview blank. 

Item nonresponse may also be encountered because responses provided by the respondent are not 

internally consistent, and this inconsistency is not discovered until after the interview is completed. In 

these cases, the items that were not internally consistent were set to missing. 
6-37

Exhibit 6-1. Relationship between the student record abstract response status and school 

characteristics: School year 2003–04 

Census region 

Northeast 71.5% 

Percent nonwhite 

enrolled 

 

 (1,647) 0-10 78.1% School affiliation 

 (903) Public 79.1% Total enrolled 

 (712) 1-499 75.0% Type of locale 

 (456) City/suburb/ 66.8% 

 large town (173) Cell 1 

 Small town/ 83.0% 

 rural (283) Cell 2 

 >=500 85.9% Cell 3 

 (256) 

 Catholic 82.2% Cell 4 

 (156) 

 Other 48.4% Cell 5 

 private (35) 

 11-49 63.6% Type of locale 

 (334) City 87.7% Total enrolled 

 (95) 1-749 83.9% Cell 6 

 (65) 

 >=750 100.0% Cell 7 

 (30) 

 Suburb/ 57.7% Cell 8 

 town/rural (239) 

 50-89 83.9% Total enrolled 

 (243) 1-499 92.9% Cell 9 

 (124) 

 >=500 78.1% Cell 10 

 (119) 

 90-100 52.6% Total enrolled 

 (167) 1-299 76.0% Cell 11 

 (31) 

 300-749 49.1% Cell 12 

 (80) 

 >=700 49.6% Cell 13 

 (56) 

See note at end of exhibit. 
6-38

Exhibit 6-1. Relationship between the student record abstract response status and school 

characteristics: School year 2003–04—Continued 

Census region 

Midwest 89.7% Type of locale 

 (2,799) 

City/suburb/ 86.5% 

Percent nonwhite 

enrolled 

 

 large town (1,752) 0-49 90.1% Total enrolled 

 (1,432) 1-749 89.2% School affiliation 

 (1,277) Public 87.6% Cell 14 

 (867) 

 Private 94.9% Cell 15 

 (410) 

 >=750 99.7% Cell 16 

 (155) 

 50-89 82.2% Total enrolled 

 (222) 1-499 76.7% Cell 17 

 (150) 

 >=500 96.0% Cell 18 

 (72) 

 90-100 67.2% Cell 19 

 (98) 

 

Small town/ 96.0% 

Percent nonwhite 

enrolled 

 

 rural (1,047) 0-49 95.9% School affiliation 

 (872) Public 97.5% Cell 20 

 (722) 

 Catholic 97.1% Cell 21 

 (117) 

 Other 65.6% Cell 22 

 private (33) 

 50-100 96.9% Cell 23 

 (175) 

See note at end of exhibit. 
6-39

Exhibit 6-1. Relationship between the student record abstract response status and school 

characteristics: School year 2003–04—Continued 

Census region 

South 84.4% 

Percent nonwhite 

enrolled 

 

 (3,401) 0-10 91.5% Total enrolled 

 (782) 1-299 75.5% Cell 24 

 (125) 

 >=300 94.4% Type of locale 

 (657) City/ 98.8% Cell 25 

 unknown (151) 

 Suburb/town/ 93.6% Cell 26 

 rural (506) 

 11-49 83.7% Total enrolled 

 (1,345) 1-299 96.8% Type of locale 

 (227) City/suburb/ 94.1% Cell 27 

 large town (125) 

 Small town/ 100.0% Cell 28 

 rural (102) 

 >=300 82.1% Cell 29 

 (1,118) 

 50-89 81.4% Total enrolled 

 (602) 1-749 86.4% Type of locale 

 (481) City/suburb/ 84.3% Cell 30 

 large town (392) 

 Small town/ 96.6% Cell 31 

 rural (89) 

 >=750 64.6% Cell 32 

 (121) 

 90-100/ 82.2% Total enrolled 

 unknown (672) 1-299 71.2% Cell 33 

 (71) 

 300-499 94.4% Type of locale 

 (250) City/suburb/ 93.6% Cell 34 

 large town (200) 

 Small town/ 100.0% Cell 35 

 rural (50) 

 500-749 86.1% Cell 36 

 (219) 

 >=750 68.3% Cell 37 

 (132) 

See note at end of exhibit. 
6-40

Exhibit 6-1. Relationship between the student record abstract response status and school 

characteristics: School year 2003–04—Continued 

Census region 

West 80.7% 

Percent nonwhite 

enrolled 

 

 (2,243) 0-10 98.9% Cell 38 

 (160) 

 11-49 82.9% Type of locale 

 (788) City/ 89.6% School affiliation 

 small town/ (488) Public 90.6% Total enrolled 

 rural (305) 1-299 100.0% Cell 39 

 (54) 

 300-499 85.8% Cell 40 

 (121) 

 >=500 92.6% Cell 41 

 (130) 

 Catholic 99.4% Cell 42 

 (122) 

 Other 74.1% Cell 43 

 private (61) 

 Suburb/ 75.6% Total enrolled 

 large town (300) 1-499 84.4% Cell 44 

 (167) 

 >=500 67.5% Cell 45 

 (133) 

 50-89 74.5% School affiliation 

 (611) Public 74.2% Cell 46 

 (515) 

 Catholic/ 79.5% Type of locale 

 other private (96) City 97.5% Cell 47 

 (50) 

 Suburb/ 65.7% Cell 48 

 Town/ (46) 

 Rural 

 90-100 78.9% Cell 49 

 (684) 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 

Class of 1998–99 (ECLS-K), spring 2004. 
6-41

Every item in the ECLS-K data file has values that indicate whether the respondent did not know the 

answer to the item (-8), or refused to give an answer (-7). The value -9 is used in all other cases where 

the answer is left blank or set to missing due to reasons mentioned above (described in the data file 

as “Not ascertained”). However, where an item is left blank due to a valid skip pattern, this is 

indicated by the value −1. Chapter 7 of the ECLS-K Combined User’s Manual for the ECLS-K Fifth 

Grade Public-Use Data Files and Electronic Codebooks. (NCES 2006–032) (Tourangeau et al. 

forthcoming) discusses in detail these special values. For each survey item, the response rate was 

computed as the unweighted number of responses not equal to any of the special values (−1, −7, −8, 

or −9) divided by the unweighted number of responses not equal to −1. Of all the ECLS-K 

instruments, only the child assessment and the parent interview have a sizable number of items with 

special values −7 (“Refused”) or −8 (“Don’t know”). Table 6-18 shows the unweighted distribution of 

the nonresponse values for each instrument. 

Table 6-18. Number of survey items and percent of nonresponse values: School year 2003–04 

 Items with nonresponse value (in percent) 

 Number of −7 −8 −9

Survey instrument items (Refused) (Don’t know) (Not ascertained)

 Total 1,758 17.8 24.3 77.9

Child assessment 85 21.2 21.2 76.5

Parent interview 786 37.5 48.7 55.2

School administrator questionnaire 215 0.0 5.6 98.6

Facilities checklist 31 0.0 0.0 96.8

Student records abstract 43 0.0 0.0 95.3

Teacher-level questionnaire 183 0.0 3.3 99.5

Child-level reading teacher questionnaire 107 0.0 0.0 99.1

Child-level mathematics teacher questionnaire 75 0.0 4.0 98.7

Child-level science teacher questionnaire 62 0.0 8.1 96.8

Special education part A 53 0.0 0.0 92.5

Special education part B 119 0.0 0.8 98.3

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 

Class of 1998–99 (ECLS-K), spring 2004. 

For most of the data items collected in the ECLS-K, the item response rate unweighted 

was very high, with an overall median item response rate of 98.8 percent. The median item response 

rate for each of the instruments ranges from 91.2 for the student records abstract to 99.8 for the child 

assessment. Table 6-19 shows the number of items, the median item response rate, the lowest item 

response rate, the highest item response rate, and the number of items with response rates of less than 

85 percent for all instruments. Items with less than 85 percent response rates are listed in table 6-20 
6-42

by instrument and in ascending order of item response rate. The number of cases for which each item 

was attempted is also shown in this table. The tables in this chapter show the item response rates for 

items on the restricted-use file. Rates in tables 6-19 and 6-20 are unweighted. 

Table 6-19. Item response rates for items on the ECLS-K fifth-grade restricted-use data file: School 

year 2003–04 

Instrument 

Number of 

items

Median 

response

rate 

(percent)

Lowest 

response 

rate

(percent)

Highest 

response 

rate 

(percent) 

Number of 

items with 

response rate

 < 85 percent

 Total 1,758 98.8 0.0 100.0 163

Child assessment data 85 99.8 93.3 100.0 0

Parent interview 786 99.6 0.0 100.0 96

School administrator questionnaire 215 98.0 49.1 100.0 33

Facilities checklist 31 95.2 34.7 100.0 12

Student records abstract 43 91.2 43.9 100.0 14

Teacher-level questionnaire 183 98.2 63.5 100.0 5

Child-level reading teacher questionnaire 107 98.6 89.3 100.0 0

Child-level mathematics teacher questionnaire 75 99.3 83.3 100.0 1

Child-level science teacher questionnaire 62 99.0 93.4 100.0 0

Special education part A 53 97.0 92.3 100.0 0

Special education part B 119 98.5 76.6 100.0 2

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 

Class of 1998–99 (ECLS-K), spring 2004. 
6-43

Table 6-20. Items on the ECLS-K fifth-grade restricted-use data file with less than 85 percent 

response rates: School year 2003–04 

Variable name Description 

Number 

eligible

Item response rate 

(percent)

Parent interview 

P6DGNELE P6 CHQ137 DIAGNOSIS MADE BEFORE ELEM SCH 59 0.0

P6DGHR02 P6 CHQ226 DIAGNOSIS MADE BEFORE 2002 60 0.0

P6DGNELS P6 CHQ227 DIAGNOSIS MADE BEFORE ELEM SCH 60 0.0

P6IMPT02 P6 CHQ252 WAS IT BEFORE 2002 60 0.0

P6IMPELM P6 CHQ253 WAS IT BEFORE ELEM SCHOOL 60 0.0

P6DNBF02 P6 CHQ346 DIAGNOSIS MADE BEFORE 2002 61 0.0

P6BELDGN P6 CHQ347 DIAGNOSIS MADE BEFORE ELEM SCH 61 0.0

P6SERVRV P6 CHQ536 SERVICES RCVD BEFORE 2002 66 0.0

P6DGSYR P6 CHQ376 DIAGNOSIS MADE BEFORE 2002 62 1.6

P6DGSESC P6 CHQ377 DIAGNOSIS MADE BEFORE ELEM SCH 62 1.6

P6DGN02 P6 CHQ136 DIAGNOSIS MADE BEFORE 2002 60 1.7

P6SRVRCV P6 CHQ537 SRVCS RCVD BEFORE ELEM SCHOOL 66 3.0

P6DGSP02 P6 CHQ186 DIAGNOSIS MADE BEFORE 2002 63 4.8

P6ELMDGN P6 CHQ187 DIAGNOSIS MADE BEFORE ELEM SCH 63 4.8

P6REL_2 P6 FSQ130 2ND PERSON’S RELATION TYPE 10,996 5.8

P6CHI_N1 P6 NRQ050 CONTACT W/BIOMOM SAME AS CHD1 67 7.5

P6CHI_N3 P6 NRQ050 CONTACT W/BIODAD SAME AS CHD1 193 9.8

P6DGBF02 P6 CHQ314 DIAGNOSIS MADE BEFORE 2002 71 12.7

P6IMPLNT P6 CHQ251 YEAR OF IMPLANT 69 13.0

P6CLRUSE P6 CHQ254 USE OF COCHLEAR IMPLANT IN SCH 69 13.0

P6DGBFEL P6 CHQ315 DIAGNOSIS MADE BEFORE ELEM SCH 71 15.5

P6DIAG02 P6 CHQ076 DIAGNOSIS MADE BEFORE 2002 73 17.8

P6DIAGEL P6 CHQ077 DIAGNOSIS MADE BEFORE ELEM SCH 73 17.8

P6HEARS2 P6 CHQ260 DEVICE EFFECT ON CHD’S HEARING 76 21.1

P6AGREE3 P6 NRQ264 AGREEMENT W/ ADOPTIVE FATHER 282 38.3

P6HEARS P6 CHQ230 DEGREE OF CHILD’S DEAFNESS 111 39.6

P6OFTCUT P6 FDQ230 FREQ CHILD SKIP MEAL-NO FOOD 131 42.7

P6NOTEA2 P6 FDQ200 FREQ NOT EAT ENTIRE DAY 138 45.7

P6YYDIA5 P6 CHQ225 YR AT 1ST DIAGNOSIS-HEARING 111 45.9

P6HEARAI P6 CHQ240 IF CHILD WEARS HEARING AID 111 45.9

P6COCHLE P6 CHQ250 IF CHILD HAS COCHLEAR IMPLANTS 111 45.9

P6LRSRVY P6 CHQ535 YR LAST RECEIVED SERVICES 152 56.6

P6MTHAGE P6 COQ010 AGE OF MOTHR WHEN MOVED TO U S 213 56.8

P6AGREE1 P6 NRQ264 AGREEMENT W/ BIOLOGICAL FATHER 145 57.2

P6NHROTH P6 CCQ250 # HRS/WK RECVS CARE OTHER NREL 95 57.9

P6CHROTH P6 CCQ403 # HRS/WK AT OTHER PROGRAMS 105 58.1

P6SCHSA P6 WPQ106 ATTEND SCHOOL -SA 187 58.8

P6LKSA P6 WPQ106 LOOK FOR WORK - SA 187 58.8

P6OTHSA P6 WPQ106 OTHER - SA 187 58.8

P6UNJBSA P6 WPQ106 WORK FOR NO PAY - SA 187 58.8

P6PDJBSA P6 WPQ106 WORK FOR PAY - SA 187 58.8

P6DIFFH3 P6 CHQ210 IF HEAR DIFFICULTY DIAGNOSED 149 59.1

See note at end of table. 
6-44

Table 6-20. Items on the ECLS-K fifth-grade restricted-use data file with less than 85 percent 

response rates: School year 2003–04—Continued 

Variable name Description 

Number 

eligible

Item response rate 

(percent)

Parent interview (continued) 

P6FTHAGE P6 COQ025 AGE OF FATHR WHEN MOVED TO U S 619 62.0

P6DIFFH2 P6 CHQ200 IF HEAR DIFFICULTY EVALUATED 185 66.5

P6FMTHRS P6 CHQ780 REASON FOR FAMILY THERAPY 237 68.8

P6SEEHLP P6 CHQ800 TIMES FAMILY SAW HELP 237 68.8

P6SEEPSY P6 CHQ790A FAMILY SEE A PSYCHIATRIST 237 69.2

P6SEPSYC P6 CHQ790B FAMILY SEE A PSYCHOLOGIST 237 69.2

P6SEESOC P6 CHQ790C FAMILY SEE A SOCIAL WORKER 237 69.2

P6SEECSL P6 CHQ790D FAMILY SEE A COUNSELOR 237 69.2

P6SEESOM P6 CHQ790E FAMILY SEE SOMEONE ELSE 237 69.2

P6MTHCTZ P6 COQ015 IS MOTHER A U S CITIZEN 213 70.0

P6FPT_2 P6 PEQ040 PERS 2 COURSE FULL/PART TIME 211 70.6

P6DGNACT P6 CHQ120 WHAT 1ST DIAGNOSIS - ACTIVITY 226 72.6

P6YYDIA2 P6 CHQ135 YR AT 1ST DIAGNOSIS-ACTIVITY 226 73.5

P6TAK_2 P6 EMQ100 PERS 2 JOB AVAILABLE LAST WEEK 234 73.5

P6DO3_2 P6 EMQ070 PERS 2 CHKD W/EMPLOYR DIRECTLY 248 73.8

P6DO4_2 P6 EMQ070 PERS 2 CHKD W/FRIENDS & REL 248 73.8

P6DO2_2 P6 EMQ070 PERS 2 CHKD W/PRIV EMP AGNCY 248 73.8

P6DO1_2 P6 EMQ070 PERS 2 CHKD W/PUB EMPL AGNCY 248 73.8

P6DO7_2 P6 EMQ070 PERS 2 DID SOMETHING ELSE 248 73.8

P6DO5_2 P6 EMQ070 PERS 2 PLACED OR ANSWERED ADS 248 73.8

P6DO6_2 P6 EMQ070 PERS 2 READ WANT ADS 248 73.8

P6BESTEY P6 CHQ320 WHAT CAN CHILD BEST SEE 372 74.5

P6DGNBEH P6 CHQ337 1ST DIAGNOSIS-BEHAVIOR 244 74.6

P6DGBEYY P6 CHQ345 YR AT 1ST DIAGNOSIS-BEHAVIOR 244 75.0

P6TINCTH P6 PAQ120 TOTAL HOUSEHOLD INCOME ($-LOW) 1,816 75.7

P6SCHFS P6 WPQ130 ATTEND SCHOOL -FS 327 76.5

P6LKFS P6 WPQ130 LOOK FOR WORK - FS 327 76.5

P6OTHFS P6 WPQ130 OTHER - FS 327 76.5

P6UNJBFS P6 WPQ130 WORK FOR NO PAY - FS 327 76.5

P6PDJBFS P6 WPQ130 WORK FOR PAY - FS 327 76.5

P6TAK_1 P6 EMQ100 PERS 1 JOB AVAILABLE LAST WEEK 449 77.7

P6CCNMCH P6 CCQ400 CC # CHILDREN AMOUNT FOR 194 78.4

P6RCNMCH P6 CCQ135 RC # CHILDREN AMOUNT FOR 182 79.1

P6DIABEH P6 CHQ335 BEHAVIOR PROBLEM DIAGNOSED 297 79.5

P6DO3_1 P6 EMQ070 PERS 1 CHKD W/EMPLOYR DIRECTLY 488 79.7

P6DO4_1 P6 EMQ070 PERS 1 CHKD W/FRIENDS & REL 488 79.7

P6DO2_1 P6 EMQ070 PERS 1 CHKD W/PRIV EMP AGNCY 488 79.7

P6DO1_1 P6 EMQ070 PERS 1 CHKD W/PUB EMPL AGNCY 488 79.7

P6DO7_1 P6 EMQ070 PERS 1 DID SOMETHING ELSE 488 79.7

P6DO5_1 P6 EMQ070 PERS 1 PLACED OR ANSWERED ADS 488 79.7

See note at end of table. 
6-45

Table 6-20. Items on the ECLS-K fifth-grade restricted-use data file with less than 85 percent 

response rates: School year 2003–04—Continued 

Variable name Description 

Number 

eligible

Item response rate 

(percent)

Parent interview (continued) 

P6DO6_1 P6 EMQ070 PERS 1 READ WANT ADS 488 79.7

P6PROFFD P6 CHQ110 IF ACTIVITY PROBLEM DIAGNOSED 301 80.4

P6MTHBTH 

P6 COQ005 COUNTRY BIOLOGCAL MTHR WAS 

BRN 1,273 80.5

P6FTHCTZ P6 COQ030 IS FATHER A U S CITIZEN 619 80.6

P6FPT_1 P6 PEQ040 PERS 1 COURSE FULL/PART TIME 498 80.7

P6RCCOST P6 CCQ125 COST OF RELATIVE CARE 301 82.1

P6NCNMCH P6 CCQ240 NC # CHILDREN AMOUNT FOR 219 82.2

P6YYDIA4 P6 CHQ185 YEAR AT 1ST DIAGNOSIS-SPEECH 373 83.1

P6DGNEMO P6 CHQ365 1ST DIAGNOSIS-EMOTIONAL BEH 424 83.3

P6REQSA P6 WPQ105 ANY RQ FOR STATE AID (SA) 491 83.9

P6WHENAF P6 WPQ102 # MNTHS RECEIVED TANF/AFDC 491 84.1

P6MIN_N1 P6 NRQ250 BIOMOM LIVES # MINUTES AWAY 641 84.2

P6TWNCAR P6 CCQ003 TWIN SAME CARE AS CHILD 236 84.7

P6EVCUT P6 FDQ150 FREQ CUT MEAL SIZE 488 84.8

School administrator questionnaire 

S6RTCHPT S6 Q38A2 # REG CLASSROOM TCHR-PART 10,916 49.1

S6READPT S6 Q38E2 # READING TCHR/SPECIALIST-PART 10,916 54.2

S6LIBRPT S6 Q38J2 # LIBRARIANS-PART 10,916 54.7

S6SPEDPT S6 Q38C2 # SPECIAL ED TCHR-PART 10,916 56.8

S6PARAPT S6 Q38I2 # PARAPROFESSIONALS-PART 10,916 57.9

S6ESLPT S6 Q38D2 # ESL/BILINGUAL TCHR-PART 10,916 58.5

S6PRABRK S6 Q19A2 PARTICIPATE ANY SCH BREAKFAST 7,378 58.8

S6PAALUN S6 Q20A2 PARTICIPATE ANY SCH LUNCH 10,916 60.6

S6GIFTPT S6 Q38F2 # GIFTED/TALENTED TCHR-PART 10,916 60.7

S6NURSPT S6 Q38G2 # SCH NURSE HEALTH PROF-PART 10,916 65.2

S6PSYCFL S6 Q38H1 SCH PSYCH/ SOCIAL WORKER-FULL 10,916 65.6

S6MSARPT S6 Q38B2 # GYM DRAMA MUSIC ART TCHR-PART 10,916 67.4

S6OTHER S6 Q15F SCH HAS OTHER REASON 3,499 68.0

S6GIFTFL S6 Q38F1 # GIFTED/TALENTED TCHR-FULL 10,916 72.4

S6NURSFL S6 Q38G1 # SCH NURSE HEALTH PROF-FULL 10,916 73.1

S6PSYCPT S6 Q38H2 SCH PSYCH/SOCIAL WORKER-PART 10,916 74.0

S6ELRPBK S6 Q19C1 ELIGIBLE RED-PRICE BREAKFAST 7,378 74.2

S6ELIBRK S6 Q19B1 ELIGIBLE FOR FREE BREAKFAST 7,378 74.5

S6ELIRED S6 Q20C1 ELIGIBLE IN REDUCED-PRICE LUNCH 10,916 75.9

S6ESLFL S6 Q38D1 # ESL/BILINGUAL TCHR-FULL 10,916 76.5

S6ELILNC S6 Q20B1 ELIGIBLE FOR FREE LUNCH 10,916 76.6

S6PCTMTH S6 Q35B MATHEMATICSEMATICS SKILLS % 10,625 78.5

S6PARPBK S6 Q19C2 PARTICIPATE RED-PRICE BREAKFAST 7,378 78.6

S6PCTRD S6 Q35A READING OR VERBAL SKILLS % 10,625 79.2

S6HWLONG S6 Q39E RESP (NOT PRINCIPAL) YR AT SCH 2,607 79.4

See note at end of table. 
6-46

Table 6-20. Items on the ECLS-K fifth-grade restricted-use data file with less than 85 percent 

response rates: School year 2003–04—Continued 

Variable name Description 

Number 

eligible

Item response rate 

(percent)

School administrator questionnaire (continued) 

S6PARBRK S6 Q19B2 PARTICIPATES IN BREAKFAST 7,378 81.7

S6AMBUSL S6 Q12 TIME LAST BUS AM 10,916 82.0

S6MSARFL S6 Q38B1 # GYM DRAMA MUSIC ART TCHR-FULL 10,916 82.7

S6PARRED S6 Q20C2 PARTICIPATES IN RED-PRICE LUNCH 10,916 83.2

S6PARAFL S6 Q38I1 # PARAPROFESSIONALS-FULL 10,916 83.8

S6LIBRFL S6 Q38J1 # LIBRARIANS-FULL 10,916 84.0

S6PARLNC S6 Q20B2 PARTICIPATES IN FREE LUNCH 10,916 84.5

S6READFL S6 Q38E1 # READING TCHR/SPECIALIST-FULL 10,916 85.0

Facilities checklist 

K6SEXITS K6 Q1E2 SCH CNFRM SECURED EXIT DOORS 3,485 34.7

K6FENCES K6 Q1F2 SCH CNFRM FENCING AROUND SCHOOL 4,669 44.2

K6BARSS K6 Q1D2 SCH CNFRM WINDOW AND DOOR BARS 9,596 45.4

K6METDTS K6 Q1B2 SCH CNFRM METAL DETECTORS 10,568 48.2

K6GUARDS K6 Q1A2 SCH CNFRM SECURITY GUARD 9,902 48.2

K6SIGNOS K6 Q1H2 SCH CNFRM SIGN-IN AT OFFICE 3,083 48.6

K6SCAMS K6 Q1C2 SCH CNFRM SECURITY CAMERAS 8,465 49.2

K6SIGNPS K6 Q1G2 SCH CNFRM SIGN-IN POLICIES 874 53.5

K6SPRNKS K6 Q1L2 SCH CNFRM FIRE SPRINKLERS 5,357 54.2

K6FREXTS K6 Q1K2 SCH CNFRM FIRE EXTINGUISHERS 287 63.8

K6INTCMS K6 Q1I2 SCH CNFRM INTERCOMS 578 70.1

K6ALARMS K6 Q1J2 SCH CNFRM FIRE ALARMS 183 71.6

Student records abstract

U6WHYLFT U6 Q4 WHY CHILD LEFT SCHOOL 171 43.9

U6MMLEFT U6 Q3A MONTH LEFT SCHOOL 171 62.0

U6DDLEFT U6 Q3B DAY LEFT SCHOOL 171 62.0

U6YYLEFT U6 Q3C YEAR LEFT SCHOOL 171 62.0

U6YY01 U6 Q9C3 YEAR 2001-2002 IEP SIGNED 885 63.8

U6MM01 U6 Q9C2 MONTH 2001-2002 IEP SIGNED 885 64.2

U6YY02 U6 Q9B3 YEAR 2002-2003 IEP SIGNED 947 71.0

U6MM02 U6 Q9B2 MONTH 2002-2003 IEP SIGNED 947 71.3

U6IEP01 U6 Q9C1 PRESENCE OF 2001-2002 IEP RECORD 1,081 75.2

U6IEP02 U6 Q9B1 PRESENCE OF 2002-2003 IEP RECORD 1,081 80.9

U6YY03 U6 Q9A3 YEAR 2003-2004 IEP SIGNED 1,049 82.4

U6MM03 U6 Q9A2 MONTH 2003-2004 IEP SIGNED 1,049 82.9

U6BUXTAR U6 Q7C CHILD UNEXCUSED TARDIES 10,064 83.7

U6BXTARD U6 Q7B CHILD EXCUSED TARDIES 10,064 83.9

Teacher-level questionnaire

J61TXFOR J61 Q1H2 TIME FOR FOREIGN LANGUAGE 3,129 63.5

J61TXREF J61 Q1I2 TIME FOR REFERENCE SKILLS 10,512 78.7

J61TXART J61 Q1G2 TIME FOR ART 9,563 80.6

See note at end of table. 
6-47

Table 6-20. Items on the ECLS-K fifth-grade restricted-use data file with less than 85 percent 

response rates: School year 2003–04—Continued 

Variable name Description 

Number 

eligible 

Item response rate 

(percent)

Teacher-level questionnaire (continued)

J61TSTPR J61 Q19 HOURS SPENT IN STD TEST PREP 9,688 81.1

J61TXMUS J61 Q1F2 TIME FOR MUSIC 8,940 82.6

Child-level mathematics teacher questionnaire

M6LNGOS M6 Q17D LANGUAGE OF MATHEMATICS INSTRUCTION 6 83.3

Special education teacher questionnaire B

E6EVLOTH E6 Q22H OTHER EVALUATION 981 76.6

E6DKMTHD E6 Q14M DON’T KNOW METHODS USED 981 81.2

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 

Class of 1998–99 (ECLS-K), spring 2004. 
6-48

This page is intentionally left blank. 
7-1 

7. WEIGHTING AND VARIANCE ESTIMATION 

The ECLS-K data were weighted to compensate for differential probabilities of selection at 

each sampling stage and to adjust for the effects of nonresponse. In the ECLS-K base year, weights were 

computed at the child, school and teacher levels. Estimates using the base year weights are representative 

of all kindergarten children, all schools with kindergarten programs and all kindergarten teachers. After 

the base year, only child-level weights were computed. The use of these weights was essential to produce 

estimates that are representative of the cohort of children who were in kindergarten in 1998–99 or in first 

grade in 1999–2000. In first grade, the sample was freshened with first-graders who had not been sampled 

in kindergarten or first grade. Freshening was not done in third grade or in fifth grade. Estimates from the 

ECLS-K fifth-grade data are representative of the population cohort rather than of all fifth-graders in 

2003–04. While the vast majority of children in fifth grade in the 2003–04 school year are members of the 

cohort, fifth-graders who repeated fourth or fifth grade, children who were homeschooled before enrolling 

in fifth grade, and recent immigrants are not covered. Data were collected from teachers and schools to 

provide important contextual information about the school environment for the sampled children. 

Similarly, home environment data were collected from parents. Data from these sources are not 

representative of all fifth-grade parents, teachers, and schools in 2003–04. For this reason, the only 

weights produced from the study are child-level weights for making statements about children, including 

statements about the parents, teachers, and schools of those children. 

The different types of weights are discussed in section 7.1, followed by a detailed 

description of the computation of the weights in section 7.2. Section 7.3 describes the variance estimation 

methods suitable for the ECLS-K. 

7.1 Types of Weights 

Two sets of weights were computed for fifth grade, cross-sectional and longitudinal. The use 

of these weights is essential to produce estimates that are representative of the cohort of children who 

were in kindergarten in 1998–99 or in first grade in 1999–2000. Since the ECLS-K sample was not 

freshened after the first-grade year with third- or fifth-graders who did not have a chance to be sampled in 

kindergarten or first grade (as was done in first grade), estimates from the ECLS-K third- and fifth-grade 

data are representative of the population cohort rather than all third-graders in 2001–02 or all fifth-graders 
7-2 

in 2003–04. While the vast majority of children in third grade in the 2001–02 school year and in fifth 

grade in the 2003–04 school year are members of the cohort, third-graders who repeated second or third 

grade, fifth-graders who repeated third or fourth grade, and recent immigrants are not covered. The fifth 

grade cross-sectional weights are used for analyses of data from the fifth-grade data collection round. The 

fifth grade longitudinal weights are used for analyses of data from a longitudinal file including fifth-grade 

data in conjunction with data from one or more other rounds. 

As in previous years, there are several survey instruments administered to sampled children 

and their parents, teachers and schools: cognitive and physical assessments for children; self-description 

child questionnaire (third and fifth grade only), parent instruments; several types of teacher instruments 

completed by reading, mathematics, science and special education teachers; and school instruments. The 

stages of base year sampling in conjunction with differential nonresponse at each stage and the diversity 

of survey instruments require that multiple fifth-grade cross-sectional sampling weights be computed for 

use in analyzing the fifth-grade ECLS-K data. Several combinations of kindergarten through fifth-grade 

longitudinal weights were also computed. Exhibit 7-1 summarizes the different types of cross-sectional 

weights. 
7-3 

Exhibit 7-1. ECLS-K fifth-grade cross-sectional weights: School year 2003–04 

Weight To be used for analysis of ...

C6CW0 Fifth-grade direct child assessment data, alone or in conjunction with any combination of 

(a) a limited set of child characteristics (e.g., age, sex, race/ethnicity), (b) teacher-level 

data from any fifth-grade teacher questionnaire without child-level teacher data, or (c) 

data from the school administrator questionnaire or school facilities checklist. 

C6PW0 Fifth-grade parent interview data alone or in combination with (a) fifth-grade child 

assessment data, (b) data from any fifth-grade teacher questionnaire (teacher-level or 

child-level), or (c) data from the school administrator questionnaire or school facilities 

checklist. 

Exception: If data from the parent AND child assessment AND teacher (child- and/or 

teacher-level) are used together, then either C6CPTR0, C6CPTM0 or C6CPTS0 should 

be used. 

C6CPTR0 Fifth-grade direct child assessment data combined with fifth-grade parent interview data 

AND fifth-grade teacher-level data with or without child-level data from the reading 

teacher, alone or in conjunction with data from the school administrator or facilities 

checklist. 

C6CPTM0 Fifth-grade direct child assessment data combined with fifth-grade parent interview data 

AND fifth-grade child data from mathematics teacher (with or without teacher-level 

data), alone or in conjunction with data from the school administrator or facilities 

checklist. 

C6CPTS0 Fifth-grade direct child assessment data combined with fifth-grade parent interview data 

AND fifth-grade child data from science teacher (with or without teacher-level data), 

alone or in conjunction with data from the school administrator or facilities checklist. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

The ECLS-K longitudinal file is created by merging data from the base year, first grade, 

third grade, and fifth grade. Longitudinal weights were created to use in analyzing data in this 

longitudinal file. These weights are described in exhibit 7-2. All longitudinal weights are child-level 

weights. 
7-4 

Exhibit 7-2. ECLS-K: K–5 longitudinal weights, spring-fifth grade: School year 2003–04 

Weight To be used for analysis of ... 

C56CW0 child direct assessment data from BOTH spring-third grade and spring-fifth grade, alone 

or in conjunction with any of the school or teacher data, or a limited set of child 

characteristics (e.g., age, sex, and race/ethnicity). 

C56PW0 parent interview data from BOTH spring-third grade and spring-fifth grade, alone or in 

conjunction with any of the child assessment, school, or teacher data. 

C456CW0 child direct assessment data from spring-first grade AND spring-third grade AND springfifth grade, alone or in conjunction with any of the school or teacher data, or a limited set 

of child characteristics (e.g., age, sex, and race/ethnicity). 

C456PW0 parent interview data from spring-first grade AND spring-third grade AND spring-fifth 

grade, alone or in conjunction with any of the child assessment, school, or teacher data. 

C2_6FC0 child direct assessment data from FOUR rounds of data collection involving the FULL 

sample of children (spring-kindergarten, spring-first grade, spring-third grade, and 

spring-fifth grade), alone or in conjunction with any of the school or teacher data, or a 

limited set of child characteristics (e.g., age, sex, and race/ethnicity). 

C2_6FP0 parent interview data from FOUR rounds of data collection involving the FULL sample 

of children (spring-kindergarten, spring-first grade, spring-third grade, and spring-fifth 

grade), alone or in conjunction with any of the child assessment, school, or teacher data. 

C1_6FC0 child direct assessment data from FIVE rounds of data collections involving the FULL 

sample of children (fall-kindergarten, spring-kindergarten, spring-first grade, spring-third 

grade, and spring-fifth grade), alone or in conjunction with any of the school or teacher 

data, or a limited set of child characteristics (e.g., age, sex, and race/ethnicity). 

C1_6FP0 parent interview data from FIVE rounds of data collections involving the FULL sample 

of children (fall-kindergarten, spring-kindergarten, spring-first grade, spring-third grade, 

and spring-fifth grade) alone or in conjunction with any of the child assessment, school, 

or teacher data. 

C1_6SC0 child direct assessment data from ALL SIX rounds of data collection (fall-kindergarten, 

spring-kindergarten, fall-first grade, spring-first grade, spring-third grade, and spring-fifth 

grade), alone or in conjunction with any of the school or teacher data, or a limited set of 

child characteristics (e.g., age, sex, and race/ethnicity). 

C1_6SP0 parent interview data from ALL SIX rounds of data collection (fall-kindergarten, springkindergarten, fall-first grade, spring-first grade, spring-third grade, and spring-fifth 

grade), alone or in conjunction with any of the child assessment, school, or teacher data. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

As mentioned in the introduction, teachers and schools are not representative of fifth-grade 

teachers and schools in 2003–04. For this reason, there are no cross-sectional weights computed to 

provide estimates at the school or teacher level. Consequently, there are no longitudinal weights 

computed at the school or teacher level. 
7-5 

Each set of weights created to be used with the ECLS-K data consists of a full sample weight 

that is used in computing survey estimates and replicate weights that are used in variance estimation with 

a jackknife replication method. First-stage stratum and primary sampling unit (PSU) identifiers are also 

created so that variance estimation using the Taylor series approximation method can be produced using 

the full sample weights. See section 7.2.6 for a description of how replicate weights were created. Section 

7.3 discusses variance estimation methods. 

The data file includes the final full sample weight (described in section 7.2.4) and the final 

replicate weights (described in section 7.2.6) but not the intermediate weights leading to the final weights. 

The names of the full sample weights in the file are as described in exhibits 7-1 and 7-2 (e.g., C6CW0). 

The names of the replicate weights have the same prefix as the full sample weight with the last digit 

indicating the replicate (e.g., C6CW1 to C6CW90 are the 90 replicate weights to be used with the full 

sample weight C6CW0). 

7.2 Computation of the Fifth-Grade Weights 

Among the 21,357 children who were eligible for the study after the base year (21,292 base 

year respondents and 165 children sampled in first grade), the fifth-grade sample excluded 5,214 children 

as explained in section 3.6.1. As in third grade, only a subsample of children who moved from the schools 

they were attending when they were sampled originally was followed into their new schools. However, 

children who moved into a destination school1

 because they had completed the highest grade at the 

originally sampled school were all followed. The fifth-grade subsampling of movers continues to give 

more weight to children in the language minority group (i.e., movers in this group were subsampled at 

higher rates than non-language minority movers). Other smaller groups of movers were also subsampled 

at lower rates, such as selected groups of movers who were sampled in first grade (as compared with base 

year respondent movers), and movers who did not have full longitudinal data. Differential sampling rates 

of movers are presented in section 3.6.1. Another feature of the fifth-grade sample is the subsampling of 

children for the administration of the mathematics or science questionnaires as discussed in section 3.6. 

In the weighting procedures, children excluded from the fifth-grade data collection are 

considered ineligible if they became ineligible in an earlier round (because they had died or moved out of 

the country), as movers not subsampled for follow-up if they were subsampled out in previous rounds 

 

1

 A destination school is a school that received at least four students from the school where they had just completed the highest grade. 
7-6 

because they moved out of the original sample, or of unknown eligibility if they were hard refusal cases 

or if they had neither first-grade nor third-grade data. Excluded children are properly adjusted for in the 

weighting procedures. 

The weighting procedures for both cross-sectional and longitudinal weights are similar, 

although weighting cells were defined differently for each type of weight. For example, any longitudinal 

weight that contains data from fall-first grade may have used different cells due to sample size 

constraints. 

The weighting procedures for the fifth grade were divided into three main stages. These 

procedures were followed for creating each weight shown in exhibit 7-1 and exhibit 7-2. The change in 

the procedures pertains only to the change in the eligibility of children for whom the weight applies. For 

example, weight C6CW0 pertains to children with completed assessments in fifth grade; weight C56PW0 

pertains to children with completed parent interview in both third grade and fifth grade. In the base year, 

children who were not assessed because of a disability or because they were language minority children 

had positive C1CW0 and C2CW0 weights because they had data such as age, sex, race/ethnicity, height 

and weight, and characteristics of parents, teachers, and classrooms. In subsequent rounds of data 

collection, they continued to be treated the same. 

Weights that include any fall-first grade data (such as C1_6SC0, which is the weight for 

children for whom child assessments were obtained in all six rounds) were computed using the same 

procedures, but the cells for the weighting adjustments were more restricted because only the fall-first 

grade subsample was included. The replication scheme for data that include the fall-first grade panel is 

also different as described in section 7.2.6. 

The first stage of weighting was to compute an initial child weight that reflects the 

following: 

 Adjustment of the school base weight for base year school-level nonresponse; 

 Adjustment of the child weights for base year child-level nonresponse; and 

 Adjustment of the base year child weight for subsampling of schools for freshening in 

first grade (for children sampled in first-grade only). 
7-7 

The procedures used in this first stage are the same as in all rounds of data collection after 

the base year because the same sample of children (base year respondents and children sampled in first 

grade) is eligible for subsequent rounds of data collection. 

The second stage of weighting was to adjust the initial child weight computed in the first 

stage for the following: 

 Subsampling of movers; and 

 Child-level nonresponse. 

For the mathematics and science child-parent-teacher weights, an additional adjustment was 

necessary (before the second stage adjustment for the subsampling of movers and for nonresponse) to 

adjust for the subsampling of children for whom mathematics or science teacher data questionnaires were 

administered. This adjustment is described in section 7.2.5. 

The third and last stage was to rake the weights adjusted in the second stage to sample-based 

control totals. Raking is a multivariate poststratification of the weights, explained in section 7.2.4. 

In general, in each adjustment to the weight, the adjustment factor is multiplied by the 

weight in the prior step to get the adjusted weight. This fact is not repeated in the discussions of the 

weight adjustments in the following sections; only the computation of the adjustment factor is discussed. 

7.2.1 Initial Child Weights 

As mentioned earlier, the first stage of weighting was to compute an initial child weight that 

reflects: (1) the adjustment of the school base weight for base year school-level nonresponse (school-level 

weights), (2) the adjustment of the child weights for base year child-level nonresponse (child-level 

weights), and (3) the adjustment of the base year child weight for subsampling of schools for freshening 

in first grade (child-level weights, for children sampled in first grade only). These weights were already 

computed for spring-first grade. For completeness, they are described below, in section 7.2.1.1 for the 

school-level weights, and in section 7.2.1.2 for the child-level weights. 
7-8 

7.2.1.1 Base Year Nonresponse-Adjusted School Weights 

The school base weight2 SCHLBWi was the same as that computed for previous rounds of 

data collection. It is the inverse of the probability of selection of the school from a stratified probability 

proportional to size (PPS) design: 

,

SCHLi POS PSU j POS i SCHLBW 1 1 = ×

where 

POSPSU j is the probability of selection of the PSU j, and 

POSSCHL i is the probability of selection of school i within the PSU j,

where the probability of selection of the PSU j, POSPSU j , was defined as 

1 if is a self-representing (SR) PSU,

2

if is a non self-representing (NSR) PSU,

PSU j

POSPSU j M j PSU j Mh

⎧

⎪⎪ = ⎨⎛ ⎞

⎪⎜ ⎟

⎪⎩⎝ ⎠

where 

M j is the measure of size (MOS) of PSU j (i.e., count of five-year-old children in the PSU as 

described in section 3.2.1), and 

Mh is the total MOS in stratum h, PSU h j ∈ , 

and the probability of selection of school i within PSU j , POSSCHL i , was defined as 

,

i kj i m

i m

kj n SCHLi POS

∑

∈

= ×

where 

k j n is the target number of schools in stratum k in PSU i, and 

mi is the MOS of the school i in stratum k in PSU i. 

For schools sampled using the new school sampling procedure,3

 the school base weight 

SCHLBWi was computed as 

 

2

 Only schools sampled in the base year have base weights. Transfer schools do not have base weights, but children in transfer schools carry with 

them the base weights of the original sampled schools. 
7-9 

,

SCHLi POS i

f PSU j POS i SCHLBW 1 1 1 = × ×

and the factor i f is defined as 

1 if the school is a new non-Catholic private school,

if the school is a new public school,

if the school is a new Catholic school,

f p i LEA

pDIO

⎧

⎪ = ⎨

⎪

⎩

where 

pLEA is the within stratum selection probability of the school district, and 

pDIO is the within stratum selection probability of the diocese. 

The school base weight was adjusted for base year school-level nonresponse. A base year 

responding school was an original sample school with at least one child with a positive C1CW0, C2CW0, 

C1PW0, or C2PW0 weight. C1CW0 is positive for language minority (not Spanish) children who were 

screened for English proficiency (regardless of whether they went on to take the assessments), children 

with disabilities, and children with at least one direct cognitive test score in fall-kindergarten. C1PW0 is 

positive for children whose parents completed the family structure questions of the parent interview in 

fall-kindergarten. C2CW0 and C2PW0 weights are positive under similar circumstances but for springkindergarten. Schools that did not meet this condition are nonrespondents and their weights were 

distributed across responding units (at the school level) in this stage. The base year school weight was 

adjusted within nonresponse weighting classes. The base year nonresponse-adjusted school weight 

SCHLADWi was computed as 

, i SCHLADWi = SCHLADFc × SCHLBW

where 

SCHLBWi is the school base weight, and 

SCHADFc is the base year school nonresponse adjustment factor for schools in cell c, calculated 

as 

,

c

i ER i m i SCHLBW

c ,ENRc

i ER i m i SCHLBW

i SCHLADF

∑

∈

×

∑

∈

×

=

 3

 The sample was expanded to account for newly opened schools and schools not found in the sampling frame. See chapter 3 for more 

details on how these schools were identified. 
7-10 

where 

ERc denotes the set of eligible school respondents in cell c, 

ENRc denotes the set of eligible school nonrespondents in cell c, and 

mi is the measure of size for school i (i.e., count of students in the school as described in section 

3.2.2.2) 

Nonresponse cells were created using the Chi-squared Automatic Interaction Detector 

(CHAID) and variables with known values for both respondents and nonrespondents. Base year school 

characteristics used for constructing nonresponse cells were the school affiliation (public, Catholic 

private, non-Catholic religious private, or nonreligious private), the school locale (large city, midsize city, 

suburb of large city, suburb of midsize city, large town, small town, or rural area), the region where the 

school is located (Northeast, Midwest, South, or West), and the size classification of the school in terms 

of school enrollment as described in table 7-1. 

Table 7-1. Size classification for school nonresponse adjustment: 

School year 1998–99 

Size School enrollment 

classification Public Private

1 1 – 24 1 – 11 

2 25 – 39 12 – 23 

3 40 – 49 24 – 35 

4 50 – 59 36 – 47 

5 60 – 69 48 – 59 

6 70 – 79 60 or more 

7 80 – 89 † 

8 90 – 99 † 

9 100 – 119 † 

10 120 – 139 † 

11 140 – 179 † 

12 180 – 219 † 

13 220 or more † 

† Not applicable. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early 

Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spring 1999. 
7-11 

7.2.1.2 Base Year Child Weights 

Two groups of children were fielded in fifth grade: base year respondents, and eligible 

children who were sampled in first grade as part of the sample freshening procedure. The base year child 

weights for the two groups were the same as those computed for the first-grade year. A description of the 

two weights follows. 

7.2.1.2.1 Base Year Child Weights for Base Year Respondents 

A base year respondent is a sampled child with a positive C1CW0, C2CW0, C1PW0, or 

C2PW0 weight. The C1CW0 weights are positive for children belonging in the language minority (not 

Spanish) group who were screened for English proficiency (regardless of whether they went on to take the 

assessments), children assessed in fall-kindergarten, and children excluded from assessment because of a 

disability. The C1PW0 weights are positive for children whose parents completed the family structure 

questions of the parent interview in fall-kindergarten. The C2CW0 and C2PW0 weights are positive 

under similar circumstances, but apply to data from spring-kindergarten. 

The base year child weight i BYCHLDW was computed as 

(1 ), i / CHLD _ PROB i BYCHLDWi = SCHLADW ×

where 

i SCHLADW is the base year nonresponse-adjusted school weight described in section 7.2.1.1, 

CHLD_PROBi is the probability of selection of the child within a school. 

To account for base year nonresponse—children who were not assessed in the base year and 

whose parent interviews were not completed (i.e., children who did not have at least one positive weight 

among C1CW0, C2CW0, C1PW0 and C2PW0)—the base year child weight was adjusted for 

nonresponse. The child weight adjusted for base year child-level nonresponse, i ABYCHLDW , was 

computed as 

, i ABYCHLDWi = BYADFc × BYCHLDW

where BYADFc , the adjustment factor, was calculated as 
7-12 

,

BY_Rc i i BYCHLDW

c i BY_Rc , BY_NR i BYCHLDW

BYADFc ∑

∈

∑

∈ =

where 

BY_Rc denotes the set of base year child respondents in cell c, and 

BY_NR cdenotes the set of base year child nonrespondents in cell c.

The base year child weights were adjusted using weighting classes similar to those 

developed for the cross-sectional spring-kindergarten child weights. These classes were created with 

CHAID, using the school characteristics from the school nonresponse adjustments (i.e., school affiliation, 

locale, region, school enrollment classified into size category), and a set of child characteristics (i.e., year 

of birth, sex, and race/ethnicity). Data on year of birth were obtained from the parent interviews, while 

data on sex and race/ethnicity were from the child sampling information, which was provided by the 

schools. If year of birth was missing from the parent interview, then it was taken from the child sampling 

information. If sex or race/ethnicity was missing from the child sampling information, then they were 

obtained from the parent interview data. Any remaining missing data were imputed with the modal value 

from the school from which the child was sampled for this purpose. 

7.2.1.2.2 Base Year Child Weights for Children Sampled in First Grade 

In spring-first grade the student sample was freshened to include first-graders who had not 

been enrolled in kindergarten in 1998–99 and, therefore, had no chance of being included in the ECLS-K 

base year kindergarten sample. For this group of children who entered the sample in first grade, their 

weights need to have additional adjustments to account for the freshening procedure. See chapter 3 for a 

discussion of the student freshening in spring-first grade. 

Since each child sampled in first grade was directly linked to a child sampled in 

kindergarten, the first step was to compute a weight for the children who were sampled in kindergarten 

that reflects the school freshening subsampling and the school freshening nonresponse (some schools 

refused to provide the complete alphabetical roster of all students enrolled in first grade needed for 

freshening). This weight was then linked back to the child sampled in first grade and further adjusted for 

nonresponse because the data (e.g., assessment data, parent interview data) had not been obtained from 
7-13 

the sample of freshened children. The procedures for computing the base year child weights for children 

sampled in first grade are described next. 

School Weight Adjusted for Subsampling of Schools for Freshening. The school base 

weight adjusted for base year school-level nonresponse ( SCHLADWi ) computed in section 7.2.1.1 was 

adjusted for the subsampling of schools for freshening. As noted earlier, student freshening was done in a 

50 percent subsample of schools. The school weight adjusted for school freshening subsampling, 

FR1SCHWi , was calculated as 

, i FR1 SCHWi = FR1ADFc × SCHLADW

where FR1 , the adjustment factor, was computed as ADFc

⎪

⎪

⎩

⎪

⎪

⎨

⎧

∉

∈

∑

∈

∑

∈ ∪

=

0 if school ,

if school , 1

c

i F

c

i F

Fc i i SCHLADW

Fc Fc i

i SCHLADW

FR ADFc

where 

Fc denotes the set of schools subsampled for freshening, and 

c

F denotes the set of schools not subsampled for freshening. 

This adjustment was done within cells defined by school affiliation (public, Catholic private, 

non-Catholic religious private, or nonreligious private) and census region (Northeast, Midwest, South, or 

West). Adjustment cells were created using CHAID. 

School Weight Adjusted for Freshening Nonresponse. The freshening procedure could 

not be applied in all designated schools because some schools did not provide the information needed for 

freshening (see chapter 3 for more details on the freshening procedures). These schools were considered 

nonrespondents. The school weight adjusted for freshening school-level nonresponse, FR2 , was SCHWi

computed as 

, i FR2SCHWi = FR2ADFc × FR1SCHW
7-14 

where FR2 , the adjustment factor, was computed as ADFc

,

c

i FER i m i FR SCHW

c

FENR

c

i FER i m i FR SCHW

i FR ADF

∑

∈

×

∑

∈ ∪

×

= 1

1

2 

where 

i m is the original school MOS, 

c

FER denotes the set of freshening school respondents in cell c, and 

FENRc denotes the set of freshening school nonrespondents in cell c. 

In both the numerator and denominator of this factor, the school measure of size (i.e., the 

count of students in the school as described in section 3.2.2.2) was incorporated; the school measure of 

size is relevant because the weights will be used for child-level estimates, not school-level estimates. The 

nonresponse cells for this adjustment were created with CHAID using school affiliation (public, Catholic 

private, non-Catholic religious private, or nonreligious private) and type of locale (large city, midsize city, 

suburb of large city, suburb of midsize city, large town, small town, or rural area). 

Base Year Child Weight. Next, the school-adjusted weight was multiplied by the inverse of 

the probability of sampling the child in the base year to obtain a base year child weight for freshening. 

The base year child weight was BYCHLDWi : 

2 (1 ), i / CHLD _ PROB i BYCHLDWi = FR SCHW ×

where i CHLD _ PROB is the within-school child selection probability. 

The base year child weight was then adjusted for base year child nonresponse because 

children who did not respond in the base year could not be linked to children in first grade in spring 2000. 

The adjusted weight, ABYCHLDWi was computed as 

, i ABYCHLDWi = BYADFc × BYCHLDW
7-15 

where BYADFc , the adjustment factor, was calculated as 

,

BY_Rc i i BYCHLDW

c i BY_Rc , BY_NR i BYCHLDW

BYADFc ∑

∈

∑

∈ =

where 

BY_Rc denotes the set of base year child respondents in cell c, and 

BY_NR cdenotes the set of base year child nonrespondents in cell c.

The nonresponse cells were created with CHAID using the school characteristics school 

affiliation, locale, region, and school enrollment size, and the child characteristics age, sex, and 

race/ethnicity. 

Base Year Child Weights Adjusted for Movers. Only children who did not move from 

their original school were designated as links to children in the freshening procedure. The children who 

moved and were followed into their new schools were not identified to participate in the freshening 

process in their new schools. As a result, all children who moved were considered nonrespondents for the 

freshening process. Additionally, nonmovers and movers who were not in first grade were not eligible for 

freshening (e.g., if the child was in kindergarten in spring 2000, he or she would be linked only to other 

kindergarten children and thus was not eligible for the freshening of first-graders). An adjustment was 

necessary to account for these two groups of children and was done in two steps. 

In the first step, an adjustment was made for movers whose grade was unknown. A portion 

of the movers was assumed to be in first grade. In the second step, the weights were adjusted for children 

who were in first grade and who were not identified to participate in the freshening process because they 

moved into a new school. For this two-step adjustment, each child was classified as in table 7-2. 
7-16 

Table 7-2. Groups of children defined for mover adjustments: 

School year 1999–2000 

Groups Mover status 

MOV1 Mover enrolled in first grade 

MOVOTH Mover enrolled in another grade 

MUNK Mover enrolled in an unknown grade 

NM1 Nonmover enrolled in first grade 

NMOTH Nonmover enrolled in another grade 

SOURCE: U.S. Department of Education, National Center for Education 

Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 

(ECLS-K), spring 2000. 

The first step adjustment for movers whose grade in spring 2000 was unknown was 

computed as 

, i R4 MOVWi = R4MOVF1c × ABYCHLDW

where R4 , the adjustment factor, was computed as MOVF1c

⎪

⎪

⎪

⎩

⎪

⎪

⎪

⎨

⎧

∈ ∪

∈

∈ ∪

∑

∑

= ∈ ∪

∈ ∪ ∪

NM NMOTH c.

MUNK c

MOV MOVOTH c

i ABYCHLDW

i ABYCHLDW

R MOVF c

i MOV c MOVOTHc

i MOV c MOVOTHc MUNKc

1 if child 1 in cell

0 if child in cell ,

if child 1 in cell ,

4 1

1

1

The second step adjustment for movers who could not be used as links for freshening was 

computed as 

, MOVF i R MOVF R MOVW i R c 4 2 = 4 2 × 4 1

where R4 , the adjustment factor, was computed as MOVF2c

⎪

⎪

⎪

⎩

⎪

⎪

⎪

⎨

⎧

∈ ∪

∈

∈

∑

∑

= ∈

∈ ∪

1 if child in cell .

0 if child 1 in cell ,

if child 1 in cell , 4 1

4 1

4 2

1

1 1

NMOTH MOVOTH c

MOV c

NM c

MOVW i R

MOVW i R

R MOVF c

i NM c

i NM c MOV c
7-17 

This two-step adjustment was done within cells defined by school affiliation and census 

region. 

The weights thus created for children sampled in kindergarten were then linked to the 

children that they brought into the sample in first grade through sample freshening. In other words, the 

weight of the child sampled in first grade was defined at this point to be the weight computed for the child 

sampled in kindergarten that was responsible for bringing the first-grader into the sample. 

For the next step in the computation of the fifth-grade child weights, the two groups of 

children—base year respondents and children sampled in first grade through sample freshening—were 

put together, and a common variable and label were used to designate the initial child weight. This is the 

base year child weight as computed above for each group of children: 

⎩

⎨

⎧ = 4 2 if sampledin first grade.

if base year respondent,

i

i

i R MOVW

ABYCHLDW

ICHLDW

The initial child weights i ICHLDW were adjusted for movers between the base year and fifth 

grade and for nonresponse in fifth grade, and raked to sampled-based control totals to obtain the final 

fifth-grade child weights. These adjustments and raking procedures are described below. 

7.2.2 Adjustment for Movers Between the Base Year and Fifth Grade 

First, the initial child weights were adjusted to reflect the subsampling of movers. In the 

ECLS-K, a child could move more than once and at different times. For example, a child could move out 

of his original sample school because the school did not have grades higher than kindergarten. Then he 

could move again between first and third grade, first and fifth grade, or third and fifth grade. Once a child 

was identified as a mover, he stayed a mover unless he moved back to the original sample school. For 

example, a child who moved between kindergarten and third grade, but stayed in that same school 

between third and fifth grade, was considered a mover for the fifth grade. 

Each mover in the fifth grade had a flag indicating whether he was followed into the new 

school. These flags were set according to the mover subsampling plan described in section 3.6.1. Children 

who were excluded from the fifth-grade data collection because they had moved out of the original 
7-18 

schools and were subsampled out for followup in previous rounds had their flag set to “not followed.” In 

fifth grade, children were fielded as described in exhibit 7-3. 

Exhibit 7-3. Movers and nonmovers by retention status: School year 2003–04 

Child moved out of original school Child subsampled for followup 

Before 

fifth grade 

During

fifth grade

Before

fifth grade

During 

fifth grade 

Child fielded

in fifth grade

No No † † Yes 

No Yes † No No 

No Yes † Yes Yes 

Yes No, did not move again No † No 

Yes No, did not move again Yes No No 

Yes No, did not move again Yes Yes Yes 

Yes Back in original school † † Yes 

†Not applicable. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), fall 1998, spring 1999, fall 1999, spring 2000, spring 2002, and spring 2004. 

The initial child weight described in section 7.2.1 was adjusted to reflect this subsampling 

movers. The initial child weight adjusted for movers i R6 was computed as CHILDW1

, i R6 CHLDW1i = R6ADF1c × ICHLDW

where i R6 , the adjustment factor, was computed as ADF1

⎪

⎪

⎪

⎪

⎩

⎪

⎪

⎪

⎪

⎨

⎧

=

= ∑

∑

=

=

∈

∈

0 if childin cell is a moverand 0,

if childin cell is a moverand 1,

1 if childin cell is not a mover,

6 1

1

with

in (0,1)

with

c FOLLOW_F

c FOLLOW_F

i ICHLDW

i ICHLDW

c

R ADF c

FOLLOW _ F

c ii MOVER

FOLLOW _ Fc i MOVER

and 

c

MOVER denotes the set of children that are movers in cell c. 

For the cross-sectional weights, the mover adjustment factor was computed within cells 

created with CHAID using the following characteristics: whether children were sampled in kindergarten 
7-19 

or first grade, and whether they were language minority children.4

 For the longitudinal weights, a 

longitudinal mover follow status was created that takes into account whether the child moved from his 

original school in fall-first grade, spring-first grade, or spring-third grade (for longitudinal weights 

involving the fall-first grade data) or whether the child moved from his or her original school in springfirst grade or spring-third grade (for the other longitudinal weights). If a child moved in either round, he 

was considered a mover. Appendix A gives the cell definitions for the mover adjustment for crosssectional and longitudinal weights. 

Twelve children with large weights had their weights trimmed by 40 percent. The trimming 

procedure was the same as in previous years. However, the weights were not redistributed because the 

total sum of weights was re-established in the raking procedure that came later. 

7.2.3 Adjustment for Fifth-Grade Nonresponse 

After the adjustment for subsampling movers, the child weights were adjusted for 

nonresponse. As in spring-first grade and spring-third grade, the nonresponse adjustment was done in two 

steps. In the first step, the adjustment was for children whose eligibility was not determined (unknown 

eligibility). A portion of children of unknown eligibility was assumed to be ineligible, equal to the 

proportion of children of known eligibility who are ineligible. In the second step, the adjustment was for 

eligible nonrespondents. To carry out these adjustments, each child was classified as (a) an eligible 

respondent, (b) an eligible nonrespondent, (c) ineligible (out of the country or deceased) or (d) of 

unknown eligibility (mover who could not be located), as shown in table 7-3. 

Table 7-3. Groups of children defined for nonresponse adjustments: 

School year 2003–04 

Groups Eligibility and response status 

ER Eligible respondent 

ENR Eligible nonrespondent 

IN Ineligible (out of the country or deceased) 

UNK Unknown eligibility (mover who could not be located) 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early 

Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 

 

4

 Fewer characteristics were used than in previous years to create cells for mover adjustments. This is due to cells with a small number of records, 

requiring them to be collapsed in order to avoid large adjustment factors. This resulted in fewer cells, hence fewer characteristics being used. 
7-20 

The child weight adjusted for nonrespondents with unknown eligibility, i R6 , was CHLDW 2

computed as 

, i R6CHLDW 2i = R6ADF2c × R6CHLDW1

where R6 , the adjustment factor, was computed as ADF2c

⎪

⎪

⎩

⎪

⎪

⎨

⎧

∈

∈ ∪ ∪

∑

∑

= ∈ ∪ ∪

∈ ∪ ∪ ∪

0 if child in cell .

if child in cell , 6 1

6 1

6 2

UNK c

ER ENR IN c

i R CHLDW

i R CHLDW

R ADF c

i ERc ENRc INc

i ERc ENRc INc UNKc

In the second adjustment, the child weight was adjusted for eligible nonrespondents. The 

child weight adjusted for eligible nonrespondents, i R6 , was computed as CHLDW 3

6 3 6 3 6 2 , i R CHLDW i = R ADF c × R CHLDW

where R6ADF3c , the adjustment factor, was computed as: 

⎪

⎪

⎪

⎩

⎪

⎪

⎪

⎨

⎧

∈

∈

∈

∑

∑

= ∈

∈ ∪

IN c.

ENR c

ER c

i R CHLDW

i R CHLDW

R ADF c

i ERc

i ERc ENRc

1 if child in cell

0 if child in cell ,

if child in cell , 6 2

6 2

6 3

In both steps of the adjustment, separate nonresponse classes were created for movers and 

nonmovers with CHAID using various combinations of response status of child assessments and parent 

interviews in the base year as well as whether children belong to the language minority group, the type of 

household collected from the parent interviews (all cross-sectional weights except C6CW0), and the 

school affiliation including whether the child was homeschooled (C6CPTR0, C6CPTM0 and C6CPTS0 

only). The adjustment cells for each type of weight are shown in appendix B. Where applicable, very 

large nonresponse adjusted weights were trimmed by 40 percent. As before, the weights were not 

redistributed in each case because the total sum of weights was re-established in the raking step that came 

next. 
7-21 

7.2.4 Raking to Sample-Based Control Totals 

To reduce the variability due to the subsampling of schools and movers, the child weights 

were then raked (i.e., calibrated) to sample-based control totals computed using the initial child weights 

described in section 7.2.1. A file was created with the initial child weights and school and child 

characteristics collected in the base year or first-grade year (such as school affiliation, region, urbanicity, 

sex, age, race/ethnicity, SES, language minority status, whether sampled in kindergarten or first grade, 

and if sampled in kindergarten, mover status in spring-first grade) to be used in the computation of the 

control totals. The child records included in this file are records of base-year respondents and records of 

eligible children sampled in first grade, including records of children who became ineligible in fifth 

grade. The sum of the initial weights thus calculated is the estimated number of children who were in 

kindergarten in 1998–99 or first grade in 1999–2000. In the previous steps, the weights of the 

nonresponding children were distributed to the responding children while the weights of the ineligible 

children were not affected by this weighting step. The weights of the ineligible children were set to zero 

at the end of this process because these children are not meant to be included in the analysis of the fifthgrade data. The reason for including the ineligible children in the raking step is that these children were 

included in the sampled-based control totals. 

The raked child weight or fifth-grade final child weight, i R6 , was calculated as CHLDW 4

6 4 6 4 6 3 , i R CHLDW i = R ADF c × R CHLDW

where R6 , the raking factor for raking cell ADF4c c, was computed as 

, 6 36 4

∑∈

=

i c

i

c

R CHLDW

SMP _CNT R ADF

where SMP _CNTc is the sample-based control total for raking cell c. Weights of children who become 

ineligible in fifth grade were set to zero after this step. 

This raking procedure is essentially a multivariate poststratification. Raking cells (also 

known as raking dimensions because they typically involve more than one variable, for example, sex by 

age) were created with CHAID using school and child characteristics collected in the base year or firstgrade year: school affiliation, region, type of locale, sex, age, race/ethnicity, socio-economic scale (SES), 

language minority status, whether sampled in kindergarten or first grade, and if sampled in kindergarten, 
7-22 

mover status. These characteristics come from the fifth-grade data collection or previous rounds of data 

collection if fifth-grade data are missing. Appendix C gives the raking dimensions used for fifth grade. 

There was no restriction set in the number of iterations during the raking procedure. The 

procedure was allowed to run until complete convergence was achieved within a control total. This 

occurred after 12 to 19 iterations. 

7.2.5 Additional Adjustment for Child-Parent-Teacher Cross-Sectional Weights 

A new feature of the fifth-grade sample is the subsampling of children for the administration 

of the mathematics and science teacher questionnaires. While all children had child-level questionnaires 

filled out by their reading teachers, half were subsampled to have child-level questionnaires filled out by 

their mathematics teachers and the other half had child-level questionnaires filled out by their science 

teachers. For this reason, there are three child-parent-teacher weights that will be used to analyze direct 

child assessment data combined with parent interview data and child data provided by teachers (in 

conjunction with school level or teacher level data). In all three weights, the presence of at least one 

completed teacher-level questionnaire determines whether a child would have a positive child-parentteacher weight in the two subjects to which they were assigned (i.e., reading and mathematics or reading 

and science.). A child could have one teacher who taught all subjects, in which case the teacher would be 

asked to fill out both the reading questionnaire and the mathematics questionnaire (if the child was 

selected for mathematics) or science questionnaire (if the child was selected for science). A child could 

also have different teachers teaching different subjects, in which case the child may have a reading 

teacher filling out the reading questionnaire and a mathematics teacher filling out the mathematics 

questionnaire, and both teachers could have filled out the teacher-level questionnaire. No children have 

both completed mathematics and science questionnaires because of the subsampling. 

Table 7-4 shows the distribution of children who have direct child assessment data, parent 

interview data and child-level data from the mathematics teacher by the number of teachers they had who 

filled out the teacher-level questionnaire. The first column in this table shows the number of teachers that 

each child had: only one teacher who taught both reading and mathematics, or two teachers, one teaching 

reading and the other teaching mathematics. The second column shows the type of teacher who filled out 

the teacher-level questionnaire. If the child had only one teacher, then it was this teacher—identified in 

the table as the reading teacher—who filled out the teacher-level questionnaire (3,142 cases out of 5,009 
7-23 

or 63 percent). If the child had two teachers, then in the majority of cases, both teachers filled out the 

teacher-level questionnaire (1,803 cases out of 5,009 or 36 percent). There are very few cases where only 

one of the two teachers filled out the teacher-level questionnaire. Table 7-5 shows the same information 

for science. Since C6CPTM0 and C6CPTS0 are used for the analysis of child and parent data with data 

from mathematics and science teachers, another option to define these weights is to use the presence of 

child-level data from the mathematics/science teachers. However, tables 7-4 and 7-5 show that by 

considering the presence of teacher-level data in constructing the child-parent-teacher weights, there are 

more records with positive weights for analysis (5,017 cases with nonzero C6CPTM0 compared with 

5,009 in table 7-4; and 5,103 cases with nonzero C6CPTS0 compared with 5,088 in table 7-5).5

 Using 

teacher-level data to define the child-parent-teacher weights is also consistent with previous years’ 

practice. 

Table 7-4. Number of children with direct child assessment, parent interview and child-level data from 

mathematics teacher, by number of teachers who filled out teacher-level questionnaire: 

School year 2003–04 

Number of 

teachers that each 

child had 

Teachers who completed 

teacher-level questionnaire 

Number of children with child-parent-mathematics 

data from the child-level mathematics questionnaire

Total 5,009

1 Reading 3,142

2 Reading 25

2 Mathematics 39

2 Reading and Mathematics 1,803

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

 

5

 See section 4.7.7 of the Combined User’s Manual for the ECLS-K Fifth-Grade Data Files and Electronic Codebooks (NCES 2006–032) 

(Tourangeau et al. forthcoming) for a description of the characteristics of the cross-sectional sample weights. 
7-24 

Table 7-5. Number of children with direct child assessment, parent interview, and child-level data from 

science teacher, by number of teachers who filled out teacher-level questionnaire: School 

year 2003–04 

Number of 

teachers that each 

child had 

Teachers who completed 

teacher-level questionnaire 

Number of children with child-parent-science data 

from the child-level science questionnaire

Total 5,088

1 Reading 2,999

2 Reading 42

2 Science 35

2 Reading and Science 2,012

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

An additional adjustment is necessary to adjust for the subsampling of children for whom 

mathematics or science teacher data questionnaires were administered.6

 Since only half of the children in 

fifth grade were eligible to have a completed mathematics teacher questionnaire and the other half to have 

a completed science teacher questionnaire, the weights before adjustment for movers and nonresponse 

adjustments (described in sections 7.2.2 and 7.2.3, respectively) were adjusted to account for the 

subsampling of children. 

The initial child weight described in section 7.2 was adjusted to reflect the subsampling of 

children for the mathematics teacher questionnaire as 

6 0 6 0 , i R CHLDW i = R ADF c × ICHLDW

where R6ADF0c , the adjustment factor, was computed as 

, ∑

∈

∑

=

i MTH i ICHLDW

i i ICHLDW

R6ADF0

where MTH denotes the set of children subsampled for mathematics teacher questionnaires. 

 

6

 Note that this adjustment occurred prior to the adjustment for subsampling movers, the adjustment for nonresponse, and raking, but it is 

discussed after section 7.2.4 on raking because it only applies to the child-parent-math/science weights. 
7-25 

Likewise, the adjustment factor for subsampling children for the science teacher 

questionnaire was computed as 

, ∑

∈

∑

=

i SCI i ICHLDW

i i ICHLDW

R6ADF0

where SCI denotes the set of children subsampled for science teacher questionnaires.

This adjustment was followed by the adjustments for subsampling movers as discussed in 

section 7.2.2 ( i R6 with CHLDW1 i ICHLDW replaced by i R6 ), the adjustment for nonresponse CHLDW 0

as discussed in sections 7.2.3 ( i R6 and CHLDW 2 i R6 ), and raking as discussed in section 7.2.4 CHLDW3

( i R6 ). CHLDW 4

7.2.6 Replicate Weights 

For each set of cross-sectional and longitudinal weights included in the fifth-grade data file, 

a set of replicate weights was computed. All adjustments to the full sample weights were repeated for the 

replicate weights. The replication scheme used for the base year was used for all of the fifth-grade 

weights that did not contain any fall-first grade component. If a fall-first grade component was included 

in the definition of the respondents for the weight, then the replication scheme used for fall-first grade 

estimates was used. 

Replicate weights are needed to estimate the standard errors of survey estimates. A total of 

90 replicate weights were computed using the paired jackknife method (denoted as JK2) for the fifthgrade weights if no fall-first grade component was included. These replicates take into account the Durbin 

method of PSU selection (Durbin 1967). A total of 40 replicates using the paired jackknife method were 

created for the weights that contain a fall-first grade component. The number of replicates is smaller 

because only 30 percent of the full sample of schools was included in the fall-first grade subsample. Only 

one of the two sampled PSUs in the non-self-representing strata was kept in the sample. Consequently, 

the fall-first grade weights do not account for the Durbin PSU sampling method, which required two 

PSUs per stratum. 
7-26 

The procedures used to compute the replicate weights took into account each step of the 

weighting process. One feature that is somewhat uncommon in practice is the use of sample-based raking 

as described in section 7.2.4. The control totals ( SMP _CNTc ) used for raking are estimates calculated 

using the initial child weights ( i ICHLDW ). When population-based raking is used, these totals are 

assumed to be numbers that are known and without sampling error. To reflect the variability of the control 

totals in the sample-based raking, a set of replicate control totals was calculated rather than having a 

constant set of totals. Each replicate weight was then raked to the corresponding replicate-based control 

total. The result of this process was that each replicate retained the variability associated with the original 

sample estimates of the control totals. As with the full sample weight, the raking procedure was allowed 

to run until complete convergence. For fifth grade, full convergence was achieved after 12 to 19 iterations 

for each replicate weight. 

7.2.6.1 Replicate Weights for Samples Not Involving Fall-First Grade 

For the original ECLS-K design in the base year, replicate weights were created taking into 

account the Durbin method of PSU selection. The Durbin method selects two first-stage units per stratum 

without replacement, with probability proportional to size and a known joint probability of inclusion. 

In the ECLS-K PSU sample design, there were 24 self-representing (SR) strata and 38 nonself-representing (NSR) strata. Among the 38 NSR strata, 11 strata were identified as Durbin strata and 

were treated as SR strata for variance estimation. The purpose of the Durbin strata is to allow variances to 

be estimated as if the first-stage units were selected with replacement. This brings the number of SR PSUs 

to 46 (24 original SR PSUs and 22 Durbin PSUs from the 11 Durbin strata). The remaining 54 NSR PSUs 

are in 27 NSR strata; thus 27 replicates were formed, each corresponding to one NSR stratum. For the SR 

strata, 63 replicates were formed. The 90 replicates will yield about 76 degrees of freedom for calculating 

confidence intervals for many survey estimates. 

As stated earlier, the sample of PSUs was divided into 90 replicates or variance strata. The 

27 NSR strata formed 27 variance strata of two PSUs each; each PSU formed a variance unit within a 

variance stratum. All schools within an NSR PSU were assigned to the same variance unit and variance 

stratum. Sampled schools in the 46 SR PSUs were grouped into 63 variance strata. In the SR PSUs, 

schools were directly sampled and constituted PSUs. Public schools were sampled from within PSU while 

private schools were pooled into one sampling stratum and selected systematically (except in the SR 
7-27 

PSUs identified through the Durbin method where private schools were treated as if they were sampled 

from within PSU). Schools were sorted by sampling stratum, type of school (from the original sample or 

newly selected as part of freshening), type of frame (for new schools only), and their original order of 

selection (within stratum). From this sorted list, they were grouped into pairs within each sampling 

stratum; the last pair in the stratum may be a triplet if the number of schools in the stratum is odd. This 

operation resulted in a number of ordered preliminary variance strata of two or three units each. The first 

ordered 63 strata were then numbered sequentially from 1 to 63; the next ordered 63 strata were also 

numbered sequentially from 1 to 63, and so on until the list was exhausted, thus forming the desired 63 

variance strata. 

In strata with two units, a unit being a PSU in the case of NSR PSUs and a school in the case 

of SR PSUs, the base weight of the first unit was doubled to form the replicate weight, while the base 

weight of the second unit was multiplied by zero. In strata with three units, two variance strata were 

created: in the first variance stratum, the base weight of two of the three units was multiplied by 1.5 to 

form the replicate weight, and the base weight of the last unit was multiplied by zero; in the second 

variance stratum, the base weight of a different group of two units was multiplied by 1.5, and the base 

weight of the third unit was multiplied by zero. Multiplying the base weight in a unit by zero is equivalent 

to dropping one unit as required by the jackknife method. All adjustments to the full sample weights were 

repeated for the replicate weights. For each full sample weight, there are 90 replicate weights with the 

same weight prefix. 

A child sampled in first grade through the freshening process was assigned to the same 

replicate as the originally sampled child to whom the child was linked. When the child sampled in first 

grade was assigned a full sample weight, he or she was assigned the replicate weights in the same manner. 

7.2.6.2 Replicate Weights for Samples Involving Fall-First Grade 

For the two longitudinal weights involving fall-first grade (C1_6SC0 and C1_6SP0), there 

are 40 replicate weights. The reason for the smaller number of replicates is that only a subsample of 

schools was included in the fall-first grade sample. The weights associated with the fall-first grade data do 

not account for the Durbin method of selecting PSUs, since it no longer applied. Rather, they reflect the 

fact that only one of the two sampled PSUs in the NSR strata was kept in the subsample. To account for 

this feature, pairs of similar NSR PSUs were collapsed into 19 variance strata. The SR PSUs account for 
7-28 

the remaining 21 variance strata. The 40 replicates will yield about 40 degrees of freedom for calculating 

confidence intervals for many survey estimates. 

7.3 Variance Estimation 

7.3.1 Jackknife Method 

The final full sample and the adjusted replicate weights can be used to compute estimates of 

variance for survey estimates using WesVar, AM, SUDAAN, or other software that handles replicate 

weights. The estimate of variance is the sum of the squared deviations of the replicate estimates from the 

full-sample estimate: 

) , ˆ ˆ ) ( ˆ v(

G

g

∑ ( g )

=

= −

1

2 θ θ θ

where 

θ is the population statistic of interest, 

θ

ˆ is the estimate of θ based on the full sample, 

G is the number of replicates, and 

( g ) ˆ

θ is the g

th replicate estimate of θ based on the observations included in the gth

replicate. 

7.3.2 Taylor Series Method 

Variance stratum and variance unit (first-stage sample unit) identifiers were also created to 

be used in statistical software that computes variance estimates based on the Taylor series method (e.g., 

SUDAAN, Stata, SAS, SPSS Complex Samples Module, and AM). In this method, a linear 

approximation to a statistic is formed and then substituted into the formula for calculating the variance of 

a linear estimate appropriate for the sample design. 
7-29 

If ( )' Yp Y Y ,..., = 1 denotes a p-dimensional vector of population parameters, ( )'

Yp

ˆ Y ,..., ˆ Yˆ = 1 is 

the corresponding vector of estimators based on a sample s of size n(s), θ = g( ) Y is the population 

parameter of interest, and (Y ) ˆ g ˆ

θ = is an estimator of θ , then 

( )

( ) j j

p

j j

Y Y ˆ

y

g Y ˆ − ∂

∂ − =∑=1

θ θ 

and 

( ) ( )( ) ( ) ( ) { } Y . ˆ Y , Cov ˆ

y

g Y

y

g Y Y Y ˆ

y

g Y

v ˆ v j i

i

p

j

p

i j

p

j

j j

i ∂

∂

∂

∂ =⎟

⎟

⎠

⎞

⎜

⎜

⎝

⎛ − ∂

∂ = ∑ ∑∑ =1 = = 1 1

θ 

The Taylor series method relies on a simplified procedure for estimating the variance for a 

linear statistic even with a complex sample design and is valid in large samples in which the first stage 

units are sampled with replacement. For the ECLS-K, this simplified method does not capture the 

variance related to the Durbin sampling method, the effects of the adjustments of the weights for 

nonresponse, or the sample-based raking procedures. These effects are not captured in the Taylor series 

variance estimates mainly because each adjustment corresponds to a different estimator that the variance 

estimation software does not support. In some cases these adjustments may have only a minor effect on 

the variance estimates, but in other cases the effects could be more substantial. 

For software that uses the Taylor series method, the variance strata and PSUs must be 

defined. For the fifth-grade ECLS-K, the Taylor variance strata were assigned by sequentially numbering 

the sampling strata and collapsing any stratum with one PSU with an adjacent stratum. In theory, any 

variance stratum with fewer than two responding units would be combined with an adjacent stratum, but 

this did not happen in the ECLS-K. The variance units were assigned by sequentially numbering the firststage sampling units within sampling strata. For example, for C6CW0, Taylor variance strata were 

numbered sequentially from 1 to 90. Within each Taylor stratum, Taylor units were numbered 

sequentially from 1 to the total number of units in the stratum. This procedure was done separately for 

each cross-sectional and longitudinal weight. 
7-30 

This page is intentionally left blank. 
R-1 

REFERENCES 

Broughman, S., and Colaciello, L. (1998). Private School Universe Survey, 1995-96 (NCES 98–229). 

U.S. Department of Education. Washington, DC: National Center for Education Statistics. 

Durbin, J. (1967). Design of Multi-Stage Surveys for the Estimation of Sampling Errors. Journal of the 

Royal Statistical Society, Series C (Applied Statistics), Volume 16, pp. 152–164. 

Kish, L. (1965). Survey Sampling. New York: John L. Wiley & Sons. 

Pollack, J.M., Najarian, M., Rock, D.A., and Atkins-Burnett, S. (2005). Early Childhood Longitudinal 

Study, Kindergarten Class of 1998–99 (ECLS-K), Psychometric Report for the Fifth Grade (NCES 

2006–036). U.S. Department of Education. Washington, DC: National Center for Education 

Statistics. 

Pollack, J.M., Rock, D., Weiss, M., Atkins-Burnett, S., Tourangeau, K., West, J., and Germino Hausken, 

E. (2005). Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), 

Psychometric Report for the Third Grade (NCES 2005–062). U.S. Department of Education. 

Washington, DC: National Center for Education Statistics. 

Rock, D., and Pollack, J. (2002). Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 

(ECLS-K), Psychometric Report for Kindergarten Through First Grade (NCES 2002–05). U.S. 

Department of Education. Washington, DC: National Center for Education Statistics. 

Tourangeau, K., Brick, M., Lê, T., Wan, S., Weant, M., Nord, C., Vaden-Kiernan, N., Hagedorn, M., 

Bissett, E., Dulaney, R., Fowler, J., Pollack, J., Rock, D., Weiss, M.J., Atkins-Burnett, S., Germino 

Hausken, E., West, J., Rathbun, A., and Walston, J. (2004). Early Childhood Longitudinal Study, 

Kindergarten Class of 1998–99 (ECLS-K), User’s Manual for the ECLS-K Third Grade Public-Use 

Data File and Electronic Code Book (NCES 2004–001). U.S. Department of Education. 

Washington, DC: National Center for Education Statistics. 

Tourangeau, K., Burke, J., Lê, T., Wan, S., Weant, M., Nord, C., Vaden-Kiernan, N., Bissett, E., Dulaney, 

R., Fields, A., Byrne, L., Flores-Cervantes, I., Fowler, J., Pollack, J., Rock, D., Atkins-Burnett, S., 

Meisels, S., Bose, J., West, J., Denton, K., Rathbun, A., and Walston, J. (2002). Early Childhood 

Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), User’s Manual for the ECLS-K 

First Grade Public-Use Data Files and Electronic Codebook (NCES 2002–135). U.S. Department 

of Education. Washington, DC: National Center for Education Statistics. 

Tourangeau, K., Burke, J., Lê, T., Wan, S., Weant, M., Brown, E., Vaden-Kiernan, N., Rinker, E., 

Dulaney, R., Ellingsen, K., Barrett, B., Flores-Cervantes, I., Zill, N., Pollack, J., Rock, D., AtkinsBurnett, S., Meisels, S., Bose, J., West, J., Denton, K., Rathbun, A., and Walston, J. (2004). Early 

Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), Base Year Public-Use 

Data Files and Electronic Codebook: User’s Manual (NCES 2001–029r). U.S. Department of 

Education. Washington, DC: National Center for Education Statistics. 

Tourangeau, K., Nord, C., Lê, T., Vaden-Kiernan, N., Wan, S., Weant, M., Pollack, J., and AtkinsBurnett, S. (Forthcoming). Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 
R-2 

(ECLS-K), Combined User’s Manual for the ECLS-K Fifth-Grade Data Files and Electronic 

Codebooks (NCES 2006–032). U.S. Department of Education. Washington, DC: National Center 

for Education Statistics. 

U.S. Department of Education, National Center for Education Statistics (1995–96), Common Core of 

Data, Public School Universe Survey. Washington, DC: Author. 

U.S. Department of the Interior, Bureau of Indian Affairs (1995–96). Office of Indian Education 

Programs Education Directory. (Unpublished document). 
A-1 

APPENDIX A: MOVER ADJUSTMENT CELLS 

Variables used to create mover adjustment cells: 

FRESHEE = 0, sampled in kindergarten 

 1, sampled in first grade 

LMSTATUS = 1, home language is not English 

 2, home language is English 

These variables are in the weighting working files. If they are included in the final data file, they do not 

have the same variable names. 

Table A-1. Mover adjustment cells for cross-sectional weights C6CW0, C6PW0 and C6CPTR0: 

School year 2003–04 

Cell FRESHEE LMSTATUS Adjustment factor

1 0 1 2.18502

2 0 2 6.39362

3 1 all 2.46993

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

Table A-2. Mover adjustment cells for cross-sectional weight C6CPTM0: 

School year 2003–04 

Cell LMSTATUS Adjustment factor

1 1 1.51759

2 2 3.26127

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood 

Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 

Table A-3. Mover adjustment cells for cross-sectional weight C6CPTS0: 

School year 2003–04 

Cell LMSTATUS Adjustment factor

1 1 1.48393

2 2 3.38600

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood 

Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 
A-2 

Table A-4. Mover adjustment cells for longitudinal weights not involving fall-first grade: School 

year 2003–04 

Cell FRESHEE LMSTATUS Adjustment factor

1 0 1 2.11188

2 0 2 5.87163

3 1 all 2.35774

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten 

Class of 1998–99 (ECLS-K), spring 2004. 

Table A-5. Mover adjustment cells for longitudinal weights involving 

fall-first grade: School year 2003–04 

Cell LMSTATUS Adjustment factor

1 1 2.04280

2 2 4.42024

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood 

Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 
B-1 

Appendix B: UNKNOWN ELIGIBILITY AND NONRESPONSE ADJUSTMENT CELLS 

Variables used to create unknown eligibility and nonresponse adjustment cells: 

HHTYPE = 1, in family of 2 parents plus siblings 

2, in family of 2 parents and no siblings 

3, in family of 1 parent plus siblings 

4, in family of 1 parent and no siblings 

5, in other type of family 

9, in unknown type of family 

HOMESCH = 0, not homeschooled 

1, in home school 

LFMOVST = 0, not a longitudinal (full sample) mover 

1, longitudinal (full sample) mover 

LMSTATUS = 1, home language is not English 

2, home language is English 

LSMOVST = 0, not a longitudinal (subsample) mover 

1, longitudinal (subsample) mover 

R6MOVST = 0, not a spring-fifth grade mover 

1, spring-fifth grade mover 

RC12P12 = 1, not assessed due to a disability or language minority (LM) or respondent to child 

assessment and parent interview in fall-kindergarten and spring-kindergarten (i.e., 

C1CW0>0 and C2CW0>0 and C1PW0>0 and C2PW0>0) 

2, not a respondent to child assessment or parent interview in fall-kindergarten or 

spring-kindergarten (i.e., C1CW0=0 or C2CW0=0 or C1PW0=0 or C2PW0=0) 

RC1245P1245 = 1, not assessed due to a disability or language minority (LM) or respondent to child 

assessment and parent interview in fall-kindergarten and spring-kindergarten and 

spring-first grade and spring-third grade (i.e., C1CW0>0 and C2CW0>0 and 

C4CW0>0 and C5CW0>0 and C1PW0>0 and C2PW0>0 and C4PW0>0 and 

C5PW0>0) 

2, not a respondent to child assessment or parent interview in fall-kindergarten or 

spring-kindergarten or spring-first grade or spring-third grade(i.e., C1CW0=0 or 

C2CW0=0 or C4CW0=0 or C5CW0=0 or C1PW0=0 or C2PW0=0 or C4PW0=0 or 

C5PW0=0) 

RC125P125 = 1, not assessed due to a disability or language minority (LM) or respondent to child 

assessment and parent interview in fall-kindergarten and spring-kindergarten and 

spring-third grade (i.e., C1CW0>0 and C2CW0>0 and C5CW0>0 and C1PW0>0 and 

C2PW0>0 and C5PW0>0) 
B-2 

2, not a respondent to child assessment or parent interview in fall-kindergarten or 

spring-kindergarten or spring-third grade (i.e., C1CW0=0 or C2CW0=0 or C5CW0=0 

or C1PW0=0 or C2PW0=0 or C5PW0=0) 

RC5 = 1, not assessed due to a disability or LM or respondent to spring-third grade child 

assessment (i.e., C5CW0>0) 

2, not a respondent to spring-third grade child assessment (i.e., C5CW0=0) 

RP4 = 1, a respondent to parent interview in spring-first grade (i.e., C4PW0>0) 

2, not a respondent to parent interview in spring-first grade (i.e., C4PW0=0) 

RP5 = 1, a respondent to parent interview in spring-third grade (i.e., C5PW0>0) 

2, not a respondent to parent interview in spring-third grade (i.e., C5PW0=0) 

STYPE = 1, Catholic 

2, Other religious private 

3, Nonreligious private 

4, public 

9, unknown 

These variables are in the weighting working files. If they are included in the final data file, they do not 

have the same variable names. 
B-3 

Table B-1. Unknown eligibility and nonresponse adjustment cells for C6CW0: School year 2003–04 

Adjustment factor 

Cell R6MOVST RC125P125 RP4 LMSTATUS

Unknown 

eligibility Nonresponse

1 0 1 all All 1.00000 1.00520

2 0 2 1 All 1.00780 1.04758

3 0 2 2 1 1.45563 1.01915

4 0 2 2 2 2.02103 1.13393

5 1 1 all All 1.04393 1.19431

6 1 2 all All 1.12945 1.80780

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

Table B-2. Unknown eligibility and nonresponse adjustment cells for C6PW0: School year 2003–04 

Adjustment factor 

Cell R6MOVST HHTYPE RP5 LMSTATUS RC5

Unknown 

eligibility Nonresponse

1 0 1, 2, 4, 5 1 1 all 1.00000 1.06209

2 0 1, 2, 4, 5 1 2 all 1.00000 1.03967

3 0 1, 2, 4, 5 2 1 all 1.26456 1.38246

4 0 1, 2, 4, 5 2 2 all 1.40014 1.57344

5 0 3, 9 1 all all 1.00000 1.08564

6 0 3, 9 2 1 all 1.29265 1.37617

7 0 3, 9 2 2 all 1.79527 1.85378

8 1 1, 2, 4, 5 1 all 1 1.03383 1.07452

9 1 1, 2, 4, 5 1 all 2 1.05715 1.14946

10 1 1, 2, 4, 5 2 all all 1.28633 1.60367

11 1 3, 9 1 all 1 1.08367 1.11359

12 1 3, 9 1 all 2 1.17165 1.02198

13 1 3, 9 2 all all 1.16237 1.81606

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 
B-4 

Table B-3. Unknown eligibility and nonresponse adjustment cells for C6CPTR0: School year 2003–04 

Adjustment factor 

Cell HOMESCH R6MOVST HHTYPE RP5 LMSTATUS STYPE 

Unknown 

eligibility Nonresponse 

1 1 all all all all all 1.00000 1.58484

2 0 0 1,2,4,5 1 1 1,2,3 1.00000 1.07445

3 0 0 1,2,4,5 1 1 4,9 1.00000 1.12005

4 0 0 1,2,4,5 1 2 1,2,3 1.00000 1.04118

5 0 0 1,2,4,5 1 2 4,9 1.00000 1.06871

6 0 0 1,2,4,5 2 1 all 1.26456 1.45233

7 0 0 1,2,4,5 2 2 all 1.40014 1.64009

8 0 0 3,9 1 all all 1.00000 1.10832

9 0 0 3,9 2 1 all 1.29265 1.48494

10 0 0 3,9 2 2 all 1.79527 1.89980

11 0 1 all all all 1,2,3 1.00000 1.21747

12 0 1 all all 1 4,9 1.11289 1.64403

13 0 1 all all 2 4,9 1.08163 1.84413

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 

(ECLS-K), spring 2004. 

Table B-4. Unknown eligibility and nonresponse adjustment cells for C6CPTM0: School year 2003–04 

Adjustment factor 

Cell HOMESCH R6MOVST HHTYPE STYPE RP5 LMSTATUS 

Unknown 

eligibility Nonresponse 

1 1 All all all all all 1.00000 2.07187

2 0 0 1,5 1,2,3 all 1 1.00000 1.08125

3 0 0 1,5 1,2,3 all 2 1.00000 1.05655

4 0 0 1,5 4,9 1 1 1.00000 1.11565

5 0 0 1,5 4,9 1 2 1.00000 1.07240

6 0 0 1,5 4,9 2 1 1.00000 1.43444

7 0 0 1,5 4,9 2 2 1.00000 1.62500

8 0 0 2,3,4,9 1,2,3 all 1 1.00000 1.35998

9 0 0 2,3,4,9 1,2,3 all 2 1.00000 1.16894

10 0 0 2,3,4,9 4,9 1 all 1.00000 1.10150

11 0 0 2,3,4,9 4,9 2 1 1.00000 1.39477

12 0 0 2,3,4,9 4,9 2 2 1.00000 1.55784

13 0 1 all 1,2,3 all all 1.00000 1.15601

14 0 1 all 4,9 1 1 1.07308 1.51809

15 0 1 all 4,9 1 2 1.04336 1.87984

16 0 1 all 4,9 2 all 1.20402 2.13094

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 

(ECLS-K), spring 2004. 
B-5 

Table B-5. Unknown eligibility and nonresponse adjustment cells for C6CPTS0: School year 2003–04 

Adjustment factor 

Cell HOMESCH R6MOVST RP5 STYPE LMSTATUS HHTYPE 

Unknown 

eligibility Nonresponse 

1 1 All all all all all 1.00000 1.10612

2 0 0 1 1,2,3 1 all 1.00000 1.06891

3 0 0 1 1,2,3 2 all 1.00000 1.04568

4 0 0 1 4,9 1 1,2,3 1.00000 1.11746

5 0 0 1 4,9 1 4,5,9 1.00000 1.20287

6 0 0 1 4,9 2 1,2,4 1.00000 1.06330

7 0 0 1 4,9 2 3,5,9 1.00000 1.10587

8 0 0 2 all 1 all 1.00000 1.47316

9 0 0 2 all 2 all 1.00000 1.83441

10 0 1 all 1,2,3 all all 1.00000 1.28755

11 0 1 all 4,9 1 all 1.11629 1.69071

12 0 1 all 4,9 2 all 1.09946 1.81234

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 

(ECLS-K), spring 2004. 
B-6 

Table B-6. Unknown eligibility and nonresponse adjustment cells for C56CW0: 

School year 2003–04 

Adjustment factor 

Cell LFMOVST RC125P125 LMSTATUS 

Unknown 

eligibility Nonresponse 

1 0 1 all 1.00000 1.00499 

2 0 2 all 1.20681 1.09388 

3 1 1 all 1.04013 1.17679 

4 1 2 1 1.16114 1.81191 

5 1 2 2 1.09921 2.14838 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal 

Study, Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 

Table B-7. Unknown eligibility and nonresponse adjustment cells for C56PW0: School year 2003–04 

Adjustment factor 

Cell LFMOVST RC125P125 STYPE LMSTATUS HHTYPE

Unknown 

eligibility Nonresponse 

1 0 1 1, 2, 3 all all 1.00000 1.02039 

2 0 1 4, 9 all all 1.00000 1.04605 

3 0 2 all 1 all 1.14175 2.19706 

4 0 2 all 2 all 1.22822 1.86073 

5 1 1 all all all 1.04013 1.07404 

6 1 2 all all 1, 4, 5 1.10370 1.66966 

7 1 2 all all 2, 3, 9 1.13694 2.13831 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–

99 (ECLS-K), spring 2004. 

Table B-8. Unknown eligibility and nonresponse adjustment cells for C456CW0: School year 2003–04 

Adjustment factor 

Cell LFMOVST RC1245P1245 STYPE RC12P12 LMSTATUS

Unknown 

eligibility Nonresponse 

1 0 1 all all all 1.00000 1.00498

2 0 2 all 1 all 1.06836 1.27287

3 0 2 all 2 all 1.25666 1.09852

4 1 all 1, 2, 3 all all 1.00000 1.18872

5 1 all 4, 9 1 all 1.07156 1.51868

6 1 all 4, 9 2 1 1.13642 1.63119

7 1 all 4, 9 2 2 1.08907 1.97756

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 
B-7 

Table B-9. Unknown eligibility and nonresponse adjustment cells for C456PW0: School year 2003–04 

Adjustment factor 

Cell LFMOVST LMSTATUS HHTYPE STYPE RC12P12

Unknown 

eligibility Nonresponse 

1 0 1 1, 5 all all 1.03557 1.29913

2 0 1 2, 3, 4, 9 all all 1.06620 1.51069

3 0 2 1, 2 1, 2, 3 all 1.04308 1.09147

4 0 2 1, 2 4, 9 all 1.02968 1.17355

5 0 2 3, 4, 5, 9 all all 1.12449 1.36726

6 1 1 1 all all 1.08017 1.41610

7 1 1 2, 3, 4, 5, 9 all all 1.13332 1.77362

8 1 2 1, 2, 4, 5 all 1 1.04744 1.22519

9 1 2 1, 2, 4, 5 all 2 1.09761 1.73074

10 1 2 3, 9 all 1 1.10180 1.53178

11 1 2 3, 9 all 2 1.06019 2.05338

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 
B-8 

Table B-10. Unknown eligibility and nonresponse adjustment cells for C2_6FC0: School year 2003–04 

Adjustment factor 

Cell LFMOVST RC1245P1245 STYPE HHTYPE RC12P12

Unknown 

eligibility Nonresponse 

1 0 1 all all all 1.00000 1.00498

2 0 2 1,2,3 1,2,5 1 1.10270 1.27759

3 0 2 1,2,3 1,2,5 2 1.23869 1.08868

4 0 2 1,2,3 3,4,9 all 1.48098 1.37939

5 0 2 4,5,9 1,2,5 1 1.06672 1.33018

6 0 2 4,5,9 1,2,5 2 1.17624 1.14951

7 0 2 4,5,9 3,4,9 all 1.30405 1.17559

8 1 all all 1,2,3 all 1.07001 1.53865

9 1 all all 4,5,9 all 1.09007 1.95635

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 

Table B-11. Unknown eligibility and nonresponse adjustment cells for C2_6FP0: School year 

2003–04 

Adjustment factor 

Cell LFMOVST LMSTATUS HHTYPE STYPE 

Unknown 

eligibility Nonresponse 

1 0 1 all all 1.04424 1.37476 

2 0 2 all 1, 2, 3 1.06020 1.11509 

3 0 2 1-2 4, 9 1.03026 1.19437 

4 0 2 3-9 4, 9 1.11064 1.40976 

5 1 1 1 all 1.07866 1.43354 

6 1 1 2-9 all 1.13166 1.86774 

7 1 2 1, 2, 4, 5 all 1.05586 1.33576 

8 1 2 3, 9 all 1.09101 1.74837 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 

Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 
B-9 

Table B-12. Unknown eligibility and nonresponse adjustment cells for C1_6FC0: School year 

2003–04 

Adjustment factor 

Cell LFMOVST STYPE RC1245P1245 LMSTATUS 

Unknown 

eligibility Nonresponse 

1 0 1, 2, 3 all all 1.06076 1.19946

2 0 4, 9 1 all 1.00000 1.00476

3 0 4, 9 2 1 1.12199 1.48187

4 0 4, 9 2 2 1.20797 1.86898

5 1 1, 2, 3 all all 1.00000 1.29333

6 1 4, 9 all 1 1.10195 1.65526

7 1 4, 9 all 2 1.07065 1.77467

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 

Kindergarten Class of 1998–99 third (ECLS-K), spring 2004. 

Table B-13. Unknown eligibility and nonresponse adjustment cells for C1_6FP0: School year 

2003–04 

Adjustment factor 

Cell LFMOVST LMSTATUS HHTYPE STYPE

Unknown 

eligibility Nonresponse 

1 0 1 all all 1.04424 1.51744

2 0 2 1-2 all 1.03228 1.29147

3 0 2 3-9 all 1.11400 1.56542

4 1 1 1 all 1.07866 1.60947

5 1 1 2-9 all 1.13166 2.08877

6 1 2 all 1, 2, 3 1.00000 1.33747

7 1 2 1, 2, 5 4, 9 1.06203 1.42833

8 1 2 3, 4, 9 4, 9 1.08873 1.94624

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, 

Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 
B-10 

Table B-14. Unknown eligibility and nonresponse adjustment cells for 

C1_6SC0: School year 2003–04 

Adjustment factor 

Cell LSMOVST STYPE HHTYPE

Unknown 

eligibility Nonresponse 

1 0 1, 2, 3 all 1.08002 1.10377

2 0 4, 9 1, 3, 5 1.04662 1.14636

3 0 4, 9 2, 4, 9 1.20938 1.29705

4 1 1, 2, 3 all 1.00000 1.30250

5 1 4, 9 all 1.06725 1.86940

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood 

Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 

Table B-15. Unknown eligibility and nonresponse adjustment cells for C1_6SP0: School year 

2003–04 

Adjustment factor 

Cell LSMOVST LMSTATUS STYPE HHTYPE

Unknown 

eligibility Nonresponse 

1 0 1 all all 1.04988 1.58359 

2 0 2 1, 2, 3 all 1.07635 1.25392 

3 0 2 4, 9 1, 2 1.04169 1.30604 

4 0 2 4, 9 3-9 1.15726 1.64778 

5 1 all 1, 2, 3 all 1.00000 1.29231 

6 1 all 4, 9 1, 2 1.06374 1.56525 

7 1 all 4, 9 3-9 1.07412 2.20842 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal 

Study, Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 
C-1 

APPENDIX C: RAKING DIMENSIONS 

The raking dimensions are in the weighting working files. They are not included in any delivery files. 

Table C-1. Raking dimension 1—sex by age: School year 2003–04 

DIM11

 NEWGENDER NEWAGE Description 

11 1 1 Male; age 7 or 8 

12 1 2 Male; age 9 or missing 

13 1 3 Male; age 10 or 11 

21 2 1 Female; age 7 or 8 

22 2 2,3 Female; age 9, 10, 11 or missing 

1

For C1_6SC0 and C1_6SP0, cell 13 was merged with cell 12. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal 

Study, Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 

Table C-2. Raking dimension 2—region by locale: School year 2003–04 

DIM2 NEWREGION NEWLOCALE Description 

11 1 1 Northeast; central city 

12 1 2 Northeast; suburb/large town 

13 1 3 Northeast; small town/rural 

21 2 1 Midwest; central city 

22 2 2 Midwest; suburb/large town 

23 2 3 Midwest; small town/rural 

31 3 1 South; central city 

32 3 2 South; suburb/large town 

33 3 3 South; small town/rural 

41 4 1 West; central city 

42 4 2 West; suburb/large town 

43 4 3 West; small town/rural 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal 

Study, Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 
C-2 

Table C-3. Raking dimension 3—race/ethnicity by SES quintile: School year 2003–04 

DIM31

 NEWRACE NEWSESQ5 Description 

11 1 1 White; SES Q1 

12 1 2 White; SES Q2 

13 1 3 White; SES Q3 

14 1 4 White; SES Q4 

15 1 5 White; SES Q5 

21 2 1 Black; SES Q1 

22 2 2 Black; SES Q2 

23 2 3 Black; SES Q3 

24 2 4,5 Black; SES Q4 or Q5 

31 3 1 Hispanic; SES Q1 

32 3 2 Hispanic; SES Q2 

33 3 3 Hispanic; SES Q3 

34 3 4 Hispanic; SES Q4 

35 3 5 Hispanic; SES Q5 

41 4 1,2 API; SES Q1 or Q2 

42 4 3,4 API; SES Q3 or Q4 

43 4 5 API; SES Q5 

51 5 all American Indian/Alaskan 

61 6 all Other race 

1

For C1_6SC0 and C1_6SP0, cell 23 was merged with cell 22; cell 35 was merged with cell 34; cells 42 

and 43 were merged with cell 41

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal 

Study, Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 

Table C-4. Raking dimension 4—school affiliation: School 

year 2003–04 

DIM4 NEWSTYPE Description 

10 1 Catholic 

20 2 Other religious private 

30 3 Nonreligious private 

40 4 Public 

SOURCE: U.S. Department of Education, National Center for Education 

Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 

(ECLS-K), spring 2004. 
C-3 

Table C-5. Raking dimension 5—language minority status: 

School year 2003–04 

DIM5 LMSTATUS Description 

10 1 Language minority 

20 2 Non-language minority 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early 

Childhood Longitudinal Study, Kindergarten Class of 1998–99 (ECLS-K), spring 2004. 

Table C-6. Raking dimension 6—mover status in spring-first grade: School 

 year 2003–04 

DIM61 MOVERST Description 

10 0 Sampled in KG, spring-first grade nonmover 

20 1 Sampled in KG, spring first-grade mover 

30 9 Sampled in first grade 

1

For longitudinal weights involving spring-kindergarten data, cell 30 was not applicable. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Early Childhood Longitudinal Study, Kindergarten Class of 

1998–99 (ECLS-K), spring 2004. 
C-4 

This page is intentionally left blank.