National Longitudinal Survey of 

Youth 1997 (NLSY97)

Technical Sampling Report

Whitney Moore, Steven Pedlow,

Parvati Krishnamurty, and Kirk Wolter 

National Opinion Research Center 

Chicago, IL 

November 2000
TABLE OF CONTENTS 

 Page

ACKNOWLEDGMENTS ......... ............ ............ ............ ............ ............ ............ ............ ......... 1

LIST OF TABLES AND FIGURES....... ............ ............ ............ ............ ............ ............ .......... 2

CHAPTER 1: INTRODUCTION ........... ............ ............ ............ ............ ............ ............ ........ 11

CHAPTER 2: DESIGN OF THE NLSY97 SAMPLE...... ............ ............ ............ ............ ........ 12

2.1 The Eligible Population ............ ............ ............ ............ ............ ............ ............ ........ 12

2.2 Overview of the Sample ............ ............ ............ ............ ............ ............ ............ ........ 14

2.3 Screening Questions and Forms. ............ ............ ............ ............ ............ ............ ........ 18

CHAPTER 3: SELECTION OF THE NLS97 SAMPLE . ............ ............ ............ ............ ........ 20

3.1 Selection of the Cross-Sectional (CX) Sample .... ............ ............ ............ ............ ........ 20

3.2 Selection of the Supplemental (SU) Sample........ ............ ............ ............ ............ ........ 24

CHAPTER 4: WEIGHTING THE SAMPLE...... ............ ............ ............ ............ ............ ........ 29

CHAPTER 5: ACCURACY AND PRECISION OF THE SAMPLE RESULTS ... ............ ........ 38

5.1 Potential Impact of Nonresponse ............ ............ ............ ............ ............ ............ ........ 38

5.2 Refielding......... ............ ............ ............ ............ ............ ............ ............ ............ ........ 46

5.3 Representativeness of the Sample........... ............ ............ ............ ............ ............ ........ 49

5.4 Estimates of the NLSY97 Shortfall ........ ............ ............ ............ ............ ............ ........ 71

5.5 Estimates of Variance ... ............ ............ ............ ............ ............ ............ ............ ........ 81

CHAPTER 6: SUMMARY OF THE ACCURACY AND REPRESENTATIVENESS 

OF THE NLSY97 SAMPLES ... ............ ............ ............ ............ ............ ............ ............ ........ 96

APPENDIX A: Construction of Demographic Variables for the NLSY97 ............ ............ ........ 99

APPENDIX B: This appendix is blank... ............ ............ ............ ............ ............ ............ ...... 111

APPENDIX C: This appendix is blank... ............ ............ ............ ............ ............ ............ ...... 112

APPENDIX D: Analysis of Language Barrier Cases ...... ............ ............ ............ ............ ...... 113

APPENDIX E: Review of Proxy Interviews ...... ............ ............ ............ ............ ............ ...... 131

2
APPENDIX F: PSU-Level Socio-Economic-Status Analysis ....... ............ ............ ............ ...... 135

APPENDIX G: Comparison of the NLSY97 Family Income Distribution to the CPS

 Family Income Distribution....... ............ ............ ............ ............ ............ ...... 141

APPENDIX H: Introduction and Paper Screener ............ ............ ............ ............ ............ ...... 156

APPENDIX I: Introduction and Age Questions from Extended Screener.. ............ ............ ...... 160

APPENDIX J: Proxy Introduction and Paper Screener .... ............ ............ ............ ............ ...... 167

APPENDIX K: Advance Letters and Brochure... ............ ............ ............ ............ ............ ...... 170

APPENDIX L: Review of Gatekeeper Interviews............ ............ ............ ............ ............ ...... 177

APPENDIX M: NIR Reports NLSY97 Round 1. ............ ............ ............ ............ ............ ...... 181

APPENDIX N: Report on NLSY97 Computer Crashes ... ............ ............ ............ ............ ...... 196

APPENDIX O: Report on NLSY97 Debriefing Interviews .......... ............ ............ ............ ...... 204

APPENDIX P: Analysis of Interview Validation ............ ............ ............ ............ ............ ...... 215

APPENDIX Q: Analysis of Parent Education for the NLSY97 .... ............ ............ ............ ...... 220

APPENDIX R: Two Comparisons of Expected vs. Observed Youth Counts ......... ............ ...... 226

APPENDIX S: Adjusted Screener Weights Used for Age Distribution Analysis ... ............ ...... 243

APPENDIX T: Urban/Rural Age Distributions... ............ ............ ............ ............ ............ ...... 247

3
Acknowledgments

We would like to acknowledge the help of the following people in preparing this report: Michael 

Horrigan, Bureau of Labor Statistics; Clessen J. Martin and Linda Curran, Defense Manpower Data

Center; and Gail Hoff and Harrison Greene, National Opinion Research Center. Some of the appendices 

have been written by Rashna Ghadialy, Lisa Lee, and Sally Murphy of the National Opinion Research 

Center.

1
List of Tables and Figures 

Tables Page 

2.1 Target Sample Sizes for the NLSY97........... ............ ............ ............ ............ ............ ........ 14

2.2 Sizes of the CX and SU Samples..... ............ ............ ............ ............ ............ ............ ........ 15

2.3 Results of Screening Operations...... ............ ............ ............ ............ ............ ............ ........ 16

2.4 Achieved Sample Sizes for the NLSY97...... ............ ............ ............ ............ ............ ........ 17

3.1 Distribution of CX PSUs, by Census Region and Metropolitan Status ............ ........ 20

3.2 Projected Screening Sample Sizes and Yields Used in Planning the NLSY97 ............ ........ 23

3.3 Actual Screening Sample Sizes........ ............ ............ ............ ............ ............ ............ ........ 24

3.4 Sample Sizes and Sampling Rates Used in Determining the

 Measure for Size of the SU Sample . ............ ............ ............ ............ ............ ............ ........ 25

3.5 Distribution of SU PSUs, by Census Region and Metropolitan Status ............ ............ ........ 25

3.6 Summary of the Design of the CX and SU Samples . ............ ............ ............ ............ ........ 28

4.1 Unadjusted and Capped Missed Housing Unit Adjustments .. ............ ............ ............ ........ 31

4.2 s Used for Precision Weighting to Combine the CX and SU Samples........... ............ ........ 33

4.3 Cell (Weighted) Response Rates (in Percent) Used for the NLSY97

 Nonresponse Weighting Adjustment ............ ............ ............ ............ ............ ............ ........ 34

4.4 Population Totals Used for NLSY97 Poststratification ......... ............ ............ ............ ........ 35

4.5 Coverage Ratios for NLSY97 Sample .......... ............ ............ ............ ............ ............ ........ 36

4.6 Summary Table of Housing Unit and Household Weights..... ............ ............ ............ ........ 37

4.7 Summary Table for Youth Weights. ............ ............ ............ ............ ............ ............ ........ 38

5.1 Final Response Status for All Screeners ....... ............ ............ ............ ............ ............ ........ 38

5.2 Screener Completion Rates. ............ ............ ............ ............ ............ ............ ............ ........ 39

5.3 Final Response Status for NLSY97 Youths.. ............ ............ ............ ............ ............ ........ 39

5.4 ASVAB Test Status for NLSY97 Youths..... ............ ............ ............ ............ ............ ........ 40

5.5 Summary of Response Rates (in Percent), Overall and by Domain .... ............ ............ ........ 41

2
5.6 Distributions (Unweighted) of Screener Variables for CPS Youths, Completed

 Screener Respondents, and NLSY97 Interview and Test Completes .. ............ ............ ........ 45

5.7 True Overall Proportions of Youths Given Varying Proportions for Nonparticipants and

 Varying Response Rates ..... ............ ............ ............ ............ ............ ............ ............ ........ 46

5.8 Screener Response Status Before and After Refielding.......... ............ ............ ............ ........ 47

5.9 Screener Response Status Before and After Refielding: CX Sample . ............ ............ ........ 48

5.10 Screener Response Status Before and After Refielding: SU Sample ............ ............ ........ 48

5.11 Response Status for the Youth Interview Before and After Refielding.......... ............ ........ 48

5.12 Weighted-up 1990 Census Counts of Youths Under Age 18 and of Housing 

 Units for the CX Segments ............ ............ ............ ............ ............ ............ ........ 50

5.13 Weighted-up 1990 Census Counts of Minority Youths Under Age 18

 for the SU Segments .......... ............ ............ ............ ............ ............ ............ ............ ........ 51

5.14 Weighted-up Listing Counts for the CX Sample ..... ............ ............ ............ ............ ........ 52

5.15 Comparison of Census and Listing Counts in the SU Sample........... ............ ............ ........ 54

5.16 Sums of Base Weights for Selected Housing Units. ............ ............ ............ ............ ........ 54

5.17 Comparison of the NLSY97 Vacancy Status of Housing Units ........ ............ ............ ........ 55

5.18 Effect of the Weighting Caps on the Sums of the Household Weights .......... ............ ........ 56

5.19 Distribution of Households by Reported Number of Youths Aged 12 to 16.. ............ ........ 58

5.20 Conditional Distribution of Households (HHs) by Reported Number of Youths Aged 

 12 to 16, Given at Least One Such Youth... ............ ............ ............ ............ ............ ........ 59

5.21 Conditional Mean Youths, Age 12 to 16, Per Household..... ............ ............ ............ ........ 60

5.22 Average Household Size .. ............ ............ ............ ............ ............ ............ ............ ........ 60

5.23 Distribution of Households by Size of Household... ............ ............ ............ ............ ........ 61

5.24 Youths as a Proportion of Total Persons Screened .. ............ ............ ............ ............ ........ 61

5.25 Race by Gender Percentages for CPS vs. NLSY97-Eligible Sample ............ ............ ........ 62

5.26 Race by Gender Percentages for CPS vs. NLSY97 Interviews ......... ............ ............ ........ 62

5.27 Race by Gender Percentages for CPS vs. NLSY97 Test Completes . ............ ............ ........ 63

3
5.28 Estimated 1990 Census Coverage Ratios (March, 1998)...... ............ ............ ............ ........ 66

5.29 Estimated CPS Coverage Ratios (March, 1998)...... ............ ............ ............ ............ ........ 67

5.30 Estimated NLSY97/PAY97 Coverage Ratios.......... ............ ............ ............ ............ ........ 68

5.31 Analysis of Actual versus Expected Youths in CX Screening Sample: Method 1A ... ........ 73

5.32 Analysis of Actual versus Expected Youths in SU Screening Sample: Method 1A.. ........ 75

5.33 Analysis of Actual versus Expected Youths in NLSY97 Screening Sample

 (CX + SU): Method 1A.... ............ ............ ............ ............ ............ ............ ............ ........ 76

5.34 Estimates of the Shortfall of Youths Aged 12-16 in the NLSY97 Screening

 Sample: Method 2........... ............ ............ ............ ............ ............ ............ ............ ........ 77

5.35 Expected Youths versus Actual Youths Screened: Method 3A......... ............ ............ ........ 78

5.36 Five Estimates of the Shortfall of Eligible Youths .. ............ ............ ............ ............ ........ 78

5.37 Variance Estimates for Overall NLSY97 Sample: Proportions ....... ............ ............ ........ 83

5.38 Variance Estimates for Overall NLSY97 Sample: Means ... ............ ............ ............ ........ 84

5.39 Variance Estimates for NLSY97 Males: Proportions ........... ............ ............ ............ ........ 85

5.40 Variance Estimates for NLSY97 Males: Means ...... ............ ............ ............ ............ ........ 86

5.41 Variance Estimates for NLSY97 Females: Proportions........ ............ ............ ............ ........ 87

5.42 Variance Estimates for NLSY97 Females: Means... ............ ............ ............ ............ ........ 88

5.43 Variance Estimates for NLSY97 Hispanics: Proportions ..... ............ ............ ............ ........ 89

5.44 Variance Estimates for NLSY97 Hispanics: Means ............ ............ ............ ............ ........ 90

5.45 Variance Estimates for NLSY97 Non-Hispanic Blacks: Proportions ............ ............ ........ 91

5.46 Variance Estimates for NLSY97 Non-Hispanic Blacks: Means........ ............ ............ ........ 92

5.47 Variance Estimates for NLSY97 Non-Hispanic Nonblacks: Proportions ...... ............ ........ 93

5.48 Variance Estimates for NLSY97 Non-Hispanic Nonblacks: Means ............ ............ ........ 94

5.49 Comparison of Summary Statistics of NLSY97 DEFTS..... ............ ............ ............ ........ 95

A-1 Crosstabulation of Screener Hispanic Origin by OLQ Ethnicity (recoded) ............ ...... 101

A-2 Crosstabulation of Screener Hispanic Origin by OLQ Origin (recoded).... ............ ...... 102

A-3 Final Hispanic Origin Frequencies ......... ............ ............ ............ ............ ............ ...... 102

4
A-4 Crosstabulation of Screener Sex by Youth Questionnaire Sex...... ............ ............ ...... 104

A-5 Crosstabulation of Screener Sex by Parent Questionnaire Sex...... ............ ............ ...... 104

A-6 Final Sex Frequencies... ............ ............ ............ ............ ............ ............ ............ ...... 104

A-7 Crosstabulation of Screener Race by OLQ Origin (recoded) ........ ............ ............ ...... 106

A-8 Final Race Frequencies. ............ ............ ............ ............ ............ ............ ............ ...... 106

A-9 Crosstabulation of OLQ Year of Birth by Screener Year of Birth ............ ............ ...... 109

A-10 Crosstabulation of Youth Questionnaire Year of Birth by Screener Year of Birth . ...... 109

A-11 Age Frequencies............ ............ ............ ............ ............ ............ ............ ............ ...... 110

D-1 Distribution (Percent) of American Youths by Ability to Speak English

and by Hispanic Origin . ............ ............ ............ ............ ............ ............ ............ ...... 115

D-2 Summary of Total Student and LEP Enrollments, 1996-97 .......... ............ ............ ...... 116

D-3 Number and Percent of LEP Students in Each Grade Level.......... ............ ............ ...... 117

D-4 Percentage of LEP Students by Language Group, 1991-92........... ............ ............ ...... 118

D-5 Percentages of LEP Students Identified and Excluded from the NAEP Sample..... ...... 119

D-6 ETP Status* Sample Crosstabulation....... ............ ............ ............ ............ ............ ...... 121

D-7 ETP: OOS, Language Barrier* Hispanic Origin Crosstabulation... ............ ............ ...... 122

D-8 STP Status * Sample Crosstabulation ............ ............ ............ ............ ............ ...... 123

D-9 STP: OOS, Language Barrier* Hispanic Origin Crosstabulation... ............ ............ ...... 124

D-10 Youths Screened into the ETP: PSUs with Proportion Language Barrier

Greater Than or Equal to 0.10.... ............ ............ ............ ............ ............ ............ ...... 128

D-11 Estimates of English Language Deficiency ......... ............ ............ ............ ............ ...... 129

E-1 Review of Proxy Record of Calls ........... ............ ............ ............ ............ ............ ...... 134

F-1 PSU-level Socio-Economic Correlations: Education ...... ............ ............ ............ ...... 136

F-2 PSU-level Socio-Economic Correlations: Income........... ............ ............ ............ ...... 138

G-1 Total NLSY97 Household Wages and Salary Income...... ............ ............ ............ ...... 148

G-2 Total NLSY97 Household Self-Employment Income ...... ............ ............ ............ ...... 148

G-3 Total NLSY97 Household Earned Income (Wages/Salary + Self-Employment).... ...... 148

5
G-4 Total NLSY97 Household Income (All Sources) ............ ............ ............ ............ ...... 148

G-5 Total NLSY97 Household Wages and Salary Income: Cross-Sectional Sample ... ...... 149

G-6 Total NLSY97 Household Self-Employment Income: Cross-Sectional Sample.... ...... 149

G-7 Total NLSY97 Household Earned Income: Cross-Sectional Sample........ ............ ...... 149

G-8 Total NLSY97 Household Income: Cross-Sectional Sample ....... ............ ............ ...... 149

G-9 Total CPS Household Wages and Salary Income ............ ............ ............ ............ ...... 150

G-10 Total CPS Household Self-Employment Income ............ ............ ............ ............ ...... 150

G-11 Total CPS Household Earned Income ... ............ ............ ............ ............ ............ ...... 150

G-12 Total CPS Household Income.... ............ ............ ............ ............ ............ ............ ...... 150

G-13 Total CPS Household Wages and Salary Income (Reported Data Only) ... ............ ...... 151

G-14 Total CPS Household Self-Employment Income (Reported Data Only).... ............ ...... 151

G-15 Total CPS Household Earned Income (Reported Data Only)........ ............ ............ ...... 151

G-16 Total CPS Household Income (Reported Data Only)....... ............ ............ ............ ...... 151

G-17 Total NLSY97 Household Wages and Salary Income by Race/Ethnicity.. ............ ...... 152

G-18 Total NLSY97 Household Self-Employment Income by Race/Ethnicity .. ............ ...... 152

G-19 Total NLSY97 Household Earned Income by Race/Ethnicity ...... ............ ............ ...... 153

G-20 Total NLSY97 Household Income by Race/Ethnicity...... ............ ............ ............ ...... 153

G-21 Total CPS Household Wages and Salary Income by Race/Ethnicity ......... ............ ...... 154

G-22 Total CPS Household Self-Employment Income by Race/Ethnicity.......... ............ ...... 154

G-23 Total CPS Household Earned Income by Race/Ethnicity. ............ ............ ............ ...... 155

G-24 Total CPS Household Income by Race/Ethnicity ............ ............ ............ ............ ...... 155

L-1 Review of Gatekeeper Record of Calls... ............ ............ ............ ............ ............ ...... 180

M-1 NIR Sample Questions Divided Into Three Main Sections ........... ............ ............ ...... 185

M-2 Reasons for a Non-Interview .... ............ ............ ............ ............ ............ ............ ...... 187

M-3 Comparison NIR Reports Due to Refusals .......... ............ ............ ............ ............ ...... 188

M-4 All Inhabited Households Compared to Those Where Potentially

Eligible Youth Were Identified.. ............ ............ ............ ............ ............ ............ ...... 190

6
M-5 Composition of NIR Household ............ ............ ............ ............ ............ ............ ...... 191

M-6 Ratio of Youths to Household Units and to Household Members. ............ ............ ...... 192

M-7 Determination of the Number of Potentially Eligible Youths ....... ............ ............ ...... 192

N-1 PSU 721..... .................... .................... ................... .................... .................... ................... .................... .................... ............. 198

N-2 PSU 764..... .................... .................... ................... .................... .................... ................... .................... .................... ............. 199

N-3 PSU 795..... .................... .................... ................... .................... .................... ................... .................... .................... ............. 200

N-4 PSU 862..... .................... .................... ................... .................... .................... ................... .................... .................... ............. 201

N-5 Location of Largest Numbers of Crashed/Technical Problems and “No HU” Cases .... 203

O-1 Number of Times a Computer Problem Prevented Completion

of a Screening Interview. .................... ................... .................... .................... ................... .................... ..............207

O-2 Types of Computer Problems Experienced by FIs . .................... .................... ................... ..............207.................... ................... ....................

O-3 Computer Problems in NLSY97 and Prior CAPI Experience ..... .................... ................... ..............208

O-4 What FIs Told Informants About Who is Eligible to Participate in NLSY97 . ............. 208

O-5 Number of FIs Giving Each Rating of Difficulty in Gaining Cooperation...... ................... ..............208....................

O-6 FIs’ Ratings on How Often Informants Read Advance Letter and Brochure .. ............. 209....................

O-7 FIs’ Ratings on How Often Informants Did Not Report Accurately

on Household Roster ...... .................... ................... .................... .................... ................... .................... .................... ............. 209

O-8 FI Reports of Questions and Concerns Expressed by NLSY97 Informants..... ............. 209

P-1 NLSY97 Round 1 Validation Interviews ............ ............ ............ ............ ............ ...... 218

P-2 NLSY97 Round 1 Validation Interviews Ineligible Screeners Only.......... ............ ...... 218

P-3 NLSY97 Round 1 Validation Interview Respondent Comments Regarding the 

Survey or Experience with Interviewer .. ............ ............ ............ ............ ............ ...... 219

P-4 Elapsed Time Between Screener and Validation Interview for 105 

Screeners for Which Validation was Other Than Successful ........ ............ ............ ...... 219

Q-1 CPS and NLSY97 Components of Final Grade Categories........... ............ ............ ...... 222

Q-2 Youths by Presence of Parent – Comparison of Data Sets ............ ............ ...... 223

Q-3 Extent of Missingness of the Data .......... ............ ............ ............ ............ ............ ...... 223

7
Q-4 Father’s Highest Grade Attained ............ ............ ............ ............ ............ ............ ...... 224

Q-5 Mother’s Highest Grade Attained........... ............ ............ ............ ............ ............ ...... 224

R-1 Expected Counts of Youths Aged 12-16 to be Screened.. ............ ............ ............ ...... 229

R-2 Actual Counts of Youths Aged 12-16 Screened .. ............ ............ ............ ............ ...... 229

R-3 Youths Aged 12-16 Actually Screened as a Percent of Expected Youths.. ............ ...... 229

R-4 Some PSU-Level Percentile Statistics on Percentage Variables

(Percentage of Expected Youths Screened) ......... ............ ............ ............ ............ ...... 232

R-5 Classification of Four PSU-Level Percentage Variables .. ............ ............ ............ ...... 234

R-6 Expected Youths Aged 12-16 to be Screened for 10 “Lowest” CX and SU PSUs.. ...... 235

R-7 Actual Youths Aged 12-16 Screened for 10 “Lowest” CX and SU PSUs.. ............ ...... 236

R-8 Actual Youths Aged 12-16 Screened as a Percentage of Expected

for 10 “Lowest” CX and SU PSUs ......... ............ ............ ............ ............ ............ ...... 237

S-1 The Four Groups Used in Step 2b of the Age-Distribution Weighting ...... ............ ...... 245

S-2 Os Used for Precision Weighting for Age-Distribution Weights... ............ ............ ...... 246

T-1 Unweighted NLSY97 Age Distributions for Urban/Rural Groups ............ ............ ...... 253

T-2 Weighted NLSY97 Age Distributions for Urban/Rural Groups.... ............ ............ ...... 254

8
Figures Page 

2.1 Map of CX and SU Samples for NLSY97..... ............ ............ ............ ............ ............ ........ 17

5.1 Screener Completion by PSU ........... ............ ............ ............ ............ ............ ............ ........ 42

5.2 Distribution of Screener Type A1 Nonresponse by PSU......... ............ ............ ............ ........ 42

5.3 Screener Completion by Segment..... ............ ............ ............ ............ ............ ............ ........ 42

5.4 Distribution of Screener Type A1 Nonresponse by Segment .. ............ ............ ............ ........ 42

5.5 Weighted Counts of Persons, Total Sample... ............ ............ ............ ............ ............ ........ 57

5.6 Weighted Counts of Persons, CX Sample ..... ............ ............ ............ ............ ............ ........ 58

5.7 Weighted Counts of Persons, SU Sample...... ............ ............ ............ ............ ............ ........ 58

D-1 ETP: Language Barrier versus Hispanic Origin, PSU Level. ............ ............ ............ ...... 125

D-2 ETP: Language Barrier versus Hispanic Origin, PSU Level, Cross-Sectional Sample ...... 126

D-3 ETP: Language Barrier versus Hispanic Origin, PSU Level, Supplemental Sample ... ...... 127

F-1 Scatterplot of PSU Screener Response Rate by PSU Bachelor Degree (or Higher) Rates ... 137

F-2 Scatterplot of PSU Screener Response Rate by PSU Median Family Income.. ............ ...... 139

G-1 Percentages of Youths within Income Categories, CPS vs. NLSY ..... ............ ............ ...... 146

M-1 Number of Cases: Screeners and Non-Interviews.... ............ ............ ............ ............ ...... 183

M-2 Counting the Number of Youths from the Inhabited Households ...... ............ ............ ...... 195

R-1 Histogram of CX Non-Minority Youth Percentages . ............ ............ ............ ............ ...... 232

R-2 Histogram of CX All Youth Percentages...... ............ ............ ............ ............ ............ ...... 233

R-3 Histogram of SU All Youth Percentages ...... ............ ............ ............ ............ ............ ...... 233

R-4 Relationship of PSU Shortfalls to PSU Screener Response Rates – CX .......... ............ ...... 238

R-5 Relationship of PSU Shortfalls to PSU Screener Response Rates – SU........... ............ ...... 238

R-6 Relationship of PSU Shortfalls to PSU Education – CX ........ ............ ............ ............ ...... 240

R-7 Relationship of PSU Shortfalls to PSU Education – SU ........ ............ ............ ............ ...... 240

R-8 Relationship of PSU Shortfalls to PSU Median Family Income – CX ............ ............ ...... 241

R-9 Relationship of PSU Shortfalls to PSU Median Family Income – SU ............ ............ ...... 241

T-1 Weighted Percentages of Persons, NLSY97 Urban Blocks and CPS .. ............ ............ ...... 249

9
T-2 Weighted Percentages of Persons, NLSY97 Rural Blocks and CPS ... ............ ............ ...... 249

T-3 Weighted Percentages of Persons, NLSY97 Urban Central Place

 Blocks and CPS... ............ ............ ............ ............ ............ ............ ............ ............ ...... 250

T-4 Weighted Percentages of Persons, NLSY97 Urban/Fringe Blocks and CPS.... ............ ...... 250

FT-5 Weighted Percentages of Persons, NLSY97 Urban,

 Outside UA Blocks and CPS ....... ............ ............ ............ ............ ............ ............ ...... 251

10
Chapter 1. Introduction 

The National Longitudinal Survey of Youth (NLSY97) is the latest in a series of surveys

sponsored by the U.S. Department of Labor (DOL), Bureau of Labor Statistics (BLS) to examine issues 

surrounding youth entry into the work force and subsequent transitions in and out of the work force. 

Originally done in 1979, the study followed a cohort of approximately 12,700 youths who completed a 

round 1 baseline interview. The NLSY97 will follow a new cohort of approximately 9,000 youths who

completed an interview in 1997 (the base year).

The overall study design for NLSY97 incorporated the Profile of American Youth (PAY97)1

 and 

resulted in one large screening sample of over 90,000 housing units to generate youth participants for 

both NLSY97 and PAY97. The NLSY97 sample was selected to represent the civilian, noninstitutional 

population of the United States within the eligible age range – 12 to 16 years of age as of December 31,

1996 -- with oversamples of Hispanics and non-Hispanic blacks. Eligible youths in the screening sample

(and their parents) were administered a questionnaire that covered a range of topics, including work 

experience, education, work-related attitudes, and other labor force and human capital issues.

This Technical Sampling Report gives detailed descriptions of the NLSY97 sample, including the 

large screening sample. We also describe the estimation procedures employed to enable valid inferences 

from the sample to the universe of all eligible American youths. Chapter 2 discusses the design of the 

sample, and Chapter 3 the selection and implementation of the sample. Estimation procedures appear in 

Chapter 4. In Chapter 5, we provide information about the potential impact of nonresponse on the study

results, the representativeness of the sample, and the precision of the survey estimators. We close the 

report in Chapter 6 with a brief summary.

1

 The PAY97, sponsored by the Department of Defense (DoD), is the second round of a large-scale social research

project designed to assess the vocational aptitudes of contemporary American youth and to establish current

national norms for the Armed Services Vocational Aptitude Battery (ASVAB) and for the Interest Finder (IF). This 

project is in two parts; the Enlistment Testing Program (ETP) provided an assessment of roughly 6,000 youths aged 

18-23, and the Student Testing Program (STP) provided an assessment of approximately 4,700 youths enrolled in

grades 10, 11, or 12 as of Fall 1997. 

11
Chapter 2. Design of the NLSY97 Sample

2.1 The Eligible Population

In this section, we define exactly who was included in the universe for the NLSY97. Careful 

definition of the universe is critical for purposes of sampling, designing and administering the screening

interview, and construction of weights. 

The universe definitions are driven fundamentally by two concepts: (1) the housing unit (HU) and 

(2) the usual place of residence. The Census Bureau provides the following definitions: 

1. A housing unit is a single room or group of rooms intended as separate living quarters by a

family, by a group of unrelated persons living together, or by a person living alone. Separate 

living quarters are those for which the occupant does not live or eat with other persons within the 

structure and has direct access from the outside of the building or through a common hall which 

is used or intended to be used by the occupants of another unit or the general public. 

2. A person's usual place of residence is the place where the person lives and sleeps most of the

 time.

These definitions prevailed throughout the NLSY97 project. 

All housing units (HU's) within the land area of the 50 States and the District of Columbia were 

considered within the in-scope universe of structures for NLSY97. The universe explicitly excluded all

group quarters structures, (e.g., prisons, college dormitories, military barracks, and nursing homes).

However, youths living in these group quarters, as specified below, were linked to an in-scope housing 

unit.

All structures within the in-scope universe were subjected to area probability sampling. This 

means that every HU within the universe had a known, nonzero probability of selection. 

The sampling universe for the NLSY97 included 

(a.1) persons aged 12-16 as of the reference day, December 31, 1996 

who met at least one of the following criteria: 

(r.1) persons whose usual place of residence was within the universe of structures set forth in 

 the previous paragraph 

Including

x persons who were temporarily away from their in-scope residence on vacation or in

a general hospital 

12
x persons with no other home who were staying at an in-scope HU at the time of the 

screening interview 

Excluding

x persons who were visiting temporarily and usually live somewhere else, such as 

overseas or in a mental hospital 

(r.2) children in a boarding school (both domestic and foreign schools) below the college level 

(r.3) college students in dormitories

(r.4) persons in a jail, prison, or similar detention facility.

Youths in the latter three categories were included and linked to their parent's or guardian's HU, 

provided that a parent or guardian was alive and identifiable and that the HU was within the in-scope

universe.

Youths who could be linked to two or more HU's (e.g., children of divorced or separated parents)

were linked to the mother's HU, provided the mother was alive and her HU was in-scope. Else, such 

youths were linked to the father's HU, provided the father was alive and his HU was in-scope. Else, such 

youths were linked to one of the linkable HU's at random.

All eligibility criteria were evaluated as of the time of the screening interview. 

Eligible youths who screened into the sample were administered the NLSY97 questionnaire, the 

ASVAB, and the Interest Finder (IF). The ASVAB and IF were administered under standardized

conditions at Sylvan Learning Centers.

Thus far, we have been discussing the sampling universe for the NLSY97. Yet sometimes a 

survey's sampling universe is slightly different from its weighting universe, i.e., the universe to which one 

plans to make inferences. Typically, the weighting universe may be a bit broader than the sampling

universe, reflecting the fact that some domains at the margin are extremely difficult or costly to survey.

For the NLSY97, the weighting universe consisted of all age-eligible youths in the civilian, 

noninstitutional population of the U.S. Thus, the sampling and weighting universes were nearly identical, 

but the weighting universe did include a few small categories not included in the sampling universe, such

as

x homeless youths

x youths living in noninstitutional group quarters who were not linkable to a household, e.g., 

youths living with their families in rooming houses or in agriculture workers' dormitories.

The sampling universe also included youths in a jail, prison, or a similar detention facility who were not 

included in the weighting universe. 

13
2.2 Overview of the Sample

The NLSY97 samples were selected in two broad phases. First, we specified a large sample of 

more than 90,000 housing units. Through fieldwork, we determined which housing units were occupied 

and which were vacant, and for the occupied housing units, we conducted brief screening interviews. In 

the second phase, we selected subsamples of the eligible youths identified during screening for

participation in the main NLSY97 interview.

Table 2.1 presents the target samples sizes, overall and by demographic domain, we used for 

planning the NLSY97. The table also shows the domain sample sizes we would achieve in simple

random samples of the same overall size. Clearly, the NLSY97 targeted sizeable oversamples of Hispanic 

and non-Hispanic, black youths.

We designed the sampling methods and procedures to yield a database of eligible youths that

could be projected (with known confidence levels) to represent the entire eligible population of American

youths. For the large screening sample, our approach involved the selection of two independent areaprobability samples:

1. a cross-sectional (CX) sample designed to represent the various segments of the eligible 

population in their proper population proportions, and 

2. a supplemental (SU) sample designed to produce, in the most statistically efficient way, the 

required oversamples of Hispanic and non-Hispanic, black youths.

Table 2.1 Target Sample Sizes for the NLSY97

Domain

Proportion of the 

Eligible Population

Hypothetical Sample 

Size in a Simple 

Random Sample

Target

Sample Size

Total 1.000 10,000 10,000

Hispanic 0.129 1,290 1,667

Non-Hispanic Black 0.154 1,540 2,500

Non-Hispanic Nonblack 0.717 7,170 5,833

Source: Proportions are from the March 1997 Current Population Survey (CPS).

As an alternative, we could have used one area-probability sample large enough to meet the needs of both 

the proportionate sample and the oversamples. After consideration, however, we decided to design one 

area-probability sample to meet the needs of the proportionate sample, and a second area-probability

sample expressly to meet the needs of the oversamples. Our decision was based on the view that two

samples would provide greater value than one in terms of supplying precise statistical information at

lower overall cost. 

14
Both the cross-sectional (CX) and supplemental (SU) samples were selected by standard areaprobability sampling methods.2 Sampling was in three essential stages: primary sampling units (PSUs), 

segments, and HUs. NORC listed all addresses in the selected segments prior to the third stage of 

sampling. The CX sample resulted from an essentially self-weighting design. Stratification was 

introduced at the last stage for the SU sample to increase the hit rate for Hispanic and non-Hispanic, black

youths. A half-open interval procedure was used in both samples at the time of screening to include

persons living in housing units missed during the initial listing. 

For the CX sample, we defined PSUs as Metropolitan Statistical Areas (MSAs) and as single

counties or clusters of neighboring counties in nonmetropolitan areas. For the SU sample, to enable 

improved targeting of Hispanic and black youths, we defined PSUs as single counties. We selected 100 

PSUs for each sample.

For both samples, we divided the selected PSUs into segments consisting of single census blocks 

or clusters of neighboring blocks. Then, we selected 1,151 and 600 segments for the CX and SU samples,

respectively. In Fall 1995, NORC field workers visited the selected segments and listed all of the 

addresses found therein. To guide their work, we prepared and provided field workers with block maps

drawn by Mapinfo software and driven by the Census Bureau’s TIGER database.

At the third and final stage of sampling, we selected addresses (presumed to be HUs) from the 

NORC listing. For the SU sample, we classified the selected segments by high or low density (of 

Hispanic or black youths in the 1990 Census) and sampled addresses in the high-density segments at 10

times the rate of sampling in the low-density segments.

Table 2.2 summarizes the sizes of the CX and SU samples. Overall, we originally selected

90,139 HUs from the NORC listing of 270,197 HUs. 

Table 2.2 Sizes of the CX and SU Samples

Stage of Sampling CX Sample SU Sample Total Sample

PSUs Selected 100 100 200

Segments Selected 1,151 600 1,751

Listed HUs 176,673 93,524 270,197

Selected HUs 65,269 25,688 90,957

 Originally Selected HUs 64,654 25,485 90,139

 HUs Obtained via Half-Open Interval 

 Procedure

615 203 818

2

 This report assumes basic familiarity with these sampling methods. Readers in need of additional information on 

area probability sampling should consult a standard survey sampling textbook.

15
Beginning in late January 1997 and continuing through October 1997, NORC conducted face-toface screening interviews at the selected HUs using a screen-and-go approach. During these field 

operations, NORC field interviewers implemented a half-open interval procedure and identified and

screened an additional 818 HUs that were not included in the original listing. 

Table 2.3 gives the results of the screening operation. From the last column on the right, it is

evident that we achieved an overall screener response rate of about 94 percent. 

Table 2.3 Results of Screening Operations

Status of HUs Count of HUs

Percent of 

All HUs

Percent of 

Eligible HHs 

Eligible to be 

Interviewed

Ineligible

 Type C Noninterview 2,139 2.4

 Type B Noninterview 8,401 9.2

 TOTAL INELIGIBLES 10,752 11.6

Eligible to be Interviewed

 Type A1 Noninterview 4,795 5.3 6.0

 Completed Screener 75,410 82.9 94.0

 SUBTOTAL 80,205 88.2 100.0

Other Eligible 

 Type A2 Noninterview 212 0.2

 TOTAL OVERALL 90,957 100.0

NOTE: Type C noninterviews include businesses and demolished units. Type B noninterviews include

vacant HUs or HUs occupied by persons ineligible for the screening interview. Type A noninterviews 

include HHs eligible for the screening interview for which the interviewer was unable to obtain a 

complete interview. A2 includes HHs eligible for the survey but not eligible for the survey interview,

including “too ill or handicapped” and “language barrier.” A1 includes HHs eligible for the survey and 

for the survey interview.

Figure 2.1 depicts our sampling and interviewing strategies for the NLSY97. For the CX sample,

all eligible youths identified in screening were designated for the main NLSY97 interview. For the SU

sample, however, only Hispanic and black youths were designated for the main interview. Non-Hispanic, 

nonblack youths in the SU sample were screened out and not included in the NLSY97 sample of youths.

All youths designated for the interview were also designated for the ASVAB and IF tests, except for a 

few cases coded as out-of-scope. 

16
Figure 2.1 Map of CX and SU Samples for NLSY97

CX Sample SU Sample

Hispanic

Non-Hispanic Black

Non-Hispanic, Nonblack

We included all eligible youths within each respondent household. Even in the case of

multiracial, multiethnic households, we kept whole households together. This decision actually resulted 

in obtaining a few NLSY97 interviews from non-Hispanic, nonblack youths in the SU sample who lived 

with eligible black or Hispanic youths. For certain technical reasons, however, we chose to disregard 

these few cases in weighting the overall sample of non-Hispanic, nonblack youths. Each such youth was 

finally assigned a zero weight. 

Table 2.4 presents achieved sample sizes at the main interview and testing stages. For example,

9,808 age-eligible youths screened into the NLSY97, 8,984 completed an interview, and 7,168 

participated in the ASVAB and IF testing. Differences between the number of youths identified in 

screening and the number of youth participants are due to several factors, including classification of some

youths as out-of-scope, and nonparticipation including refusal. More details about these differences are 

presented later. 

Table 2.4 Achieved Sample Sizes for the NLSY97 

Domain

Located in 

Screening#

Subsampled for 

Participation

Completed

Main

Interview*

Out-of-Scope

for Testing

Completed

Tests

Total 9,808 9,808 8,984 178 7,168

Hispanic 2,128 2,128 1,905 79 1,366

Non-Hispanic Black 2,546 2,546 2,333 45 1,807

Non-Hispanic Nonblack 5,134 5,134 4,746 54 3,921

#

These counts exclude non-Hispanic, nonblack youths, age 12-16, discovered in the SU sample. *

Three non-Hispanic, nonblack individuals were screened into the supplemental sample and completed the

interview, but are excluded from the counts in this table and elsewhere in this volume. These individuals were 

interviewed because they were siblings of eligibles, or else their race or ethnicity was unknown or misrepresented at 

the time of the initial interview. NLSY97 databases may contain interview data for these cases, yet each has been

assgined an estimator weight of zero. 

Response rates based on eligible youths can be determined with the aid of Table 2.4. Of the 

17
eligible youths screened into the NLSY97 sample, 92 percent completed the youth interview and 73 

percent participated in the ASVAB test. We present detailed information about participation rates in 

Chapter 5. 

2.3 Screening Questions and Forms

We designed and implemented two screening forms for NLSY97: a brief paper screener and an 

extended CAPI screener. The paper form -- a copy of which is attached as Appendix H -- basically

sought to determine whether any residents were age eligible. 

If age eligibles were reported, we then administered the extended screener, which sought more

explicit information about date of birth, age, other demographic characteristics, grade, and the like for 

each resident (and linked person) of the household (HH). If no age eligibles were reported, then we did 

not administer the extended screener and, by implication, did not collect demographic and other 

information for HH members. A copy of the date of birth and age questions from the extended screener 

appears in Appendix I. 

In the foregoing paragraph, we described the general usage of the two screening forms. There 

were exceptions where the paper form was omitted.

To administer the screener, interviewers went to the door of the housing unit and attempted to 

gain the cooperation of a responsible adult who usually lived at the residence. Once cooperation was 

gained, the interviewer administered the three minute household screener to determine if there were any

potentially eligible youths in the household. If the screener indicated the presence of such youths, the 

interviewer attempted to complete the extended screener.

During a pretest for NLSY97, NORC and BLS realized that many households were extremely

costly to reach. Field interviewers made ten to twenty visits and still found no one home. Subsequently,

we decided to allow the interviewers to receive help from specified neighbors in determining whether

eligible youths lived at the sample households. Specifically, we decided to permit proxy and gatekeeper 

interviews, under strict protocols, as described in Appendices E and L. Forms used by proxy and 

gatekeeper respondents appear in Appendix J. 

To achieve a high screener response rate, we also employed advance letters and a brochure, all of 

which appear in Appendix K. The first advance letter, signed by the BLS project director Michael W. 

Horrigan, was sent by first-class mail prior to the screening interview. It alerted the HH to the purpose of

the NLSY97, its importance, a small $10 incentive payment, sponsorship, and confidentiality.

The second letter, signed by NORC project director Harrison Greene, was handed to the HH

respondent at the time of the screening interview. This letter was specific to PAY97 and told of the $75 

incentive payment for participation in the testing program.

The brochure, handed to the screener respondent at the screening interview, repeated much of the 

information sent in the first advance letter. It also gave answers to some frequently asked questions. 

18
Screening achieved considerable apparent success. During planning stages, for the overall 

project, we assumed that 91 percent of occupied HUs would respond to the screening interview. In actual 

practice, NORC obtained a 94 percent screener response rate. We present more details about screener, 

interview, and test participation rates in Chapter 5. 

19
Chapter 3. Selection of the NLSY97 Sample

In this chapter, we provide details of the procedures followed in selecting and implementing the 

screening sample. We also describe the selection of youths for the NLSY97 sample.

3.1 Selection of the Cross-Sectional (CX) Sample

Selection of PSUs. We used NORC's 1990 National Sample for the CX PSUs. This sample

includes 100 PSUs selected to represent all 50 states and the District of Columbia. Each PSU is defined 

as either a metropolitan area or one or more counties with a minimum size of 2,000 housing units (HUs). 

Nineteen of the metropolitan PSUs contained such a large population they were included in the sample

with certainty; we will refer to these 19 as the certainty PSUs. Table 3.1 gives a tabulation of the PSUs 

by type and region.

To increase the precision of sample estimates, the noncertainty PSUs were implicity stratified by

sorting them on several variables: whether the PSU was a metropolitan statistical area (MSA) or set of 

non-MSA counties, state or census division, proportion of the 1990 population that was black or Hispanic

(grouped into quartiles), and per capita income. Sample PSUs were selected using a systematic sampling

scheme, with the selection probability for a PSU proportional to the number of housing units counted in 

the 1990 Census. Therefore, the inclusion probability for the i-th PSU is given by

%i = 1 , if i is one of the 19 certainty PSUs 

 

 

n

X

X

i , otherwise, 

where n = 81 is the number of noncertainty selections, Xi+ is the 1990 count of HUs in the i-th PSU, and 

X++ is the 1990 count of HUs across all noncertainty PSUs in the universe. 

Table 3.1 Distribution of CX PSUs, by Census Region and Metropolitan Status

Type of PSU

Census Regions

Total

Northeast Midwest South West

Certainty Metropolitan 4 4 6 5 19

Other Metropolitan 8 13 21 9 51

Nonmetropolitan 4 9 13 4 30

Total 16 26 40 18 100

20
Selection of Segments. The second stage sampling units, or segments, consisted of one or more

neighboring blocks within PSUs. Blocks were linked to assure that each segment included at least 75 

HUs based on the relevant 1990 Census data. In total, we selected 1,151 segments within the 100 sample

PSUs -- with 9 or 10 in each of the noncertainty PSUs and up to 80 in the largest certainty PSU. For the 

19 certainty PSUs (really strata from a technical point of view) and the one noncertainty stratum

consisting of all other areas, we allocated integer numbers of segments in rough proportion to the 1990 

HU counts of the strata. 

Prior to selection, we sorted the segments (implicity stratified) successively by (i) PSU, (ii) 

whether or not they were in the central city (for metropolitan PSUs), (iii) state, (iv) county, (v) place

within county, (vi) percent minority quartile, and (vii) census tract. We sampled segments independently

from PSU to PSU using a systematic sampling scheme with probabilities proportional to the 1990 Census 

count of housing units. 

The selection probability for segment j within PSU i (conditional on the selection of the PSU) is 

given by

S ij i

ij

i

m

X

X 

 ,

where mi is the number of segments selected from the PSU and Xij is the 1990 count of HUs. 

There was one segment in the selected PSUs so large that 

X Xm ij i i !  / .

We designated this segment as a certainty segment, included this segment in the sample and set S ij 1.

In the CX sample, 76 segments required additional subsampling prior to the selection of HUs for

screening. At NORC, we refer to this intermediate stage of sampling as chunking. The need for chunking 

arises during a listing operation when field workers discover segments that have grown a lot since the

time of the original measure of size. We instructed the field workers to partition each of the 76 segments

based upon visible, replicable boundaries and to give a count of the HUs within each cell of the partition.

Subsequently, we selected one cell (or chunk) at random per segment, with probability proportional to the

field counts. For simplicity, we will continue to refer to the unit of sampling at the stage prior to the HU

stage as the "segment," even though for 76 of the segments there is actually an additional stage of

sampling. Henceforth, we shall also assume that the chunking probability has been incorporated in the 

%ij.

Mapping and Listing the Selected Segments. Once the segments were selected, maps were 

produced for each one using software that manipulates the Census Bureau's TIGER files. The maps

displayed the segment boundaries, roads, and other important features of the segment needed to manage

field work. NORC field staff then compiled a list of all the HUs within each sample segment.

21
Selection of HUs for Screening. The final stage of sampling involved the selection of HUs for

screening. We planned the CX screening sample so that it would produce the 5,833 interviews with nonHispanic, nonblack youths targeted by the NLSY97. The same sample, of course, would provide

proportional representation of Hispanic and non-Hispanic, black youths. We planned the SU screening 

sample so that it would be large enough to produce the oversamples of Hispanic and non-Hispanic, black 

youths.

In light of the foregoing plans, 5,833 interviews with non-Hispanic, nonblack youths became the 

driver for the CX screening sample size. We estimated the number of HUs needed to be screened to 

achieve this goal: 

(1) Based on data from the March 1993 Current Population Survey (CPS), we estimated that 

for each occupied housing unit in the US, there would be .131 non-Hispanic, nonblack 

youths between the ages of 12 and 16 years old; 

(2) Based on experience in recent surveys, we assumed that approximately 85 percent of the 

HUs fielded would be occupied; 

(3) We assumed that, as in the NLSY79, screening interviews would be completed at 91 

percent of these occupied HUs; and 

(4) We assumed that, as for NLSY79, 89 percent of youths selected for the sample would 

complete the NLSY97 interview. 

Given, these conditions, we selected and subsequently screened 64,654 HUs (approximately 5,833 over 

the product of .131 × .85 × .91 × .89) for the CX sample.

To achieve a self-weighting sample of HUs, we determined that the conditional sampling rate for

HUs k in segment (i, j) should be

S ijk o i ij f / ( ) , S S

where fo = qo / Q is the overall sampling rate for HUs, qo = 64,654 is the sample size for HUs, and Q is an 

estimate of the nation's housing stock. The within-segment sampling rate was constant across all housing 

units (subscripted by k) within the segment. This method of sampling caused the final cluster sizes within 

the CX sample to be roughly constant, varying only due to any discrepancies between the 1990 Census 

figures and the actual number of listed HUs.

In the cross-sectional sample, three segments had a calculated %ijk > 1. For these segments, %ijk

was limited to 1 (i.e., the selection of all listed housing units within these segments). It should be pointed 

out that the reason for %ijk to be calculated as greater than 1 is that the probability of selection for the 

segment is already smaller than the desired equalized probability for housing units. Even by choosing 

every housing unit in these segments, the selection probability for a housing unit cannot exceed the

selection probability for its segment.

22
We sampled HUs using systematic sampling, according to the rates %ijk, using the order of HUs 

determined by the NORC listing. Sampling was independent from segment to segment.

In Table 3.2, we present the sample sizes calculated for the CX sample during the planning stages 

of the project. The difference between the actual number of HUs selected and fielded, 64,654, and the 

number, presented in this table, 64,684, is due to rounding. At the planning stage of the project, it looked 

like the CX sample would supply 9,001 youths aged 12-16. The SU sample would be designed to supply

the remaining black and Hispanic youths needed for the NLSY97. 

Screening for Eligibility. NORC field staff conducted screening interviews at the specified HUs

to find out the number of persons residing there, as well as their ages, races/ethnicities, grades, genders,

and other eligibility information. The answers to these questions determined which, if any, household

members were eligible for the main interview. 

In the course of the screening operation, we executed the half-open interval (HOI) procedure to

detect and screen both HUs missed by the original listing operation and HUs resulting from new 

construction. The HOI procedure is standard practice at NORC for area probability samples. A detailed 

description is available upon request. If one, two, or three new HUs were discovered at an original 

selection, then each was added to the screening sample. If four or more new HUs were discovered at an 

original selection, then we specified a systematic sample of three of the new HUs for the screening 

sample and incorporated the sampling probability into the corresponding %ijk.

Table 3.2 Projected Screening Sample Sizes and Yields Used in Planning the NLSY97

CX Sample SU Sample

Sampling Units Number Rate Number Rate

Total

Number

HUs Selected 64,684 – 31,753 – 96,437

Occupied HUs 54,981 0.8500 26,990 0.8500 81,971

Completed Screeners 50,033 0.9100 24,561 0.9100 74,594

Total Person Rostered 131,086 2.6200 64,350 2.6200 195,436

Persons Aged 12-16 9,001 0.0687 4,419 0.0687 13,420

 Hispanics 1,027 0.0078 1,008 0.0157 2,035

 Non-Hispanic Blacks 1,420 0.0108 1,389 0.0216 2,809

 Non-Hispanic Nonblacks 6,554 0.0500 2,022 0.0314 8,576

NLSY97 Youth Interviews 8,011 0.8900 2,133 0.8900 10,144

 Hispanics 914 0.8900 897 0.8900 1,811

 Non-Hispanic Blacks 1,264 0.8900 1,236 0.8900 2,500

 Non-Hispanic Nonblacks 5,833 0.8900 – – 5,833

23
Selecting Youths for Participation in NLSY97. The actual numbers of eligible youths

identified in the screening operation appear in Table 3.3. Overall, 9,808 NLSY97-eligible youths

screened in. These actual counts are notably smaller than the projected counts from Table 3.2. We will 

return to this disparity later. But first, we consider the SU sample.

Table 3.3 Actual Screening Sample Sizes

Youths CX Sample Number SU Sample Number Total Number

NLSY97-Eligible Persons 7,335 2,473 9,808

 Hispanics 1,026 1,102 2,128

 Non-Hispanic Blacks 1,175 1,371 2,546

 Non-Hispanic Nonblacks 5,134 0 5,134

3.2 Selection of the Supplemental (SU) Sample

Selection of PSUs. The SU screening sample was selected using procedures very similar to those 

used to select the CX sample, but with the intention of oversampling black and Hispanic youths. The 

PSUs were defined differently from those for the CX sample: all PSUs for the SU sample were single 

counties, not clusters of counties or MSAs. We chose not to use MSAs because of our goal to oversample

black and Hispanic youths; MSAs often have counties with widely differing minority rates. Thus, it made

sense to separate counties with higher percentages of minorities from those with lower percentages. The 

easiest way to accomplish this was simply to use counties as the PSUs. 

We merged counties into PSUs to meet a minimum size criterion of 2,000 HUs. To guarantee

that the PSUs with the highest minority youth rates were properly represented, we implicitly stratified the

frame by sorting PSUs first by minority youth density (number of blacks and Hispanics, aged 17 and 

under, per housing unit) grouped into thirds. Within each third, we sorted by region, division, 

metropolitan status, state, and per capita income. Then, we selected a systematic sample of 100 PSUs,

with the selection probability for each PSU proportional to its measure of size. 

We used as the PSU measure of size the weighted number of black and Hispanic youths aged 17 

and younger in the PSU:

X fN i H Hi  f NB Bi max ( , ) 1 1000 1000 

where fH and fB refer to the overall sampling rates for Hispanic and non-Hispanic, black youths in the 

NLSY97-eligible age range, and NHi and NBi refer to 1990 Census counts of Hispanic and black youths

within the PSU. The sampling rates are shown in Table 3.4. As it happened, the sampling rates for the 

two groups were similar, and thus the measure of size was essentially proportional to the simple sum of

blacks and Hispanics in the eligible age range within the PSU. For the NHi and NBi, we would have 

preferred to use nonoverlapping race/ethnicity categories, such as Hispanic and non-Hispanic black, but 

the relevant 1990 Census files did not provide separate counts at the county level of persons by age, race, 

24
and Hispanic origin.3

 Instead, we decided to use the overlapping categories black and Hispanic. This 

treatment has little statistical consequence, because very few Hispanics (less than 3.5 percent of the total) 

classify themselves as black in the decennial census. 

Table 3.4 Sample Sizes and Sampling Rates Used in Determining

the Measure of Size for the SU Sample 

Domain Hispanic Non-Hispanic Black All Others

Age 12-17 

 Target Sample Size 2,000 3,000 7,000

 Sampling Rate .000822 .000896 .000451

The sampling rates in Table 3.4 are based on a total sample size of 12,000 youths aged 12-17, 

which were the figures contemplated at the time of sample implementation. Later, just before NLSY97 

went into the field, in an effort to reduce cost, the BLS cut the age range to 12-16 and the sample size to

10,000. Notwithstanding the cut, Table 3.4 reflects the fH and fB we actually used in defining the measure

of size. 

The selected sample of 100 PSUs appears in Table 3.5, broken down by region, certainty, and 

metropolitan status. All 16 nonmetropolitan PSUs were noncertainty PSUs, and all 17 certainty PSUs 

were metropolitan PSUs. Clearly, the overall SU sample is heavily concentrated in the South and the

West, reflecting well-known concentration patterns of the black and Hispanic populations. 

Table 3.5 Distribution of SU PSUs, by Census Region and Metropolitan Status

Type of PSU

Census Regions

Total

Northeast Midwest South West

Certainty Metropolitan 5 2 5 5 17

Other Metropolitan 9 12 33 13 67

Nonmetropolitan 2 0 12 2 16

Total 16 14 50 20 100

3

In defining the measure of size, we worked with county-level data from the 1990 Census, specifically the STF 1C

data.

25
Selection of Segments. At the next stage of sampling, we selected a total of 600 segments from

a sorted list of all the segments within the selected PSUs. In defining segments, census blocks were 

collapsed to reach a minimum size of 75 HUs. The measure of size was defined by

X fN ij H Hij Bij max ( , ) 1 1000 1000  f NB

and the conditional inclusion probability by

S ij i

ij

i

m

X

X 

 ,

for the j-th segment in the i-th PSU, where NHij and NBij refer to 1990 counts of Hispanic and nonHispanic black youths, respectively, aged 0-17.4

Prior to sampling, the list of segments was sorted by (i) PSU (i.e., county), (ii) minority youth

density grouped in thirds, (iii) percentage of minority youths who were Hispanic (grouped into thirds), 

(iv) place within county, (v) percent minority quartile, and (vi) census tract. Note that sort (iii) was done

to guarantee that the sample of segments would be representative of both minority groups (blacks and

Hispanics). Sampling was independent from PSU to PSU. 

Mapping and Listing of the Selected Segments. Just as for the CX sample, once the SU 

segments were selected, maps were produced for each one. NORC field staff then compiled a list of all

the HUs within each sample segment.

Selection of HUs for Screening. We observe, once again, that the screening sample size was 

driven by the needs of the NLSY97. Because the sampling rate for non-Hispanic blacks was highest 

among the three targeted race/ethnicity domains (see Table 3.4), we designed the SU screening sample to 

provide us with enough eligible non-Hispanic blacks to achieve 2,500 completed youth interviews. Such 

a sample would be large enough, we figured, to achieve the targeted 1,667 Hispanic interviews. Using 

the numbers from the CX sample given in Table 3.2, we determined that the SU sample must screen in an

additional 1,389 eligible non-Hispanic blacks aged 12 to 16. From the March 1993 CPS, we expected 

there to be .057 such youths per household. Using the same rates of occupancy (85%) and of screener

response (91%), we planned to sample and screen about 31,753 HUs (approximately 1,389 over the 

product of .057 × .85 × .91). With an assumed 89 percent response rate, the planned screening sample of

2,809 non-Hispanic, black youths would yield exactly 2,500 completed interviews as targeted. 

Table 3.2 lays out all of the sample sizes calculated for the SU sample during the planning stage 

of the overall project. There, we see that the SU sample was projected to yield 897 completed Hispanic

interviews, more than enough to achieve the overall target of 1,667 completed Hispanic interviews. 

4

 In defining the measure of size, we worked with county-level data from the 1990 Census, specially the STF 1C

data.

26
As screening operations were launched for NLSY97/PAY97, cost considerations came into play,

and NORC and the BLS decided to release only about 80 percent of the planned SU sample of 31,753 

HUs, reserving the balance for a later possible release. Thus, we released an SU screening sample of 

exactly 25,485 HUs. Our thinking was that money could be saved by releasing the smaller sample. Our 

plan, at that time, was to monitor the yields of youths as screening interviews proceeded, and to release 

the reserved HUs in the event that yields were falling short of targets. Ultimately, we never did release 

the reserve, despite an early and obvious shortfall in the number of youths.

To achieve a self-weighting sample of HUs, we determined that the conditional sampling rate for

HUs k in segment (i, j) should be 

S ijk i ij f1 /( ) , S S

where is the overall sampling rate for HUs, f qQ 1 1 / q1 = 31,753 is the sample size for HUs, and Q is a 

current estimate of the nation's housing stock. Rather than a self-weighting design, however, we decided 

to oversample HUs within segments with large concentrations of black or Hispanic youths by creating

two classes of segments based on the segment-specific rates of minority youths in the 1990 Census. A 

segment was classified as “high-minority” if either the rate of black youths under age 18 per housing unit 

or the rate of Hispanic youths under age 18 per housing unit was at least 0.20 (i.e., one youth for every

five housing units). Otherwise, a segment was classified as “low-minority” (even if the sum of the two 

rates was at least 0.20).

We determined two different sampling rates and for the two classes. The proper ratio

between and was a key decision. Increasing the ratio would increase the sampling efficiency

in targeting minority youths, but would also increase the variability in final sampling weights. The ratio 

chosen was 10:1. This means that housing units in “high minority” segments would have a final 

probability of selection 10 times that of housing units in “low minority” segments. The application of this 

decision resulted in separate sampling rates for “high minority” and “low minority” segments, with

fhigh flow

fhigh flow

S ijk high i ij f / ( ) , S S if (i, j) is in the “high minority” class;

 f , low i ij /( ) S S if (i, j) is in the “low minority” class.

Sampling of HUs was independent from segment to segment.

Unlike the CX sample, in which only three segments had a calculated S ijk > 1, there were many

such segments in the SU sample, especially among the “high-minority” segments. As in the CX sample,

S ijk was limited to 1 (i.e., all listed housing units were selected). 

27
Table 3.6 summaries the key features of the actual CX and SU screening samples.

Table 3.6 Summary of the Design of the CX and SU Samples

Design CX Sample SU Sample

First Stage

 Number of Selections 100 PSUs 100 PSUs 

 Measure of Size (MOS) Housing units (HUs) Weighted sum of black and 

Hispanic youths, aged 17 and 

below

 Minimum Size 2,000 HUs 2,000 HUs 

 Method of Selection Systematic selection with 

probabilities proportional to 

size (pps) 

Systematic pps selection 

 Sort (implicit stratification) Metropolitan status, 

division/state, percent minority

quartile, per capita income

Minority youth density thirds, 

region, division, metropolitan

status, state, per capita income

Second Stage

 Number of Selections 1,151 600

 Measure of Size HUs Weighted sum of black and 

Hispanic youths aged 17 and 

below

 Minimum Size 75 HUs 75 HUs 

 Method of Selection Systematic pps selection Systematic pps selection 

 Sort (implicit stratification) PSU, central city, state, county,

place, percent minority

quartile, census tract/BNA 

PSU, minority youth density

thirds, percentage of minority

youths who are Hispanic, 

place within county, percent 

minority quartile, census 

tract/BNA

Third Stage

 Actual Number of HUs 64,654 25,485

Screening for Eligibility. NORC field staff conducted screening interviews at the selected HUs

to find out the number of persons residing there, as well as their ages, races/ethnicities, grades, genders,

and other eligibility information. The answers determined which, if any, household members were 

eligible for the main interview.

Selecting Youths for Participation in NLSY97. The actual numbers of eligible youths located 

in the screening operation appear in Table 3.3. Overall, 9,808 eligible youths screened in, and we 

attempted to complete a main interview with each. 

28
Chapter 4. Weighting the Sample

Data from large-scale national samples typically need to be weighted to achieve an unbiased

estimator of the population total. The weights are needed for four main reasons. First, the weights 

compensate for differences in the selection probabilities of individual cases, which often arise by design, 

as in the NLSY97/PAY97, where different overall sampling rates were required for Hispanics, nonHispanic blacks, and others within the eligible age ranges. Second, weighting compensates for subgroup 

differences in participation rates; even if the sample as selected were representative of the larger 

population, differences in participation rates can compromise the representativeness of the sample. For 

example, different geographic areas may experience different rates of screener nonresponse. Such

differences in participation rates can introduce nonresponse bias into the results; weighting can reduce 

these biases. Third, weights compensate for random fluctuations from known population totals due to 

sampling. For instance, if one sex were overrepresented in the NLSY97 sample purely by chance, it 

would be possible to use data from the Decennial Census or the Current Population Survey to adjust for 

this departure from the population distribution. And fourth, adjusting the data to known population totals 

can help reduce the impact of survey undercoverage (such as undercoverage arising from the omission of

persons in partially enumerated households). 

The weights for the NLSY97 were computed in six steps: 

Step 1. Computation of the base weight, reflecting the housing unit's selection probability for the

 screening sample.

Step 2. Adjustment for household nonresponse to the screener. 

Step 3. Adjustment to reflect subsampling of youths in screened households.

Step 4. Development of a combination weight to allow youths from the CX and SU samples to

be merged into one combined sample.

Step 5. Adjustment for youth nonresponse to the main interview. 

Step 6. Poststratification of the nonresponse-adjusted weights. 

This approach implicitly treats the CX and SU screening samples as stand-alone samples, which are then

combined in the fourth step. This approach was taken in NLSY79, and its soundness was attested to at 

that time by a panel of experts including Joseph Sedransk, Leslie Kish, and Benjamin King (see Frankel

& McWilliams, 1981).

Step 1: Computation of the base weight. The base weight (W ) for the k-th housing unit in the 

screening sample is the inverse of the probability of selecting the unit:

1

W k

k

1

1 S ,

where %k denotes the relevant inclusion probability. For the CX sample, the base weights are mostly

identical because the sampling was designed to equalize the probability of selection for each housing unit. 

However, as described in Chapter 3, three segments (out of 1,151) were selected with certainty, and 

29
therefore, the housing units within these segments have a slightly larger base weight, and all other

housing units have a slightly lower base weight. 

In the SU sample, segment probabilities were often smaller than the desired equalized 

probabilities, especially in the segments “low” in minority youths (see Chapter 3). This resulted in larger 

base weights for housing units within these segments. Segments “low” in minority youths already had an 

equalized base weight 10 times larger than segments “high” in minority youths, and for some “low” 

segments, the base weight threatens to be 70 times larger. To prevent extreme weights, we limited the 

base weight to 2.0 u 9993.74 (the equalized base weight for “low” segments). It should be noted that this 

cap reduces the sum of the weights in the SU sample by over 25 percent (see Section 5.2 for more details

on the effect of the weighting caps), which also reduces the coverage ratios. This is the price for the 

efficiency of the supplemental sample, which did an excellent job of reducing the fieldwork necessary to 

obtain the black and Hispanic oversamples.

There is one additional complication to this step. NORC utilized the half-open interval

procedure, as explained in Chapter 3. Since this procedure subsamples missed housing units, an 

adjustment is necessary. This adjustment is simply multiplication by the number of missed housing units 

found divided by the number selected. For example, if nine are found (three are always selected if there 

are more than three), then each of the three selected have their base weight multiplied by 9/3 = 3 (i.e.

these three housing units represent the nine found in the sample). Unfortunately, this adjustment also has 

the risk of causing extreme weights. In particular, one segment had 334 missed housing units attached to a 

single sampled housing unit (the adjustment factor would be 111.33). To combat potentially extreme

weights, we truncated the adjustment to 5.0. This cap reduced the sum of the weights by only 0.40 

percent (see Section 5.2 for more details on the effect of the weighting caps). The following table lists the 

unadjusted and capped missed housing unit adjustments, and how many housing units had the adjustment

made to their weights. 

30
Table 4.1 Unadjusted and Capped Missed Housing Unit Adjustments 

Unadjusted Factor Capped Factor Number of HUs 

1.00 1.00 702

1.33 1.33 24

1.67 1.67 21

2.00 2.00 12

2.33 2.33 12

2.67 2.67 3

3.00 3.00 5

3.67 3.67 3

4.33 4.33 9

5.00 5.00 3

6.67 5.00 3

7.33 5.00 3

10.00 5.00 3

13.00 5.00 3

15.00 5.00 3

16.33 5.00 3

16.67 5.00 3

111.33 5.00 3

Total 818

Step 2: Adjustment for screener nonresponse. The next step was to adjust the screener base 

weights for nonresponse to the screening interviews. The nonresponse-adjusted weight (W ) is the base 

weight (W ) inflated by the inverse of the weighted response rate within an adjustment cell: 

2

1

W

W

W k W

j

j E

j

j S

2 k

1

1

 1





¦

¦

D

D

 ,

where ED is the set of eligible units within the D-th cell and SD is the set of screener respondents in the D-th

cell. Because few variables, if any, are available for the screener nonrespondents, we based the 

adjustment cells on simple geographic criteria. The cells were defined to be segments. Unfortunately,

some segments were too small to have a reliable nonresponse weighting adjustment. If there were fewer 

than 25 respondents in a segment, we instead used the nonresponse weighting adjustment for the entire

PSU. Using the PSU nonresponse rate did prevent some large nonresponse adjustments, but the effect on

the sum of weights was less than 0.50 percent (see Section 5.2 for more details on the effect of various

weighting “caps”). 

Four PSUs, however, had weighted PSU response rates that were considered too low to be used 

for adjustment. Within these four PSUs, the nonresponse adjustment was capped at 2.0 to prevent extreme

weights. This is equivalent to saying that if the weighted segment (PSU if fewer than 25 respondents in 

31
the segment) response rate was less than 50 percent, it was set to 50 percent for purposes of the weight 

adjustment. This cap reduced the sum of the weights by only 0.22 percent in the SU sample (see Section 

5.2 for more details on the effects of the weighting caps).

W2 represents the final screening sample weight and the final household weight. 

Step 3: Adjustment for subsampling youths. We now turn to youth weights. Because eligible 

youths were not subsampled for the youth interview, the initial youth weight is given by

W W 3 2 k k ,

for the k-th youth, where W is the step-2 weight assigned to the youth’s household. This youth

weighting derives from the fact that the youth’s probability of selection is equal to the youth’s

household’s probability of selection. 

2k

Step 4: Combining the cross-sectional and supplemental samples. Thus far, the weights have 

treated the CX and SU sreening samples as stand-alone samples. The next step was to adjust the weights 

from step 3 so that the two samples could be used together. In effect, we created a “precision” weight, O ,

to combine estimators from the two samples:

  ( )  T OT O T  c s 1

in which represents a statistic derived from the CX sample and represents the corresponding 

statistic from the supplemental sample. Because the two samples are independent, the optimum O is 

proportional to the effective sample size in the CX sample:



Tc Ts

 ,

O

O

 

 

n d

nd nd

n d

nd nd

c c

cc ss

s s

cc ss

/

/ /

/

/ / 1

where nc and ns are the nominal sample sizes for the CX and SU samples and and represent the 

design effects for the estimators from each sample. It is inconvenient to use the design effects 

themselves, since they will vary from one variable to the next. Instead, we used one plus the relative 

variance of the weights within each sample (as was done for NLSY79); this factor captures the impact of 

unequal weighting on sample efficiency:

dc ds

 / 

/  / 

 / 

/  / 

O

O

 

 

n d

nd nd

n d

nd nd

c c

cc ss

s s

cc ss

1

32
 (, )

(, )

 (, )

(, )

d

sW i

W i

d

sW i

W i

c

i

i

s

i

i

 





a

¬

« o

1⁄4

»

 





a

¬

« o

1⁄4

»

1

1

3

3

2

3

3

2

 CX

M CX

 SU

M SU ,

where M(.) and s(.) signify the simple mean and standard deviation of the weights, respectively.

The merging of the samples was done separately for each race/ethnicity by sex domain (e.g.,

Hispanic males). Race/ethnicity was defined as Hispanic, non-Hispanic black, or non-Hispanic, nonblack.

At the level of individual members of the sample, the weight at this stage (W ) is the sample member's

weight from the previous step times the relevant combination factor ( ):

4



OE

WW i

W i

k k

k

4 3

1 3

 

  



(  )

O

O

E

E

CX

SU ,

where  signifies the sex by race/ethnicity domain. Shown below is a table listing the that were used 

during the weight calculations.



Os

Table 4.2 Os Used for Precision Weighting to Combine the

CX and SU Samples 

Domain CX () SU (1-)

Male .75 .25 Hispanic Female .75 .25

Male .68 .32 Non-Hispanic, Black Female .66 .34

Male 1.00 .00 All Other Female 1.00 .00

It is clear that the CX cases are being given more weight for the minority categories; three-quarters for

Hispanics, and two-thirds for non-Hispanic blacks. This is because there is more variability in the 

weights in the SU sample (caused by the oversampling for minority youths), reducing the effective

sample size. It is important to remember than non-minorities in the SU sample were ineligible for the 

NLSY97 interview.

33
Step 5: Adjusting for nonresponse in the main data collection. For each sample, we adjusted the 

weights computed in the third step for nonresponse in the main data collection. We used a fairly simple

scheme to create nonresponse adjustment cells. Cells were based on screening sample (CX, SU), 

race/ethnicity (Hispanic, non-Hispanic black, and all other youths), age, and sex. The adjusted weight 

(W ) is just the weight from Step 4 inflated by the inverse of the weighted response rate, with separate 

adjustments computed for each cell. This step does not affect the sum of the weights. The only change is 

the redistribution of the nonrespondents’ weights to the respondents. No combining of cells was required.

Also, no caps on the nonresponse weighting adjustment were necessary or used. 

5

Table 4.3 shows the cell response rates used for the nonresponse weighting adjustment.

Table 4.3 Cell (Weighted) Response Rates (in Percent) Used for the

NLSY97 Nonresponse Weighting Adjustment 

Domain Hispanic

Non-Hispanic

Black All Other

Age 12 88.87 91.57 94.20

Age 13 90.95 90.12 93.30

Age 14 93.67 92.35 90.50

Age 15 90.51 86.60 92.74

Male

Age 16 82.77 90.02 89.88

Age 12 89.90 95.46 92.54

Age 13 91.27 92.21 91.69

Age 14 91.67 90.97 92.68

Age 15 97.01 96.51 94.32

CX Sample

Female

Age 16 82.13 93.00 91.52

Age 12 86.90 94.25 Not applicable

Age 13 86.35 94.94 Not applicable

Age 14 91.45 92.68 Not applicable

Age 15 84.61 94.76 Not applicable

Male

Age 16 85.05 89.83 Not applicable

Age 12 89.25 88.62 Not applicable

Age 13 91.81 92.94 Not applicable

Age 14 94.67 94.90 Not applicable

Age 15 91.57 92.55 Not applicable

SU Sample

Female

Age 16 85.96 92.11 Not applicable

34
Step 6: Poststratifying the weights. The last step in the computation of the weights was to

adjust the weights to bring them into agreement with independent estimates of the population size:

W W

T

W k k

k

k

6 5

5

 u

c

c

¦

J .

For each cell (or poststratum), J, the adjustment factor, is the ratio between an (independent and presumed

superior) estimate of the size of the population (TJ) and the sum of the weights for sample members in that 

cell. Cells were defined in terms of age, race/ethnicity, and sex. Benchmarks for the poststratification 

adjustments were obtained from the March 1996 and 1997 Current Population Surveys (CPS), excluding 

persons outside the household population, as in Table 4.4 below.

Table 4.4 Population Totals Used for the NLSY97 Poststratification

Hispanic Non-Hispanic

Black

All Other

12-Year-Olds Male 249,654 301,005 1,483,161

Female 236,321 291,417 1,365,341

13-Year-Olds Male 250,174 298,899 1,354,138

Female 233,112 300,464 1,331,628

14-Year-Olds Male 269,949 314,067 1,426,387

Female 243,954 293,977 1,376,555

15-Year-Olds Male 295,675 293,540 1,385,961

Female 216,934 299,701 1,370,011

16-Year-Olds Male 271,631 316,369 1,432,737

Female 225,988 272,632 1,377,071

The ratio of the sum of the sample weights after Step 5 (W ) for a specific domain to the corresponding 

CPS estimate of the size of that domain (i.e., the reciprocal of the poststratification factor) is one possible 

estimate of the coverage of the NLSY97. Table 4.5 presents these coverage ratios. 

5

35
Table 4.5 Coverage Ratios for NLSY97 Sample

Hispanic Non-Hispanic Black All Other

12-Year-Olds Male .75 .62 .62

Female .77 .70 .62

13-Year-Olds Male .72 .68 .71

Female .75 .68 .67

14-Year-Olds Male .62 .74 .67

Female .81 .64 .64

15-Year-Olds Male .65 .69 .68

Female .85 .63 .69

16-Year-Olds Male .69 .65 .62

Female .71 .81 .57

This table suggests that the coverage rate was about two-thirds (67 percent). In our judgement, however, 

these ratios are not the best estimates of the coverage of the NLSY97 because they confound the coverage 

effect and the effect of capping weights, which is intended to limit extremes and reduce sampling

variabilility. We analyze this matter again in Section 5.2, where we present our best estimates of 

coverage and compare the coverage of NLSY97 to the coverage of the 1990 Census and the CPS.

We close this chapter with some summary tables for the NLSY97 intermediate and final weights. 

Table 4.6 Summary Table of Housing Unit and Household Weights 

W1 W2

N (> 0) 90,957 75,410

Sum 200,471,047 177,917,390

Mean 2,204.02 2,359.33

Standard Deviation 2,892.12 3,042.33

Minimum (> 0) 1,000.24 1,000.24

5th percentile 1,000.24 1,010.45

25th percentile 1,652.35 1,652.35

Median 1,652.35 1,699.56

75th percentile 1,652.35 1,784.53

95th percentile 9,992.01 10,120.11

Maximum 98,937.13 102,765.79

36
Table 4.7 Summary Table for Youth Weights 

W3 W4 W5 W6

N (> 0) 9,808 9,808 8,984 8,984

Sum 16,619,189 12,858,858 12,858,858 19,378,453

Mean 1,694.45 1,311.06 1,431.31 2,157.00

Standard Deviation 1,037.86 635.56 683.26 1,070.78

Minimum (> 0) 1,000.24 248.88 264.68 323.30

5th percentile 1,004.02 264.09 298.39 420.26

25th percentile 1,652.35 1,089.75 1,170.47 1,518.77

Median 1,697.00 1,652.35 1,782.82 2,601.05

75th percentile 1,768.71 1,731.03 1,881.93 2,922.39

95th percentile 1,996.69 1,893.44 2,052.48 3,298.97

Maximum 22,385.97 9,056.13 9,785.96 15,759.42

37
Chapter 5. Accuracy and Precision of the Sample Results 

5.1 Potential Impact of Nonresponse 

Given the typical rates of completion currently obtained in U.S. household surveys, completion

rates for the various stages of the NLSY97 are fairly high. In what follows, we examine noncooperation at 

the various stages of the survey process and discuss its potential impact on the survey results. 

Participation Rates. Participation will be examined at several stages of the study, including 

screening, interviewing, and testing. The screening process is the first opportunity for participation in the 

NLSY97; final response rates for all screeners are given in Table 5.1. Among the ineligibles are Type C 

noninterviews, consisting of condemned and demolished residences as well as businesses and nonexistent 

housing units, and Type B noninterviews, which include group quarters, vacation homes, properties under 

construction, and vacant housing units. Type A2 includes HHs eligible for the survey but not eligible for 

the survey interview, including “too ill or handicapped” and “language barrier.” The eligibles for the 

interview are made up of Type A1 noninterviews (refusals, breakoffs, and computer crashes) and of 

completed screeners. As Table 5.1 shows, eligibles for the interview make up 88.2 percent of the total 

screeners, among which 94.0 percent completed a screener. Among all cases, completions were 82.9

percent. Apparently, the screener response rate is slightly higher in the CX sample than in the SU sample.

Table 5.1 Final Response Status for All Screeners 

Sample

CX SU Both Samples 

Status of HUs Count

Percent

of CX 

Cases Count

Percent

of SU 

Cases Count

Percent

of Cases 

Percent of 

HHs Eligible 

to be 

Interviewed

Ineligible

Type C Noninterview 1,427 2.19 712 2.77 2,139 2.35

Type B Noninterview 6,146 9.42 2,255 8.78 8,401 9.24

Total Ineligibles 7,573 11.61 29,067 11.55 10,540 11.59

Eligible to be Interviewed

Type A1 Noninterview 3,309 5.07 1,486 5.79 4,795 5.27 5.98

Completed Screener 54,253 83.12 21,157 82.36 75,410 82.91 94.02

Subtotal 57,562 88.19 22,643 88.15 80,205 88.18 100.00

Other Eligible 

 Type A2 Nonresponse 134 0.20 78 0.30 212 0.23

 Total Overall 65,269 100.00 25,688 100.00 90,957 100.00

Proxy and gatekeeper screeners make up a portion of the completed screeners (proxies and 

gatekeepers were first discussed at the end of Chapter 2; see also Appendices E and L for reviews of 

38
gatekeeper and proxy interviews); their rates, along with final screener completion rates, are given in 

Table 5.2 below. Approximately 11.5 percent of the total eligible screeners and 12.2 percent of the total 

completed screeners were proxy or gatekeeper interviews.

Table 5.2 Screener Completion Rates 

Total Sample 

Disposition Count

Percent of Eligible 

Screeners

Percent of Completed 

Screeners

Proxy Screeners 5,175 6.45 6.86

Gatekeeper Screeners 4,055 5.06 5.38

Other Completes 66,180 82.51 87.76

Total Completed Screeners 75,410 94.02 100.00

Response rates for the NLSY97 interview and test are shown in Tables 5.3 and 5.4. For the 

NLSY97 interview, ineligible youths are defined as those cases that, subsequent to the screening 

interview, were determined to be age ineligible on the basis of updated age reports in the youth interview 

or other detailed analyses of screening information. Table 5.3 indicates that 1.5 percent of all cases that

were thought to be NLSY97-eligible at the time of screening were, in fact, not eligible to participate in

the NLSY97 interview; an additional fraction of a percent were determined to be language barrier cases or 

to be too ill or handicapped to participate. About 92.2 percent of the remaining eligible cases completed

the interview, while 90.3 percent of all cases completed an interview. 

Table 5.3 Final Response Status for NLSY97 Youths 

Sample

CX SU Both Samples

Status of Youths Count

Percent

of CX 

Cases Count

Percent

of SU 

Cases Count

Percent

of Cases 

Percent of 

Eligibles

Ineligible 100 1.34 45 1.79 145 1.46

Eligible to be Interviewed 

Type A1 Nonresponse 541 7.28 218 8.66 759 7.62 7.79

Completed Interview 6,748 90.76 2,236 88.80 8,984 90.26 92.21

Subtotal 7,289 98.04 2,454 97.46 9,943 97.89 100.00

Other Eligible 

Type A2 Nonresponse 46 0.62 19 0.75 65 0.65

Total Overall 7,435 100.00 2,518 100.00 9,953 100.00

Those youths who completed an NLSY97 interview were then asked to take the ASVAB test.

Table 5.4 displays unconditional participation rates for the test as well as conditional participation rates 

given that an interview was completed. Noninterviews are made up of ineligibles and Type A1 and A2 

39
nonrespondents, as defined earlier. Interviews also include out-of-scope youths, which are defined as 

youths who, at the time scheduled for testing, were incapacitated, in jail, dead, out of the country, in the 

military, or had a language barrier. (See Appendix D for a detailed discussion of out-of-scope youths,

particularly language barrier cases; Tables D-6 and D-8 in Appendix D give full breakouts of out-ofscope youths by type.) About 90.2 percent of youths completed an interview; of those, 78.0 percent also

completed an ASVAB test, and an additional one percent completed the AFQT5 portion of the test. 

Further, if we look at response in terms of in-scope (for testing) youths only, nearly 80.6 percent of the inscope youths completed at least the AFQT, and 79.6 percent completed the full ASVAB battery.

Table 5.4 ASVAB Test Status for NLSY97 Youths 

Sample

CX SU Both Samples

Status of Youths Count

Percent

of CX 

Cases Count

Percent

of SU

Cases Count

Percent

of

Cases

Percent

of Total 

Interviews

Percent

 of InScopes

No Interview 

 Ineligible 100 1.34 45 1.79 145 1.46

 Type A2 Nonresponse 46 0.62 19 0.75 65 0.65

 Type A1 Nonresponse 541 7.28 218 8.66 759 7.62

 Total Non-Interviews 687 9.24 282 11.20 969 9.74

Interview

 Out-of-Scope for Testing 118 1.59 60 2.38 178 1.79 1.98

 Test Nonparticipation 1,151 15.48 488 19.38 1,639 16.47 18.24 18.60

 Incomplete AFQT6 44 0.59 30 1.19 74 0.74 0.82 0.84

 AFQT Complete7 63 0.85 22 0.87 85 0.85 0.95 0.96

 Complete Battery 5,301 71.30 1,710 67.91 7,011 70.44 78.04 79.59

 Total In-Scope for

 Testing 6,559 88.22 2,250 89.36 8,809 88.50 98.05 100.00

 Total Interviews 6,748 90.76 2,236 88.80 8,984 90.26 100.00

Total Overall 7,435 100.00 2,518 100.00 9,953 100.00

A final summary table, Table 5.5, displays response rates8 overall and for several breakouts, 

including type of area, region, gender, race/ethnicity, and age. Nonmetropolitan youths have consistently

5

 AFQT stands for the Armed Forces Qualification Test, calculated as the sum of the number of correct answers to 

the Word Knowledge, Paragraph Comprehension, Arithmetic Reasoning, and Mathematics Knowledge subtests.

This represents a portion of the entire ASVAB battery, which consists of twelve subtests in total. 6

 An incomplete AFQT is a case for which some but not all of the subtests have been completed; specifically, one 

or more of the subtests which make up the AFQT portion of the ASVAB is not complete. 7

 An AFQT Complete is a case for which some of the subtests are not complete, but all of the subtests used in the

AFQT are complete.

8

 For the purposes of Table 5.5, an AFQT Complete is treated as a response, while an Incomplete AFQT is treated

as a nonresponse. 

40
higher response rates than their metropolitan counterparts. Region, which corresponds to the four Census 

regions, also shows consistent, though small, differences. While the Midwest has the highest response 

rates across the regions, the response rates for all regions are very close and probably do not differ by

more than sampling variability. Differences by gender show females with slightly higher response rates 

than males. Race/ethnicity breakdowns show the most disparate response rates; Hispanics respond less 

frequently than any other racial/ethnic group. Non-Hispanic, nonblack youths have the highest response

rates, while non-Hispanic blacks fall somewhere in between. Age differences are minimal, though 16-

year-olds tend to respond at a lower rate than youths of other ages. 

Table 5.5 Summary of Response Rates (in Percent), Overall and by Domain 

Domain

Household

Screener

Youth

Interview

Given

Screener

ASVAB

Given

Interview

ASVAB

Given

Screener

Total

94.02 92.19 78.88 72.72

Type of Area MSA* 93.65 91.87 77.72 71.40

Non-MSA 95.67 93.31 82.79 77.25

Northeast 94.87 90.89 78.64 71.48

Midwest 95.08 94.31 80.75 76.16

South 93.45 91.72 78.74 72.22

Region

West 93.21 91.91 77.37 71.11

Gender Male 92.03 78.03 71.81

Female 92.37 79.77 73.68

Hispanic 90.23 71.71 64.70

Non-Hispanic Black 92.31 77.28 71.34

Race/Ethnicity

Non-Hispanic Nonblack 92.95 82.53 76.71

12 93.00 79.69 74.11

13 92.54 79.19 73.28

14 92.41 80.85 74.71

15 93.24 79.54 74.16

Age

16 89.67 76.61 68.70

* Metropolitan Statistical Area or Consolidated Metropolitan Statistical Area

41
Screener completion rates were also calculated by PSU and segment and are displayed in Figures 

5.1 through 5.4 below. As seen in Figure 5.1, screener participation by PSU was higher than 85 percent 

for almost all of the PSUs; only eight of the 200 fell below that rate. Figure 5.2 shows the distribution of 

nonresponse by PSU, indicating that 90 percent of the PSUs had a nonresponse rate of 10 percent or less.

Figures 5.3 and 5.4 display similar results for segments. While a larger proportion of the segments fall

below the 85 percent completion rate, we can still see in Figure 5.4 that 90 percent of the segments have a 

nonresponse rate below 15 percent. 

Figure 5.1 Screener Completion

by PSU

700

725

750

775

800

825

850

875

900

0% 20% 40% 60% 80% 100%

Completion Rate

Figure 5.2 Distribution of Screener Type A1

Nonresponse by PSU

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

Nonresponse Rate

Figure 5.3 Screener Completion

by Segment

0

300

600

900

1200

1500

1800

0% 20% 40% 60% 80% 100%

Completion Rate

Figure 5.4 Distribution of Screener Type A1 

Nonresponse by Segment

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

Nonresponse Rate

Obviously, there were some PSUs and segments that had very low response rates. In fact, four 

PSUs and 56 segments fell below a 70 percent screener completion rate, and 11 segments fell below 30 

percent. These low rates are due in part to problems with the computers used for screening; in some

segments, computer crashes were the overwhelming cause of screener nonresponse. Appendix N details 

the issues regarding crashed cases during the screening process; Appendix M gives a full analysis of 

noninterview reports (NIRs) and examines further reasons for screener nonresponse.

Impact of Nonresponse. The impact of nonresponse on the survey results (in this case the 

NLSY97 interview responses and the ASVAB test scores) depends on the product of two factors: (1) the 

proportion of the sample which is nonresponding, and (2) the magnitude of the difference between

42
respondents and nonrespondents. A large difference between respondents and nonrespondents combined

with a small percentage of nonresponse may produce minimal impact on the survey results. Similarly, a

high rate of nonresponse may have minimal impact on survey results as long as the difference between

respondents and nonrespondents is small. Since the impact of nonresponse is a function of the product of 

the two factors, if either factor is small the impact may be minimal.

The major difficulty in estimating the impact of nonresponse derives from the fact that although

the rate of nonresponse (factor 1) is usually calculable, the magnitude of the difference between 

respondents and nonrespondents (factor 2) can seldom be known with any confidence. Several methods

for estimating the characteristics of nonrespondents have been suggested in the literature, but since none 

is entirely satisfactory, one can assume that the impact of nonresponse is negligible only when the rate of 

nonresponse (factor 1) is very small.

One possible way to examine the potential for differences between respondents and

nonrespondents is to examine response rates among subgroups, as shown earlier in Table 5.7. A common

method for reducing the impact of nonresponse is to adjust the weights for nonresponse. For the 

NLSY97, corrections for nonresponse were introduced in the weighting process (see Chapter 4). A 

correction for screener nonresponse was made at Step 2 and for interview nonresponse at Step 5. 

Screener nonresponse was adjusted at the segment level for all respondents in the segment while

interview nonresponse was adjusted separately within design cohorts based upon sample (CX or SU), age,

race/ethnicity, and sex. Thus, in correcting for screener nonresponse we assumed no difference between 

responding and nonresponding households in the same segment. In the correction for interview 

nonresponse we assumed no difference between respondents and nonrespondents who were in the same

design cohort. 

As described in Chapter 4, a further adjustment to correct for differences between study

participants and the appropriate reference population (the CPS was used) was made in the calculation of a

post-stratification adjustment factor. The use of age, sex, and race/ethnicity in the post-stratification

adjustment was based on theoretical considerations as well as previous survey experience which suggests 

that these factors are related to many survey variables of interest.

It is important to point out that all surveys with less than 100 percent response are forced to make

assumptions about the characteristics of nonrespondents. An unweighted (or self-weighting) survey

assumes that all nonrespondents taken as a group are like all respondents taken as a group. We found it 

preferable on both theoretical and empirical grounds to make such assumptions at a disaggregated level, 

i.e., at the segment or design cohort level. We feel that these more exacting corrections treat nonresponse

with the care it deserves.9

Having made the various adjustments for nonresponse described above, we carried out a simple

but fairly standard test to examine differences between the final completed sample and the sample at 

various stages in the overall survey process, in addition to differences between the NLSY97 results and a

nationally representative reference population. Specifically, we compared the sample distribution for a 

few variables for screener respondents, NLSY97 eligibles who completed the interview, and NLSY97 

9

 Needless to say, we also did all we could through field operations to keep the response rate high.

43
respondents who completed the ASVAB test to families with resident youths aged 12 to 16 responding to 

the March 1997 Current Population Survey (CPS). Table 5.6 shows the results of these comparisons for 

the NLSY97 sample. The rationale for making the comparisons is to see whether there are any

differences between the CPS sample of youths and the NLSY97 sample, and whether any significant

changes in sample composition appear from one stage of the survey to the next as nonresponse attrition 

reduces the number of participants. Distributions are also given separately for the CX and the SU 

samples; this will help to illustrate the differences between these two distinct groups and illuminate

where, in particular, any differences between NLSY97 and CPS may occur. As Table 5.8 shows, the 

overall sample of screener respondents is almost identical to the overall sample of NLSY97 interviews 

and test completes in terms of the distribution across household size, urbanicity, and region of the 

country.10 Further, NLSY97 displays a somewhat higher proportion than CPS of youths in small

households and in metropolitan areas. Most of the differences by region are due to the concentration of

supplemental cases in the South; however, even the CX sample has more youths from the South and 

fewer from the West than the CPS. Remember that the SU sample consists only of black and Hispanic 

youths and was sampled with probability proportional to the number of blacks and Hispanics; thus, this 

sample is very different from the CPS and the CX samples, which are balanced, representative cross 

sections of the U.S. population. Because the SU sample distribution is skewed toward black and Hispanic

households, the numbers for this sample will be quite different from the other samples shown. For this

reason, the reader should focus attention on a comparison of the CPS sample to the CX sample.

10 Since sample weighting was only applied to respondents who participated in the interview, the comparison

between the screener sample and the interview sample is shown on an unweighted basis.

44
Table 5.6 Distributions (Unweighted) of Screener Variables for CPS Youths, Completed Screener Respondents, 

and NLSY97 Interview and Test Completes11

Screener Respondents 

(NLSY97 Eligibles) NLSY97 Test Completes CPS

Youths

Aged 12-16

CX SU

Distribution (in percent) among 

Variable NLSY97 Interviews 

Total CX SU Total CX SU Total

Persons in Household  1 – 2 4.8 5.6 7.1 5.9 5.4 6.5 5.6 4.9 6.4 5.2

17.8 18.4 16.5 18.0 18.7 15.7 18.0 18.6 15.5 17.9 3 

 4 33.4 33.6 24.9 31.4 33.5 25.2 31.4 34.2 26.4 32.3 5 24.1 25.1 20.4 23.9 25.4 20.4 24.2 25.4 20.3 24.2 6 or more 19.9 17.3 31.1 20.8 17.0 32.2 20.8 16.9 31.4 20.4Type of Area  MSA 74.1 74.2 87.1 77.5 73.9 86.8 77.2 72.9 85.6 76.0 Non-MSA 25.7 25.8 12.9 22.5 26.1 13.2 22.8 27.1 14.4 24.0Region Northeast 18.7 19.0 15.3 18.0 18.7 14.6 17.7 18.1 15.8 17.5 Midwest 23.4 25.8 12.2 22.3 26.3 13.0 22.9 26.6 12.5 23.3 South 29.8 33.9 47.1 37.3 33.9 47.3 37.2 34.0 48.0 37.3 West 28.1 21.3 25.3 22.4 21.1 25.1 22.1 21.1 23.7 21.911 These percentages are slightly inaccurate due to ineligibles found in Round 2 and subsequent reweighting for Round 1. 

45
Table 5.6 shows that nonresponse did not produce any major changes in sample composition at

later stages of the survey process, at least for the variables presented here. Yet the limited number of 

screener variables available for comparison should induce caution in concluding that the distributions of

test scores and study variables among nonrespondents would be identical to the distributions among

respondents.

More importantly, these analyses show that the NLSY97 sample is not dramatically different 

from another nationally representative sample, the CPS. Further evidence of the similarities between the 

NLSY97 and the CPS based on family income and educational attainment of the youths’ parents can be 

found in Appendices F and G.

In conclusion, we would like to point out that the best overall evidence that the impact of 

nonresponse must be small is the relatively high response rate attained. Whatever differences there may

be between respondents and nonrespondents are diluted by the fact that there are many more respondents 

than nonrespondents. For example, imagine that we wanted to consider the equation 

p  rp1 2 ( ) 1 r p

where p is the true proportion of youths with a certain attribute, is the proportion for participants, is

the proportion for nonparticipants, and r is the response rate. Normally, because we don’t have any

information about nonparticipants, we employ an unbiased estimator of . As Table 5.7 shows, for p = 

.5 and for response rates ranging from 72 to 82 percent, even if is as much as 10 percentage points 

higher than , the true proportion is still quite close to . In other words, a large difference between

respondents and nonrespondents still produces only a small difference in the true versus observed 

proportion, given that the response rate is relatively high. 

p1 p2

p1

p2

p1 p1

Table 5.7 True Overall Proportions of Youths Given Varying Proportions for 

Nonparticipants and Varying Response Rates 

p1 : proportion for 

participants

p2: proportion for 

nonparticipants r: response rate 

p: true overall 

proportion

0.50 0.55 0.82 0.509

0.50 0.55 0.77 0.512

0.50 0.55 0.72 0.514

0.50 0.60 0.82 0.518

0.50 0.60 0.77 0.523

0.50 0.60 0.72 0.528

5.2 Refielding 

While all of the response rates presented in Section 5.1 are final, the rates of response achieved 

by the end of the field period in October of 1997 were actually somewhat lower. After completion of the 

field period, NORC discovered that many of the screeners that were not completed had, in fact, been 

46
broken off in midstream or otherwise left unfinished; furthermore, there was evidence that these 

incomplete cases were rich in NLSY97-eligible youths. In an effort to capture at least some of these 

youths, NORC went back into the field in what we refer to as the “refielding” period to complete the 

screening of these households and to complete the youth and parent interviews for NLSY97-eligible 

youths found therein. We refielded 1,175 such households in April-May 1998. 

Also, because we were going back into the field, we decided to reapproach 640 youth cases in 

527 households who were screened in during the original field period, but for whom the youth interview

had been refused or was otherwise incomplete. During refielding, we sought to obtain the youth and

parent interviews for these cases. Overall, we refielded 1,702 (1,175 + 527) households. 

Table 5.8 displays screener response statuses before and after refielding; the corresponding rates

by sample type (CX and SU) are shown in Tables 5.9 and 5.10. Of the 1,702 households refielded, 973 

completed a screener, compared to 763 completed screeners before refielding. This corresponds to 210 

additional completed screeners, 161 (697-536) of which were achieved in the CX sample and 49 (276-

227) in the SU sample.

Table 5.8 Screener Response Status Before and After Refielding 

Final Screener Response Status 

Response Status

Before Refielding 

Type C

NonInterview

Type B 

NonInterview

Type A2 

NonResponse

Type A1 

NonResponseCompleted

Screener

TOTAL

Type C Non-Interview 4 1 0 6 9 20

Type B Non-Interview 0 7 0 21 7 35

Type A2 Non-Response 0 0 13 21 27 61

Type A1 Non-Response 1 3 0 556 263 823

Completed Screener 3 1 2 90 667 763

TOTAL 8 12 15 694 973 1,702

Table 5.9 Screener Response Status Before and After Refielding: CX Sample 

Final Screener Response Status 

Response Status 

Before Refielding

Type C

NonInterview

Type B 

NonInterview

Type A2 

NonResponse

Type A1 

NonResponse

Completed

Screener

TOTAL

Type C Non-Interview 2 1 0 6 6 15

Type B Non-Interview 0 5 0 13 5 23

Type A2 Non-Response 0 0 11 14 17 42

Type A1 Non-Response 0 3 0 450 201 654

Completed Screener 1 0 1 66 468 536

TOTAL 3 9 12 549 697 1,270

47
Table 5.10 Screener Response Status Before and After Refielding: SU Sample 

Final Screener Response Status 

Response Status

Before Refielding 

Type C

NonInterview

Type B 

NonInterview

Type A2 

NonResponse

Type A1 

NonResponse

Completed

Screener

TOTAL

Type C Non-Interview 2 0 0 0 3 5

Type B Non-Interview 0 2 0 8 2 12

Type A2 Non-Response 0 0 2 7 10 19

Type A1 Non-Response 1 0 0 106 62 169

Completed Screener 2 1 1 24 199 227

TOTAL 5 3 3 145 276 432

Table 5.11 gives NLSY97 response status before and after refielding, illustrating the NLSY97-

eligible cases screened in and interviewed in the refielding operation. As seen here, 278 new NLSY97 

youth cases were spawned during refielding, 2 of which were ultimately found to be ineligible, and 222 of 

which completed an interview. Further, of the 640 NLSY97 cases coded as type-A1 nonrespondents

before refielding, 12 were found to be ineligible, and 180 actually completed an interview during 

refielding. Thus, the refielding operation resulted in an additional 402 completed interviews. 

5.11 Response Status for the Youth Interview Before and After Refielding 

Final Response Status 

Response Status

Before Refielding Ineligible

Type A2 NonResponseType A1 NonResponse Completed

Interview TOTAL

Screened in Refielding 2 1 53 222 278

Type A1 Non-Response 12 2 446 180 640

14 3 499 402 918

The refielded cases were clearly hard to interview, either because they initially refused the youth

interview or because the original screener was broken off or incomplete. Yet, we ultimately found 918 

youths in the 1,702 refielded households, or a youth rate of 0.54 youths per household. This youth rate is 

much larger than the rate for American households overall. The refielded households were clearly rich in 

eligible youths. And since this is so, the refielding experience begs the question, “Were other hard to 

enumerate households, not refielded, richer than average in youths?” Modern professional experience 

and judgement suggests that the answer to this question must be “yes,” yet at this writing we have no hard

data to confirm or refute this judgement. We conclude, based only on judgement, that some portion of the 

shortfall between youths planned and youths achieved may be due to hard to enumerate households, rich 

in youths, who did not complete the NLSY97 screener. 

48
5.3 Representativeness of the Sample 

Overview. In this section, we examine the representativeness of the NLSY97 sample, including 

the large screening sample and the youths who actually participated in the NLSY97 interview and testing 

program. We begin by reviewing the screening sample in terms of the coverage of housing units and the 

designation of housing units as occupied or vacant. This review is intended to shed light on the quality of 

the sampling of PSUs and segments, the listing of housing units or addresses, the sampling of housing

units or addresses for screening, and the screening for occupancy status. 

Next, we examine the screening sample in terms of the coverage of persons, focusing on the 12-

23 age range. We present the age distribution of all persons who were screened in our large screening 

sample, and we discuss the distribution of screened households by the number of eligible youths reported. 

Third, we analyze the age/sex/race distribution of the youths who screened in and of the youths

who participated in the NLSY97. This analysis is intended to provide an understanding of the 

demographic balance of the samples and of how the sample compares to known or presumed known

national norms. We also review the composition of the sample with respect to its socio-economic

balance, including a review of variables such as parent education and household income.

Fourth, we examine once again the coverage of youths by age, sex, and race, and we compare and

contrast the coverage of NLSY97, the Current Population Survey (CPS), and the Decennial Census. 

By this point, it will have become clear to even the most casual reader that NLSY97 experienced 

some notable undercoverage of youths aged 12-23. Finally, we summarize some special studies we 

performed to try to explain why the undercoverage occurred. Such studies include validation interviews 

and debriefing interviews. 

Coverage of HUs. For every segment selected, we have the 1990 Census count of housing units 

as well as counts of youths under the age of 18 in each of three race/ethnicity categories: Hispanic, nonHispanic black, and non-Hispanic nonblack. We also have, for each segment, a sampling base weight, 

which is the inverse of the segment’s probability of selection.

The sampling base weight indicates the number of units “represented” by each unit in the 

segment. Therefore, if a segment has five Hispanic youths under the age of 18 years and a sampling base 

weight of 1000, that segment is said to “represent” (5 * 1000 = ) 5000 Hispanic youths under the age of 

18. A “representative” sample would therefore “represent” the entire population of the United States. We 

refer to this process as “weighting up the segment-level information.”

We analyze the CX and SU samples separately because they were selected independently. Table 

5.12 below shows the outcome of this analysis for the CX sample for all four counts mentioned above. 

The analysis is done separately for each of the four Census regions in addition to the national analysis.

Four rows of data are given for each analysis. The first line is the data from our selected segments

weighted up by the segment base weight. The second line is the 1990 Census count of youths aged 0 to 

17 or housing units. The third line is the difference, and the fourth line is this difference as a percentage 

of the data from the 1990 Census.

49
Table 5.12 Weighted-up 1990 Census Counts of Youths Under Age 18 

and of Housing Units for the CX Segments 

Region Total Youths Hispanic

Non-Hispanic

Black All Other Housing Units

National

“Represented” 62,210,920 7,071,404 9,772,687 45,366,828 102,184,234

Census 63,606,544 7,637,394 9,566,954 46,402,196 102,263,678

Difference -1,395,624 -565,990 205,733 -1,035,368 -79,444

Percent Difference -2.19 -7.41 2.15 -2.23 -0.08

Northeast

“Represented” 11,889,060 1,059,730 1,726,455 9,102,875 21,232,281

Census 11,911,083 1,164,579 1,652,022 9,094,482 20,810,637

Difference -22,023 -104,849 74,433 8,393 421,644 

Percent Difference -0.18 -9.00 4.51 0.09 2.03

Midwest

“Represented” 15,267,945 574,694 1,704,364 12,988,887 24,304,554

Census 15,630,137 634,725 1,883,154 13,112,258 24,492,718

Difference -362,192 -60,031 -178,790 -123,371 -188,164 

Percent Difference -2.32 -9.46 -9.49 -0.94 -0.77

South

“Represented” 21,130,305 1,722,438 5,408,463 13,999,403 35,692,925

Census 22,017,465 2,247,486 5,167,288 14,602,691 36,065,102

Difference -887,160 -525,048 241,175 -603,288 -372,177 

Percent Difference -4.03 -23.36 4.67 -4.13 -1.03

West

“Represented” 13,923,610 3,714,542 933,405 9,275,663 20,954,475

Census 14,047,859 3,590,604 864,490 9,592,765 20,895,221

Difference -124,249 123,938 68,915 -317,102 59,254 

Percent Difference -0.88 3.45 7.97 -3.31 0.28

50
Table 5.13 below shows a similar analysis for the SU sample. However, because the SU sample

was intended to be representative only for black and Hispanic youths, we do not present this analysis for 

the housing units or the non-Hispanic, nonblack youths in the SU sample.

Table 5.13 Weighted-up 1990 Census Counts of Minority Youths

Under Age 18 for the SU Segments

Region

Total Minority

Youths Hispanic

Non-Hispanic

Black

National

“Represented” 17,067,764 7,521,903 9,545,861

Census 17,204,348 7,637,394 9,566,954

Difference -136,584 -115,491 -21,093 

Percent Difference -0.79 -1.51 -0.22

Northeast

“Represented” 2,733,764 1,113,334 1,620,430

Census 2,816,601 1,164,579 1,652,022

Difference -82,837 -51,245 -31,592 

Percent Difference -2.94 -4.40 -1.91

Midwest

“Represented” 2,505,257 726,056 1,779,202

Census 2,517,879 634,725 1,883,154

Difference -12,622 91,331 -103,952

Percent Difference -0.50 14.39 -5.52

South

“Represented” 7,489,582 2,182,927 5,306,655

Census 7,414,774 2,247,486 5,167,288

Difference 74,808 -64,559 139,367 

Percent Difference 1.01 -2.87 2.70

West

“Represented” 4,339,161 3,499,587 839,574

Census 4,455,094 3,590,604 864,490

Difference -115,933 -91,017 -24,916 

Percent Difference -2.60 -2.53 -2.88

The results for youths are quite encouraging, especially for the SU sample. The number of black 

youths under the age of 18 is only off nationally by 0.2 percent. Even at the regional levels (for which the 

sample was not designed to be representative), none of the differences for black youths are over 6 percent.

The national number of Hispanic youths under the age of 18 is only off by 1.5 percent. The discrepancy

of 14.4 percent in the Midwest looks large, but the absolute differential of 91,331 is not much larger than 

the discrepancy of 91,017 in the West. It is not surprising that the SU sample shows smaller percentage 

discrepancies than the CX sample. The targeting of the SU sample to black and Hispanic youths results

51
in more accuracy for these two groups at the expense of accuracy for non-minority youths and housing 

units.

The CX sample is extremely close with respect to housing units, coming within 0.08 percent of 

the Census count. In fact, this difference would be zero except that one segment was selected with

certainty. The CX sample is very representative with respect to black and non-minority youths under the

age of 18. There are 2.2 percent more black and 2.2 percent fewer non-Hispanic, nonblack youths

represented by the sample than appear in the 1990 Census. There is a shortfall among Hispanic youths,

however. There are 7.4 percent fewer Hispanic youths under the age of 18 represented by the sample

than appear in the 1990 Census. This difference is almost entirely due to a large shortfall in the South 

region. It is important to remember, though, that the CX sample is not targeted at minorities; a segment’s

selection probability is proportional to its total number of housing units. In a nationally representative 

sample, there is often variability in the size of subgroups. 

When NORC sent listers to the segments selected in the Fall of 1995, there were usually not the 

same number of housing units in the segment as the 1990 Census indicated. Many segments had

undergone significant change between 1990 and 1995. In order to show the representativeness of the

housing units that were actually found in the selected segments, we can weight up the number of housing 

units listed in the segments (again using the segment base weight, the reciprocal of the segment

probability of selection). The number of housing units thus Arepresented@ is shown below, compared to 

the 1990 Census as well as more recent Census Bureau estimates. This analysis tests the quality of the 

NLSY97 listing operation. The results for the SU sample are more difficult to interpret because of the 

measure of size used in selecting PSUs and segments.

Table 5.14 below shows weighted-up counts of HUs for the CX sample.

Table 5.14 Weighted-Up Listing Counts for the CX Sample 

Region

(1)

1990

Census

(2)

Sample

Census

(2)/(1)

Ratio

(in

percent)

(3)

Sample

Listing

(3)/(2)

Ratio

(in

percent)

(4)

Population

Estimates

Program

July, 1996 

(3)/(4)

Ratio

(in

percent)

TOTAL 102,263,678 102,184,234 99.92 106,944,539 104.66 109,800,000 97.40

Northeast 20,810,637 21,232,281 102.03 21,672,550 102.07 21,530,000 100.66

Midwest 24,492,718 24,304,554 99.23 25,189,511 103.64 26,014,000 96.83

South 36,065,102 35,692,925 98.97 38,441,114 107.70 39,416,000 97.53

West 20,895,221 20,954,475 100.28 21,641,365 103.28 22,840,000 94.75

52
Column (1) contains 1990 Census housing unit counts, while column (2) contains the weightedup 1990 Census housing unit counts from the selected NLSY97 segments (these numbers also appear in

Table 5.12). The third column shows that columns (1) and (2) are within one percent overall, and within 

two percent for every region. Column (4) shows (revised) estimates of the numbers of housing units as

of July 1, 1996, from the Census Bureau’s Population Estimates Program (PEP) 12. The PEP estimates the 

number of housing units by updating the 1990 Census counts of housing units. The main sources for the 

updates are construction and demolition permits, where these permits are available. These estimates are 

larger than those in column (1), indicating that there are more housing units now than in 1990.

Column (3) shows the weighted-up sums of actual housing unit counts from the NLSY97 listing 

operation. The weighted-up sums of housing unit counts from the NLSY97 listing effort are greater than

the 1990 Census counts, but are smaller than the July 1996 estimates. This is to be expected because the 

listing operation was done in the Fall of 1995. 

Under the assumption that growth in the housing stock was uniform between 1990 and 1996, we

might expect the numbers in column (3) to be very close to the July 1996 PEP estimates. In fact, the

numbers in column (3) are closer to the July 1996 PEP estimates than to the 1990 Census counts, and 

there are a few factors that could explain why they are not even closer to the July 1996 PEP estimates (we 

would not expect our listing-based estimate to be equal because of a half-year of growth between the Fall 

of 1995 and July 1996). First, the PEP estimates themselves are estimates, and therefore subject to error. 

Second, the numbers in column (3) are based on a sample, and are therefore subject to sampling

variability. Finally, there is a possibility of NLSY97 or PEP coverage error. We note, however, that 

NLSY97 used a half-open interval procedure (see Chapter 3) during the main field effort to correct any

such listing errors as well as to include new construction in our sample.

Some of the comparisons made above for the CX sample are inappropriate for the SU sample

because the probabilities of selection were based upon minority youths rather than housing units. This 

means that the weight (the inverse of the probability of selection) will be large for a non-minority

segment and small for a minority segment. Therefore, non-minority segments will have their housing unit 

counts multiplied by a large weight, regardless of how many housing units they contain. Any nonminority segments with a large number of housing units will dominate the sum, which results in an 

unbiased estimator of the 1990 Census count of housing units, but with a very large sampling variance. 

One extra or fewer non-minority segment will have a large effect on the total sum of weights. However, 

it is still useful to compare the weighted-up listing operation counts with the weighted-up 1990 Census 

segment counts. This comparison is shown in Table 5.15. 

12 The Census estimates given in Table 5.14 can be found at 

http://www.census.gov/population/estimates/housing/prhuhht1.txt. More information can also be found at this 

website, and the methodology used is described in Census report P25-1127.

53
Table 5.15 Comparison of Census and Listing Counts in the SU Sample 

(2)

Sample

Census

(3)

Sample

Listing

(3)/(2)

Ratio

(in percent) 

TOTAL 115,405,337 118,265,606 102.4785

Northeast 13,808,939 15,745,940 114.0272

Midwest 54,819,502 52,826,805 96.3650

South 32,416,072 33,438,963 103.1555

West 14,360,825 16,253,898 113.1822

The weighted-up Census housing unit counts for the SU sample shown as column (2) of Table 

5.15 can be used in order to obtain a test of the listing operation for the SU sample. This is because 

weighting-up the actual listing counts to a national total uses the same weights as those used in column

(2). The weighted-up actual listing counts are shown in column (3) of Table 5.15. The rightmost column

in Table 5.15 shows that these two columns are within 3 percent overall, and within 2,000,000 housing 

units in every region. These differences do seem to show a general growth from the 1990 Census 

[column (2)] to the 1995 listing operation [column (3)], which was expected. It is also true that 

differences found in segments with large weights would be exaggerated by the large weights. This could

explain the apparent decrease in the Midwest, which is where the large weights (i.e., areas with low 

counts of minority youths) seem to be concentrated. 

Of the 270,197 housing units listed, 90,139 were selected into the sample; a further 818 housing

units were added to the sample through the half-open interval procedure. We can weight up these 90,957 

housing units using their (untruncated) base weights. This is done in Table 5.16 below. 

Table 5.16 Sums of Base Weights for Selected Housing Units 

Sample Region Sum of Weights 

TOTAL 108,965,394

Northeast 22,070,137

Midwest 24,579,151

South 39,489,719

CX

West 21,926,387

TOTAL 121,182,555

Northeast 15,761,767

Midwest 55,976,275

South 33,140,334

SU

West 16,304,180

These numbers are very similar to the numbers in column (3) of Tables 5.14 and 5.15, indicating that a 

54
representative sample of the listed housing units was selected. It has already been stated that the sum of 

the weights for the SU sample is too high because of the large weights assigned to housing units in nonminority areas, and that the sum of the weights for the CX sample is close to the national total number of 

housing units.

We now move from an examination of the sampling of PSUs, segments, and HUs to an

examination of the results from screening, where field interviewers classified HUs as occupied or vacant. 

Prior to the field work, it was hypothesized that 85 percent of the housing units selected would be 

occupied. We can examine the actual rate because of the very complete set of screener disposition codes 

used for NLSY97. Table 5.17 shows the NLSY97 occupied rate, and compares it to two Census Bureau

standards. One standard used is the 1996 Housing Vacancy Survey. This survey comes from Current 

Population Survey household estimates averaged over a twelve-month period. The second standard is the 

Population Estimation Program used already in this chapter. NLSY97 has a very similar rate of vacant 

housing units (11.3%) to the rate shown in the Census Bureau’s 1996 Housing Vacancy Survey (11.5%), 

and is slightly larger than the rate shown in the Census Bureau’s (revised) July 1996 Population Estimates

Program (10%).

Table 5.17 Comparison of the NLSY97 Vacancy Status of Housing Units 

NLSY97/PAY97

Housing Vacancy 

Survey*

Population Estimates 

Program#

Cases Percent Cases Percent Cases Percent

All Lines 

90,957

Not an HU 327

 * Business 233

 * Group Quarters 94

HU 90,630

Vacancy not determined 157

 * HU inaccessible 157

Vacancy determined 90,473 100.00 114,139,000 100.00 109,800,000 100.00

 ++Vacant 10,214 11.29 13,155,000 11.53 11,049,000 10.06

 * Condemned 163 0.18

 * Demolished 163 0.18

 * No such address, HU 1,580 1.75

 * Vacation cabin 600 0.66

 * Unusable as perm. Resid. 121 0.13

 * Transient use 102 0.11

 * Under construction 28 0.03

 * Vacant 7,457 8.24

 ++Occupied 80,259 88.71 100,984,000 88.47 98,751,000 89.94

*

These data are available at http:www.census.gov/hhes/www/housing/hvs/historic/histab7.html

#

From the (revised) Population Estimates Program: July, 1996. 

As stated in Chapter 4, the preliminary coverage ratios that suggest that “one out of every three 

NLSY97 eligibles” were missed are very misleading. The caps used during weighting steps 1 and 2 to 

55
prevent extreme weights have a negative effect on the coverage ratios. The effect of these weighting caps 

can be seen most effectively from the effects on the sum of the household weights, examined in Table 

5.18 below. 

Table 5.18 Effect of the Weighting Caps on the Sums of the Household Weights 

CX Sample SU Sample 

Sum of 

Samples

Census estimate of housing units, 1996 (PEP) 109,800,000 109,800,000 219,600,000

NLSY97 estimate of housing units 108,965,394 121,182,555 230,147,950

Census estimate of occupied housing units, 1996 (PEP) 98,751,000 98,751,000 197,502,000

NLSY97 estimate of occupied housing units 96,413,677 109,531,766 205,945,443

Percent of difference 2.37 -10.92 -4.28

Sum of weights (base weight only capped) 96,413,677 83,471,399 179,885,076

Percent of weights truncated by base weight cap 0.00 26.39 13.19

Sum (base weight and missed HU adjustment capped) 95,696,559 83,374,766 179,071,325

Percent of weights truncated by missed HU adjustment 0.73 0.10 0.41

Sum of weights after nonresponse (no cap of 2.0 used) 95,092,819 83,048,494 178,141,313

Percent of weights truncated by using PSU response rate 0.61 0.33 0.47

Sum of weights (with truncation within 4 PSUs) 95,092,819 82,824,571 177,917,390

Percent of base weights truncated within 4 PSUs 0.00 0.23 0.11

Total reduction by caps in sum of weights 1,320,858 26,707,195 28,028,053

Total percent of base weights truncated by caps 1.34 27.04 14.19

The third column in Table 5.18 is simply the sum of the CX and SU weight sums; it is used to 

examine an “overall” weighting cap effect. The top row in the table is the Census Bureau’s estimate of the 

total number of U.S. housing units, as taken from the 1996 Population Estimates Program (PEP). The 

NLSY97 estimate is the total sum of the untruncated base weights after the untruncated missed housing 

unit adjustment. The next two rows remove housing units from the rest of the table. The Census estimate

of occupied housing units is also taken from the 1996 PEP. The NLSY97 estimate is the total sum of the

untruncated base weights after the untruncated missed housing unit adjustment for only occupied housing 

units. The denominator for all percentages is the PEP estimate of occupied housing units. Base weights 

were capped only in the SU sample, but this resulted in a 26.4 percent reduction in the sum of the 

supplemental weights.

Other truncations made were: 1) truncating the missed housing unit adjustment to 5.0; 2) (not 

technically a truncation) using the PSU response rate instead of the segment response rate in segments

with fewer than 25 respondents to reduce the sum of weights by preventing some large adjustments; and 

3) truncating the nonresponse weighting adjustment within the four supplemental PSUs with a low PSU

response rate. These other truncations had only small effects on the sum of the weights (all of less than 

one percent). 

Taken together, weighting caps decreased the sum of the CX weights by 1.3 percent, the sum of 

the supplemental weights by 27.0 percent. Of course, the reader will recall that the weight is restored by

poststratification, the last step in weighting.

56
Coverage of Persons. Chapter 3 touches on the shortfalls in the numbers of youths screened in 

to the NLSY97 samples. This section furthers that discussion. We have conducted various analyses to 

determine both the coverage and representativeness of the NLSY97 samples. Some of those analyses

were presented in Section 5.1; several more will be illustrated here. The March 1997 CPS was used in all 

of the following analyses as a comparison; weighted CPS numbers include the Hispanic supplement,

while unweighted analyses using CPS data do not.

Figures 5.5, 5.6, and 5.7 display weighted13 counts of persons for the overall screening sample as 

well as the CX and SU samples compared to the CPS. These figures show distributions for ages 0 to 35

and illustrate the shortfalls in the age ranges critical to the NLSY97. While screening identified most of 

the expected persons below age 12, and almost exactly the number of expected persons aged 24 to 35,

there is a large undercount of youths aged 12 to 23. 

The reader will recall that youths aged 12-16 were eligible for the NLSY97 and that older youths

up to and including age 23 were eligible for components of PAY97, the DoD study that employed the 

NLSY97 screening sample. Thus, the hole in the age distribution coincides exactly with the range of ages 

for which the screening sample was intended. No natural age distribution exhibits this type of pattern. 

Evidently, the NLSY97 screening sample experienced an undercoverage mechanism specific to its 

targeted ages. 

Figure 5.5 Weighted Counts of Persons,

Total Sample

0

1,000,000

2,000,000

3,000,000

4,000,000

5,000,000

0 4 8 12 16 20 24 28 32

Age

CPS-Expected

NLSY

13 This weighted analysis relies on a special set of household weights for the screener completed cases. These 

weights are different from the final youth weights calculated for youth completes. The special weights, described in

detail in Appendix S, make appropriate allowance for the fact that proxy and gatekeeper screeners did not result in

HH rosters. 

57
Similar patterns exist by sample type, as seen below. The CX and SU samples both have

noticeable gaps in coverage for youths aged 12 to 23; for the SU sample, these gaps are even more severe 

in the 20- to 23-year-old age range. For all three figures, notice the spike in the number of persons aged

11, and the marked drop following it for youths aged 12. This pattern could indicate that the respondents 

were somehow aware of the ages sought for the survey, and thus reported youths who were actually 12 

(or older) as 11 years old. Similar age misreporting may also have happened at the other end; the low 

number of 23-year-olds reported followed by a sharp spike at age 24 supports this hypothesis.

Figure 5.6 Weighted Counts of Persons,

CX Sample

0

1,000,000

2,000,000

3,000,000

4,000,000

5,000,000

0 5 10 15 20 25 30 35

Age

CPS-Expected

NLSY

Figure 5.7 Weighted Counts of Persons,

SU Sample

0

1,000,000

2,000,000

3,000,000

4,000,000

5,000,000

0 5 10 15 20 25 30 35

Age

CPS-Expected

NLSY

As these figures show, undercoverage is differential by age—specifically by the eligible ages—

therefore, the undercoverage mechanism cannot be one that generally misses households or people. In 

other words, the mechanism is not related to sampling but is likely related to reporting in the screening 

interview.

Table 5.19 displays the distribution of households by the number of 12- to 16-year-old youths for 

the CPS compared to the NLSY97 sample. In this and the tables that follow, please note that a 

comparison of the CX sample to the CPS is appropriate because the SU sample is dominated by black and 

Hispanic households. As seen here, the NLSY97 sample has a larger proportion of households with no 

youths in the age range, further suggesting that households did not report youths when there were in fact

resident youths in the age range.

Table 5.19 Distribution of Households by Reported Number of Youths Aged 12 to 16 

Proportion of Households 

Number of NLSY97-Weighted

Youths

CPS

 Weighted Total CX SU

0 .848 .892 .894 .876

1 .114 .079 .077 .091

2 .034 .026 .026 .029

3 .004 .004 .004 .003

4 .000 .000 .000 .000

 5+ .000 .000 .000 .000

58
Table 5.20 gives conditional household distributions, given there was at least one resident youth

in the 12- to 16-year-old age range, this time overall and by race/ethnicity of the householder. After 

controlling for the existence of at least one youth, the overall NLSY97 sample has slightly fewer 

households with only one such youth, and slightly more with two or more such youths. Again, this 

implies that households with only one youth in the age range were less likely to report that youth, while 

households with multiple youths were more likely to report all of their youths. This finding holds for the

sample overall and for each of the race/ethnicity groups. 

Table 5.20 Conditional Distribution of Households (HHs) by Reported Number of Youths Aged 

12 to 16, Given at Least One Such Youth

Proportion of Households 

Number of NLSY97-Weighted

Youths

CPS

 Weighted Total CX SU

1

Total HHs

.748 .725 .722 .733

2 .224 .238 .240 .235

3 .024 .033 .034 .028

4 .003 .003 .003 .003

 5+ .001 .001 .001 .000

1

Hispanic

Householder

.733 .675 .673 .694

2 .237 .276 .280 .232

3 .025 .045 .044 .066

4 .006 .003 .002 .007

 5+ .000 .000 .000 .000

1

Non-Hispanic,

Black Householder

.750 .738 .739 .737

2 .214 .211 .210 .208

3 .026 .041 .041 .039

4 .006 .007 .006 .015

 5+ .004 .003 .003 .001

1

Non-Hispanic,

Nonblack

Householder

.754 .731 .728 .737

2 .221 .238 .239 .240

3 .023 .029 .030 .022

4 .002 .002 .002 .001

 5+ .001 .000 .000 .000

The next table, Table 5.21, displays the mean number of youths in each household that has at 

59
least one youth aged 12 to 16. Again, these tables show that the NLSY97 sample has larger numbers of

youths in the 12- to 16-year-old age range given that they reported at least one youth in that age range, 

once again suggesting that households with more than one youth were more likely to report their youths

than households with only one youth. Again, this is true overall and for each of the races/ethnicities. 

Table 5.21 Conditional Mean Youths, Age 12 to 16, Per Household 

Mean Number of Youths, Age 12-16,

Per Household With >= 1 Such Youth 

NLSY97-Weighted

Race/Ethnicity CPS Weighted Total CX SU

Hispanic 1.303 1.377 1.376 1.386

Non-Hispanic, Black 1.300 1.328 1.324 1.337

All Other 1.275 1.303 1.308 1.288

Total 1.282 1.316 1.320 1.302

A tabulation of average household sizes is shown in Table 5.22, and the distribution of 

households by the size of the household is given in Table 5.23. Compared to the CPS, NLSY97

households are reported smaller. Table 5.22 shows that the average household size for the March, 1993 

CPS, which was used in planning the NLSY97 sample size, is 2.62; this falls between the mean household 

size for the March, 1997 CPS (2.89) and for the NLSY97 (2.34). Also, Table 5.23 indicates that more

NLSY97 households report one or two residents, and fewer households report three or more residents,

than CPS. This is in accordance with the evidence presented earlier, suggesting that households with 

small numbers of youths are less likely to report them.

Table 5.22 Average Household Size

Sample Number of Persons 

CPS, March 1993– Weighted 2.62

CPS, March 1997 – Weighted 2.89

NLSY97 – Weighted 2.34

NLSY97 – Weighted (CX PSUs) 2.34

NLSY97 – Weighted (SU PSUs) 2.41

60
Table 5.23 Distribution of Households by Size of Household 

Number of NLSY97/PAY97-Weighted

Residents CPS Weighted Total CX SU

1 .252 .317 .318 .290

2 .324 .339 .340 .340

3 .169 .146 .145 .158

4 .152 .123 .121 .135

5 .067 .051 .051 .056

 6+ .036 .024 .024 .022

Table 5.24 shows that the NLSY97 screening sample captured fewer youths between the ages of 

12 and 23 than we expected to exist in the population at large. The CPS numbers indicate that youths

between ages 12 and 23 make up between 16.3 and 16.7 percent of total people, while for the NLSY97 

sample they make up only 12.7 percent. Again, the results of the 1993 CPS fall somewhere between

those of the 1997 CPS and the NLSY97/PAY97, except for 18- to 23-year-olds.

Table 5.24 Youths as a Proportion of Total Persons Screened 

Age Range of NLSY97/PAY97-Weighted

Youths

March 1993 

CPS-Weighted

March 1997

CPS-Weighted Total CX SU

12 to 16 .069 .073 .061 .060 .067

18 to 23 .081 .079 .055 .055 .050

12 to 23 .163 .167 .127 .126 .128

Examination of Demographic Balance. Up to this point, we have presented figures and tables

detailing the shortfalls in the NLSY97 counts of youths. Tables 5.25, 5.26, and 5.27 illustrate the 

population proportions in various demographic domains compared to the CPS. The tables demonstrate

that, though there are obvious shortfalls in overall counts, there is little evidence that the sample is not 

demographically balanced. 

61
62

Table 5.25 Race by Gender Percentages for CPS vs. NLSY97-Eligible Sample14

CPS Ages 12-16 NLSY97 Total Eligible NLSY97 CX Eligible NLSY97 SU Eligible 

Gender Gender Gender Gender

Male Female Total Male Female Total Male Female Total Male Female Total 

% w/in race 53.7 46.3 100.0 51.5 48.5 100.0 51.5 48.5 100.0 51.4 48.6 100.0

Hispanic % w/in sex 10.6 9.8 10.2 21.8 21.6 21.7 14.0 14.0 14.0 45.0 43.7 44.4

% w/in race 51.2 48.8 100.0 50.4 49.6 100.0 50.6 49.4 100.0 50.1 49.0 100.0 Non-Hispanic 

Black % w/in sex 12.5 12.8 12.7 25.5 26.5 26.0 15.7 16.3 16.0 54.7 56.1 55.4

% w/in race 51.6 48.4 100.0 51.7 48.3 100.0 51.8 48.2 100.0 -- -- --

Race/

Ethnicity 

Other % w/in sex 76.9 77.4 77.1 52.7 51.9 52.3 70.3 69.6 70.0 -- -- --

% w/in race 51.7 48.3 100.0 51.3 48.7 100.0 51.6 48.4 100.0 50.7 49.3 100.0 Total 

% w/in sex 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0

Table 5.26 Race by Gender Percentages for CPS vs. NLSY97 Interviews 

CPS Ages 12-16 NLSY97 Total Interviews NLSY97 CX Interviews NLSY97 SU Interviews 

Gender Gender Gender Gender

Male Female Total Male Female Total Male Female Total Male Female Total 

% w/in race 53.7 46.3 100.0 51.4 48.6 100.0 51.0 49.0 100.0 51.7 48.3 100.0

Hispanic % w/in sex 10.6 9.8 10.2 21.3 21.1 21.2 13.6 13.7 13.7 44.5 43.1 43.8

% w/in race 51.2 48.8 100.0 50.1 49.9 100.0 49.8 50.2 100.0 50.3 49.7 100.0 Non-Hispanic 

Black % w/in sex 12.5 12.8 12.7 25.4 26.6 26.0 15.6 16.5 16.0 55.4 56.7 56.1

% w/in race 51.6 48.4 100.0 51.6 48.4 100.0 51.6 48.4 100.0 -- -- --

Race/

Ethnicity 

Other

% w/in sex 76.9 77.4 77.1 53.3 52.3 52.8 70.8 69.8 70.3 -- -- --

% w/in race 51.7 48.3 100.0 51.1 48.9 100.0 51.2 48.8 100.0 50.9 49.1 100.0 Total 

% w/in sex 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0

14 The percentages in Tables 5.25, 5.26, and 5.27 are slightly outdated due to small changes in the weighted and unweighted totals for the NLSY97 sample after 

these tables were produced. 
63

Table 5.27 Race by Gender Percentages for CPS vs. NLSY97 Test Completes15

CPS Ages 12-16 NLSY97 Total Tested NLSY97 CX Tested NLSY97 SU Tested 

Gender Gender Gender Gender

Male Female Total Male Female Total Male Female Total Male Female Total 

% w/in race 53.7 46.3 100.0 50.9 49.1 100.0 51.0 49.0 100.0 50.9 49.1 100.0 Hispanic

% w/in sex 10.6 9.8 10.2 19.5 19.3 19.4 11.9 12.0 12.0 44.7 42.3 43.5

% w/in race 51.2 48.8 100.0 48.6 51.4 100.0 48.6 51.4 100.0 48.5 51.5 100.0 Non-Hispanic 

Black % w/in sex 12.5 12.8 12.7 24.4 26.6 25.5 15.2 16.7 15.9 55.2 57.4 56.3

% w/in race 51.6 48.4 100.0 51.5 48.5 100.0 51.5 48.5 100.0 -- -- --

Race/

Ethnicity 

Other

% w/in sex 76.9 77.4 77.1 56.0 54.1 55.1 72.9 71.4 72.2 -- -- --

% w/in race 51.7 48.3 100.0 50.6 49.4 100.0 51.0 49.0 100.0 49.5 50.5 100.0 Total 

% w/in sex 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0

15 NLSY97 Test Completes include those who completed the full ASVAB battery as well as those who completed the AFQT portion of the ASVAB.
Table 5.25 displays comparisons of CPS youths aged 12 to 16 to the screening sample of 

NLSY97-eligible youths, overall and by sample type. Because the SU sample consists only of minority

youths, percentages of youths within gender categories are inflated to compensate for the lack of nonHispanic, nonblack youths. This, in turn, inflates the overall within-gender percentages of NLSY97-

eligible youths; thus, to compare fairly the CPS and the NLSY97, we must concentrate on the CX sample

of NLSY97 eligibles and the within race/ethnicity gender ratios only for the SU sample. Table 5.25 

shows that CPS males make up about 51.7 percent and females 48.3 percent of the overall sample. The

NLSY97 CX sample has an almost identical distribution of males and females overall. By race/ethnicity,

the only difference occurs for Hispanics, and this difference is quite small. The CPS shows that 53.7 

percent of Hispanics in this age group are males, while the NLSY97 Hispanic, CX sample is 51.5 percent 

male. Within race/ethnicity, the supplemental gender ratios are almost identical to the CX sample.

Racial/ethnic percentages within gender show the only noticeable differences. According to the CPS, the 

female population of youths is 9.8 percent Hispanic, 12.8 percent non-Hispanic black, and 77.4 percent 

other races/ethnicities. The NLSY97 has 14.0 percent Hispanics, 16.3 percent non-Hispanic blacks, and 

69.6 percent other races/ethnicities making up the CX sample of females. This suggests that we have a 

surplus of Hispanic and non-Hispanic, black youths and an underrepresentation of non-Hispanic,

nonblack youths within the NLSY97 CX sample.

Similarities between CPS and NLSY97 are also evident in Table 5.26, which displays the same

percentage tabulations for NLSY97 interviews. Again, a comparison of the CPS to the NLSY97 CX 

sample and the within race/ethnicity gender ratios for the SU sample is appropriate. The results for this 

table are almost identical to those for the NLSY97 eligibles. There are very small differences by gender 

within race/ethnicity (the CPS shows that 53.7 percent of Hispanics in this age group are males, while the 

NLSY97 Hispanic, CX sample is 51.0 percent male and the SU sample is 51.7 percent male), and 

somewhat larger racial/ethnic differences within gender categories. For example, the CPS shows that the 

female population of youths is 9.8 percent Hispanic, 12.8 percent non-Hispanic black, and 77.4 percent 

other races/ethnicities, while the NLSY97 interviewed population has 13.7 percent Hispanics, 16.5 

percent non-Hispanic blacks, and 69.8 percent other races/ethnicities making up the CX sample of 

females. Again, this suggests that we have a surplus of Hispanic and non-Hispanic, black youths and an

underrepresentation of non-Hispanic, nonblack youths within the NLSY97 CX sample

Finally, Table 5.27 compares the CPS to proportions of NLSY97 youths who completed an 

ASVAB test. Overall gender and race/ethnicity distributions are almost identical for the CX sample. The 

differences that do occur among subgroups for the CX sample are similar to the NLSY97 eligibles and 

interviews; CPS Hispanics have slightly larger proportions of males than the NLSY97. Further, while 

CPS non-Hispanic, black youths are 51.2 percent male, the NLSY97 non-Hispanic blacks who completed

a test are only 48.6 percent male. Within race/ethnicity gender ratios for the SU sample are comparable to

results for the CX sample. These small differences between CPS and NLSY97 may be driven by

response rates; females tended to participate at a higher rate than males. Racial/ethnic differences within 

gender are smaller than for NLSY97 eligibles and interviews; as stated earlier, the CPS distribution of 

females is 9.8 percent Hispanic, 12.8 percent non-Hispanic black, and 77.4 percent other races/ethnicities. 

The NLSY97 distribution of CX females who completed a test is 12.0 percent Hispanic, 16.7 percent nonHispanic black, and 71.4 percent other races/ethnicities. Again, this may indicate a surplus of minority

youths and an underrepresentation of non-Hispanic, nonblack youths in the NLSY97 CX sample.

The results shown in these tables reassure us that the NLSY97 sample is balanced 

demographically; there is no evidence of a serious differential bias vis-à-vis the CPS. There is no

64
denying that there were shortfalls in overall counts of youths between the ages of 12 and 23; far fewer 

youths were found than we had projected. Despite these shortfalls, though, Tables 5.25, 5.26, and 5.27

present evidence that the sample of 12- to 16-year-old youths is balanced and distributed very similarly,

overall and among various subgroups, to the CPS.

Appendices F and G lend more support to these results. The relationship between PSU-level 

response rates and PSU-level socio-economic status variables (income and education) is examined in

Appendix F. These analyses indicate only small differences in PSU response rates for PSUs of different 

education and income levels. In particular, regression equations suggest a difference of only two percent 

in the PSU response rates of high-income/education PSUs and low-income/education PSUs. These 

relationships could be muted by the aggregation to the PSU level, since the socio-economic status of 

nonrespondents could not be assessed. Appendix G compares family income for NLSY97 youths to the 

income of families with resident youths aged 12 to 16 responding to the March 1997 CPS. The results of 

these comparisons indicate that while there are some differences in family income between NLSY97 and

CPS youths, the differences are fairly small and are most obvious in the race/ethnicity (particularly

Hispanic) breakouts. Over all of these variables, the NLSY97 sample is reasonably well-balanced in 

comparison to the CPS sample.

Examination of NLSY97/PAY97 Coverage Ratios. As stated in Chapter 4, the preliminary

coverage ratios that suggest that “one out of every three NLSY97 eligibles” were missed are very

misleading. The caps used during weighting to prevent extreme weights have a negative effect on the

coverage ratios. But the caps alone do not explain all of the undercoverage of youths observed in 

NLSY97.

All surveys suffer from some level of general undercoverage. The presumed true population is 

the decennial population adjusted for the census undercount, which is the undercoverage in the census

itself. Estimated coverage rates for the 1990 Census are shown in Table 5.28 below. 

65
Table 5.28 Estimated 1990 Census Coverage Ratios (March, 1998)*

Race Sex Age Coverage Ratios

Male 10-19 .9805

20-29 .9091

10-19 .9787

Black

Female

20-29 .9653

Male 10-19 1.0089

20-29 .9830

10-19 1.0053

Non-Black

Female

20-29 .9937

*

These coverage ratios come from Robinson et al. (1993) “Estimation of Population Coverage in the 1990 United States Census 

Based on Demographic Analysis,” Journal of the American Statistical Association, Vol, 88, pp. 1061-1071. 

Coverage ratios are not given for Hispanics, but Census experts often assume they are similar to 

coverage ratios for blacks. Among the age groups of NLSY97/PAY97, this undercoverage is greatest for 

black males aged 20-29 -- over 9% were not counted. For most other groups, the undercoverage is small,

and non-black 10-19 year olds actually show overcoverage (i.e., some counted more than once). 

The Current Population Survey has greater undercoverage than the Decennial Census. Overall,

the CPS covers about 92 percent of the adjusted decennial-census population. However, the CPS has less 

coverage of the age groups of NLSY97/PAY97, as shown in Table 5.29 below. 

66
Table 5.29 Estimated CPS Coverage Ratios (March, 1998) 

Race Sex Age Coverage Ratios

0-14

15

16-19

.916

.905

.855

Male

20-29

30-39

40-49

.823

.877

.917

0-14

15

16-19

.943

.883

.877

All Persons

Female

20-29

30-39

40-49

.884

.920

.959

0-14

15

16-19

.850

.763

.711

Male

20-29

30-39

40-49

.660

.680

.816

0-14

15

16-19

.838

.824

.802

Black

Female

20-29

30-39

40-49

.811

.845

.911

0-14

15

16-19

.929

.933

.881

Male

20-29

30-39

40-49

.847

.904

.928

0-14

15

16-19

.964

.895

.891

Non-Black

Female

20-29

30-39

40-49

.897

.966

Again, coverage ratios are not given separately for Hispanics. The coverage ratios tend to be

lowest among 20- to 29-year-olds and 16- to 19-year-olds. Undercoverage is most severe for black males

aged 20-29 (.660), which indicates that one in three is not represented. Clearly, the CPS has trouble with

some of the age groups studied by NLSY97/PAY97.

.931

67
Before proceeding, we note that CPS undercoverage is defined by comparing preliminary CPS 

estimates of the size of the various demographic domains to the Census Bureau's best demographic

estimates of the sizes of those domains. The final step in CPS weighting actually poststratifies the

preliminary CPS weights to those best estimates of population size. Thus, CPS estimates derived from

final CPS weights exhibit no residual undercoverage relative to the best estimates of population size. We 

observe that the CPS age distribution used as the standard of comparison in Figures 5.5-5.7 was created 

using the CPS final weights, and thus this age distribution essentially equates to the Census Bureau's best 

estimates of population size. 

Turning to the NLSY97/PAY97, one way to measure coverage is to look at the ratios of the sums

of weights before and after poststratification. The NLSY97 ratios are given in Table 4.5. However, the 

ratios in this table are excessively small because of caps used during weighting (to reduce variability), as 

shown in Table 5.18. We believe superior coverage ratios can be derived from the age distribution of

persons screened. We weight up these persons (we used special household weights, see Appendix S) to 

get an NLSY97/PAY97 estimate of persons of that age. Dividing these estimates by CPS controls gives

us the estimated coverage ratios in Table 5.30. 

Table 5.30 Estimated NLSY97/PAY97 Coverage Ratios 

Age Range Total CX SU

0-11 0.90 0.89 0.94

12-16 0.74 0.83

18-23 0.68 0.69 0.64

24-35 1.01 1.01 1.00

0.72

The comparable CPS overall coverage ratios are about .85, so it is clear that NLSY97/PAY97 

suffers from more undercoverage than the CPS. Even the overwhelming and costly Decennial Census

suffers from some undercoverage. Large and ongoing samples such as the CPS, however, suffer from

more undercoverage than the Decennical Census.

It is probably reasonable to conclude that both CPS and NLSY97/PAY97 achieved good 

coverage of persons age 24-35. For the eligible years, ages 12-23, it may be reasonable to conclude that 

both CPS and NLSY97/PAY97 incur a general survey undercoverage of, say, 15 percent. Then,

NLSY97/PAY97 incurs an additional 10 to 15 percent undercoverage attributable to the age-specific 

screening which was done. For younger youths age 0-11, CPS misses about 8 percent and the 

NLSY97/PAY97 screening sample missed that plus an additional 2 to 3 percent. 

Special Studies. We conducted several other special studies to examine the shortfalls in the age 

distribution and uncover possible reasons for these shortfalls. As discussed earlier in this report, studies 

of language barrier cases, proxies, and gatekeepers have been undertaken as part of the investigation. 

Further studies include analyses of crashed cases, noninterview reports, the debriefing of field 

interviewers, and an analysis of validation interviews. 

68
Appendix D details the analysis of language barrier cases. In this appendix, we focused mainly

on comparing actual PAY97 language barrier cases to national norms for English proficiency. Because 

language proficiency is a difficult concept to define and measure, different sources of data arise from

different measurement processes. Thus, the comparability of data sources may be compromised, and 

these sources are subject to a variety of differential biases and random errors. Despite these limitations,

the data are presented and compared in this appendix, and there is no evidence that the PAY97 samples

were excessively saturated with language-barrier youths. In fact, these data suggest that the PAY97 

samples found language-barrier cases at a typical to below typical rate. We found no support for the 

claim that language barrier cases compromise the representativeness of the PAY97 samples to a greater 

extent than they compromise the representativeness of other important, national surveys. We believe 

these results and conclusions carry over to the NLSY97 samples.

Analyses of proxy and gatekeeper interviews are given in Appendices E and L. Appendix E 

summarizes the results of reviewing a statistical sample of the records of calls (ROCs) for cases given a 

field disposition indicating that they were screeners completed by proxy respondents. The main purpose

of this analysis was to ensure that the proper procedures were followed and that there were no indications

of youths in the households due to improper use of this disposition code. Of the 5,175 households given 

this disposition, NORC reviewed 490. Results show that most of the time (92.2 percent), field 

interviewers correctly followed procedures for proxy screeners. In those cases where procedures were not 

followed, only 0.6 percent were households with possible youths present. The analysis of gatekeeper 

interviews in Appendix L found similar results. This report summarizes the results of reviewing a 

statistical sample of ROCs for cases coded as gatekeeper interviews. Again, the purpose of the analysis

was to verify that proper procedures had been followed. NORC reviewed 494 of the 4,055 households 

identified as gatekeepers. Again, the majority of the time (78.5 percent), field interviewers correctly

followed procedures; in the cases when procedures were not followed, only 0.6 percent were households 

with possible youths present. We conclude that gatekeeper interviews may have provided a minor

contribution to the hole in the age distribution, but that they could not be a major source of the hole.

NORC debriefed 96 of the 432 NORC field interviewers who had administered screening

interviews for NLSY97/PAY97, and an analysis of these debriefing interviews is given in Appendix O. 

The debriefing interviews found, among other things, that the household informants may have discovered 

the eligible age range through various sources and then either misreported their youths’ ages or failed to 

report eligible youths at all. For example, the NLSY97 and PAY97 brochures and letters, which were

sent to households prior to screening (see Appendix K), disclosed information about the age range or, at 

least, the study’s focus on youths. These materials also disclosed the amount of time required of

participants, requesting the youth to “spend approximately one hour with the interviewer...” and “take a 

2-hour test on a computer...” This may have discouraged participation, possibly resulting in purposeful 

underreporting of youths or misreporting of resident youths’ ages to avoid participation in the study.

Also, neighbors who had already been screened may have supplied the relevant information to other 

informants, giving them the opportunity to misreport their youths. Finally, the field interviewers 

themselves may have given this information to respondents during the screening process. In fact, 38 of

the 96 interviewers (nearly 40 percent) reported telling informants the ages of youths eligible to 

participate in NLSY97/PAY97. Twenty-two of the interviewers, about 23 percent, said that they

followed the intended script -- presented to them at training sessions -- during the screening interviews

(for examples of the script, see Appendices H and I).

69
The debriefing interviews also examined possible computer problems involved in screening. 

Seventeen of the 96 interviewers conducted only paper and pencil screening interviews, while the 

remaining 79 used the laptop computer for screening. Of the 79, 32 reported experiencing problems with 

the computer, and 26 interviewers reported that computer crashes which prevented the completion of a 

screening interview occurred at least once at a household where they thought there were eligible youths.

However, this analysis concluded that although the interviewers reported problems with the computers,

the frequency of the problems was not high; therefore, computer problems alone could not explain the

missing youths in the sample.

A thorough analysis into the computer crashes that occurred during screening can be found in

Appendix N. Here, reasons for the crashes, including low random access memory (RAM), insufficient 

space on the hard drive, inconsistent battery life, and incorrectly assigned cases, were cited. The four 

PSUs with the lowest response rates were examined, and final dispositions for each case within those 

PSUs were analyzed. The results show that in these four PSUs, the large number of crashes and other 

technical problems, ranging from 22.5 to 64.0 percent of total cases, could have contributed to the

shortfalls in the eligible age range. However, nationwide there were a total of 1,318 crashed cases, 357 of 

which fell into those four PSUs. The remaining 961 average to only 4.9 cases per PSU for the other 196

PSUs. Thus, it is unlikely that, in the remaining PSUs, computer problems were major contributors to the 

shortfalls in the age distribution. 

Appendix M documents the incidence of noninterviews in the NLSY97 screening effort and 

examines the data from the noninterview reports (NIRs). For each of the noninterview households, the 

NORC field interviewer assigned to the case was asked for his or her opinion about the makeup of the

household, including how many people they thought lived in the household, overall and in particular age

categories (10-25 and 17-25). The data collected from these reports indicated that 4,289 people were 

presumed to be living in the noninterviewed households. 1,401 household members were reported to be 

between the ages of 10 and 25, and 570 between the ages of 17 and 25. The subtraction of youths in the 

latter age group from the first gives a rough approximation of the number of youths aged 10 to 16; 

according to this calculation, 831 youths fall into that age range. In other words, 19.4 percent of the 

people in noninterviewed households were reported to be between the ages of 10 and 16, and 13.3 percent 

between 17 and 25 years of age; these rates are two to three times what we would expect in the nation as a 

whole. The NLSY97/PAY97 found that only 6.1 percent of people (in screened households) were

between ages 12 and 16, and only 5.5 percent were between 18 and 23. The March 1997 CPS estimates

are only slightly higher; 7.3 percent of people were aged 12 to 16, and 7.9 percent were aged 18 to 23.

The fact that youths in similar age groups comprised a much higher percentage of the population in 

noninterviewed households than in the screened households or in the overall population provides 

evidence that households with youths were more likely to refuse to be screened, thus contributing to the 

hole in the NLSY97/PAY97 age distribution.

An analysis of validation interviews, conducted in an effort to help explain the undercount of 

youths in the NLSY97/PAY97 eligible age range, is given in Appendix P. The validation process entails

reinterviewing a subsample of the households that were screened in the NLSY97; its main objectives are

for the respondents and NORC managers to evaluate the performance of the interviewers. A total of 

1,296 validation interviews were conducted on respondents who were originally interviewed between 

January and October 1997, including 347 screeners and 949 youth and parent interviews. A crosstabulation of interview dates and validation interview dates showed that the majority of the validation 

70
interviews occurred at least one month after the original interview. In the comments section of the 

interview, 338 respondents remarked on the interviewers’ mannerisms. Of these, 80.5 percent reported

that the interviewers were polite and courteous, 7.4 percent were neutral, and the remaining 12.1 had 

negative comments. The analyses of the validation interviews failed to reveal any poor conduct on the

part of the interviewers, as most of the comments of the respondents were positive and the few negative 

comments pointed to circumstances beyond the control of the interviewers. On the other hand, the field 

staff failed to conduct timely validation interviews, which leave the results somewhat inconclusive. 

Finally, in Appendix T, we compared the NLSY97 age distributions of various groups based on 

whether they live in urban centers, suburbs, other urban areas, or rural areas. The age distributions for all 

groups have a drop in the number of people in the 12-23 age range. The urban age distribution shows a 

gradual drop in percentages of people in the entire age range, while the rural age distribution shows a 

sharper drop for older youths. For the NLSY97, we are primarily interested in the numbers of 12- to 16-

year-olds. This group appears to be undercounted in urban rather than in rural areas. However, we 

cannot attribute the undercount to one particular type of urban area. A comparison of the CPS and 

NLSY97 age distributions indicate that the undercount is larger for older youths in rural areas and small

cities, larger for younger youths in urban centers, and uniform across the 12-23 age range in suburban 

areas. These comparisons should be interpreted cautiously, though; we are using the overall CPS as a

benchmark, since the CPS does not allow for breakdowns by different rural/urban groups. Thus, our 

analysis of rural/urban age distributions does suggest that there are some differences in the shapes of the 

distributions and undercounts of NLSY97 youths for different types of areas. However, there is no 

compelling evidence which would allow us to attribute the undercount to one particular type of 

rural/urban category.

5.4 Estimates of the NLSY97 Shortfall

In previous sections and in various appendices, we have described a shortfall of eligible youths

age 12-16 in the NLSY97 screening sample. In this section, we produce our best estimate of the size of 

the shortfall, defined as the number of eligibles expected minus the number of eligibles found in 

screening operations. While the number of eligibles found is known, the number of eligibles expected 

must be estimated. Expectations, of course, depend upon conditioning variables, if any. More on this

critical factor later. 

We have determined and now present five different methods of estimating the number of eligibles 

expected. In brief, they are as follows: 

Method 1A uses the March 1997 Current Population Survey (CPS) to determine a rate of eligible 

youths per household by race/ethnicity. The SU sample is assumed to find youths at a higher rate 

than the CX sample. The method multiplies the rates by the number of completed screeners to 

estimate the number of eligibles expected. 

Method 1B is identical to Method 1A, except it makes an alternative assumption about the 

increased success of the SU sample in finding youths.

Method 2 employs the 1990 Decennial Census totals of youths age 0-17 by race/ethnicity for 

each NLSY97 segment to calculate an estimate of the number of eligibles expected. 

Method 3A uses the March 1997 CPS to determine a rate of eligible youths per household by

71
age. The SU sample is assumed to find youths at a higher rate than the CX sample. The method

multiplies the rates by the number of completed screeners to estimate the number of eligibles 

expected.

Method 3B is identical to Method 3A, except it makes an alternative assumption about the 

increased success of the SU sample in finding youths.

In what follows, we describe each of these methods in greater detail, and we implement the methods to 

estimate the shortfall.

We also determined and implemented two additional methods (Method 4A and 4B) of estimating

the number of eligibles expected, both of which are based upon the distributions of HHs by number of

youths, as presented in Table 5.19. Because these methods give similar results to the methods presented 

here, we omit discussion of them from this report. 

For the cross-sectional (CX) sample, Method 1A is straightforward. Table 5.31 shows the actual 

field numbers, following the sample all the way from the number of addresses fielded, to the number of 

completed screeners, to the number of eligible youths. It also shows three different estimates of the

shortfall. The first estimate uses the expected sample sizes given in the original sampling plan for the 

NLSY97. The second and third estimates use expectations based upon the March 1993 CPS rates (used

for the original sampling plan) and the March 1997 CPS rates (closest in time to the NLSY97 fieldwork) 

applied to the actual number of screened households. The last column gives the source table for each of 

the actual field numbers.

Unfortunately, Method 1A is much more difficult to apply to the supplemental (SU) sample. The 

main difficulty is that CPS rates are not suitable for the SU sample. The oversampling done in the SU 

sample implies higher youth rates, especially for Hispanic and black youths, than those available from an

essentially self-weighting survey like the CPS.

72
Table 5.31 Analysis of Actual versus Expected Youths in CX Screening Sample: Method 1A 

Expectations

from the 

Original

Sampling

Plan

Expectations

(March 93 

CPS) Applied 

to Actual 

Number of 

Screened HHs

Expectations

(March 97 CPS) 

Applied to 

Actual Number 

of Screened HHs

Actual

Field

Numbers

Source for 

Actual Field 

Numbers

Addresses fielded 64,654 65,269 65,269 65,269 Table 2.2 

Proportion eligible for 

screening

0.85 0.882 0.882 0.882

Addresses eligible for 

screening

54,956 57,562 57,562 57,562 Table 5.1 

Response rate for 

screening

0.91 0.943 0.943 0.943

Completed Screeners 50,010 54,253 54,253 54,253 Table 5.1 

Number of eligible youths per screened household 

Hispanic 0.021 0.021 0.024 0.019 Calculated

Non-Hispanic black 0.028 0.028 0.029 0.022 Calculated

Non-Hispanic nonblack 0.131 0.131 0.137 0.095 Calculated

TOTAL 0.180 0.180 0.190 0.136 Sum

Number of eligible youths 

Hispanic 1,050 1,139 1,302 1,026 Table 3.3 

Non-Hispanic black 1,400 1,519 1,573 1,175 Table 3.3 

Non-Hispanic nonblack 6,551 7,107 7,433 5,134 Table 3.3 

TOTAL 9,001 9,765 10,308 7,335 Sum

Shortfall of eligible youths 

Hispanic 24 113 276

Non-Hispanic black 225 344 398

Non-Hispanic nonblack 1,417 1,973 2,299

TOTAL 1,666 2,430 2,973

We performed simulations during planning stages of the NLSY97 and estimated that the

oversampling would double the percentage of youths that would be Hispanic or black. In the actual

survey, we found the percentages more than doubled. The percentage of 12- to 16-year-olds in the SU 

who were Hispanic was 30.22 percent, which is 2.59 times the national rate of 11.67 percent. Also, the 

percentage of 12- to 16-year-olds in the SU who were non-Hispanic black was 37.80 percent, which is 

2.43 times the national rate of 15.55 percent. Therefore, the oversampling was even more efficient than 

the simulations suggested it would be. However, these simulations did not reflect one layer of 

oversampling that was added during actual sampling operations. This layer was the selection of housing

units in “high” minority segments at ten times the rate of housing units in “low-low” minority segments.

This additional layer could be the reason the SU was even more efficient in finding minority youths than 

the simulations suggested. 

73
Despite the difficulties, we have made estimates for the number of eligible youths per HH by

race/ethnicity using the following assumptions:

1. The CPS total number of eligible youths per household is .180 (1993) or .190 (1997). 

2. We found 27.2 percent more NLSY97 youths per household in the SU (3,638 eligibles in 21,157

completed screeners) than in the CX (7,335 eligibles in 54,253 completed screeners). 

3. Therefore, we assume for Table 5.32 that the total number of eligible youths per household in the SU 

is .180 x 1.272 = .229 for the 1993 CPS expected values and .190 x 1.272 = .242 for the 1997 CPS 

expected values. 

4. The SU NLSY97 sample consisted of 1,102 eligible Hispanics (30.2 percent), 1,371 eligible nonHispanic blacks (37.8 percent), and 1,165 eligible non-Hispanic nonblacks (32.0 percent).

5. Therefore, for the 1993 CPS column, we assume the numbers of eligible youths per household are 

.229 x .302 = .069 (Hispanic), .229 x .378 = .087 (non-Hispanic black), and .229 x .320 = .073 (nonHispanic, nonblack). Similar numbers for the 1997 CPS are .073, .091, and .077. 

Of course, assumption 3 is questionable because it assumes the shortfall occurred at the same rate in the 

CX and SU samples. Assumption four is based upon the premise that the shape of the race/ethnicity

distribution is the same for the undercovered SU cases as for the covered cases. 

Given these assumptions, Table 5.32 shows estimates of the shortfalls of eligible youths in the SU 

sample, using the same format as Table 5.31. One additional column has been added because 20 percent

of the SU sample represented in the original sampling plan was not fielded. The extra column represents 

information based upon the actual number of released addresses. 

Table 5.33 then shows the same information as Tables 5.31 and 5.32 for both samples combined.

The estimated shortfall, including SU non-Hispanic nonblacks, is 4,433 youths. Including only NLSY97

eligibles (excluding SU non-Hispanic nonblacks), the estimated shortfall is 3,969 youths.

74
Table 5.32 Analysis of Actual versus Expected Youths in SU Screening Sample: Method 1A 

Expectations

from the 

Original

Sampling

Plan

Expectations

from the 

Original

Sampling

Plan,

Released

Addresses

Expectations

(March 93 

CPS) Applied 

to Actual 

Number of 

Screened HH

Expectations

(March 97 

CPS) Applied 

to Actual 

Number of 

Screened HH 

Actual

Field

Numbers

Source for 

Actual

Field

Numbers

Addresses fielded 31,855 25,485 25,688 25,688 25,688 Table 2.2

Proportion eligible 

for screening 

0.85 0.85 0.881 0.881 0.881

Addresses eligible 

for screening 

27,077 21,662 22,642 22,642 22,642 Table 5.1

Response rate for 

screening

0.91 0.91 0.934 0.934 0.934

Completed

Screeners

24,640 19,713 21,157 21,157 21,157 Table 5.1

Number of eligible youths per screened household1

Hispanic 0.041 0.041 0.069 0.073 0.052 Calculated

Non-Hispanic black 0.057 0.057 0.087 0.091 0.065 Calculated

Non-Hispanic

nonblack

0.082 0.082 0.073 0.077 0.055 Calculated

TOTAL 0.180 0.180 0.229 0.241 0.172 Sum

Number of eligible youths1

Hispanic 1,010 808 1,460 1,544 1,102 Table 3.3

Non-Hispanic black 1,404 1,124 1,841 1,925 1,371 Table 3.3 

Non-Hispanic

nonblack

2,020 1,616 1,544 1,629 1,165 Screening

data

TOTAL 4,434 3,548 4,845 5,098 3,638 Sum

Shortfall of eligible youths 

Hispanic -92 -294 358 442

Non-Hispanic black 33 -247 470 554

Non-Hispanic nonblack 855 451 379 464

TOTAL 796 -90 1,207 1,460 1

This also includes non-Hispanic nonblack youths screened in the SU sample.

The estimates in Tables 5.32 and 5.33 were based fundamentally on the assumption (derived from

the actual NLSY97 yields) that the SU screening sample would to yield 27.2 percent more youths per 

household than the CX screening sample. An alternative assumption can be derived from 1990 Decennial 

Census data for NLSY97 segments. Using this alternative, Table R-1 shows that we would expect 10,413 

eligible youths from 54,253 CX screeners and 5,552 eligible youths from 21,156 SU screeners. This SU

rate is 36.72 percent larger than the CX rate. Table R-1 also shows that 26.85 percent (1,491/5,552) of 

the SU expected youths are Hispanic, 36.91 percent (2,049/5,552) are non-Hispanic black, and 36.24 

percent (2,012/5,552) are non-Hispanic, nonblack. Combining these numbers leads to larger estimates for 

the rates of eligible youths per household. For the 1993 CPS, we have .180 x 1.3672 x .2685 = .066 

75
(Hispanic), .180 x 1.3672 x .3691 = .091 (non-Hispanic black), and .180 x 1.3672 x .3624 = .089 (nonHispanic, nonblack). Similar estimated rates for the 1997 CPS are .070 (Hispanic), .096 (non-Hispanic 

black), and .094 (non-Hispanic, nonblack). These larger rates, of course, lead to larger estimates of the 

shortfall, called Method 1B, that appear in Table 5.36. 

Table 5.33 Analysis of Actual versus Expected Youths in NLSY97 Screening Sample

(CX + SU): Method 1A 

Expectations from

the Original 

Sampling Plan, 

Released

Addresses

Expectations

(March 93 CPS) 

Applied to the 

Actual Number 

of Screened HH

Expectations

(March 97 

CPS) Applied 

to the Actual 

Number of 

Screened HH 

Actual Field 

Numbers

Addresses fielded 90,139 90,957 90,957 90,957

Proportion eligible for screening 0.85 0.882 0.882 0.882

Addresses eligible for screening 76,618 80,204 80,204 80,204

Response rate for screening 0.91 0.940 0.940 0.940

Completed Screeners 69,723 75,410 75,410 75,410

Number of eligible youths per screened household2

Hispanic 0.027 0.034 0.038 0.028

Non-Hispanic black 0.036 0.045 0.046 0.034

Non-Hispanic nonblack 0.117 0.115 0.120 0.084

TOTAL 0.180 0.194 0.204 0.146

Number of eligible youths2

Hispanic 1,858 2,599 2,846 2,128

Non-Hispanic black 2,524 3,360 3,498 2,546

Non-Hispanic nonblack 8,167 8,651 9,062 6,299

TOTAL 12,549 14,610 10,973

Shortfall of eligible youths 

Hispanic -270 471 718

Non-Hispanic black 814 952

Non-Hispanic nonblack 1,868 2,763

TOTAL 1,576 3,637

1,125

15,406

-22

2,352

4,433

TOTAL ELIGIBLES 3,258 3,969 2

This also includes non-Hispanic nonblack youths screened in the SU sample.

Method 2 is explained in detail in Appendix R. In this appendix, expected counts of youths age 

12-16 were estimated using the actual 1990 Decennial Census counts for the 1,751 NLSY97 segments.

As observed moments ago in our presentation of Method 1B, Table R-1 shows the number of youths age 

12-16 that we expected to screen. Table R-2 shows the actual counts of youths screened, and Table 5.34

below shows the differences between these two tables. Therefore, the estimate of the NLSY97 shortfall, 

including SU non-Hispanic nonblacks, using this method is 4,992 youths. Including only NLSY97

eligibles, the estimate is 4,145 youths. These numbers are very similar to those from Method 1B, which 

we would expect because both methods used relationships between the CX and SU samples observed in 

76
1990 Census data. 

Table 5.34 Estimates of the Shortfall of Youths Aged 12-16 in the NLSY97

Screening Sample: Method 2 

Sample Hispanic

Non-Hispanic,

Black

Non-Hispanic,

Nonblack TOTAL

CX 138 370 2,570 3,078

SU 389 678 847 1,914

TOTAL 527 1,048 3,417 4,992

Method 3A simply uses the age distributions from the CPS and NLSY97, comparing expected

counts using the March 1997 CPS to actual eligible counts from the NLSY97. In calculating expected

counts, we first found the average rate of persons per household by single year of age in the CPS (using 

weighted data). For example, the rate for 12-year-olds per CPS household is 0.0391. Next, the number of 

NLSY97 screened households was multiplied by each of the age specific rates to determine the number of 

people of each age expected to be found within the screened households, according to the CPS rates. This 

method was used for the CX sample, but the additional factor derived for Method 1A of 1.272 was used 

for the SU sample. This factor of 1.272 reflects the fact that 27.2 percent more eligible youths per 

household were found in the SU sample than in the CX sample. The method requires the additional 

assumption that the shape of the CPS age distribution applies to both the CX and SU samples.

These calculations result in a set of expected numbers of persons by single year of age for the CX 

and SU samples. For example, the CX number of screened households (54,253) multiplied by the CPS

rate of 12-year-olds per household (0.0391) yields an expected 2,123 youths age twelve. For the SU 

sample, the expected number of youths age twelve is 21,157 x 1.272 x .03913 = 1,053. Finally, actual

NLSY97 numbers of persons screened were compared to the expected numbers to identify patterns and 

differences in various age ranges. The Method 3A estimates appear in Table 5.35. 

Table 5.35 Expected Youths versus Actual Youths Screened: Method 3A 

Age March

1997

CPS

Rate

Expected

Total

Sample

Expected

CX

Sample

Expected

SU

Sample

Actual

Total

Sample

Actual

CX

Sample

Actual

SU

Sample

Shortfall

Total

Sample

Shortfall

CX

Sample

Shortfall

SU

Sample

0.03913 3,177 2,123 2,139 1,435 1,037 688 349

13 0.03773 3,063 2,047 1,016 2,179 1,504 675 884 543 341

14 0.04028 3,269 2,185 1,084 2,229 1,494 735 1,040 691 349

15 0.03811 3,094 2,068 1,026 2,253 1,498 755 841 570 271

16 0.03860 3,134 2,094 1,040 2,173 1,404 769 961 690 271

Total 0.19385 15,737 10,517 5,219 10,973 7,335 3,638 4,763 3,182 1,581

12 1,053 704

77
We estimate a total shortfall of 4,763 youths using this method. Please note that the totals in

Table 5.35 include all 12- to 16-year-olds, even the non-Hispanic nonblacks in the SU sample who were 

not eligible for the NLSY97. This method does not yield an estimate of the shortfall of eligible youths

because it divides the shortfall by age rather than by race/ethnicity. In theory, we could devise yet

another method of estimating the shortfall via CPS youth rates by age by race/ethnicity. However, time

prevented us from pursuing this approach. 

Recalling Method 1B, we can obtain alternative estimates of expected youths by substituting 

1.3672 for 1.272 as the increase in youths per household associated with the SU sample. This method -- 

which we call Method 3B -- leads to an expected SU sample size of 5,610 (total sample size of 16,127), 

and an estimated SU shortfall of 1,972 (total shortfall of 5,154). This estimate of the total shortfall is very

close to the estimates from Method 1B and Method 2. 

Table 5.36 Five Estimates of the Shortfall of Eligible Youths 

Method 1A Method 1B Method 2 Method 3A Method 3B

SU youth rate increase 27.2% 36.72% 36.72% 27.2% 36.72%

Shortfall of eligible youths, CX 

Hispanic 276 276 138

Non-Hispanic black 398 398 370

2,299 2,299 2,570

CX TOTAL 2,973 2,973 3,078 3,182 3,182

442 379 389

Non-Hispanic black 554 660 678

Non-Hispanic nonblack 464 824 847

1,460 1,863 1,914 1,581 1,972

SU TOTAL ELIGIBLE 996 1,039 1,067

Shortfall of eligible youths,

TOTAL

718 655 527

952 1,058 1,048

2,763 3,123 3,417

4,433 4,836 4,992 4,763 5,154

3,969 4,012 4,145

Non-Hispanic nonblack 

Shortfall of eligible youths, SU 

Hispanic

SU TOTAL 

Hispanic

Non-Hispanic black 

Non-Hispanic nonblack 

TOTAL

TOTAL ELIGIBLE 

Table 5.36 displays estimates for all five methods. All methods suggest a similar level of

shortfall in the CX sample. It seems clear from these data that a critical factor is the assumed increase in 

youths per household applicable to the SU sample. The three methods that employ the 36.72-percent 

assumption all give similar estimates of the total shortfall (including SU non-Hispanic, nonblack youths).

And the two methods, Methods 1A and 3A, that employ the 27.2-percent assumption, also give similar

and somewhat lower estimates of the shortfall. 

78
A reasonable question one might ask is why there is any increase at all. The main reason lies in 

how the SU sample was selected. It was an oversample not just of minorities (Hispanics and nonHispanic blacks), but of youths. The selection probabilities for PSUs and segments were proportional to

their numbers of minority youths (according to the 1990 Census). Therefore, we expect the SU to be 

denser not just in minorities, but in youths.

In our opinion, however, the 36.72 percent higher rate, as shown by 1990 Census data, is likely to 

have been smaller by the time of NLSY97 fielding due to the regression effect. Mobility in the

population will change minority youth densities positively in some areas and negatively in others, but in 

the specific NLSY97 segments chosen because of their high minority youth density, the effect is likely to 

be negative. Therefore, the actual 1997 SU increase in youths per household above the CX rate is 

unknown, but is likely to be less than 36.72 percent. This implies that any estimate of shortfall using the 

36.72-percent assumption will be conservative (in the sense of tending to be an overestimate of the 

shortfall).

In theory, Methods 1A and 3A should probably agree more closely than they do. The observed 

differences between them arise almost entirely from differences between the 1997 CPS youth rates 

employed in Tables 5.31 (0.190) and 5.35 (0.19385). The youth rates should agree but do not because

they were produced by different analysts working at different points in time, using slightly different 

assumptions. The difference between the youth rates is quite small, but when multiplied by the 21,157 

HHs in the SU sample, even a small difference in rates implies an absolute difference of over 300 youths.

Methods 1B, 2, and 3B also have another undesirable property, namely that the assumed

race/ethnicity distribution used in calculating youth rates for the SU sample is borrowed from 1990

Census data. Thus, the estimates ignore the tremendous growth in the Hispanic population between 1990 

and 1997. 

Another reasonable estimate of the increase is 27.2 percent, which is the observed NLSY97 

increase. This estimate arises under the assumption that the shortfall affects the CX and SU samples

equally, an assumption that we cannot conclusively verify nor exclude on the basis of our data.

On balance, considering the regression effect and other evidence presented in this overall report,

we prefer the factor 1.272 rather than 1.3672, and thus we prefer Methods 1A and 3A.

We believe Method 1A is as reasonable as Method 3A, and it has the added virtue that it yields

estimates of eligible youths. Thus, we eliminate Method 3A from further discussion. 

If the rate of shortfall in the SU sample is smaller than in the CX sample, then even Method 1A

would tend towards an overestimate of the shortfall, and vice versa. Normally, in census and survey work 

we expect higher undercount rates for Hispanics and blacks than for non-Hispanic nonblacks. If this 

effect applies to the NLSY97, then we would be forced to conclude that the SU shortfall rate is higher 

than the CX rate, and thus that Method 1A underestimates the shortfall. On the other hand, in this report 

we have reviewed evidence that suggests higher undercount rates for non-Hispanic nonblacks and 

generally for affluent domains. If this effect is, in fact, real, then we would conclude that the SU shortfall 

rate is the smaller rate and that Method 1A overstates the shortfall. On balance, we would characterize 

this evidence as somewhat weak, however, and given the evidence before us, we are reluctant to deviate

far from the 1.272 assumption.

79
Thus, in our opinion, Method 1A supplies the preferred estimates of the shortfall in the NLSY97. 

After rounding, we estimate the screening samples missed a total of 4,400 youths age 12-16, and a total of

4,000 eligible youths, relative to the hypothetical true population. Dividing estimated shortfalls for total

youths by expected total youths gives the following shortfall rates: 

TOTAL 28.7 percent

Hispanic 25.2 percent

Non-Hispanic, black 27.2 percent 

Non-Hispanic, nonblack 30.5 percent. 

These rates are apparently quite consistent with the coverage ratios presented in Table 5.30 – both suggest

the NLSY97 coverage was less than 75 percent. 

Finally, at the opening of this Section 5.4, we mentioned but postponed an examination of the 

effect of conditioning on estimated shortfalls. Let us now address this important issue. The estimated

shortfalls studied here using Methods 1A, 1B, 2, 3A, and 3B are conditional on (1) the sizes of the 

screening samples and (2) the assumption that the screening samples comprise representative probability

samples of the population of in-scope American HHs. The first conditioning factor is obvious, because

all of the expected numbers of youths were built up from various youth rates applied to the actual number

of HHs screened. The second conditioning factor is required because the expected numbers of youths

were built up from CPS or Census youth rates, which we take to be representative of the population of inscope American HHs. Therefore, let us pose the following hypothetical question: 

Suppose new, representative screening samples were fielded, according to the NLSY97

design, resulting in 54,253 and 21,157 completed screening interviews in the new CX and SU 

samples, respectively. Further, suppose the screener response mechanism in these new 

samples is fully ignorable (i.e., response is unrelated to the presence of youths in the HH). 

How many youths age 12-16 would the new screening samples expect to find, and what is the 

shortfall of youths in the actual NLSY97 screening samples vis-à-vis the new, ideal screening 

samples?

The methods and estimated shortfalls studied here provide appropriate answers to this question. For later

use, let y denote the count of youths in the actual sample and let E^ ` Yn fignorable | , denote the expected 

sample size as defined in this hypothetical question. 

On the other hand, through our work in Sections 5.1-5.3 and various appendices, especially

Appendix M, we strongly believe the actual NLSY97 screening samples encountered a nonignorable 

response mechanism (i.e., response is related to the presence of youths in the HH). Our belief is that HHs 

with no eligible youths responded at a higher rate than HHs with youths, and HHs with youths had a 

lower propensity to respond to the screening interviews. Therefore, let us pose a second hypothetical

question:

Given the actual numbers of screening interviews (54,253 and 21,157) and the type of 

nonignorable response mechanism we suspect applies to the actual NLSY97 screening 

samples, how many eligible youths were expected and what is the shortfall of the actual

youths screened vis-à-vis this expectation?

80
This question seems highly relevant to us, yet the methods and estimated shortfalls studied here do not 

provide an answer. In fact, we have no solid information on which to formulate a quantitative answer to

this question.

Had we been able to estimate the shortfall given the screening sample sizes and the nonignorable 

response mechanism, we would be in the enviable position of being able to partition the shortfall into 

components due to the response mechanism and to misreporting. Let E^ ` Yn f nonignorable | , denote the 

expected sample size as defined by this second hypothetical question. Then, the shortfall can be 

decomposed as follows: 

^ `

^ `^ `^ `

Total shortfall E

EE E

Shortfall due to response mechanism Shortfall due to misreporting.

 

   

 

Yn f y

Yn f Yn f Yn f y

ignorable

ignorable nonignorable nonignorable

| ,

|, |, |,

As it stands, this decomposition is unobservable given the evidence we have available to us. 

5.5 Estimates of Variance

Two types of errors -- sampling and nonsampling--affect sample survey statistics. The standard

errors calculated for the NLSY97 reflect, for the most part, the magnitude of the sampling error. They

also take into account some of the effects of nonsampling error related to response and enumeration, but 

they do not account for any systematic biases in the data. 

Nonsampling error in surveys can be attributed to a variety of sources, including inability to

obtain information about all persons in the sample, differences in question interpretation, inability or 

unwillingness of respondents to provide accurate information, inability to recall information, processing 

errors, imputation errors, and undercoverage. Several studies were undertaken to investigate the existence 

and prevalence of nonsampling error. Undercoverage was discussed extensively throughout this chapter, 

and several special studies were discussed in Section 5.3 and are described in detail in Appedices D, E, L, 

M, N, O, and P. 

Sampling error is the name given to the between-sample variation in sample-based estimates.

These differences occur by chance, and the variability is measured by the standard errors of the 

estimates. Sample estimators from a given survey design are unbiased when an average of the estimates

from all possible samples would yield the true population value. In this case, the sample estimate and its 

estimated standard error can be used to construct approximate confidence intervals, or ranges of values,

that include the true population value with known probabilities. Specifically, approximately 95 percent of 

the intervals from two standard errors below the estimate to two standard errors above would include the

true population parameter.

81
The impact of departures from simple random sampling on the precision of sample estimates is

often measured by the design effect (DEFF). The design effect is defined as the ratio of the variance 

corrected for the sampling design to the variance that would be obtained given a hypothetical simple

random sample. Most complex, multi-stage designs result in a design effect greater than one; in other 

words, the variance of an estimator is greater than the variance that would be obtained had the data been

based on a simple random sample.

To estimate the variance using information about the sample design, it is necessary to use

statistical procedures such as Taylor series approximations, Balanced Half Samples, or Jackknife. For the 

NLSY97, NORC used the Taylor series procedure to calculate standard errors. Finally, the square root of

the design effect, referred to as DEFT, is another useful measure provided for multiplication by simple

random sample standard errors to obtain design-corrected standard errors. 

Standard errors, design effects, and root design effects were calculated for six domains in the 

NLSY97 sample, including the full sample, males, females, Hispanics, non-Hispanic blacks, and other 

races/ethnicities, and for a total of 25 binary and continuous survey variables. They can be found in

Tables 5.37 through 5.48. For the binary variables (odd-numbered tables), the sample size estimate (in

percent), design and simple random sample standard errors (in percent), and design effect and root design

effect are displayed. For the continuous variables (even-numbered tables), the total, mean, design and

simple random sample standard errors of the mean, and design effect and root design effect are given. 

Summaries of the design effect and root design effect are also given for each set of variables in each 

domain, including the mean, standard deviation, minimum, median, and maximum.

A final table, Table 5.49, summaries the root design effects among the six domains two types of 

variables. As a rough rule of thumb, we recommend that NLSY97 analysts divide the statistical test 

statistics, derived under simple random sampling assumptions, by the median DEFT (or by its square in 

the case of tests) prior to consulting the reference distributions of the tests and to determining the pvalues. This practice will approximately correct the test statistics for the fact that the NLSY97 is not 

based on a simple random sample, but instead is based upon correlated observations.16

F2

16 For more details, see Skinner, C.J., Holt, D., and Smith, T.M.F. (1989), Analysis of Complex Surveys, John Wiley

& Sons, New York. 

or

Lehtonen, R. and Pahkinen, E.J. (1994), Practical Methods for Design and Analysis of Complex Surveys, John 

Wiley & Sons, New York. 

82
Table 5.37 Variance Estimates for Overall NLSY97 Sample: Proportions 

Proportions (Binary Variables) Sample

Size

Estimate (in 

Percent)

Design SE 

(in Percent) 

SRS SE (in 

Percent) DEFF DEFT

8,983 97.45 0.21 0.17 1.11

R in labor force? 8,984 17.49 0.57 0.40 1.42 1.19

R employed? 8,984 17.17 0.55 0.40 1.38 1.17

R working >= 10 hrs/week? (alternative 1)17 8,837 9.15 0.38 1.22 0.31 1.11

R working >= 10 hrs/week? (alternative 2)18 8,837 11.99 0.45 0.34 1.32 1.15

R employed during school year? 8,984 10.58 0.47 0.32 1.47 1.21

R ever worked during school year? 8,984 19.77 0.58 0.42 1.38 1.18

R employed during summer? 8,984 12.20 0.46 0.34 1.35 1.16

R ever worked during summer? 8,984 15.99 0.53 0.39 1.36 1.16

R ever retained a grade? 7,708 15.01 0.70 0.41 1.71 1.31

R ever skipped a grade? 7,718 1.84 0.19 0.15 1.12 1.27

3,872 38.93 1.15 0.78 1.47 1.21

R have a computer at home? 5,410 0.67 1.33 58.13 1.19 1.78

0.48 1.10

7,936 88.63 0.36 2.53 1.59

R’s parent has worked at least 3 months since R 

born?

7,927 95.00 0.53 0.24 2.21 1.49

DEFF/DEFT Summary

Mean 1.52 1.22

Standard Deviation 0.37 0.14

Minimum 1.20 1.10

Median 1.18

Maximum 2.53 1.59

R enrolled? 1.24

R ever in remedial classes?

R have a quiet place to study? 5,408 90.14 0.40 1.20 

R’s parent born in US? 0.91

1.38

17 For this variable, all youths who reported working at job(s) more than 10 hours per week were coded as 1; youths reporting working less than 

10 hours per week were coded as 0; and youths reporting “hours vary” were coded as 0. An alternative to this coding scheme is given in the 

following variable and described in the footnote below.

18 For this variable, all youths who reported working at job(s) more than 10 hours per week were coded as 1; youths reporting working less than 

10 hours per week were coded as 0; and youths reporting “hours vary” were coded as 1.

83
Table 5.38 Variance Estimates for Overall NLSY97 Sample: Means 

Continuous Variables Sample Size

Estimated

Mean Design SE SRS SE DEFF DEFT

Number of weeks worked in 1996? 8.11 0.20 1.50

Days/wk have dinner with family? 5,356 5.19 0.04 1.15

Days/wk have fun with family? 5,356 2.67 0.03 0.03 1.00

Hours/week on homework? 4,723 5.99 0.10 0.07 1.20

Hours/week watching TV? 5,382 17.26 0.25 0.17 1.21

Hours/week reading for pleasure? 5,382 2.92 0.08 0.06 1.15

Parent’s income 5,211 22,474.78 443.48 261.88 1.69 1.30

Spouse’s income 3,797 37,698.98 991.44 599.69 1.65 1.28

Percent chance of R getting 4-year degree by age 

30

3,004 69.73 0.75 0.59 1.13

DEFF/DEFT Summary

Mean 1.18

Standard Deviation 

Minimum 1.00 1.00

Median 1.20

Maximum 1.30

5,398 0.30 1.22

0.03 1.33

1.00

1.43

1.47

1.33

1.27

1.41

0.21 0.09

1.43

1.69

84
Table 5.39 Variance Estimates for NLSY97 Males: Proportions 

Binary Variables Sample Size

Estimate

(in Percent) 

Design SE 

(in Percent) (in Percent) DEFF DEFT

R enrolled? 4,598 0.28 0.23 1.10

R in labor force? 4,599 18.59 0.70 0.57 1.23 1.11

R employed? 4,599 18.22 0.68 0.57 1.19 1.09

R working >= 10 hrs/week? (alternative 1) 4,517 9.37 0.50 0.43 1.16 1.08

R working >= 10 hrs/week? (alternative 2) 4,517 12.49 0.60 0.49 1.22 1.11

R employed during school year? 4,599 11.47 0.47 1.23 1.11

R ever worked during school year? 4,599 21.80 0.76 0.61 1.24 1.12

R employed during summer? 4,599 13.40 0.59 0.50 1.18 1.09

R ever worked during summer? 4,599 17.76 0.65 1.16 1.08

R ever retained a grade? 18.11 0.92 0.61 1.51 1.23

R ever skipped a grade? 3,957 1.58 0.23 0.20 1.15 1.07

R ever in remedial classes? 1,907 39.75 1.41 1.12 1.26 1.12

R have a computer at home? 2,791 57.97 1.39 0.93 1.22

R have a quiet place to study? 2,791 90.68 0.61 1.11 1.05

R’s parent born in US? 4,080 88.34 1.06 1.46

R’s parent has worked at least 3 months since R 

born?

4,074 0.58 0.34 1.70 1.31

DEFF/DEFT Summary

Mean 1.32 1.15

Standard Deviation 0.26 0.11

Minimum 1.11 1.05

Median 1.22

Maximum 2.12 1.46

SRS SE

97.40 1.22

0.58

0.56

3,950

1.49

0.55

0.50 2.12

94.97

1.11

85
Table 5.40 Variance Estimates for NLSY97 Males: Means 

Continuous Variables Sample Size

Estimated

Mean Design SE SRS SE DEFF DEFT

2,750 9.07 0.38 0.30 1.27

Days/wk have dinner with family? 2,768 5.35 0.05 0.04 1.25 1.12

Days/wk have fun with family? 2,769 2.71 0.04 1.25 1.12

Hours/week on homework? 2,407 5.36 0.12 0.09 1.33 1.15

Hours/week watching TV? 2,777 18.31 0.31 0.24 1.29 1.14

Hours/week reading for pleasure? 2,780 2.28 0.10 0.08 1.25 1.12

Parent’s income 2,659 22,633.18 481.41 365.71 1.32 1.15

Spouse’s income 2,003 37,146.39 1,011.58 743.51 1.17

Percent chance of R getting 4-year degree by age 

30

1,519 66.23 0.87 1.17 1.08

DEFF/DEFT Summary

Mean 1.28 1.13

Standard Deviation 0.06 0.02

Minimum 1.17 1.08

Median 1.27 1.12

Maximum 1.36 1.17

Number of weeks worked in 1996? 1.12

0.05

1.36

1.02

86
Table 5.41 Variance Estimates for NLSY97 Females: Proportions 

Binary Variables 

Sample

Size

Estimate

(in Percent) 

Design SE 

(in Percent) 

SRS SE 

(in Percent) DEFF DEFT

R enrolled? 0.28 0.24 1.17 1.08

16.33 0.76 0.56 1.36 1.16

4,385 16.07 0.75 0.55 1.36

R working >= 10 hrs/week? (alternative 1) 4,320 8.92 0.53 0.43 1.23

R working >= 10 hrs/week? (alternative 2) 4,320 0.61 0.48 1.27 1.13

R employed during school year? 4,385 9.64 0.59 0.44 1.34 1.16

R ever worked during school year? 4,385 17.63 0.76 1.31 1.14

4,385 10.95 0.57 0.47 1.21 1.10

R ever worked during summer? 4,385 14.12 0.65 0.52 1.25 1.12

R ever retained a grade? 3,758 11.75 0.52 1.40 1.18

R ever skipped a grade? 3,761 2.11 0.27 0.23 1.17 1.08

R ever in remedial classes? 1,965 38.11 1.40 1.10 1.27 1.13

R have a computer at home? 2,619 58.31 1.37 0.96 1.43 1.19

R have a quiet place to study? 2,617 89.56 0.70 1.17 1.08

R’s parent born in US? 3,856 88.93 0.92 0.50 1.84 1.36

R’s parent has worked at least 3 months since R 

born?

3,853 95.02 0.57 0.35 1.63 1.28

DEFF/DEFT Summary

Mean 1.15

Standard Deviation 0.18 0.08

Minimum 1.08

Median 1.29 1.14

Maximum 1.84 1.36

4,385 97.50

R in labor force? 4,385

R employed? 1.17

1.11

11.47

0.58

R employed during summer?

0.73

0.60

1.34

1.17

87
Table 5.42 Variance Estimates for NLSY97 Females: Means 

Continuous Variables Sample Size

Estimated

Design SE DEFF DEFT

Number of weeks worked in 1996? 2,648 7.09 0.36 0.28 1.28 1.13

Days/wk have dinner with family? 2,588 5.03 0.05 1.25

Days/wk have fun with family? 2,587 2.63 0.04 0.04 1.00 1.00

Hours/week on homework? 2,316 6.63 0.14 0.11 1.27 1.13

Hours/week watching TV? 2,605 16.17 0.30 0.24 1.25 1.12

Hours/week reading for pleasure? 2,602 3.58 0.10 1.05

Parent’s income 2,552 22,309.21 574.34 375.22 1.53 1.24

Spouse’s income 1,794 38,307.95 1,403.80 959.07 1.46 1.21

Percent chance of R getting 4-year degree by age 

30

1,485 73.42 0.96 0.79 1.22 1.10

DEFF/DEFT Summary

Mean 1.26 1.12

Standard Deviation 0.07

Minimum 1.00 1.00

Median 1.12

Maximum 1.53 1.24

Mean SRS SE 

0.04 1.12

0.11 1.10

0.16

1.25

88
Table 5.43 Variance Estimates for NLSY97 Hispanics: Proportions 

Binary Variables 

Sample

Size

Estimate

(in Percent) 

SRS SE 

(in Percent) DEFF DEFT

R enrolled? 1,898 95.68 0.68 0.47 1.20

R in labor force? 1,898 1.00 0.75 1.33 1.15

R employed? 1,898 11.96 1.01 0.74 1.36 1.17

1,882 6.89 0.75 0.58 1.29 1.14

R working >= 10 hrs/week? (alternative 2) 1,882 8.91 0.86 0.66 1.30 1.14

R employed during school year? 1,898 6.87 0.73 0.58 1.26 1.12

R ever worked during school year? 1,898 14.46 1.05 0.81 1.30 1.14

R employed during summer? 0.87 0.62 1.40 1.18

R ever worked during summer? 1,898 11.01 1.04 0.72 1.44 1.20

R ever retained a grade? 1,509 17.72 1.33 0.98 1.16

R ever skipped a grade? 1,512 1.77 0.41 0.34 1.20 1.10

R ever in remedial classes? 741 36.00 1.76 1.32 1.15

R have a computer at home? 1,152 37.76 2.10 1.43 1.47 1.21

R have a quiet place to study? 1,152 87.76 1.26 0.96 1.31

R’s parent born in US? 1,591 51.66 3.29 1.25 2.63 1.62

R’s parent has worked at least 3 months since R 

born?

1,586 86.82 1.50 0.85 1.76 1.33

DEFF/DEFT Summary

Mean 1.45 1.20

Standard Deviation 0.34 0.12

Minimum 1.20

Median 1.34 1.16

Maximum 1.62

(in Percent) 

Design SE 

1.45

12.33

R working >= 10 hrs/week? (alternative 1)

1,898 7.95

1.36

2.33

1.14

1.10

2.63

89
Table 5.44 Variance Estimates for NLSY97 Hispanics: Means 

Continuous Variables Sample Size

Estimated

Mean Design SE DEFF DEFT

Number of weeks worked in 1996? 1,140 4.79 0.49 0.34 1.44 1.20

Days/wk have dinner with family? 1,149 5.01 1.20

Days/wk have fun with family? 1,146 2.64 0.08 0.06 1.33 1.15

Hours/week on homework? 965 6.63 0.21 0.17 1.24 1.11

Hours/week watching TV? 1,143 18.55 0.57 0.40 1.42 1.19

Hours/week reading for pleasure? 1,143 2.83 0.18 1.28 1.13

Parent’s income 936 18,629.97 920.59 511.86 1.80 1.34

Spouse’s income 693 25,313.38 1,093.97 750.26 1.46 1.21

Percent chance of R getting 4-year degree by age 

30

573 66.01 1.54 1.34 1.15 1.07

DEFF/DEFT Summary

Mean 1.39 1.18

Standard Deviation 0.18 0.08

Minimum 1.15 1.07

Median 1.19

Maximum 1.80 1.34

SRS SE 

0.10 0.07 1.43

0.14

1.42

90
Table 5.45 Variance Estimates for NLSY97 Non-Hispanic Blacks: Proportions

Binary Variables 

Sample

Size

Estimate

(in Percent) (in Percent) 

SRS SE 

(in Percent) 

DEFF

R enrolled? 2,334 98.06 0.30 1.07 1.04

R in labor force? 2,334 10.40 0.83 0.63 1.32 1.15

R employed? 2,334 10.06 0.81 1.14

R working >= 10 hrs/week? (alternative 1) 2,318 6.13 0.61 0.50 1.22 1.10

R working >= 10 hrs/week? (alternative 2) 2,318 7.69 0.66 1.20 1.10

R employed during school year? 2,334 0.61 0.45 1.36 1.16

R ever worked during school year? 2,334 12.44 0.93 0.68 1.37 1.17

R employed during summer? 7.04 0.61 1.15 1.07

R ever worked during summer? 10.04 1.31 1.14

R ever retained a grade? 1,997 27.22 1.90 1.00 1.90 1.38

2,005 2.85 0.49 0.37 1.32 1.15

R ever in remedial classes? 35.00 2.17 1.53 1.42 1.19

R have a computer at home? 36.01 1.95 1.29 1.51 1.23

R have a quiet place to study? 1,381 87.20 1.07 1.09

R’s parent born in US? 2,060 93.14 1.29 0.56 2.30 1.52

R’s parent has worked at least 3 months since R 

born?

2,057 1.27 0.63 2.02 1.42

DEFF/DEFT Summary

Mean 1.44 1.19

Standard Deviation 0.34 0.13

Minimum 1.07 1.04

Median 1.32 1.15

Maximum 2.30 1.52

Design SE DEFT

0.28

0.62 1.31

0.55

5.05

2,334 0.53

2,334 0.81 0.62

R ever skipped a grade?

974

1,382

0.90 1.19

91.13

91
Table 5.46 Variance Estimates for NLSY97 Non-Hispanic Blacks: Means 

Continuous Variables Sample Size

Estimated

Mean SRS SE 

Number of weeks worked in 1996? 4.08 0.33 0.27 1.22 1.10

Days/wk have dinner with family? 1,355 4.57 0.09 0.07 1.28

Days/wk have fun with family? 1,361 2.79 0.08 0.06 1.33 1.15

Hours/week on homework? 1,179 5.81 0.18 0.15 1.20 1.10

Hours/week watching TV? 1,378 23.15 0.53 0.40 1.32 1.15

Hours/week reading for pleasure? 1,379 2.92 0.14 0.13 1.08 1.04

Parent’s income 1,227 20,082.21 868.83 438.59 1.98 1.41

Spouse’s income 583 26,160.00 1,184.22 714.40 1.66 1.29

Percent chance of R getting 4-year degree by age 

30

790 69.76 1.50 1.17 1.28 1.13

DEFF/DEFT Summary

Mean 1.37 1.17

Standard Deviation 0.28 0.11

Minimum 1.08 1.04

Median 1.28 1.13

Maximum 1.98 1.41

Design SE DEFF DEFT

1,428

1.13

92
Table 5.47 Variance Estimates for NLSY97 Non-Hispanic Nonblacks: Proportions 

Binary Variables 

Sample

Size

Estimate

(in Percent) 

Design SE 

(in Percent) 

SRS SE 

(in Percent) DEFF DEFT

R enrolled? 4,731 97.64 0.26 0.22 1.18 1.09

R in labor force? 4,732 19.90 0.66 0.58 1.14 1.07

R employed? 4,732 19.60 0.65 0.58 1.12 1.06

R working >= 10 hrs/week? (alternative 1) 4,617 10.22 0.48 0.44 1.09 1.04

R working >= 10 hrs/week? (alternative 2) 4,617 13.46 0.54 0.50 1.08 1.04

R employed during school year? 4,732 12.44 0.59 0.48 1.23 1.11

R ever worked during school year? 4,732 22.34 0.72 0.60 1.20 1.10

R employed during summer? 4,732 14.11 0.56 0.51 1.10 1.05

R ever worked during summer? 4,732 18.20 0.64 0.56 1.14 1.07

R ever retained a grade? 4,195 12.06 0.66 0.50 1.32 1.15

R ever skipped a grade? 4,194 1.63 0.22 0.20 1.10 1.05

R ever in remedial classes? 2,155 40.21 1.43 1.06 1.35 1.16

R have a computer at home? 2,863 66.55 1.36 0.88 1.54 1.24

R have a quiet place to study? 2,862 91.15 0.56 0.53 1.06 1.03

R’s parent born in US? 4,276 94.00 0.59 0.36 1.64 1.28

R’s parent has worked at least 3 months since R 

born?

4,275 97.19 0.46 0.25 1.84 1.36

DEFF/DEFT Summary

Mean 1.26 1.12

Standard Deviation 0.23 0.10

Minimum 1.06 1.03

Median 1.16 1.08

Maximum 1.84 1.36

93
Table 5.48 Variance Estimates for NLSY97 Non-Hispanic Nonblacks: Means 

Continuous Variables 

Sample Size Estimated

Mean Design SE SRS SE DEFF DEFT

Number of weeks worked in 1996? 2,817 9.60 0.36 0.31 1.16 1.08

Days/wk have dinner with family? 2,845 5.36 0.04 0.04 1.00 1.00

Days/wk have fun with family? 2,842 2.65 0.04 0.04 1.00 1.00

Hours/week on homework? 2,569 5.92 0.12 0.10 1.20 1.10

Hours/week watching TV? 2,848 15.75 0.24 0.21 1.14

Hours/week reading for pleasure? 2,847 2.93 0.11 0.09 1.22 1.10

Parent’s income 3,046 23,463.63 518.66 358.46 1.45 1.20

Spouse’s income 2,516 40,481.38 1,153.45 788.36 1.46 1.21

Percent chance of R getting 4-year degree by age 

30

1,638 70.35 0.92 0.80 1.15 1.07

DEFF/DEFT Summary

Mean 1.20 1.09

Standard Deviation 0.16 0.07

Minimum 1.00 1.00

Median 1.16 1.08

Maximum 1.46 1.21

1.07

As shown in the tables above, the variables with the highest design effects throughout the 

domains are some of the parent variables, including parent’s and spouse’s income, whether the parent was 

born in the U.S., and whether the parent has worked for three or more months since the respondent was 

born. Furthermore, for a few variables, means and standard error estimates were based on small sample

sizes. Variables which had sample sizes that were less than one-half of the total NLSY97 sample size 

(n=8,984) were whether the respondent has taken remedial classes, spouse’s income, and the percent 

chance, according to the parent, that the respondent will earn a four-year college degree by the age of 30. 

Finally, Table 5.49 gives a summary of the mean, minimum, and maximum design effects and 

root design effects for the six NLSY97 domains. As is typical, the largest means belong to the overall

category. Among the subgroups, the Hispanic and non-Hispanic black means are higher than those of the 

other races/ethnicities. Finally, the design effects for males and females are similar.

94
Table 5.49 Comparison of Summary Statistics of NLSY97 DEFTs 

Domain and Types of Variable 

Mean

DEFT

Minimum

DEFT

Median

DEFT

Maximum

DEFT

Total

 Binary 1.22 1.10 1.18 1.59

 Continuous 1.18 1.00 1.20 1.30

Male

 Binary 1.15 1.05 1.11 1.46

 Continuous 1.13 1.08 1.12 1.17

Female

 Binary 1.15 1.08 1.14 1.36

 Continuous 1.12 1.00 1.12 1.24

Hispanic

1.62

 Continuous 1.18 1.07 1.19 1.34

Non-Hispanic Black 

 Binary 1.19 1.04 1.15 1.52

 Continuous 1.17 1.04 1.13 1.41

Non-Hispanic Nonblack

1.12 1.03 1.08

 Continuous 1.09 1.00 1.08 1.21

 Binary 1.20 1.10 1.16

 Binary 1.36

95
Chapter 6. Summary of the Accuracy and Representativeness of the NLSY97 Samples

On the basis of all of the foregoing analyses, we find as follows: 

1. The sampling of PSUs and segments was performed in operationally correct fashion, as planned. 

2. NORC’s listing operation was apparently successful, because the weighted counts of housing units 

listed compare favorably to the Census Bureau’s contemporaneous housing unit counts. 

9. For the NLSY97, in comparison to the CPS, we find no major imbalance by sex or by race/ethnicity.

In both the CX and SU samples, there appears to be a relative deficit of Hispanic males. For nonHispanic blacks and for non-Hispanic nonblacks, the NLSY97 sex ratios match the CPS closely.

Within the CX sample, there appears to be a relative surplus of Hispanic and non-Hispanic, black

youths and a relative deficit of non-Hispanic, nonblack youths. This pattern, if true, is particularly

interesting because it is at odds with the well-known pattern of undercount in the U.S. Decennial 

Censuses.

3. The sampling of housing units was performed in operationally correct fashion, as planned. 

4. The classification done by NORC field interviewers of housing units as occupied or vacant was 

apparently successful, because our vacancy rate compares favorably to the Census Bureau’s

contemporaneous vacancy rates. 

5. NORC’s screening of occupied housing units was successful by one standard, achieving over 94 

percent response as compared to the 91 percent response rate assumed at the planning stages of

NLSY97. The screener response rate varies from PSU to PSU as one might expect, but four PSUs 

ran into computer, and perhaps other, problems and suffered unacceptably low response rates. 

6. At the planning stages of NLSY97, we made calculations of the expected number of youth

participants based upon data from the March 1993 Current Population Survey (CPS) and rates from

NLSY79 and other comparable studies. Our calculations were performed in operationally correct 

fashion such that, absent undercoverage and differential nonresponse patterns, the expected counts

should reasonably have been realized in the NLSY97 fielding. 

7. Nevertheless, the screening interviews failed to find as many eligible youths as expected, but 

succeeded in rostering older and younger persons outside the eligible age range. NLSY97 evidently

experienced an undercount specific to the eligible age range. 

8. Despite the shortfall of eligible youths, both the screening sample and the samples of NLSY97 

participants are reasonably balanced demographically and substantively, as determined by

comparison of the samples to the CPS.

10. We do not find a differential undercount by age within the eligible range of 12-16 years. Older

youths age 18-23 found in the screening samples and used in the PAY97 testing program may have

been missed at a higher rate than younger youths age 12-16. However, for the NLSY97, we find no 

major imbalance by age. 

11. There is no evidence of large substantive bias in the NLSY97 samples, as documented in Appendix F

(PSU-Level Analyses of Socio-Economic-Status Attainment), Appendix G (Comparison of the 

NLSY97 Family Income Distribution to the CPS Family Income Distribution), Appendix Q (Analysis

of Parent Education for the NLSY97), and Appendix R (Two Comparisons of Expected vs. Observed 

Youth Counts). We note, however, evidence of a modest downward shift in the location of the

96
family income distribution. 

12. There is no solid evidence of large coverage differentials in the NLSY97 screening samples by

urban/rural or metro/nonmetro status. There is, however, weak evidence that coverage was more

problematic in urban areas than in rural areas. 

20. We poststratified the sample by race/ethnicity, age, and sex. In our judgement, the poststratification

curtailed the component of undercoverage bias, if any, associated with the covariates. 

13. Coverage of the NLSY97 screening sample compares favorably to the coverage of the CPS for ages

0-11 and 24-35. 

14. Youth response rates for the main interview (92 percent overall) exceeded expectation (89 percent

overall).

15. For the NLSY97, we handled nonparticipation through a weighting adjustment, controlling for 

sample (CX, SU), race/ethnicity, age, and sex. In our judgement, the adjustment curtailed the

component of nonparticipation bias, if any, associated with the covariates. 

16. A review of systems and procedures reveals no technical or substantive errors that would have been 

the major cause of either the age hole or the deficit in eligible youths. It is possible that computer

crashes of laptop equipment employed in interviewing caused a minor portion of the deficit. 

17. Noninterview reports by field interviewers reveal that screener refusals may be relatively rich in 

eligible youths, especially in small HHs. In our judgement, there is some evidence of a differential 

response mechanism, whereby HHs with eligible youths had a lower propensity to respond to the

screener than HHs without eligible youths. It is likely that this mechanism contributed to the shortfall 

of eligible youths.

18. The advance letter mentions the survey’s focus on youths and the brochure lists the target age range

of interest. Debriefing interviews of the field interviewers show that interviewers frequently revealed 

the target group before the completion of the screening interview. All of this information may have 

invited reluctant HHs to hide eligible youths or misreport their ages, while apparently cooperating 

with the screening interview. It is likely that this mechanism contributed to the shortfall of eligible 

youths.

19. The limited evidence we have been able to assemble suggests that households tended to report either 

all of their eligible youths or none of them. Partial household misses (reporting some but not all

eligible youths) seem less plausible in the face of our data. Also, it appears the undercount may be 

skewed towards single-youth households. A variety of motivations can be ascribed to why the 

households may have behaved this way, but of course the screening data and all of our special 

analyses of coverage are silent on the intents of the householders. 

21. Survey statistics are appropriately precise. Design effects are consistent across domains, and are 

usually well below 1.5. The highest design effects occur for some of the parent variables. 

97
22. An independent review of the NLSY97/PAY97 has been conducted by Professor Eugene P. Ericksen 

of Temple University19. We have not incorporated his analyses or findings into this report. Among

other things, he examines the possibility that college students were differentially undercounted in the 

screening samples, thus contributing to the hole in the age distribution. Even if true, this problem

would have little material effect on the NLSY97 eligible age range of 12-16 years.

23. In the authors’ opinion, taking into account all of the foregoing evidence, the NLSY97 screening 

samples missed about 4,000 eligible youths, relative to ideal circumstances. The overall coverage 

rate of eligibles was less than 75 percent, while the CPS coverage rate for similar ages is about 85 

percent.

24. No natural age distribution known to us exhibits the type of age-specific hole found in the NLSY97

screening sample. It may be reasonable to conclude that the NLSY97 experienced CPS-like 

undercoverage of, say, 15 percent, plus additional undercoverage specific to the eligible age range of 

about 14 percent (= 28.7 – 15). The additional undercoverage, in our opinion, was caused mainly by

elusive respondents – reacting to information supplied to them by field interviewers and other survey

materials – who refused to report their eligible youths or misreported their ages outside of the eligible

range.

19 Ericksen, E.P. (2000), “An Evaluation of Technical Sampling Procedures of a Profile of American Youth, 1997,” 

unpublished manuscript, NORC, Chicago, IL. 

98
APPENDIX A 

Construction of Demographic Variables 

for the NLSY97 

99
In order to complete the weighting for NLSY97 as well as provide a basis for other analyses of 

the data, complete demographic information was needed. There was a small amount of missing

demographic data for race, ethnicity, sex, and age. Further, there were multiple reports for these variables 

which most of the time agreed with one another, but sometimes did not. Thus, the approach put forth in 

this appendix was designed to solve two problems. First, if the demographic information was missing, an

imputed value was provided. More importantly, the vast majority of the cases had multiple inputs for 

demographic variables; for these, a hierarchy of inputs was established and provided the final

demographic data. 

Hispanic Origin 

Parent variables:

Parent 1 

a) if all orgpar1_1-orgpar1_8 (parent origin) variables are blank, then orgpar1 is blank (missing).

b) if orgpar1_1 is ‘0’ or ‘1’ and all orgpar1_2-orgpar1_8 variables are ‘0’, then orgpar1=’O’ (NonHispanic).

c) if any of orgpar1_2-orgpar1_8 is ‘1’, then orgpar1=’H’ (Hispanic). 

The same follows for Parent 2. 

On-line questionnaire recodes:

Ethnicity

a) if ethncolq (ethnicity from the OLQ) is blank, then ethnrec (ethnicity recode) is blank (missing).

b) if ethncolq is 1, 2, 3, 4, or 5, then ethnrec=’H’ (Hispanic). 

c) if ethncolq is any other value, then ethnrec=’O’ (Non-Hispanic). 

c) if origolq is any other value, then origrec='O' (Non-Hispanic). 

x If hisposcr is missing and ethnrec is blank and origrec is ‘O’, then FHISP=0.

Origin

a) if origolq (origin from the OLQ) is blank, then origrec (origin recode) is blank. 

b) if origolq is 20, 21, 22, 23, 24, 25, 26, then origrec='H' (Hispanic). 

Hispanic Origin Variable (FHISP) Construction

1. Screener variable:

x If screener hispanic origin variable (hisposcr) is 0 or 1, FHISP=hisposcr (0=non-Hispanic, 

1=Hispanic).

2. OLQ recodes:

x If hisposcr is missing and ethnrec=’H’, then FHISP=1.

x If hisposcr is missing and ethnrec is not ‘H’ but origrec is ’H’, then FHISP=1.

x If hisposcr is missing and ethnrec is ‘O’ and origrec is not ‘H’, then FHISP=0.

3. Parent variable 1: 

x If all above variables are missing and orgpar1 is ‘H’, then FHISP=1.

x If all above variables are missing and orgpar1 is ‘O’ or ‘B’, then FHISP=0.

100
4. Parent variable 2: 

x If all above variables (including orgpar1) are missing but orgpar2 is ‘H’, then FHISP=1.

x If all above variables (including orgpar1) are missing but orgpar2 is ‘O’ or ‘B’, then FHISP=0.

5. Imputation:

x Cases for which all hispanic origin variables are missing are saved to a file and merged with other 

variables in the screening data, which is used for logical imputation. For NLSY97, there were 51 

such cases. Those with other family members who are Hispanic were given a value of 1 for 

FHISP, and all others were given a value of 0 for FHISP.

Hispanic Origin Source Variable (SHISP) Construction

1. Screener variable:

x If FHISP value is taken from screener variable hisposcr, then SHISP=1.

2. OLQ Ethnicity recode:

x If FHISP value is taken from OLQ variable ethnrec or origrec, then SHISP=2.

3. Parent variable1: 

x If FHISP value is taken from orgpar1, then SHISP=3.

4. Parent variable2: 

x If FHISP value is taken from orgpar2, then SHISP=4.

5. Imputation:

x If FHISP value is logically imputed with answer file data and given value of 1, then SHISP=5.

x For all remaining cases eligible for imputation but with no available answer file data (FHISP=0),

SHISP=6.

Table A-1 Crosstabulation of Screener Hispanic Origin by OLQ Ethnicity (recoded) 

29 5 30 64

29 5 18 52

2062 269 5383 7714

772 1111 261 2144

2892 1390 5692 9974

missing

-

Non-Hispanic

Hispanic

HISPOSCR

Total

missing Hispanic

NonHispanic

ETHNREC

Total

101
Table A-2 Crosstabulation of Screener Hispanic Origin by OLQ Origin (recoded) 

29 4 31 64

29 3 20 52

2067 114 5533 7714

773 1037 334 2144

2898 1158 5918 9974

missing

-

Non-Hispanic

Hispanic

HISPOSCR

Total

missing Hispanic

NonHispanic

ORIGREC

Total

Table A-3 Final Hispanic Origin Frequencies 

7816

2158

9974

0

1

Total

Frequency

FHISP

9858

58

7

4

47

9974

1

2

3

5

6

Total

Frequency

SHISP

Sex

Sex Variable (FSEX) Construction

1. Youth variable: 

x If youth sex variable (sex_yth) is 1 or 2, FSEX=sex_yth (1=Male, 2=Female).

2. Parent variable: 

x If sex_yth is missing and sex_par is 1 or 2, then FSEX =sex_par.

3. Screener variable: 

x If sex_yth and sex_par are missing and sex_scr is 1 or 2, then FSEX=sex_scr.

4. Imputation:

x Cases for which all sex variables are missing are saved to a file for imputation. For NLSY97, 

there were 84 cases eligible for imputation; 58 of those could be logically imputed based on first 

name.

x The remaining 26 cases were randomly imputed (13 male, 13 female).

Sex Source Variable (SSEX) Construction

102
1. Youth variable: 

x If FSEX value is taken from sex_yth, then SSEX=1.

2. Parent variable: 

x If FSEX value is taken from sex_par, then SSEX=2.

3. Screener variable: 

x If FSEX value is taken from sex_scr, then SSEX=3.

4. Imputation:

x If FSEX value is logically imputed, SSEX=4.

x If FSEX value is randomly imputed, SSEX=5.

103
Table A-4 Crosstabulation of Screener Sex by Youth Questionnaire Sex 

63 1 64

21 2 23

5056 1 19 5076

4789 21 1 4811

9929 25 20 9974

missing

-

Male

Female

SEX_SCR

Total

missing Male Female

SEX_YTH

Total

Table A-5 Crosstabulation of Screener Sex by Parent Questionnaire Sex 

64 64

23 23

5067 9 5076

4809 1 1 4811

9963 1 10 9974

missing

-

Male

Female

SEX_SCR

Total

missing Male Female

SEX_PAR

Total

Table A-6 Final Sex Frequencies

5121

4853

9974

1

2

Total

Frequency

FSEX

45

6

9839

58

26

9974

1

2

3

4

5

Total

Frequency

SSEX

104
Race

x Cases for which racescr is missing, FHISP=0, and origolq is 36, 37, 88, or missing are saved to a 

file and merged with answer file data, which is used for logical imputation. For NLSY97, there 

were 58 such cases. 17 of those had other family members with valid race values and were given 

that value for FRACE. The remaining 41 did not have any available family race information and 

were given a value of 9 for FRACE. Later, for weighting, these 41 cases were folded into the 

“Other” (non-Hispanic/non-Black) race category.

1. Screener variable:

x If FRACE value is taken from screener variable racescr, then SRACE=1.

3. OLQ origin variable: 

Race Variable (FRACE) Construction

1. Screener variable:

x If screener race variable (racescr) is 1, 2, 3, 4, or 5, FRACE=racescr (1=White, 2=Black, 

3=American Indian, 4=Asian, and 5=Other). 

2. FHISP variable:

x If racescr is missing and FHISP is 1, then FRACE=1.

3. OLQ origin variable: 

x If racescr is missing and FHISP=0 but origolq is 1, 3, 5, 6, 8, 9, 10, 13, 16, 17, 20, 21, 22, 23, 24, 

25, 26, 27, 28, 29, 30, 31, 32, or 35, then FRACE=1.

x If racescr is missing and FHISP=0 but origolq is 2 or 11, then FRACE=2.

x If racescr is missing and FHISP=0 but origolq is 14, then FRACE=3.

x If racescr is missing and FHISP=0 but origolq is 4, 7, 12, 15, 18, 19, 33 or 34, then FRACE=4.

4. Imputation:

Race Source Variable (SRACE) Construction

2. THISP variable:

x If FRACE value is based solely on FHISP (FRACE=1 for FHISP=1), then SRACE=2.

x If FRACE value is taken from origolq, then SRACE=3.

4. Imputation:

x If FRACE value is imputed, then SRACE=4.

105
 Table A-7 Crosstabulation of Screener Race by OLQ Origin (recoded) 

29 7 4 24 64

79 2 22 35 138

1473 35 508 3679 5695

807 1351 64 407 2629

22 2 8 38 70

55 1 1 125 182

433 36 551 176 1196

2898 1434 1158 4484 9974

missing

-

White

Black

American Indian

Asian

Other

RACESCR

Total

missing

NonHispanic

Black Hispanic

NonHispanic

Non-Black

ORIGREC

Total

Table A-8 Final Race Frequencies 

5837

2638

76

185

1197

41

9974

1

2

3

4

5

9

Total

Frequency

FRACE

9772

106

38

58

9974

1

2

3

4

Total

Frequency

SRACE

Age

Age Variable (FAGE) Construction

1. Youth date of birth variable:

x If the year portion (first four digits) of the youth date of birth variable (dobyth) is between 1980 

and 1984, FAGE = 1996 minus dobyth (dobyth=1980 implies FAGE=16, ..., dobyth=1984

implies FAGE=12).

x If the year portion of dobyth is some other year besides 1980-1984 and 1880 (default missing),

FAGE=–1 to indicate age ineligibility at the first stage. 

2. Youth age and birthday variables: 

x If dobyth is equal to 1880 or missing and ageyth is between 12 and 16, then if bdayyth=0

(indicating that the youth has not had a birthday since January 1, 1997), FAGE=ageyth.

x If dobyth is equal to 1880 or missing and ageyth is between 12 and 16, then if bdayyth=1

(indicating that the youth has had a birthday since January 1, 1997), FAGE=ageyth-1.

106
x If dobyth is equal to 1880 or missing and ageyth is not missing but is outside of the 12-16 age 

range (or if ageyth-1 calculated in the step above is outside of the 12-16 age range), then 

FAGE=–2 to indicate age ineligibility at the second stage. 

3. Screener date of birth variable: 

x If dobyth is 1880 or missing and ageyth is missing, then if the year portion (first four digits) of 

the screener date of birth variable (dobscr) is between 1980 and 1984, FAGE = 1996 minus

dobscr (dobscr=1980 implies FAGE=16, ..., dobscr=1984 implies FAGE=12).

x If dobyth is 1880 or missing, ageyth is missing, and the year portion of dobscr is some other year

besides 1980-1984 and 1880 (default missing), then FAGE=–3 to indicate ineligibility at the 

third stage. 

5. OLQ year of birth variable: 

4. Screener age and birthday variables: 

x If variables from steps one and two are missing and dobscr is equal to 1880 or missing, then if 

agescr is between 12 and 16 and bdayscr=0 (indicating that the youth has not had a birthday

since January 1, 1997), FAGE=agescr.

x If variables from steps one and two are missing and dobscr is equal to 1880 or missing, then if 

agescr is between 12 and 16 and bdayscr=1 (indicating that the youth has had a birthday since 

January 1, 1997), FAGE=agescr-1.

x If variables from steps one and two are missing and dobscr is equal to 1880 or missing, and if 

agescr is not missing but is outside of the 12-16 age range (or if agescr-1 calculated in the step

above is outside of the 12-16 age range), then FAGE=–4 to indicate age ineligibility at the fourth 

stage.

x If all above variables are missing (or 1880) and yob (year of birth from the on-line questionnaire) 

is 80, 81, 82, 83, or 84, then FAGE = 1996-yob.

x If all above variables are missing (or 1880) and yob is not missing and is some other year besides 

80, 81, 82, 83, and 84, then FAGE=–5 to indicate ineligibility at the fifth stage. 

6. Range variable: 

x If all above variables are missing (or 1880) and rangs1ck=1 (indicating that the youth falls into

the 12-16 age range), then FAGE=99 to indicate eligibility for random imputation.

x If all above variables are missing (or 1880) and rangs1ck=0 or missing (indicating that the youth

does not fall into the 12-16 age range), then FAGE=98 to indicate the absence of all age data. 

7. Imputation:

x Cases with FAGE=98 or 99 are saved to a file and merged with answer file data, which is used for 

logical imputation. For NLSY97, there were 53 such cases. First, all 53 cases were located

within the answer file extract, and if a valid value for age could be found for that person, FAGE

was set equal to that value. 30 cases were given valid FAGE values in this manner. Of the 10

cases given a value of 98 in step 6, only one was not among those 21 cases and was left for 

random imputation. Finally, the remaining 22 cases with no age information in the answer files 

were given a value of –7, which indicates ineligibility at the last stage. These, along with all

other negative FAGE values, were considered ineligible at the time of weighting. 

107
Age Source Variable (SAGE) Construction

1. Youth date of birth variable:

x If a valid FAGE value (12-16) is taken from dobyth, then SAGE=1. Also, if FAGE was deemed

ineligible (FAGE=-1) at the first stage, SAGE=1.

4. Screener age and birthday variables: 

x For the 10 cases for which rangs1ck=1 and FAGE was given an initial value of 98 and later

imputed, SAGE=6.

2. Youth age and birthday variables: 

x If a valid FAGE value (12-16) is taken from ageyth, then SAGE=2. Also, if FAGE was deemed

ineligible (FAGE=-2) at the second stage, SAGE=2.

3. Screener date of birth variable: 

x If a valid FAGE value (12-16) is taken from dobscr, then SAGE=3. Also, if FAGE was deemed

ineligible (FAGE=-3) at the third stage, SAGE=3.

x If a valid FAGE value (12-16) is taken from agescr, then SAGE=4. Also, if FAGE was deemed

ineligible (FAGE=-4) at the fourth stage, SAGE=4.

5. OLQ year of birth variable: 

x If a valid FAGE value (12-16) is taken from yob, then SAGE=5. Also, if FAGE was deemed

ineligible (FAGE=-5) at the fifth stage, SAGE=5.

6. Range variable: 

7. Imputation:

x Among the remaining 43 cases for which rangs1ck is not equal to 1, 21 were logically imputed

and SAGE set to 7; 22 had no available age data (FAGE=-7) and SAGE was set to 8. 

108
Table A-9 Crosstabulation of OLQ Year of Birth by Screener Year of Birth 

295 2 493 467 474 466 440 4 2 1 2644

3 4 3 7 10 8 35

1 1

1 1 2

1 1

1 1 2

1 1 2

1 1 2

8 5 1 14

106 1195 18 1 2 1322

96 18 1396 19 4 2 1535

114 7 17 1343 24 5 1510

101 1 2 18 1287 28 1437

94 2 1 3 22 1308 1430

4 1 2 25 32

1 1 2

1 1

1 1 2

824 2 1 1727 1907 1868 1820 1818 4 2 1 9974

missing

00

73

74

75

76

77

78

79

80

81

82

83

84

85

86

88

89

YOB

Total

missing 1957 1975 1980 1981 1982 1983 1984 1985 1991 1992

DOBSCR Year

Total

Table A-10 Crosstabulation of Youth Questionnaire Year of Birth by Screener Year of Birth 

395 2 1608 1753 1726 1705 1689 4 1 8883

2 1 3

1 1

87 1 95 20 1 1 1 1 207

68 12 124 17 1 2 1 225

99 7 8 105 23 3 245

94 2 1 14 82 28 221

77 2 1 4 7 93 184

2 1 2 5

824 2 1 1727 1907 1868 1820 1818 4 2 1 9974

missing

1970

1979

1980

1981

1982

1983

1984

1985

DOBYTH

Year

Total

missing 1957 1975 1980 1981 1982 1983 1984 1985 1991 1992

DOBSCR Year

Total

109
Table A-11 Age Frequencies 

22

3

7

7

9

1936

1984

2041

2049

1916

9974

-7

-5

-4

-3

-1

12

13

14

15

16

Valid

Total

Frequency

FAGE

1091

8488

196

146

10

21

22

9974

1

3

4

5

6

7

8

Valid

Total

Frequency

SAGE

110
APPENDIX B 

This appendix is blank 

111
APPENDIX C 

This appendix is blank 

112
APPENDIX D 

Analysis of Language Barrier Cases 

113
In this appendix, we analyze the extent of youths designated as Alanguage barrier@ (LB) in 

ASVAB testing. We focus mainly on the Enlistment Testing Program, but also present some analysis for 

the Student Testing Program. We begin by presenting known national norms for English proficiency.

Later, we compare PAY97 actuals to the national norms and conclude that PAY97 was not unusual in the 

extent of LB youths it encountered. In our opinion, this finding also extends to the NLSY97. 

National Norms

The 1990 Census long form contained this question: 

15a. Does this person speak a language other than English at home?

Yes

 No B Skip to 16

15b. What is this language? _________________________

 (For example: Chinese, Italian, Spanish, Vietnamese)

15c. How well does this person speak English?

 Very well

 Well 

 Not Well 

 Not at all. 

Using the 1 percent public-use file (U.S. Bureau of the Census, 1993), we cross-tabulated responses to 

this question by responses to census question 7, the Hispanic origin question, obtaining Table D-1. 

114
115

English and by Hispanic Origin 

English Speaking Ability All Youths 

Table D-1 Distribution (Percent) of American Youths by Ability to Speak 

Not of Hispanic Origin Hispanic Origin 

a. Youths Age 12-23 

Only English at Home 81.27

92.05

2.95

25.16

84.22

Very Well 

45.10

4.81

5.44

5.28 10.09

Well 1.43 3.10

1.62

1.68

14.31

9.79

Not Well 0.75

0.84

1.15 1.89

Not at All 0.04

0.05

0.66

5.64

0.70

Total 88.29 11.71 100.00

b. Youths Age 12-16 

Only English at Home 81.62

92.40

3.37

28.90

84.99

48.20

Very Well 4.65 10.27

5.26

5.62

Well 1.31

1.49

1.72

14.77

3.03

Not Well 0.73

0.82 6.27

0.73 1.46

0.03

1.85

Not at All 

0.04

0.22 0.25

Total 88.34 11.66 100.00

c. Youths Age 18-23 

Only English at Home 81.17 2.63

91.95 22.40

83.8

Very Well 4.82

5.46 42.35

4.96 9.78

14.04

Well 1.49

1.69

1.65 3.14

Not Well 0.75

0.85

1.48

12.67

2.23

Not at All 0.05

0.06 8.54

1.00 1.05

Total 88.28 100.00 11.72

NOTE: Two percentages are given in each cell. The first percent is relative to the population of all youths. The 

second percent is relative to the youths in the column domain. 

NCES (1994) consolidates persons who responded less than Avery well@ in the category Aspeak
116

English with difficulty.@ Using this consolidation, the census shows 4.75 percent of youths age 12-16 

speak English with difficulty. For youths age 18-23, the corresponding figure is 6.42 percent.

Please note that the census question refers to speaking English. The issue of speaking versus 

understanding versus writing versus reading is clearly an important one. Some people speak English 

passably but can't read it and others read foreign languages reasonably well but speak them abysmally. 

Also, the census data are reported either by the person himself or herself or by the census respondent in 

the same household. 

Macias et al. (1998) report data on limited English proficiency (LEP) reported by State Education 

Agencies (SEAs) that receive Title VII funds. We reproduce data for the 1996-97 school year in Table 

D-2. SEAs reported a total of 45,650,352 enrolled students, of which 7.4 percent are LEP students.

Table D-2 Summary of Total Student and LEP Enrollments, 1996-97

States and DC Total Enrollment LEP Students (in percent) 

Public School Students 41,704,542 8.0

Private School Students 3,945,810 0.9

Total Students 45,650,352 7.4

Macias et al. assert that their figures are conservative estimates of the true LEP enrollment, in 

part because not every SEA responds to the survey every year and the number of LEP students continues 

to increase and become a greater part of the total school enrollment of the country. 

Hopstock and Bucaro (1993) provide a careful review and analysis of the strengths and 

weaknesses of different methods of estimating the LEP population. In summarizing their own work and 

that of other analysts, they suggest that persons who are rated as speaking English less than Avery well@

are likely to be LEP, and that even some rated as speaking Avery well@ may actually be LEP as well. By 

this reasoning, census data tell us that at least 5.7 percent of youths age 12-23 should be LEP, while 

Macias et al. suggest 7.4 percent of enrolled students are LEP. However, Hopstock and Bucaro conclude 

that

What is most clear from all of these differing estimates is that analysts could not agree on criteria 

for defining LEP students. Although there was variation based on the data collection 

methodologies and population groups included, the most important factor differentiating 

estimates appeared to be the standard for determining whether a child was or was not LEP. ... In 

order to develop a reliable and consistent count of LEP students, the field will need to agree on an 

operational definition of LEP status. 

Furthermore, it is far from clear how reporting of LEP status by state agencies compares to the 

concept of Alanguage barrier@ as reported by an individual, a household respondent, or a field 

interviewer. Estimates from school-based surveys may understate language barrier cases in the general 

youth population since students who haven't learned English by high school tend to drop out at a high 

rate. Definitions of LEP are far from standard and vary from place to place, and English proficiency is a 
117

continuum such that "language barrier" tends to implicate only the more severely limited. For older 

youths age 18-23, overall English proficiency will be strongly influenced by immigration of working age 

Hispanics and others.

Fleischman and Hopstock (1993) used a mail survey of 745 school districts to estimate the LEP 

student population by grade. See Table D-3. According to their data, about 5.5 percent of all students are 

thought to be LEP. The numbers are somewhat higher for lower grades than for high school age, e.g., 8.4 

percent for kindergarteners, 6 percent for fourth graders, 4.2 percent for eighth graders, and 3.2 percent 

for 12th graders. 

Table D-3 Number and Percent of LEP Students in Each Grade Level

Grade Level Total Students in the US Percent LEP of Total Students 

Kindergarten 3,305,619 8.4

1 3,554,274 7.9

2 3,359,193 7.4

3 3,333,285 6.7

4 3,312,443 6.0

5 3,268,381 5.4

6 3,238,095 4.6

7 3,108,120 4.6

8 3,019,826 4.2

9 3,310,290 4.8

10 2,913,951 4.7

11 2,642,554 3.9

12 2,390,329 3.2

Ungraded --- ---

Total 42,000,343 5.5
118

Table D-4 gives a breakdown of the top 10 language groups among LEP students, produced by 

the Fleischman and Hopstock survey. Clearly, with 72.9 percent of all LEP students, Spanish is the 

dominant non-English language. Other language groups than those shown in Table D-4 individually 

account for less than 1 percent of the LEP-student population, and collectively account for only 11.9 

percent of the population. 

Table D-4 Percentage of LEP Students by Language Group, 1991-92

Language Groups Percentage of LEP Students 

Spanish 72.9

Vietnamese 3.9

Hmong 1.8

Cantonese 1.7

Cambodian 1.6

Korean 1.6

Laotian 1.3

Navajo 1.3

Tagalog 1.1

Russian 0.9
119

Public Schools 

Another source of information about LEP youths is the NAEP 1992 mathematics assessments. 

Table D-5 provides a summary of some of the results. 

Table D-5 Percentages of LEP Students Identified and Excluded from the NAEP Sample

Percent of Students Identified 

LEP

Percent of Students Excluded 

LEP

a. Grade 4 B 1992 

Total US 4 3

Midwest 1 1

Northeast 3 3

South 1 1

West 9 7

California 22 10

Arizona 9 2

Texas 9 4

Rhode Island 6 3

New York 5 2

b. Grade 8 B 1992 

Total US 3 2

Midwest 1 1

Northeast 3 2

South 1 1

West 8 4

California 13 5

Arizona 6 2

Texas 6 2

Three percent of eighth graders (13-year-olds) were classified as LEP and 2 percent were 

excluded owing to a language barrier. The percent LEP identified and excluded was higher in the West 

than in other regions. Several states identified 5 percent or more, including California, Arizona, Texas, 
Rhode Island, and New York. LEP percentages were a bit higher in the fourth grade than in the eighth 

grade assessments.

Generally about a third of students who speak languages other than English in the home have

difficulty speaking English (NCES, 1994, Table 46). 

Results from PAY97

Youths included in the PAY97 study were designated as in- or out-of-scope for testing by NORC 

field interviewers. Field interviewers performed test scheduling by telephone, and they made their 

designation at the time of scheduling. In turn, field interviewers coded youths as to the reason why they

were out of scope for testing, including incapacitated, in jail, language barrier, dead, out of the country, or 

in the military. They coded youths as Alanguage barrier@ in the event that the youth could not speak or

understand English on the telephone during the scheduling call, or whenever the field interviewer

otherwise reached the judgment that the youth could not reasonably take the ASVAB test administered in 

English.

We present evidence about LB youths in Tables D-6 through D-9. Older youths who screened 

into the Enlistment Testing Program (ETP) appear in Tables D-6 and D-7, while younger youths who 

screened into the Student Testing Program (STP) appear in Tables D-8 and D-9. About 2.9 percent of all 

the older youths (age 18-23) were coded as LB. After removing youths subsampled out, we find that 

about 3.1 percent of youths who were age-eligible for the ETP were coded as LB. While more than half 

of these LB youths came from the cross-sectional sample, the portion of the supplemental sample (5.3 

percent) deemed LB was more than twice as large (2.2 percent) as the corresponding portion of the crosssectional sample. As Table D-7 makes clear, most of the LB youths were of Hispanic origin. Only 7

percent of the LB youths were non-Hispanic. Among Hispanic youths, about 12.4 percent were 

designated as LB. 

Relatively fewer STP youths were designated LB B only 0.9 percent overall, and 0.8 percent and 

1.1 percent for the cross-sectional and supplemental samples, respectively. In absolute terms, the crosssection produced more LB youths, owing to its larger sample size. Over 86 percent of LB youths were 

Hispanic, but only 4 percent of Hispanic youths were designated LB. 

Because of the larger number (258) of LB youths screened into the ETP, we performed one more

analysis, breaking down the LB cases by geography. We did not perform this analysis for the STP youths

since the sample of LB cases was too small (58). The analysis of ETP youths appears in Figures D-1 to

D3. For each PSU, we calculated the proportion of youths identified as LB and the proportion identified 

as Hispanic. We present, in Figure D-1, a scatter plot of proportion LB versus proportion Hispanic across

the total sample of 200 PSUs. Figures D-2 and D-3 contain similar scatter plots for the 100 crosssectional and 100 supplemental PSUs, respectively. Most of the PSUs have a very low proportion of LB 

cases. On the other end, many of the PSUs with a high proportion LB also have a high proportion

Hispanic. This finding is not surprising and, in fact, is intimated by earlier tables. Three cross-sectional 

PSUs and 14 supplemental PSUs have a proportion LB greater than 0.1, and we list these PSUs in Table 

D-10. Of these 17 PSUs, however, 11 are based upon a very small numbers of cases B probably too small

to warrant attention. Clearly, most of the LB cases come from densely Hispanic areas and from areas of

the country known for their Hispanic populations. 

120
Table D-6 ETP Status * Sample Crosstabulation 

Sample

Cross-Sectional

Sample

Supplemental

Sample Total

ETP Status Ineligible, subsampled out Count 593 16 609

% within NEWCODE 97.4% 2.6% 100.0%

% within Sample 8.7% .8% 6.8%

% of Total 6.7% .2% 6.8%

Age ineligible or special 

circumstances

Count 38 32 70

% within NEWCODE 54.3% 45.7% 100.0%

% within Sample .6% 1.5% .8%

% of Total .4% .4% .8%

Count 20 10 30

% within NEWCODE 66.7% 33.3% 100.0%

% within Sample .3% .5% .3%

% of Total .2% .1% .3%

OOS, in jail Count 9 14 23

% within NEWCODE 39.1% 60.9% 100.0%

% within Sample .1% .7% .3%

% of Total .1% .2% .3%

OOS, language barrier Count 147 111 258

% within NEWCODE 57.0% 43.0% 100.0%

% within Sample 2.2% 5.3% 2.9%

% of Total 1.7% 1.2% 2.9%

OOS, dead Count 48 28 76

% within NEWCODE 63.2% 36.8% 100.0%

% within Sample .7% 1.3% .9%

% of Total .5% .3% .9%

OOS, out of country Count 31 7 38

% within NEWCODE 81.6% 18.4% 100.0%

% within Sample .5% .3% .4%

% of Total .3% .1% .4%

OOS, in military Count 7 3 10

% within NEWCODE 100.0% 70.0% 30.0%

% within Sample .1% .1% .1%

% of Total .1% .0% .1%

1266 488 1754

% within NEWCODE 72.2% 27.8% 100.0%

% within Sample 18.6% 23.4% 19.7%

% of Total 14.2% 5.5% 19.7%

Eligible, incomplete test 17 Count 20 37

% within NEWCODE 45.9% 54.1% 100.0%

% within Sample .2% 1.0% .4%

% of Total .2% .2% .4%

Eligible, AFQT complete 79 102

% within NEWCODE

% within Sample 1.2%

% of Total .9% .3%

Count 4566 5900

% within NEWCODE 77.4% 22.6%

% within Sample 66.9%

% of Total 51.3% 66.2%

Total Count

% within NEWCODE 23.4% 100.0%

100.0% 100.0%

% of Total 76.6% 23.4% 100.0%

OOS, incapacitated 

Eligible, nonparticipant Count

Count 23

77.5% 22.5% 100.0%

1.1% 1.1%

1.1%

Eligible, complete test 1334

100.0%

64.0% 66.2%

15.0%

6821 2086 8907

76.6%

% within Sample 100.0%

121
Table D-7 ETP: OOS, Language Barrier * Hispanic Origin Crosstabulation

Hispanic Origin

NonHispanic Hispanic Total

OOS, Language Barrier No Count 6938 1698 8636

% within OOS, language barrier 80.3% 19.7% 100.0%

% within Hispanic Origin 99.7% 87.6% 97.1%

% of Total 78.0% 19.1% 97.1%

Yes Count 18 240 258

% within OOS, language barrier 7.0% 93.0% 100.0%

% within Hispanic Origin .3% 12.4% 2.9%

% of Total .2% 2.7% 2.9%

Total Count 6956 1938 8894

% within OOS, language barrier 78.2% 21.8% 100.0%

% within Hispanic Origin 100.0% 100.0% 100.0%

% of Total 78.2% 21.8% 100.0%

122
Table D-8 STP Status * Sample Crosstabulation

Sample

Cross-Sectional

Sample

Supplemental Sample

Total

STP Status Age or grade ineligible, or 

special circumstance

Count

% within STP Status 

14

42.4%

19

57.6%

33

100.0%

% within Sample .3% .9% .5%

% of Total .2% .3% .5%

Count 11 8 19

% within STP Status 57.9% 42.1% 100.0%

% within Sample .3% .4% .3%

% of Total .2% .1% .3%

Count 11 3 14

% within STP Status 78.6% 21.4% 100.0%

% within Sample .3% .1% .2%

% of Total .2% .0% .2%

OOS, language barrier Count 35 23 58

% within STP Status 60.3% 39.7% 100.0%

% within Sample .8% 1.1% .9%

% of Total .5% .4% .9%

OOS, dead Count 50 26 76

% within STP Status 65.8% 34.2% 100.0%

% within Sample 1.2% 1.2% 1.2%

% of Total .8% .4% 1.2%

Count 3 2 5

% within STP Status 60.0% 40.0% 100.0%

% within Sample .1% .1% .1%

% of Total .0% .0% .1%

Count 1 1

% within STP Status 100.0% 100.0%

% within Sample .0%

% of Total .0%

Count 903 569 1472

% within STP Status 61.3% 38.7% 100.0%

% within Sample 21.4% 26.4% 23.1%

% of Total 14.2% 8.9% 23.1%

Eligible, incomplete test Count 19 25 44

% within STP Status 56.8% 100.0%

% within Sample .4% 1.2% .7%

% of Total .3% .4% .7%

Eligible, AFQT complete Count 42 10 52

% within STP Status 80.8% 19.2% 100.0%

% within Sample .5% .8%

% of Total .7% .2% .8%

Eligible, complete test Count 3135 1467 4602

% within STP Status 68.1% 31.9% 100.0%

% within Sample 74.2% 68.2% 72.2%

49.2% 23.0%

4224 2152 6376

% within STP Status 66.2% 33.8% 100.0%

% within Sample 100.0% 100.0% 100.0%

% of Total 66.2% 33.8% 100.0%

OOS, incapacitated 

OOS, in jail 

OOS, out of the country

OOS, in military

.0%

.0%

Eligible, nonparticipant

43.2%

1.0%

% of Total 72.2%

Total Count 

123
Table D-9 STP: OOS, Language Barrier * Hispanic Origin Crosstabulation

Hispanic Origin

NonHispanic Hispanic Total

OOS, Language Barrier No Count 5114 1204 6318

% within OOS, Language Barrier 80.9% 19.1% 100.0%

% within Hispanic Origin 99.8% 96.0% 99.1%

% of Total 80.2% 18.9% 99.1%

Yes Count 8 50 58

% within OOS, Language Barrier 13.8% 86.2% 100.0%

% within Hispanic Origin .2% 4.0% .9%

% of Total .1% .8% .9%

Total Count 5122 1254 6376

% within OOS, Language Barrier 80.3% 19.7% 100.0%

% within Hispanic Origin 100.0% 100.0% 100.0%

% of Total 80.3% 19.7% 100.0%

124
125

Figure D-1 ETP: Language Barrier versus 

Hispanic Origin, PSU Level 
126

Figure D-2 ETP: Language Barrier versus 

Hispanic Origin, PSU Level, Cross-Sectional Sample 
127

Figure D-3 ETP: Language Barrier versus 

Hispanic Origin, PSU Level, Supplemental Sample 
Table D-10 Youths Screened into the ETP: PSUs with Proportion

Language Barrier Greater than or Equal to 0.10

PSU Number Sample Number of LB 

Youths

Proportion LB Proportion

Hispanic

704 Supplemental 3 0.14 0.59

721 Supplemental 3 0.20 1.00

723 Supplemental 2 0.12 0.29

747 Supplemental 20 0.44 0.89

761 Supplemental 22 0.13 0.91

766 Supplemental 17 0.36 0.83

785 Supplemental 4 0.20 1.00

792 Cross-Sectional 57 0.18 0.58

795 Supplemental 2 0.25 0.25

812 Cross-Sectional 13 0.19 0.25

825 Supplemental 1 0.13 1.00

835 Supplemental 2 0.12 0.53

841 Supplemental 11 0.11 0.59

863 Supplemental 2 0.18 1.00

868 Cross-Sectional 4 0.12 0.70

872 Supplemental 2 0.13 1.00

896 Supplemental 1 0.50 1.00

Findings

As we have remarked previously, language proficiency is a difficult concept to define and 

measure well. Even in this brief appendix, we have introduced three specific terms B LB, LEP, and speak 

English with difficulty B which are related to language proficiency, and it is unclear exactly how these 

terms relate definitionally to one another. We suppose, however, that LB could represent the most

seriously English deficient level. 

128
Even if the terminology were well defined and understood, different sources of data arise from

different measurement processes. Some data are self-reported by the person. Some data are reported by a 

household informant; others by an interviewer; and still others by a school or school district. 

Comparability of data sources is also compromised by the fact that different sources cite data for 

different years. Some sources are not even internally consistent as to reference period.

We conclude, without question, that the data on English proficiency displayed in this appendix 

are subject to a variety of differential biases and random errors. 

Despite these limitations, we summarize the data in Table D-11. From these data, there is no

evidence that the ETP or STP samples were excessively saturated with LB youths. Quite to the contrary,

these data suggest that the PAY97 samples found LB youths at a typical to below typical rate. 

Furthermore, census data suggest that there should be more LBs among youths age 18-23 than among

youths 12-16, and this general pattern is present in the ETP and STP data. 

Table D-11 Estimates of English Language Deficiency

Source Percent English Deficient 

STP, LB Youths, Grades 10-12 0.9

1990 Census, ANot Well@ plus ANot at All@, Age 12-16 1.7

NAEP, LEP Students, Grade 8 3.0

ETP, LB Youths, Age 18-23 3.1

Fleischman and Hopstock, LEP Students, Grade 12 3.2

1990 Census, ANot Well@ plus ANot at All@, Age 18-23 3.3

Fleischman and Hopstock, LEP Students, Grade 11 3.9

NAEP, LEP Students, Grade 4 4.0

Fleischman and Hopstock, LEP Students, Grade 10 4.7

1990 Census, ASpeak English with Difficulty@, Age 12-16 4.8

Fleischman and Hopstock, LEP Students, Age 5-17 5.5

1990 Census, ASpeak English with Difficulty@, Age 18-23 6.4

Macias et al., LEP Students 7.4

National norms further suggest that the majority of English language deficient cases should be 

Hispanic, and that the West, and California in particular, should display higher rates of deficiency than 

other geographic areas. These patterns too are present in ETP and STP data. For example, Fleischman

and Hopstock=s data suggest that around 73 percent of LEP students are Hispanic. On the other hand, the 

ETP finds that 93 percent of LBs are Hispanic. That ETP finds more Hispanics than Fleischman and 

Hopstock is not surprising because Hispanics were oversampled for the ETP. 

129
Among Hispanic youths age 18-23, the census reports that 12.7 percent speak English Anot well@

and another 8.5 percent speak Anot at all.@ By comparison, in the ETP, 12.4 percent of Hispanic youths

were designated LB. Among Hispanic youths age 12-16, the census reports 0.7 percent speak Anot well@

and another 0.2 percent speak Anot at all.@ For STP youth, 4.0 percent of Hispanics were designated LB.

The differences between census and PAY97 data are not large, and in any event are confounded by

definitional issues, reporting issues, and the PAY97 oversample.

In light of the foregoing evidence, we find no support for the claim that the PAY97 samples are 

excessively saturated with LB cases. We find no support for the claim that LBs compromise the 

representativeness of the PAY97 samples to a greater extent than they compromise the representativeness 

of other important, national surveys.

References

Fleischman, H.L. & Hopstock, P.J. (1993), Descriptive Study of Services to Limited English Proficient

Students. Development Associates, Inc., Arlington, VA. 

U.S. Bureau of the Census (1993), Census of Population and Housing, 1990: Public Use Microdata, 1 

Percent Sample. CD-ROM, 2 disks, 2 the Bureau of the Census, Washington, 

D.C.

Hopstock, P.J., and Bucaro, B.J. (1993), A Review And Analysis of Estimates of the LEP Student

Population. Development Associates, Inc., Arlington, VA. 

Macias, R.F., et al. (1998), Summary Report of the Survey of the States’ Limited English Proficiency 

Students and Available Educational Programs and Services, 1996-97. Washington, DC: National

Clearinghouse for Bilingual Education. 

Mullis, I.V.S., et al. (1993), NAEP 1992 Mathematics Report Card for the Nation and the States.

National Center for Education Statistics, Washington, D.C. 

National Center for Education Statistics (1994), The Condition of Education, 1994. National Center for

Education Statistics, Washington, D.C. 

nd release, distributed by

130
APPENDIX E 

Review of Proxy Interviews 

131
Introduction

This report summarizes the results of reviewing a statistical sample of the record of calls (ROCs) 

for cases given a field disposition of 74, proxy screener, by NORC field interviewers. There were 5,175 

households identified as “proxy” during the data collection and 490 were reviewed. The purpose of the 

review was to ensure that the proper procedures were followed and that there were no indications of 

youths in the households due to improper use of this disposition code. 

During the preset of Round 1, NORC and BLS realized that many households were extremely

hard to reach. Field interviewers were making ten to twenty visits and still finding no one home. NORC 

and BLS needed to find more cost-efficient ways to determine if there were eligible youths in households. 

After the Round 1 pretest, NORC and BLS decided to allow the field interviewers to receive help from

specified neighbors in determining whether youths lived at the sample households. While there was 

worry about misuse of this procedure, this review shows that there were few cases where errors were 

found.

Rule for Using a Proxy Screener 

The field interviewer must have had no contact with anyone in the selected housing unit and have 

attempted to contact the household at least three times (one daytime attempt not on the weekend, one 

evening attempt not on a Friday, Saturday or Sunday, and one weekend attempt), and then the field 

interviewer was allowed to contact a neighbor to determine if the housing unit was occupied, if the 

residents were on vacation, whether they were just hard to find, and when it would be a good time to find

them at home. In addition, the field interviewers were to attempt to determine the household’s 

composition from the neighbor. 

Basically, the purpose of talking to the neighbor and administering the proxy screener was to 

assess whether there were eligible members residing in the sampled household and to determine what 

steps were needed to complete the screener roster. 

In order for field interviewers to complete the CAPI screener with a proxy they had to make sure

that the proxy was a responsible adult, lived either next door to or directly across the street (or hallway)

from the selected housing unit, was aware of the exact number of persons living in the selected housing 

unit, and were positive that there were no persons between the ages of 8 and 28 living in the selected

housing unit. If the field interviewers were unable to locate a proxy who met all of these criteria, they

had to continue attempting to reach and screen the housing unit. 

If the field interviewer learned from the proxy screener that there was no one in the eligible age

range in the sample household, they had to make at least three more attempts to contact and screen the

sample household. If they still could not make contact with anyone in the household, they had to consult 

with their field managers before finalizing the case based on the proxy data and assigning a proxy code. 

However, if the proxy household appeared to contain someone within the specified age range, or

the neighbor did not know for sure if there were youths in the household, the field interviewer was to 

continue his or her efforts to contact the sampled household based on the suggestion of the nieghbor as to 

the best time to reach the household members. If they were still unsuccessful after having done all they

could to contact the household, they were to consult with their field manager to determine if they should 

end their attempts to contact this household. If the field manager determined the case should be finalized, 

the case would have been assigned a final noninterview code, but would not have been given a proxy

code.

Records of Calls

132
NORC uses a Case Management System (CMS) to control the disposition of cases. In the CMS, 

the field interviewers have the ability to enter information about efforts to get an interview. They are not 

required to enter a record of call (ROC) for every attempt, phone call or visit. For example, they may go

by a house several times in one day, but only enter one ROC. Therefore ROCs can’t be used to determine

number of contacts. The field interviewers were not told to enter the number of household members or

whether there were youths. However, many field interviewers did in fact enter information that some

number of youths were in the household, even when they were unable to get a completed interview. They

tended to make statements like, “I saw teenagers coming out of the house” or that a neighbor said there 

were eligible youths in the household. The fact that they did not enter such information does not mean

they made a mistake since there was no requirement to do so, just as there was no request to enter a ROC 

for every contact with the household. It was thought to be too time consuming to enter a ROC for every

visit, since they often stop by a sample address several times a day when they are in the area. Opening 

and closing the case in a laptop computer takes valuable time. In future screenings we would recommend

that they be required to enter any possible information about household members in a note to the ROC

and to enter a ROC for every household contact. This information is very useful when reviewing 

information about the sample.

Sample

The 5,175 proxy screeners were sorted by field interviewers with increasing numbers of proxy

cases. Five samples of approximately 98 cases each were drawn. Starting with case seven, using a 

selection interval of 53, the first sample was selected. This process was repeated until five independent 

samples were drawn. The purpose of drawing five samples was to decrease the work of the person 

reviewing the ROCs. If no problems were found in the first sample, then review of the other samples

might not be necessary. However, the reviewer decided to review all five samples to be certain there 

were no problems. As a result, 490 screener ROCs were reviewed. 

Results of Review

A review of the ROCs for a statistical sample of 490 households given a code 74, proxy screener, 

found that one household had three youths in the 17- to 23-year-old age range (ASVAB-eligible). Two 

households were found that had one or more eligible youths in the 12- to 16-year-old age range 

(NLSY97-eligible). If you extrapolate to the total number of 5,175 proxy screeners, one would find only

11 ASVAB-eligible households and 21 NLSY97-eligible households. Approximately 77 percent of the

sample had specific notes in either the ROCs or the Noninterview Report that there were no eligible 

youths in the household and slightly over 22 percent had no such statement, but there were also no notes

indicating youths were present. Since such a note was not required, one should assume these were done

correctly and there are no youths. Previous reviews of ROCs have indicated that field interviewers are

much more likely to have written a note when there were youths than when there were not. 

Slightly over three percent of the time, the field interviewer gave a disposition of proxy when the 

respondent told her/him that there were no eligible youths. These cases should have been coded as

refusals. If one extrapolates to the full sample, there would have been approximately 169 such cases. 

Likewise, field interviewers accepted statements from housekeepers and babysitters about one percent of 

the time. These should not have been proxy households. 

133
Approximately two percent of the sampled cases should have been codes as vacant, demolished

or some other Type B or C noninterview. 

Most of the time (92.2 percent), field interviewers correctly followed the procedure. In those

cases where procedures were not followed, only 0.6 percent were households with possible youths

present.

Table E-1 Review of Proxy Record of Calls 

Description*

Number of 

Households in 

Sample

Percent of 

Households in 

Sample

100

No kids, definitive statement 343 70.00

No definitive statement, but no kids indicated 109 22.24

Respondent said no one eligible**** 16 3.27

Housekeeper, babysitter said no one eligible 5 1.02

No kids, crash 4 0.82

No kids, should be vacant 8 1.63

No kids, should be Type C (demolished, etc) 2 0.41

No kids, bad address 0 0.00

Possible age 17-23 in household 1 0.20

Possible age 12-16 in household 2 0.41

*Interviewers were not instructed to write any particular type of note in the record of 

calls, so the fact that they did not indicate youths does not mean they erred. 

****Technically these should have been refusals. 

#

Totals may not add due to rounding. 

Total# 490

134
APPENDIX F 

PSU-level Socio-Economic Status Analysis 

135
In this appendix, we discuss how PSU-level screener response and other rates depend on socioeconomic status, as represented in this report by education and income. For each category, we actually

use two different variables (although they are correlated). For education, we use two summary variables: 

percent of PSU adults who are at least high school graduates, and percent of PSU adults who have at least 

a bachelor’s degree. It is important to note that these percentages, taken from the 1990 Census, include 

all persons, male and female, who are at least 25 years of age. Therefore, these percentages are likely to 

be smaller than the percentages would be for just NLSY97/PAY97 parents (e.g., older generations went to 

college less, especially females).

For income, we use the median family income for the PSU and the median household income for 

the PSU. Family income is larger (which seems counter-intuitive if families are subsets of households) 

because single, unrelated persons would be considered separate households, not families (families consist 

of two or more related persons).

The PSU-level NLSY97/PAY97 rates studied are the screener response rate, the percentage of

gatekeeper cases, the percentage of out-of-scope cases, and the percentage of proxy cases. These four 

variables will often be referred to as the dependent variables. The income and education variables will 

often be referred to as the independent variables. 

Education

 As stated above, the two education variables studied are percent of high school graduates and 

percent of bachelor degree recipients. The correlation between the two percentages is .727, but the 

bachelor degree recipients variable seems to have a stronger relationship with the dependent variables, as 

shown by the correlation matrix below: 

Table F-1 PSU-level Socio-Economic Correlations: Education 

PSU

Screener

Response

Rate

PSU

Screener

Gatekeeper

Rate

PSU

Screener

Proxy

Rate

PSU

Screener

Out-ofScope Rate 

Percent High Pearson Correlation -.065 .058 .084 -.331

School Graduate 

or Higher Significance (2-tailed) .361 .412 .237 .000

Percent Bachelors Pearson Correlation -.164 .190 .139 -.358

Degree or Higher Significance (2-tailed) .021 .007 .050 .000

136
Table F-1 shows that the correlations with the independent variables are in the same direction for 

both dependent variables, but that the relationships are much stronger with the percent bachelor=s degree 

variable. The basic relationships seem to suggest that more educated PSUs are less likely to respond, 

more likely to have gatekeeper communities, more likely to need proxy information, and less likely to 

have out-of-scope cases. For PSUs with more high school graduates, these relationships are quite weak, 

except for the out-of-scope rate. This suggests that more educated people live in residential areas (e.g., 

there are few businesses, group quarters, etc.), and that these residential areas probably have fewer

condemned, demolished, and vacant housing units.

With regard to the bachelor=s degree variable, all four correlations are significant, although the 

correlation with the PSU proxy rate is only marginally significant (p=.050). Of particular interest, of 

course, is the relationship with the PSU screener response rate. Here is a scatterplot of PSU screener

response rates by PSU college graduate percentages: 

Figure F-1 Scatterplot of PSU Screener Response Rate by PSU Bachelor Degree 

(or Higher) Rates 

Y = 96.9 - .09 X 

It should be noted that the outlier PSUs with less than an 80 percent screener response rate are omitted

from the above graph so that there can be more differentiation among the other data points. A downward 

pattern seems clear, but the outline shape of the data overstates the relationship because of the increasing

variability among PSU screener response rates as the percentage of bachelor degree recipients rises. A 

simple unweighted regression (line shown in graph) gives the following equation: 

where Y= Screener Response Rate (percentage) 

 and X= Percentage of Bachelor Degree holders 

137
This equation implies that a PSU with 10 percent bachelor degree recipients would have a 96.0 percent 

response rate, as compared to a 94.2 percent response rate for a PSU with 30 percent bachelor degree 

recipients.

Income

As stated above, the two response variables studied are two different measures of PSU median

income: median family income and median household income. A family is defined as two or more

related persons living together. Unrelated single persons living together are considered to be different 

households, and the housing unit is considered to have zero families. Nevertheless, the median incomes

by family or by household are highly correlated. The correlation between the two income variables is 

.987, so there is very little difference in their relationships with the independent variables, as shown by

the correlations below. 

Table F-2 PSU-level Socio-Economic Correlations: Income 

Out-ofScope Rate 

PSU

Screener

Response

Rate

PSU

Screener

Gatekeeper

Rate

PSU

Screener

Proxy Rate 

PSU

Screener

Median Pearson Correlation -.105 .137 .193 -.404

Household

Income, 1990 

Census

Significance (2-tailed) .140 .054 .006 .000

Median Family Pearson Correlation -.103 .126 .198 -.406

Income, 1990 

Census Significance (2-tailed) .148 .075 .005 .000

It can be seen that as income rises, there is a slight drop in the screener response rate, but not a significant 

drop. However, income relates more directly to the proxy and out-of-scope rates than the education 

variables do. While the p-values for the education variable correlations with the proxy rate were not

significant or marginally significant, the p-values in Table F-2 for the proxy rate are very significant. 

Proxies were needed more often in higher-income areas. Also, the correlation between the out-of-scope 

rate and income is -.40, as compared to a correlation of -.35 between the out-of-scope rate and education. 

Just as more educated persons are more likely to live in Astable@ residential areas, these numbers suggest 

that high income persons are even more likely.

138
Figure F-2 is a scatterplot of the PSU screener response rate with median family income.

Figure F-2 Scatterplot of PSU Screener Response Rate by 

PSU Median Family Income

Once again, the outlier PSUs with less than an 80 percent screener response rate are omitted from the 

above graph, and the outline shape of the data overstates the downward trend of the data, as shown by the 

regression line. The simple regression line corresponds to the following equation: 

Y = 97.3 - .6 X 

where Y= Screener Response Rate (percentage) 

 and X= Median Family Income (in $10000s) 

This equation implies that a PSU with a median family income of $20,000 would have a 96.1 percent

response rate, as compared to a 94.3 percent response rate for a PSU with a median family income of

$50,000.

139
Summary

 = 25.9 - 3.96 X 

Differences in socio-economic status do lead to differences in certain screener rates. In particular, 

PSUs with a higher socio-economic status have many fewer out-of-scope cases. Out-of-scope cases 

include condemned and demolished buildings, businesses, unlocatable addresses, group quarters, vacation 

cabins and other non-permanent residences, and transient and vacant buildings. It is not surprising that 

PSUs with a higher socio-economic status seem to have more of a purely residential mix. In fact, the 

linear regression equation involving the median family income and the out-of-scope rate is: 

where Y= PSU Out-of-Scope Rate (percentage) 

and X= Median Family Income (in $10000s) 

This equation implies that a PSU with a median family income of $20,000 would have an 18.0 percent 

out-of-scope rate, as compared to a 6.1 percent out-of-scope rate for a PSU with a median family income

of $50,000. This is a much larger difference than we saw above for the PSU screener response rates. 

This large difference, of course, does not suggest an NLSY97/PAY97 bias, since out-of-scope cases are 

just that . . . out-of-scope, and eliminated from all analyses.

PSUs with a higher socio-economic status did have higher proxy and gatekeeper rates on both 

education and income. This means they were harder to find at home, but these two codes were only to be

used when the information provided was that there were no persons between 8 and 28 years old in the 

housing unit. The eight correlations (two rates by four socio-economic variables) all range between 0 and 

0.2. Neither the proxy nor gatekeeper rates are significantly correlated with percent of high school 

graduates. However, the gatekeeper rate is significantly correlated with percent of bachelor degree 

recipients, and the proxy rate correlation is marginally significant. For the income variables, the

gatekeeper correlations are almost significant (p=.054 for household income and p=.075 for family

income), but the proxy correlations are quite significant (both p < .01). These differentials in the proxy

and gatekeeper rates could lead to bias if significant numbers of eligible youths were missed within proxy

and gatekeeper cases. However, NORC examined 490 proxy cases and 494 gatekeeper cases. Only three 

Apossible@ eligibles were discovered in each sample, making it likely that any possible bias (or 

contribution to the Amissing@ youths problem) due to missed eligibles in proxy and gatekeeper cases is

very small.

For the most important dependent variable studied, there was a decrease in the screener response 

rate in PSUs with higher socio-economic status. However, the only screener response rate correlation that 

was significant was with percent of bachelor degree recipients. In both education and income, the 

difference in screener response rates was about 96 percent for low-SES PSUs, and 94 percent for highSES PSUs. This 2 percent difference is about the same as differences seen between metropolitan and

non-metropolitan areas, and is not much larger than the 1.5 percent difference seen among the four 

Census regions. Therefore, there does not appear to be a serious bias here, although there is certainly a 

slight bias in that it was more difficult to collect screener data from high-SES households. Also, as a final

caution, it should be noted that the correlations could be muted by the fact that we only have PSU-level

data, as opposed to non-response rates for individual households that have different socio-economic

statuses.

140
APPENDIX G 

Comparison of the NLSY97 Family

Income Distribution to the CPS Family 

Income Distribution

141
In order to further explore the possible differences between NLSY97 youths and the general

population of 12- to 16-year-olds, NORC conducted an analysis based on family income. Income was 

examined according to four classifications: (1) wages and salary income, (2) self-employment income

(which includes both non-farm and farm income), (3) total earned income (the sum of wages and salary

and self-employment income), and (4) total income from all sources. The frequencies of NLSY97

families within these income classifications were tabulated overall, for the cross-sectional sample, and by

race/ethnicity. For comparison, the same frequencies were run for families with resident youths aged 12 

to 16 responding to the March 1997 Current Population Survey (CPS), overall and by race/ethnicity.

Framework for Income Analyses. The analyses that follow were conducted at the youth level, 

and were based on family, rather than household, income. NLSY97 data provided detailed income

information for the youths’ parents and parents’ spouses, but not necessarily for all household members.

These family-level income data matched most closely to the CPS family income; thus, the family was

chosen as the basis for the income analysis. However, rather than calculating income distributions across 

families, we viewed family income as a youth characteristic, and then calculated weighted distributions 

across all youths. For the NLSY97, family income was calculated based on the income of the parent or

parents living with the youth. For the CPS, all 12- to 16-year-old youths were selected for analysis, and 

family-level income was calculated for each. 

In the case of the CPS, final person weights were used for computing income distributions. For 

the NLSY97, we used the final youth weights, which combine the cross-sectional and supplemental

samples. A separate weight was used when analyzing the cross-sectional sample alone; this crosssectional weight reapportions the weights across only the cross-sectional sample, giving all supplemental

cases a weight of zero. Note that comparisons have not been made between the supplemental sample and 

the CPS; because the supplemental sample is made up only of Hispanic and non-Hispanic black youths, it

would be unfair to compare their families to the families of all CPS youths. A comparison based on

race/ethnicity will reveal any differences between NLSY97 and CPS minority and non-minority youths.

Both the CPS and NLSY97 surveys ask similar income questions. The CPS asks two questions

about each type of income for which it collects data. First it asks whether any income was received; then 

it asks for the amount (value) of income received. The NLSY97 also asks these two questions; 

furthermore, if the respondent refuses to give the exact income amount, it asks them to select, from a list 

of ranges, one into which their income falls. Due to this variation, the construction of the income

variables for the two surveys entailed slightly different approaches. As mentioned above, the four 

household income variables constructed for comparison purposes were wages and salary income, selfemployment income, total earned income, and total income from all sources. For each income variable in

the CPS, the given income value was used as the raw income, and an additional variable was created to 

group this value into the seven categories used in the tables in this analysis: $1-$5,000; $5,001-$10,000; 

$10,001-$25,000; $25,001-$50,000; $50,001-$100,000; $100,001-$250,000; and more than $250,000.

For the NLSY97, if the income value was given, it was used as the raw income. If instead a range was

given, the midpoint of the range was used as the raw income value. (For example, if the respondent 

selected the range $5,001-$10,000 for a particular income type, they were given a raw value of $7,500 for

that income type.) Finally, an additional variable was created to collapse raw income into the same seven

analysis categories described for the CPS above. 

The raw income values discussed above may, in some cases, be topcoded. Because the NLSY97 

data come from the public use file, the values above a certain threshold are subject to a topcoding scheme,

devised by CHRR, whereby the top two percent of the income values were truncated. Those values 

which exceeded the truncation level were averaged, and each of those cases was given the average value. 

We are also aware that CPS values above a particular threshold are topcoded; however, the method by

which this is accomplished is complicated. The March 1997 CPS documentation contains an explanation 

of their topcoding system.

142
In this analysis, up to two forms of null data were coded as missing. For the first three income

classifications, families which refused to give an income value (or range) had their income coded as 

missing; all other values were considered valid. For total income, not only were refusals coded as 

missing, but so too were zero-income values (i.e., families reporting having received no income

whatsoever). In other words, families that either refused to disclose income amount or that claimed to 

have no income from any income source were given a missing status for that variable. Because both the 

CPS and the NLSY97 surveys ask numerous questions about types of income received, and both have a 

catchall “all other income” question, it is highly unlikely that a family would have no source of income at

all. (Note that public assistance/welfare, social security, and child support are among the income types in 

both surveys.) Thus, while most families do not receive income from all sources, we would also not

expect a family to have received income from no source, and for this reason their income value was 

treated as invalid.

The amount of missing/null data in the two surveys differs quite notably on the surface; NLSY97 

has quite a bit more invalid total income data than the CPS does. (Nearly 22 percent of total income data 

for the families of NLSY97 youths are missing or zero, while only 2 percent of total income data for 

families of CPS youths are null.) However, this can be partially explained by the fact that the CPS

imputes missing data, while the NLSY97 does not. The CPS imputation is two-fold. First of all, if a CPS 

respondent does not indicate whether he or she received income, that indicator variable is imputed.

Second, if a respondent is coded as receiving a particular type of income but gives no value for that 

income type, this value is imputed. Thus, the 2 percent of CPS total income data that, for the purposes of 

these analyses, we consider missing come solely from families that were coded as receiving no income at 

all. Conversely, the 22 percent of income data for the NLSY97 that is considered missing in these 

analyses is made up both of families indicating they received no income (55.5 percent of the missing

cases) and of those who did receive income but refused to provide the amount or a range into which the

amount fell (44.5 percent of the missing cases). In order to further equalize the comparisons between the 

two surveys, we identified imputed total income values in the CPS, removed them from analysis, and ran 

the comparisons based on reported data only20. (Incidentally, only an additional 3.4 percent of the data 

were imputed, resulting in a total 5.4 percent missing/zero total income rate for the CPS.) The section 

describing the overall analysis discusses the results of this additional analysis.

In a further effort to reduce the effects of the differences in the amounts of missing data and the

way they are treated, we have provided two percentage columns in each of the attached tables. The first 

of the percent columns displays overall percentages of youths with families in each income range, with a 

cell indicating the percentage of missing/null income data. The second column shows valid percentages, 

giving the percent of youths with families in each income category based solely on those families with 

valid income data. These valid percentages allow us a more accurate comparison of NLSY97 and CPS 

data.

20 The CPS provides imputation flags for every source of income collected. In order to code cases as imputed or not

for a particular income type, a set of rules was necessary. See the next section, “Construction of the Estimates,” for 

details.

143
While all four income categories are displayed in the tables that follow, we will focus our 

analysis on the final category, total income from all sources. The reader may make other comparisons on 

his or her own; however, we urge caution, especially with the self-employment income category where 

the limited sample sizes cause sampling variability to be enormous.

Construction of the Estimates. In this section, we present a detailed description of how each 

estimate was constructed. As described throughout this document, four variables were constructed for 

comparison between the NLSY97 and the CPS; these variables include wages and salary income, selfemployment income, total earned income, and total income from all sources. Each of these variables was 

created for the NLSY97 and the CPS.

For the NLSY97, up to three income questions created the basis for the final variables used in our

analysis. The first of these asked if the respondent received any income from a particular source. If not, 

no further questions were asked, and the raw income value for that income source was coded as zero. If 

the respondent did receive income, the next question asked for a dollar value for that income type. If the 

respondent provided a dollar value, that value was saved as the raw income value for that variable. If not, 

a final question requested the respondent to provide the range into which his or her income from that 

source fell. If the respondent provided a range, then the midpoint of that range was saved as the raw 

income value for that source. If no range was given, then the raw income value was set to missing for

that source. Next, the sources were combined to form the four variables used for comparison. Wages and 

salary income only includes income from the wages and salary source. Self-employment income includes 

income from both farm and non-farm self-employment. Total earned income includes wages and salary

income and self-employment income. Finally, total income from all sources is the sum of total earned

income and income from interest, AFDC, food stamps, supplemental security, child support, and all other 

income. Once these income values were summed for each of the four income types, the raw values were 

converted to ranges for comparison with the CPS. The eight range categories are: None, $1-$5,000; 

$5,001-$10,000; $10,001-$25,000; $25,001-$50,000; $50,001-$100,000; $100,001-$250,000, and 

Missing. For individual income sources, “None” is a valid value. However, as described earlier, if total 

income from all sources has a value of zero, this is considered invalid. Only in the case where all 

contributing income sources are missing is the actual value for a particular income variable coded as 

missing. (For example, if the wages and salary source had a missing value, then the wages and salary

income variable would be coded as missing. In order for the total income from all sources value to be

missing, though, every one of the sources included would have to be missing.) Tables G-1 through G-8 

and G-17 through G-20 display the income variables for the NLSY97 overall, for the cross-sectional 

sample, and by race/ethnicity.

The CPS variables were constructed in an almost identical manner, with only a few minor

differences. First of all, while the NLSY97 had three potential questions for each income source, the CPS 

had two. The respondent was asked whether he or she received income from a particular source, and 

then, if so, what the dollar value of that income was. There was no range question asked in the CPS.

However, as described earlier, in the case of missing data, the values were imputed. Thus, if the 

respondent did not report whether he or she received income from a particular income source, that 

variable was imputed. If the imputed value indicated that income was received, then the income (dollar) 

value was also imputed. Likewise, if the respondent did report having received income from a particular 

source but the actual value was not reported, then that value was imputed. (For most of the analyses that 

follow, CPS data using imputed values for missing data was used.) Finally, the four comparison variables

were constructed just as the NLSY97 variables were constructed, converting all raw values into the eight 

range categories. 

144
In order to conduct an analysis based on reported CPS data only (excluding all imputed

information), we constructed four additional variables for the CPS. Because there were many ways in

which some of the income types could be classified as imputed, we constructed a set of rules for this 

classification procedure. The following were the rules we used in identifying imputes for each income

variable. For the first three income types: if the portion which made up the majority of that income type

was imputed, then the case was coded as imputed. (For example, if total earned income was made up of 

60 percent wages and salary, and 40 percent self-employment income, and if the wages and salary income

value was imputed, then the total earned income variable would be classified as imputed. If the wages

and salary income was not imputed, but the self-employment income was, this case would not be 

classified as imputed.) For total income from all sources, the process was two-fold. If the majority of 

total income was earned, then if the portion which made up the majority of the total income was imputed,

the case was coded as imputed. Otherwise, if the majority of income was not earned, then if one or more

types of unearned income was imputed, the case was coded as imputed. (For example, if the majority of

income was earned and came from wages and salary, and wages and salary was imputed, then the variable

would be coded as imputed. Further, if the majority of income was not earned, but income that was

received from an unearned source such as public assistance was imputed, then the variable would be 

coded as imputed.)

This calculation of the variables excluding imputed values for the CPS resulted in only 3.4 

percent of the data being coded as imputed. This, in addition to the 2 percent of CPS families which 

reported no income from any sources results in a total 5.4 percent missing or invalid total income rate. As 

a note of warning, though, please be aware that other authors have cited much higher imputation

percentages for the CPS; these authors have considered an income value for which any portion was

imputed to be imputed, while we have used a majority rule. 

Overall Analysis. The comparison data for CPS and NLSY97 are summarized in tables G-1

through G-24. The results of these comparisons indicate that, overall, there are some differences between 

the NLSY97 family income distributions and the CPS family income distributions; however, these

differences are most obvious in the race/ethnicity (particularly Hispanic) breakouts.

Tables G-1 through G-4 and G-9 through G-12 display the NLSY97 (full sample) and CPS 

youths’ family income distributions for each of the four income categories. In comparing these 

distributions, there are small but significant differences between the two groups. For total income (tables 

G-4 and G-12), both NLSY97 and CPS families tend to be concentrated near the center of the income

distribution, the majority falling into the $25,001 to $50,000 and $50,001 to $100,000 ranges. However, 

NLSY97 has more families than CPS in the lowest income categories, slightly more than CPS in the

middle ranges, and fewer than CPS at the higher income levels. The following graph illustrates the 

similarities and differences in total family income between NLSY97 and CPS. 

145
Figure G-1 Percentages of Youths within 

Income Categories, CPS vs. NLSY

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

35.0%

Total Income

CPS

NLSY

While significance testing is generally inappropriate for sample sizes of this magnitude due to the 

tendency for even small differences to become greatly exaggerated, and although the criteria for a formal

test, including simple random sampling, are not met here, a t-test was performed to compare the means of 

the NLSY97 and CPS distributions and their variances. At this writing we have not yet estimated the 

design effects for the NLSY97 sample, but we are certain that they will be greater than one. The t-test 

calculated here incorporates a design effect borrowed from the full NLSY79 sample of youths (deff

approximately equal to 2). The results of the test show NLSY97 with a significantly lower mean total 

income than the CPS. The families of NLSY97 youths have a mean total income of $47,199, while 

families of CPS youths have a mean total income of $53,967. The t-value associated with the difference 

in means, taking into account the design effect, is 5.1, which is significant at the 0.01 level; still, the

variances of these distributions were found to be equal. We must emphasize that these samples are quite 

large, and with any sample large enough many differences appear to be significant. However, regardless 

of the inadequacy of this test under these circumstances, it does give at least some evidence that these 

distributions are not identical. 

A second analysis was conducted to compare the raw CPS income distribution to the NLSY97

distribution by removing all imputed CPS income values. Tables G-13 through G-16 display these

revised weighted CPS income distributions. A second t-test was also run in order to compare the revised 

mean total income to the NLSY97 mean total income. As mentioned earlier, only 3.4 percent of CPS 

cases had total income data imputed, and an additional 2 percent claimed to have received no income at 

all; thus, a total of 5.4 percent of the CPS cases were left out of this analysis. The results of the t-test 

were almost identical to the results seen above. The NLSY97 again displays a significantly lower mean

income ($47,199 compared to $54,637 for the CPS), and the t-value associated with this difference was 

5.6, also significant at the 0.01 level. Because these results are so close to the original analysis for total 

income, we have not conducted any other analyses based solely on reported CPS data. 

Cross-sectional Analysis. The NLSY97 cross-sectional sample behaves similarly to the overall

sample. Tables G-5 through G-8 display cross-sectional income distributions for the four income

146
categories, and valid percentages are almost identical to those of the overall sample. As an example, table 

G-8, which displays total income percentages for the cross-sectional sample, parallels the overall sample

in its comparison to the CPS total income distribution: families of NLSY97 cross-sectional youths are 

more prevalent in the lower income categories (11.7% of families of NLSY97 youths earn between $1

and $10,000 compared to 9.2% of families of CPS youths), similarly distributed in the middle income

groups (62.0% of NLSY97 youths’ families and 62.5% of CPS youths’ families fall into the $25,001-

$100,000 range), and less prevalent in the upper income categories than CPS (7.2% of NLSY97 youths’

families earn more than $100,000, compared to 9.4% of CPS youths’ families).

Race/Ethnicity Analysis. Race breakouts, displayed in tables G-17 through G-24, show a fairly

consistent picture, across variables, between NLSY97 and CPS. Hispanics have the most notable 

differences; NLSY97 has more families than CPS in the lowest income extreme (9.6% of NLSY97 

Hispanic youths’ families vs. 4.6% of CPS Hispanic youths’ families earn $5,000 or less), more in the 

upper-middle and upper ranges (for NLSY97, 22.4% earn more than $50,000, compared to 16.6% for

CPS), and fewer in the lower and middle (68.0% of NLSY97 Hispanic youths’ families earn between

$5,001 and $50,000, versus 78.7% of CPS Hispanic youths’ families). Family incomes of NLSY97 black 

youths are distributed approximately the same as for the CPS, with slightly more at the lower end (10.7% 

in the $5,000 or less range for NLSY97, compared to 8.3% for CPS) and slightly fewer at the higher end 

(2.2% in the over $100,000 range for NLSY97, compared to 3.1% for CPS). Finally, NLSY97 families of 

non-Hispanic, non-black youths were also very similar to their CPS counterparts. While more do fall into 

the lower income groups (8.3% earning less than $10,000 for NLSY97, compared to 5.5% for CPS) and 

slightly fewer into the higher income groups (8.9% earning more than $100,000 for NLSY97 versus 

11.9% for CPS), the majority are distributed similarly among the middle income groups. 

Summary. Given sampling variability and definitional and procedural differences, these 

comparisons are encouraging. The CPS and NLSY97 total income distributions have equal variances, 

and although the test conducted on the overall sample does show a significant difference between the 

mean incomes of the two populations, it does not necessarily indicate an important discordance; sample

sizes of this magnitude very frequently result in significant results for even minor differences. 

Furthermore, the income distributions do not look considerably different to the naked eye. The most

visible distinction may be that the NSLY97 distribution looks as if it were the CPS distribution shifted 

slightly to the left. The results of these analyses were also consistent across sample subdivisions; findings

were similar by race/ethnicity and for the cross-sectional compared to the overall sample. Thus, in 

closing, while this study does find significant differences between the means of the total income

distributions, individual users must judge for themselves whether this difference is substantively

important.

As one additional note of warning, we would like to draw attention to the “None” category for 

each of the income types, specifically for total earned income. According to tables G-3 and G-11, nearly

21 percent of the NLSY97 sample responded that they did not earn any income that year, compared to

only 8 percent of CPS cases. On common sense grounds, it seems quite unlikely that such a large 

percentage of the population would be living on interest and dividends or public welfare. Thus, we 

suggest that the “None” category may in some cases be another form of refusal; some respondents may

say that they received no income in order to move through the interview more quickly. In comparing

tables G-3 and G-11, one can see that the major difference between the CPS and NLSY97 samples lies in 

this category; the remaining income category percentages are much closer to each other. Thus, users of

our dataset must be aware of these “None” cases and have a strategy in place for dealing with them.

147
148
149
150
151
152
153
154
155
APPENDIX H 

Introduction and Paper Screener

156
NLSY97 SCREENER INTRODUCTION

Good morning/afternoon/evening!

My name is __________, and I am in your neighborhood today working on a national study for the 

United States Department of Labor, Bureau of Labor Statistics. I am a field interviewer for the National

Opinion Research Center at the University of Chicago (show identification). I understand that you

received a letter about this important study a few days ago? (HAVE A COPY OF LETTER AND 

BROCHURE READY TO PRESENT IF RESPONDENT INDICATES LETTER NOT

RECEIVED; ALLOW A MOMENT IF HE/SHE WANTS TO GLANCE AT THEM.)

Now, before I start the interview, I’d like to tell you some important information about this study.

- any others who are staying here such as foster children 

- and any babies or small children 

This study is sponsored by the U.S. Department of Labor, Bureau of Labor Statistics, under authority of 

Title 29 USC 2. Your participation is vital to the success of the study and is voluntary. This survey is

conducted to understand the labor market experience of young adults. This information will be used by

BLS and other researchers. The information you provide is protected under the Privacy Act. Personal 

identifiers, such as name, address, or Social Security numbers will be held in confidence and will not be 

released to the public. 

1. First I’d like to do a 3 minute interview to determine if any persons in this household are eligible 

for the survey. Let’s begin by asking a few questions about the people who usually reside in this 

household, including persons: 

- who are away at school or college 

- lodgers, boarders, or persons in your employ who live here 

- any others who usually live here but are away at present, such as someone travelling, or someone

in a hospital, correctional facility or any other type of institution 

START PAPER SCREENER. GET NAME FOR EACH HOUSEHOLD MEMBER. 

2a. Could you tell me the names of the people who usually live here, beginning with the eldest? 

(ENTER NAMES, INCLDING NICKNAMES/INITIALS IN COLUMN 1. CIRCLE NAME OF THE 

HOUSEHOLD INFORMANT.) 

2b. I have the following people listed as living here (READ NAMES IN COLUMN 1.) 

 Have I missed anyone? (ADD NAMES.) 

NEXT, GET DATE OF BIRTH FOR EACH HOUSEHOLD MEMBER: CONTINUE 

WITH Q.3 ON REVERSE SIDE.

- OVER -

157
3. Now I would like to ask you about the ages of the people you have just mentioned.

3a. What is ____________ date of birth? (READ FIRST/NEXT NAME) 

 (ENTER D.O.B. IN COLUMN 2–ENTER EVEN PARTIAL INFORMATION SUCH AS THE 

 YEAR OF BIRTH–AND PUT AN “X” IN COLUMN 4, 5, OR 6 AS APPROPRIATE.) 

 ASK 3b. IF DATE OF BIRTH INFORMATION IS INCOMPLETE FOR THAT NAME. 

 OTHERWISE, REPEAT 3a. FOR NEXT NAME, THEN GO TO 4 WHEN D.O.B.

 INFORMATION IS COMPLETE FOR ALL NAMES. 

3b. What is __________ age? (READ FIRST/NEXT NAME) (ENTER AGE ON THE

 APPROPRIATE.) 

 -- IF THE ANSWER TO 3b. IS 17......................................................GO TO 3c. 

 - 32 to 40 years old? ..........(IF YES, WRITE “32-40” IN COLUMN 3) 

GO BACK TO 3a. UNTIL D.O.B. INFORMATION IS COMPLETE FOR ALL NAMES, THEN GO TO 4 

4. FOR EACH PERSON WITH AN “X” IN COLUMN 6, ASK: 

 (CIRCLE THE Y, N, OR DK AS APPROPRIATE IN COLUMN 7.) 

CHECK FOR ELIGIBLE HOUSEHOLDS

 APPROPRIATE LINE IN COLUMN 3 AND PUT AN “X” IN COLUMN 4, 5, OR 6 AS

 -- IF THE ANSWER TO 3b. IS ANY AGE OTHER THAN 17....................GO TO 3a. 

-- IF THE INFORMANT DOES NOT KNOW AGE OR HESITATES............GO TO 3d.

3c. Has _______________ (READ FIRST/NEXT NAME) had a birthday since January 1st 1997? 

 (MARK “Y,” “N,” OR “DK” NEXT TO THE NUMBER 17 IN COLUMN 3, AND GO TO 3a.) 

3d. Would you say that ___________ (READ FIRST/NEXT NAME) is 12 to 17 years old? 

 (IF YES, WRITE “12-17” IN COLUMN 3, AND MARK AN “X” IN COLUMN 4; IF NO ASK) 

 Would you say that ___________ (READ FIRST/NEXT NAME) is 18 to 23 years old? 

 (IF YES, WRITE “18-23” IN COLUMN 3, AND MARK AN “X” IN COLUMN 5; IF NO ASK) 

 Would you say that ___________ (READ FIRST/NEXT NAME) is

 - under 12 years old?..........(IF YES, WRITE “UNDER 12” IN COLUMN 3) 

 - 24 to 31 years old? ...........(IF YES, WRITE “24-31” IN COLUMN 3) 

 - 41 or older?....................(IF YES, WRITE “41 OR OLDER” IN COLUMN 3)

LIVING IN THE U.S. QUESTION

 (Were/Was) you/name living in the United States in 1978? 

5a. INTERVIEWER: IF THERE IS AN “X” MARKED IN COLUMN 4 OR COLUMN 5, SAY:

It appears that members of your household are eligible to participate in our study. I will need to enter some

information into my laptop computer, and continue the interview. May I come in to do that or if you prefer, I will 

proceed from out here. TRY TO COMPLETE FULL SCREENER ON CAPI. 

5b. INTERVIEWER: IF THERE IS NO “X” MARKED IN COLUMN 4 OR COLUMN 5, SAY: 

It does not appear that anyone in your household will be eligible to participate in this study, but I would like to 

thank you for taking the time to speak with me. My supervisor randomly checks my work, and someone MAY call 

to verify that I was actually here. Could I please get your telephone number for that verification call? (ENTER 

PHONE NUMBER ON PAPER SCREENER FORM.) Thank you again. Goodbye.

158
Date ___________________________________ 

Time ___________________________________ 

NLSY PAPER SCREENER

HOUSEHOLD ID # 

PSU: _______________ FI NAME _____________________

SEG: _______________ FI ID#: _______________________

 ________________________________________ 

(4) (5)

LINE#/ADDRESS: ________________________________________ 

CITY/STATE/ZIP: ________________________________________

(1) (2) (3) (6) (7)

Name: First/Last 

Interviewer: Circle

Household information’s 

Name)

DATE OF 

BIRTH

(D.O.B)

AGE Interviewer: Put

an X in box if 

D.O.B between

1/1/80 and 

12/31/84 or age 

12 through 17 

Interviewer: Put 

an X in box if 

D.O.B between

6/1/73 and 

12/31/79 or age 

18 through 23 

Interviewer: Put 

an X in box if 

D.O.B between

1/1/57 and 

12/31/64 or age 

32 through 40 

Interviewer: If X in 

column 6 ask: 

“(Was/were)

you/name living in the 

United States in 

1978?

(Circle Response) 

Y N DK 

Y N DK

Y N DK

Y N DK

Y N DK

Y N DK

Y N DK

Y N DK

Y N DK

APPOINTMENT DATE: ________________________ TIME: _____________ TELEPHONE NUMBER: ________________ 

Y N DK

OTHER INFORMATION: __________________________________________________________________________________ 

159
APPENDIX I 

Introduction and Age Questions from 

Extended Screener

160
161
162
163
164
165
166
APPENDIX J 

Proxy Introduction and Paper Screener 

167
168
169
APPENDIX K 

Advance Letters and Brochure

170
171
172
173
174
175
176
APPENDIX L 

Review of Gatekeeper Interviews 

177
Introduction

This report summarizes the results of reviewing a statistical sample of the records of calls (ROCs) 

for cases given a field disposition of 71, “gatekeeper,” by NORC Field interviewers. There were 4,055 

households identified as “gatekeeper” during the data collection and 494 were reviewed. The purpose of 

the review was to ensure that the proper procedures were followed and that there were no indications of 

youths in the households due to improper use of this disposition code. 

During the pretest of Round 1, NORC and BLS realized that many households were extremely

hard to reach, especially those in multi-unit buildings, locked buildings or gated communities. In 

addition, it was found that many of these types of units were housing for seniors or that in order to live 

there household occupants had to be at least 55 years of age and children were not allowed to live there as 

residents. Field interviewers made multiple visits and still were unable to get past security guards, call 

boxes and building managers; hence, we used the term gatekeeper. NORC and BLS needed to find more

cost-efficient ways to determine if there were eligible youths in these households. After the Round 1 

pretest, NORC and BLS decided to allow the field interviewers to receive help from gatekeepers (persons 

who blocked the field interviewer from talking to a respondent) in determining whether any youths lived 

at the sample households. While there was worry about misuse of this procedure, this review shows that 

there were few cases where errors were found.

Rules for Using the Gatekeeper Code

To use the gatekeeper code, the field interviewer must have had no contact with anyone in the 

selected housing unit. They were to try everything possible to get into the building. However, if a

building manager or other building gatekeeper absolutely refused entrance, and the building manager told 

them it was a “seniors” building or some other such statement, the field interviewer had to get permission

from their field manager before assigning the gatekeeper code. 

However, if the field interviewer had any indication that there were youths within the specified 

age range, or the gatekeeper did not know for sure if there were youths in the household, the field

interviewer was to continue his or her efforts to contact the sampled household. If they were still 

unsuccessful after having done all they could do to contact the household, they were to consult with their 

field manager to determine if they should end their attempts to contact this household. If, by talking to

the field interviewer, the field manager determined that nothing else could be done and the case should be 

finalized, the field manager would assign a final noninterview code. 

178
Records of Calls

NORC uses a Case Management System (CMS) to control the disposition of cases. In the CMS, 

the field interviewers have the ability to enter information about efforts to get an interview. They are not 

required to enter a record of call (ROC) for every attempt, phone call or visit. For example, they may go

by a house several times in one day, but only enter one ROC. Therefore ROCs can’t be used to determine

number of contacts. The field interviewers were not told to enter the number of household members or

whether there were youths. However, many field interviewers did in fact enter information that some

number of youths were in the household, even when they were unable to get a completed interview. They

tended to make statements like, “I saw teenagers coming out of the house,” “a neighbor said there were

eligible youths in the household,” or “a neighbor says only an 80-year old woman lives there.” The fact 

that they did not enter such information does not mean they made a mistake since there was no 

requirement to do so, just as there was no requirement to enter a ROC for every contact with the 

household. It was thought to be too time consuming to enter a ROC for every visit, since they often stop

by a sample address several times a day when they are in the area. Opening and closing the case in a 

laptop computer takes valuable time. In future screenings we would recommend that they be required to 

enter any possible information about household members in a note in the ROC and to enter a ROC for 

every household contact. This information is very useful when reviewing information about the sample.

Sample

The 4,055 gatekeeper screeners were sorted by field interviewers with increasing numbers of 

gatekeeper cases. Five samples of 98 or 99 cases each were drawn. Starting with case one, using a

selection interval of 41, the first sample was selected. This process was repeated until five independent 

samples were drawn. The purpose of drawing five samples was to decrease the work of the person 

reviewing the ROCs. If no problems were found in the first sample, then review of the other samples

might not be necessary. However, the reviewer decided to review all five samples to be certain there 

were no problems. As a result, 494 screener ROCs were reviewed. 

Results of Review

A review of the statistical sample of 494 households given a code 71, gatekeeper, found that one 

household had possible youths in the 17- to 23-year-old age range (ASVAB eligible). Two households 

were found that had one or more eligible youth in the 12- to 16-year-old age range (NLSY97-eligible). If 

you extrapolate to the total number of 4,055 gatekeepers, one would estimate eight ASVAB-eligible 

households and 16 NLSY97 households. Slightly over 69 percent of the sample had specific notes in

either the ROCs or the Noninterview Report that there were no eligible youths in the household and 

approximately 29 percent had no such statement, but there were also no notes indicating youths were 

present. Since such a note was not required, one should assume these were done correctly and there are

no youths. Previous reviews of ROCs have indicated that field interviewers are much more likely to have 

written a note when there were youths than when there were not. 

Slightly over 19 percent of the time, the field interviewer gave a disposition of gatekeeper when, 

in fact, a household respondent told her/him that there were no eligible youths. These cases should have

been coded as refusals. If one extrapolates to the full sample, there would have been approximately 788 

such cases.

Slightly over one percent of the sampled cases should have been coded as vacant, demolished or

some other type B or C noninterview. 

179
The majority of the time (78.5 percent), field interviewers correctly followed the procedure. In 

those cases where procedures were not followed, only 0.6 percent were households with possible youths

present.

Table L-1 Review of Gatekeeper Record of Calls 

Description*

Number of 

Households in 

Sample

Percent of 

Households in 

Sample

Total 494 100

No kids, definitive statement*** 240 48.58

No definitive statement, but no kids indicated*** 148 29.96

Household respondent said no one eligible**** 96 19.43

Housekeeper, babysitter said no one eligible 0 0.00

1

4 0.81

1 0.20

1 0.20

1

2 0.40

*Interviewers were not instructed to write any particular type of note in the record of calls, so the

***About 2-3 percent of these should have been a proxy screener and not a gatekeeper. Even

though it was generally in an apartment building, they went to a neighbor, so therefore had gotten 

in the building. The gatekeeper code was reserved for inability to get into a building or 

community due to locked doors. 

No kids, crash 0.20

No kids, should be vacant 

No kids, should be Type C (demolished, etc.) 

No kids, bad address 

Possible age 17-23 in household 0.20

Possible age 12-16 in household 

 fact that they did not indicate youths does not mean they erred. 

****Technically these should have been refusals. Once a field interviewer talked to a household 

respondent, it was no longer a gatekeeper case. 

180
APPENDIX M 

NIR Reports, NLSY97 Round 1

181
1. Introduction21

This report summarizes analyses of Non-Interview Respondent (NIR) Report data for households 

selected for NLSY97 screening. NIR reports were used during the Round 1 screening effort to document

screener cases in which interviews were not completed during the screener field period from January

through October of 1997. NORC Field Interviewers finalized a case as an NIR only with Field Manager 

permission. Out of 90,957 sampled housing units, NORC completed screeners with 75,410, including 

screeners completed with household respondents or with gatekeepers and proxy respondents. NIR reports 

were completed for 13,810 of the 15,547 non-interviewed housing units. Among these 13,810 NIRs are

10,436 units which were either vacant or uninhabitable at the time of the screening interview. Thus, the 

NIR reports offer information on 3,374 sampled housing units that were eligible for the screener but were 

finalized as non-interviews. The purpose of the NIR Report analysis is to obtain some impressions of the 

non-interview housing units in our screening sample, to highlight possible trends among screener 

nonrespondents, and to identify potential sources of bias in our youth sample. Figure M-1 shows the

number of households that completed screeners, NIR reports, and the reasons they provided for noninterviews.

21 Final updating of the NLSY97, Round 1 disposition codes was done in November 1999, in light of all evidence 

on eligibility collected to that point in time, including certain evidence collected during Round 2 operations. 

Because the tabulations cited in this appendix reflect an earlier generation of the Round 1 disposition codes, there 

may be small discrepancies between counts cited here and the final counts cited in the main body of this NLSY97

Technical Sampling Report.

182
Figure M-1 Number of Cases: Screeners and Non-interviews 

Sampled Housing Units

90,957

Did not complete screener

15,547

Completed Screeners 

75,410

Includes proxy and 

gatekeeper cases 

Completed NIR reports No NIR report

13,810 HUs 1,737 

Vacant/

Uninhabitable Non-interviews,

eligible for

screener

HUs

10,436 3,374 HUs

Refusals 1,556 

Technical Problems 610 

Inaccessible 144 

Language Barrier 178 

Ill/ Handicapped 15 

Field period ended/ R

 never home 871

Vacant, under 

construction

7,417

Not usable as a 

residence 443

Business 231

Not a permanent 

residence 2,435

183
2. Purpose of Non-Interview Report 

For every sample housing unit, Field Interviewers (FIs) attempted to complete a household 

screener -- a roster of all usual residents of the sampled address, including their age, sex, race and

ethnicity. When potentially eligible NLSY97 youths or PAY97 youths were identified on the basis of age

and ethnicity data, the Field Interviewer then attempted to complete an Extended Screener, which

confirmed youth eligibility by collecting school enrollment information and gathered additional

background information about the household members. Finally, the Field Interviewer attempted to

complete a parent and youth interview or secure test participation. 

3.1 The Instrument 

When a household refused to participate in a screener interview or a screener was not completed

for some other reason, the project failed to identify and interview eligible youths. Even impressionistic

data about omitted households can be valuable in assessing the screener effort. The NIR report data are 

important for identifying potential sources of bias in our sample that we can then investigate more

rigorously.

3. NIR Data Collection 

The NIR was administered using computer-assisted data entry technology (CADE) and was 

completed by the Field Interviewer at home after a case had been finalized as a non-interview. The 

CADE instrument comprises three main sections, addressing the following questions: (1) Why were we 

unable to complete a screener? (2)What type of households were non-interviews? (3) How many

potentially eligible youths were missing from our sample? Table M-1 lists questionnaire items for each of 

these main questions. 

184
Table M-1 NIR Sample Questions Divided Into Three Main Sections

(1) Why were we unable to complete a screener?

& What is the reason for NIR?

& If refusal, what methods were used to convert the respondent? 

& Did the respondent / informant ask any of the following questions? 

& Did the respondent / informant make any of the following comments?

(2) What type of households were non-interviews? 

& What kind of housing unit is this? 

& How would you describe the immediate area or street (one block, both sides) where this 

housing unit is located? 

& What type of residence is most common on the street (one block, both sides) where this 

housing unit is located? 

& How well kept are most of the buildings on the street where this housing unit is located? 

& How well kept is this housing unit? 

& What is the sex of the household member / informant?

& What is the race / ethnicity of the informant / respondent? 

(3) How many potentially eligible youth were missing from our sample? 

& How many people do you think live in this housing unit? 

& How did you determine the number of people living in this housing unit? 

& Do you think anyone between the ages of 10-25 years of age lives here?

& How many people between the ages of 10-25 years live here?

& Do you think any are 17-25 years of age?

& How many people between the ages of 17-25 years live here?

& How did you determine this? 

3.2 Respondent vs. Informant

Respondents were the people who were eligible to complete a youth or parent questionnaire or the 

household screener (and as appropriate, the extended screener). A screener respondent was someone 18

years of age or older who usually resided in the household and who was a responsible adult. However, if 

there was no such respondent available at the household, an interviewer could have spoken to an 

informant outside of the household who was a responsible person 18 years of age or older and who was 

familiar with the household. Typical informants outside the household would be visitors to the household,

neighbors who live next door or directly across the street, the postal carrier or the doorman for the

building. Informants were sought by field staff to assist in securing the cooperation of respondents, 

complete proxy or gatekeeper reports, or provide information for the NIR report. 

The questions in the NIR regarding reasons for non-interview, housing units, neighborhood, and

number of household members or youths reveal information about respondent(s) only. However, the 

remaining items regarding demographics and comments or concerns about the survey refer to either an 

informant or a respondent, depending on who spoke with the interviewer. Details on respondents and 

other informants can be found in the body of this report. 

185
4. Rules for Completing a Non-Interview Report

Field interviewers were required to get the permission of a Field Manager before finalizing a case 

as non-interview. The rules for completing an NIR differ according to whether the interviewer spoke 

with a household respondent or non-household informant.

4.1 Household Informant / Respondent

To finalize a case as non-interview for a Household Respondent, the Field Interviewer must have 

contacted a household member and must have attempted to complete a screener. For example, the

Household Informant may have (1) refused to complete the Household Screener (including breaking off 

partway through the household screener interview) (2) completed the household screener but refused to 

participate in the extended screener when potentially eligible youth were identified. If the respondent 

refused, then the Field Interviewer attempted a conversion. After repeated unsuccessful conversion

attempts, a Field Manager would designate the case as a final screener non-interview. Youths were only

spawned when the entire screener was completed.

4.2 Non-Household Informant

If no contact was made with a household respondent, then the Field Interviewer sought to obtain a 

proxy screener or a gatekeeper screener. These were data collection alternatives when the residents of the

housing unit were exceedingly difficult to locate and were used only to identify a housing unit as having 

no potentially eligible youths or as a source of NIR Report but not of screener data.

A case was assigned a proxy screener code when an acceptable proxy respondent was able to

definitively state that no household members were between ages 8 and 28 years. When a proxy screener

could not provide this confirmation, then the interviewer continued to seek a screener respondent from the 

sampled unit. If still no screener respondent was available, the cases were coded as final non-interview if 

permitted by the Field Manager.

A case was assigned a final gatekeeper code when the interviewer was blocked from talking to a 

respondent and was able to obtain residential eligibility documents which confirmed that no youths aged 

8 to 28 lived there. When eligibility documents did not provide this confirmation, the Field Interviewer 

continued to attempt screening. If no screener respondent was available, the case was coded as a final 

non-interview if permitted by the Field Manager. 

Even if a Field Interviewer was unable to obtain an acceptable proxy screener or gatekeeper, a

Non-Household Informant might have been sought to assist in the completion of an NIR report or to gain

cooperation of the household. Such informants could not be used to complete a Screener and they were 

only obtained for providing information on the NIR when permitted by a Field Manager. Although the 

informant’s information was used in the NIR report, the screener case itself was designated a ‘refusal’ or 

as some other type of non-interview and we did not use the informant’s information in identifying the 

sample or in our analyses of screener data. 

186
5. Results of the NIRs

5.1 Why Were We Unable to Get an Interview?

After gaining the permission of a Field Manager to finalize as non-interview, Field Interviewers 

gave a final disposition code to the case to explain the main reason they were unable to obtain an 

interview. For over 75 percent of the 13,810 NIR reports, Field Interviewers were not able to complete a 

screener because the sampling unit was not a household unit or because it was a vacant unit--3 percent not 

usable as a residence, 2 percent business address, 17 percent not permanent residence, and 54 percent 

vacant or under construction. The next most frequent reason for no interview is a refusal (including 

break-offs or partial screeners)--11 percent of non-interview housing units resulted because a Household 

Respondent refused to complete a screener. Table M-2 shows the distribution of reasons for an NIR. 

Table M-2 Reasons for a Non-Interview 

Description Number of 

Households

with NIR data

Percent of 

Households

with NIR data

Technical Problems 610 4.4

Field Period Ended / Respondent never home 871 6.3

Not Usable As Residence

7,417

443 3.2

Business--not a household unit 231 1.7

Not a Permanent Residence (i.e. Vacation home) 2,345 17.0

Vacant / Under Construction 53.7

Household Unit Inaccessible 144 1.0

Language Barrier Cases 178 1.3

Respondent too ill / handicapped 15 0.1

Refusal 1,556 11.3

Total 13,810 100.0

5.2 Refusals

First we consider the subset of NIR reports where refusals were the reason for non-interview. 

There were 1,556 such households. When a respondent refused, Field Interviewers attempted to “convert” 

the respondent and secure permission to complete a household screener and an interview when 

appropriate. Conversion methods included sending a refusal letter or requesting a trained converter to 

contact the respondent/informant by phone or in person. Table M-3 shows a comparison of conversion 

methods and some general concerns expressed by the respondent/informant for households with 

potentially eligible youths between the ages of 10 and 25 and for households without potentially eligible 

youths.

187
Table M-3 Comparison NIR Reports Due to Refusals (1556 

Households Out of Which 789 are Missing Data on the Number of Potentially Eligible Youths) 

Description

Percentage of Households 

With No Potentially 

Eligible Youth (91)

Percentage of Households 

With Potentially Eligible 

Youth (676)

Method of Conversion

Refusal Letter Sent 

(74 Households, 17 missing)

14.9

5.4

Attempt By Phone Converter 

Attempt By Personal Converter 

Attempt By Supervisor 

Not Applicable 

Other

54.1

8.1

64.9

2.7

10.8

(607 Households, 69 

missing)

61.9

20.6

69.7

9.2

6.9

Respondent/Informant Comments 

I’m too busy / survey too long 

I don’t like / do surveys

Don’t know anything about this 

Not the sort of person you want 

I’m not interested 

This is too personal 

Survey is a waste of money

Government has no business 

Government already knows this 

Don’t want children bothered 

Let me think about it 

Negative comment or delay

None of the above 

(78 Households, 13 missing)

17.9

28.2

3.8

1.3

41.0

7.7

3.8

5.1

1.3

7.7

0.0

12.8

29.5

(598 Households, 78 missing)

29.4

20.9

7.4

4.5

52.5

11.0

4.7

10.4

6.5

27.6

6.9

16.9

12.0

Did the informant ask... 

What is the purpose of the survey?

Who is the sponsor of the survey?

How was I chosen?

(78 Households, 13 missing)

0.0

0.0

28.7

2.5

How long will the interview take? 

Who sees answers?

Can I get a copy of the results? 

Is there an incentive?

Informational question or

expression of interest?

9.0

2.6

3.8

2.6

0.0

1.3

(592 Households, 84 missing)

35.8

22.3

15.4

9.3

0.5

1.4

Overall, Table M-3 shows that Field Interviewers made additional efforts to complete a screener 

where youths were thought to be present. Of households with potentially eligible youths, 62 percent 

received a refusal letter and 21 percent received a call by a phone converter compared to 54 percent 

refusal letters and 8 percent phone converters for households without potentially eligible youths. Of 

households with potentially eligible youths, 28 percent of respondents commented that they “don’t want

children bothered” compared to 8 percent of respondents in households with no such youths. (Note that 

households without potentially eligible youths could include younger children as household members.)

188
An effective tactic for gaining cooperation is to engage the respondent in conversation. 

Comparing questions asked by the informant/respondent suggests that interviewers had more extensive

communication with informants when there were potentially eligible youths present than in other

households. For example, many more households with youths wanted to know the purpose of the survey

(36 percent), the sponsor of the survey (22 percent), how he/she was chosen (29 percent), the length of the 

interview (15 percent) and who sees the answers (9 percent) compared to households without youths-- 9 

percent wanted to know the purpose, 3 percent asked about the sponsor, 4 percent asked how he/she was

chosen, 3 percent asked about the length, and 0 percent wanted to know who sees the answers.

5.3 What Type of Households Were Non-Interviews?

The racial/ethnic composition of the non-interview housing units is similar to the national 

composition. Of the informants/respondents, 67.3 percent are Non-Hispanic White, 0.2 percent American

Indian, 7.9 percent Asian/Pacific Islander, 8.3 percent Hispanic, 14.2 percent Black/African-American,

and 2.0 percent other. The most prominent difference is that the Asian/Pacific Islander population is 

over-represented in the total NIR’s (7.9 percent), mirroring a tendency toward higher non-response

among Asian Americans in general population surveys. The majority of informants were female.

When we compared households on the basis of whether they contained potentially eligible youths

(Table M-4), we found that all inhabited households for which we had NIR reports were similar to 

households with potentially eligible youths for gender of the informant, race/ethnicity of informant, type

of respondents’ housing units, and type of respondents’ neighborhood. One notable difference was that

there were fewer Asian/Pacific Island or informants for households with potentially eligible youths (2.8 

percent) compared to all households (7.9 percent). While a high non-response rate among Asians/Pacific

Islanders was not surprising, it appears that we were able to decrease this nonresponse rate for

Asian/Pacific Islanders with potentially eligible youths, perhaps due to greater efforts by the field

interviewers.

Table M-4 shows a comparison of the non-interview households where Field Interviewers were 

able to identify potentially eligible youths aged 10-25 with all the inhabited NIR households (including

the households where they could identify potentially eligible youths). There were 3,374 inhabited 

households, out of which they could identify potentially eligible youths in 928 households. The remaining

reports either did not provide a response to the question about the number of family members (686),

reported that no one lives in the housing unit (67), provided invalid or don’t know responses (1192), 

reported no 10- to 25-year-olds (470) or didn’t know that there were any potentially eligible youths (31) 

in the household. 

As one might expect, households with potentially eligible youths were less likely to live in an 

atypical or “other” type of housing unit (1.6 percent) compared to all households (5.3 percent). Similarly,

households with youths were less likely to be in an atypical or “other” type of neighborhood (0.4 percent) 

compared to all households(4.3 percent).

189
Table M-4 All Inhabited Households Compared to Those Where

Potentially Eligible Youth Were Identified

Description

All inhabited households 

(3374)

(percent of non-missing 

households)

Households where potentially eligible 

youths 10-25 were identified (928) 

(percent of non-missing households)

Gender of Informant

Male

Female

(1664 Households, 1710 (781 Households, 147 missing)

missing)

37.9

62.1

36.5

63.5

Race/Ethnicity

Non-Hispanic White

American Indian 

Asian/Pacific Island 

Hispanic

Black-African American

Other

(1488 Households, 1886 

missing)

67.3

0.2

7.9

8.3

14.2

2.0

(706 Households, 222 missing)

71.7

0.1

2.8

7.4

16.3

1.7

Type of Housing Unit

Detached Single Family

Mobile Home Trailer 

Single Family Row 

Small Apt. or Condo

Large Apt. or Condo

Apt. Attached to Commercial

Structure

Other

(1861 Households, 1513 

missing)

72.5

4.1

4.7

7.6

5.6

0.2

5.3

(793 Households, 135 missing)

6.2

75.9

5.9

5.9

4.4

0.0

1.6

Type of Neighborhood

Rural and Agricultural

Rural and Residential

Suburban, Residential

Urban, Residential

Urban, Mixed Retail

Urban, Mixed

Wholesale

Vacant Buildings 

Other

(1885 Households, 1489 

missing)

9.3

15.5

45.2

18.7

5.7

1.2

0.0

0.1

4.3

(796 Households, 132 missing)

5.4

11.7

17.5

43.0

20.4

1.8

0.0

0.0

0.4

In general, most of the non-interviewed housing units were detached single-family homes (72.5 

percent). The most typical housing units surrounding the sampling unit closely match those of the sample

housing units.

Field Interviewers described a large majority of the household units as either very well kept (47 

percent) or well kept (32 percent), while very few of the units were characterized as poorly kept (8 

percent). The condition of the housing units in the surrounding area closely resembled these sampling

units.

190
5.4 How Many Household Members/Potentially Eligible Youths Are Missing From Our Sample?

The 3,374 inhabited units among the NIRs contained a total of 4,289 household members. Of 

greatest relevance to the NLSY97 and PAY97 samples are the number of potentially eligible youths

missing from our sample. Field Interviewers were able to estimate the number of household members for 

1,429 households and the number of potentially eligible youths for 1,398 households. Other households 

are not enumerated because the Field Interviewer was unable to secure an informant who could provide 

household counts. Using the 1,398 enumerated households, we estimate that 928 had at least one 

potentially eligible youth (10 to 25 years of age), and we have the number of youths for 923 households;

of these households with at least one potentially eligible youth, the average number of 10- to 25-year-olds

is 1.52. Households with at least one youth aged 17 to 25 years had on average 1.32 such youths. Data 

from 923 households indicate that 1,401 potentially eligible youths were associated with these NIR 

households and so omitted from the NLSY97 youth study. (The remaining 2,451 households were 

missing data for number of youths or were reported to have no youths.) We can count 430 households 

with 570 youths ages 17-25 from the enumerated households, and estimate that there were 831 youths

aged 10-16 years. The composition of the NIR households is shown in Table M-5. Figure M-2 shows 

how the number of youths was determined.

Table M-5 Composition of NIR Households

Total number of NIR housing units which are inhabited 3,374

1,429

Total number of household members 4,289

Housing Units which are believed to contain a potentially eligible youth 1,398 

Number of households with at least one 10- to 25-year-old 923

Number of youths aged 10-25 1,401

Average number of 10- to 25-year-olds per household with at least one 10- to 25-

year-old

 1.52 

Number of households with at least one 17- to 25-year-old 430

Number of youths aged 17-25 570

Average number of 17- to 25-year-olds per household with at least one 17- to 25-

year-old

 1.32 

1,401-570=831

Total number of housing units where we have a count of household members

Estimate of number of youths aged 10-16

In Table 5.24 of this report, we find that the CPS density estimates (for 12- to 23-year-olds) are 

around 16 percent while the NLSY97 estimates are lower (around 12 percent). Our estimate of 10- to 25-

year-old youths as a proportion of total household members from the NIR reports is approximately 32

percent. Even though the age ranges do not exactly match, we do find the density of youths in the NIR to 

be much higher, indicating that target youths constitute a larger percentage of the population in the noninterview group than among those completing screeners. This is consistent with the idea that households 

with youths may have been more likely to refuse to be screened, causing the shortfall of sampled youths

in the NLSY97. 

191
Table M-6 Ratio of Youths to Household Units and to Household Members

Description NIR ratio

10-25 year olds to total number of household members 

17-25 year olds to total number of household members 

10-25 year olds to number of household units 

17-25 year olds to number of household units

.32

.13

.42

.17

While making attempts to gain cooperation and complete the household screener, Field 

Interviewers exploited a variety of tactics to determine the number of household members and the number

of potentially eligible youths. For 68 percent of the cases, Field Interviewers spoke with and even began 

the household screener interview with the respondent before he/she refused and so they were able to 

secure reliable information about youth residents in the household. Out of the 1,401 youths counted, 962 

youths are counted from information given directly by the respondent. Of the remaining cases, the

number of youths in the sample households were determined by one of the following methods: asked a 

neighbor (10.7 percent), asked the postal carrier (0.2 percent), saw someone come out of the house (9

percent), or other methods (11.6 percent). Table M-7 shows which method was used to determine the 

number of potentially eligible youths for all 923 households which have at least one youth aged 10-25.

The remaining 2,451 inhabited households had missing household counts, invalid/don’t know responses 

or said they had no youths aged 10-25. 

Table M-7 Determination of the Number of Potentially Eligible Youths 

Method Used to Determine

Percentage of households 

(923 households)

Respondent Was Willing to Tell Me 

Asked a Neighbor 

Asked the Postal Carrier 

Saw Someone Come Out of the House 

Other

Don’t Know 

68.0

10.7

0.2

9.0

11.6

0.5

Apart from the 1,401 youths aged 10-25 already identified, we may have missed youths either

because we have no NIRs for some cases or because the number of household members or youths was not 

reported. Out of the non-interview households, the 1,737 missing NIR households, shown in Figure M-1,

could contain eligible youth. Also, informants often gave “don’t know” and “invalid” responses and for 

some households there is missing information on the number of household members. These households 

may have youths in the 10-25 age range too.

To calculate a range of estimates for youths ages 10-25, we assume that households for which the 

reports had missing, “don’t know” or “invalid” information on the number of household members and 

youths, could have contained some youths in the 10-25 age range. Figure M-2 shows the counts on which 

this estimate is based. There are 1,878 inhabited households (686 + 1,192 households) for which we do 

not have information on the number of household members. For example, these could be cases where a

car was seen parked in the driveway, newspapers were delivered to the household, or lights were on in the 

house. Out of the 1,496 households (1,429 +67 households) for which we do have information on the 

number of household members, 1,429 (i.e. 95.5 percent) have at least one household member and 67 do 

192
not. We assume that the 1,878 households are similar to the 1,429 households where we have reports of

the number of household members. Therefore, out of 1,878 households, we calculate that 1,793

households could have at least one household member. For the 1,429 households with household 

members, 928 (65%) have at least one 10- to 25-year-old, so for the 1,793 households we calculate that 

1,165 households would contain at least one 10- to 25-year-old. Using the youths/household ratio of 1.52 

(Table M-5), we find that 1,165 households would contain approximately 1,771 youths in the 10-25 age 

range.

In addition, there are 31 households where there is no information on the presence of 10- to 25-

year-olds. Out of 1,398 households for whom we do have information on 10- to 25-year-olds, 928 (66.38 

percent) have at least one 10- to 25-year-old and 470 have none. Therefore, out of 31 households there 

would be 20.5 households with 10- to 25-year-olds and those 20.5 households would contain 

approximately 31 youths.

Finally, there are 5 households which contain 10- to 25-year-olds but for which we do not have 

the number of youths. If we assume that these households contain youths in the same proportion (1.52 

youths per household) as the households for whom we know the number of youths, then these 5 

households would contain approximately 8 youths.

Thus, we have a combined estimate of 1,810 additional 10- to 25-year-old youths. So we can 

estimate that there were 1,401 to 3,211 missing youths in the 10-25 age range. 

6. Summary

The analysis is structured around three main questions: 

(1) Why were we unable to get an interview? The main reason for not completing a household 

screener (a non-interview) was that the sampling unit was uninhabited, while the second most common

reason was a refusal. When a respondent refused to complete a screener, the Field Interviewers attempted

a “conversion”; these conversions were pursued more aggressively if there was reason to believe an 

eligible youth resided in the housing unit.

(2)What type of households were non-interviews? The NIR demographics indicate that the 

people who were non-interview respondents look very much like our respondents who completed the 

household screener. Of the non-interview respondents, those whom we believe to have eligible children

were more likely to be non-Hispanic white and live in a detached single family housing unit than those 

whom we believe not to have eligible youth.

(3) How many household members and youths were missing from our sample? Using NIR data 

for 1,429 enumerated households, we estimate 4,289 household members are missing from the NLSY97 

sampling universe. Using youth counts for housing units, we count as missed 1,401 youths 10 to 25 years

of age. Household enumerations are reliable since they come primarily from household members who

refused to participate in the household screener but provided sufficient summary information to Field 

Interviewers for completing the NIR form. The limited NIR data do not permit us to estimate the number

of these 1,401 youths who would have been eligible for the NLSY97 or PAY97 (ages 12 to 23) and are 

thus missing from the study samples. However, we can use estimates of the number of youths counted in 

the 17-25 age range (570 youths) to count approximately 831 youths in the 10-16 age range.

We should also keep in mind that there could potentially be eligible youths whom we cannot 

count because information on the number of household members or number of youths was not provided 

by respondents/informants. In particular, they often gave “don’t know” and “invalid” responses. 

Therefore, the actual number of potentially eligible youths could be significantly higher than the number

of youths we were actually able to count from the information provided in the NIR reports. We estimate

193
that there could be at least 1,810 more 10- to 25-year-olds beyond the 1,401 youths counted from the NIR

reports, giving us an estimate that ranges from 1,401 to 3,211 such youths.

194
Figure M-2 Counting the Number of Youths from the Inhabited Households 

Inhabited NIR Households 

3,374

How many household members?

Missing

17-25 year olds Any 17-25

Don’t Know 

1,192

 Have Household

Members

 1,429 households There are

No household

members 67 information

686

Any 10-25 years old928 households s? None

470

Don’t Know 

31

Number of 10-25 year olds?

year ds? ol 22

There are

 439 households 

Don’t Know 

Number of youths 17-25?

Don’t Know 

 9 

Have youths, ages 17-25 

430 households, 

570 youths

Don’t

Know

5

10-25 year olds,

923

households

1,401 youths

195
APPENDIX N 

Report on NLSY97 Computer Crashes 

196
This appendix summarizes the findings from NORC’s research into areas that were possible

causes or contributors to the NLSY97 “hole in the age distribution” problem. The areas we researched 

were: incidences of crashed/technical problem cases (codes 72 & 73), cases coded “no housing unit” 

(code 83), and the four lowest responding PSUs. Table N-5 lists the blocks where the highest numbers of 

crashed/technical problem cases and “no housing unit” cases occurred. 

Crashed Cases 

We learned that the crashed cases were mainly due to the following: 

x Low Random Access Memory (RAM) 

x Reports of insufficient space on the hard drive 

x Inconsistent battery life 

It is important to note that there was no single cause for these crashed cases. All of the categories 

listed above have to do with the equipment used for the survey.

Low Response PSUs 

The four PSUs with lowest response rates were: 

x PSU 721 31.6 percent 

x PSU 764 37.4 percent 

x PSU 795 21.3 percent 

x PSU 862 59.5 percent . 

The following discussion provides the final disposition codes for each case within each of the four PSUs. 

The “no action” category reflects households from replicates that were never considered to be part of the

sample, and thus should be ignored. 

197
Table N-1 PSU 721 

SEG LSTFLDDI Frequency Percent

Cumulative

Frequency

Cumulative

Percent

No Action 8 4.6 8 4.6

001 Complete – Pers 8 4.6 16 9.2

Comp by Paper Sc 7 4.0 23 13.3

001 Case Crashed 8 4.6 31 17.9

001 No Address/No HU 7 4.0 38 22.0

001 Vacant HU 10 5.8 48 27.7

002 No Action 5 2.9 53 30.6

002 Complete – Pers 1 0.6 54 31.2

002 Comp by Paper Sc 11 6.4 65 37.6

002 Vacant HU 8 4.6 73 42.2

003 No Action 9 5.2 82 47.4

003 Complete – Pers 16 9.2 98 56.6

003 Comp by Paper Sc 9

1

109

0.6

118

80.3

004

R Unavail

1

2.9

153

97.7

005

001

001

5.2 107 61.8

003 Case Crashed 0.6 108 62.4

003 Proxy No 8-28 1 0.6 63.0

No Address/No HU 1 0.6 110 63.6

003 Not Perm Residen 1 0.6 111 64.2

003 Hostile Refusal 1 112 64.7

004 No Action 6 3.5

Complete – Pers 21 12.1 139

004 Comp by Paper Sc 1 0.6 140 80.9

004 Conv Phone Paper 1 0.6 141 81.5

Proxy No 8-28 3 1.7 144 83.2

004 2 1.2 146 84.4

004 Refusal 0.6 147 85.0

005 No Action 5 152 87.9

005 Complete – Phone 1 0.6 88.4

005 Case Crashed 16 9.2 169

005 Not Final inTime 1 0.6 170 98.3

005 Comp DataUnavail 1 0.6 171 98.8

Vacant HU 2 1.2 173 100.0

Original Sample: 173 

No Action: 33 

Out of Scope: 29 

 Net Sample: 111

Completes: 80 

NIRs: 6 

Response Rate: 72.1 percent 

Percent Crash/Tech: 22.5 percent 

Observations

003

68.2

004

Sample Analysis

Crash/Tech: 25 

198
In the case of PSU 721, dispositions for all segments within the PSU were given. Segment 005 

carries a 10 times greater weight than the others and only had one completed case out of 19 eligible lines. 

Of these eligible lines, 16 were crashed/technical problems, one was unavailable, one was incomplete due

to lack of time, and the other was completed. This meant that we had a 5 percent response rate in that

segment which would effect the overall response rate after weighting. It should also be noted that there 

were a total of 25 crashed cases in the whole PSU, and 16 of them were in segment 005. 

Table N-2 PSU 764 

Disposition Category Frequency Percent Percent

No Action 60 20.1 60 20.1

Complete – Pers 21 7.0 81 27.1

Complete – Phone 0.7 83 27.8

Conv Complete Ph 1 0.3 28.1

Comp by Paper Sc 30 10.0 114 38.1

Gate No one 8-28 4 1.3 118 39.5

Case Crashed 36.5 227 75.9

Unspec Tech Prob 1 0.3 76.3

Proxy No 8-28 16 5.4 244 81.6

Not Final inTime 1 0.3 245 81.9

Inelig. Data 0.3 246 82.3

No Address/No HU 12 86.3

Vacant HU 29 9.7 287

0.7 289 96.7

No Ans/NeverHome 1.7 294 98.3

0.3 98.7

Refusal 4 1.3 299 100.0

Sample Analysis

Original Sample: 299 

No Action: 60 

Out of Scope: 41 

 NIRs: 14

Crash/Tech: 110 

Response Rate: 37.4 percent 

Percent crash/tech: 55.5 percent 

Frequency

2

84

109

228

1

258

5

295

4.0

96.0

HU Inaccessible 2

R Unavail 1

 Net Sample: 198

Completes: 74 

199
Observations

There were only 14 NIR cases out of 198 sampled households. Therefore, these could not be

perceived as the cause of the low response rate. Fifty-five point five percent crashed cases could certainly

have affected the response rate and may have been a minimal contributor to the hole in the age

distribution.

Table N-3 PSU 795 

Disposition Category Percent Frequency Percent Frequency

No Action 66 20.1 66 20.1

16 4.9 82 24.9

Complete – Phone 5 1.5 87 26.4

Comp by Paper Sc 6 1.8 93

14 4.3 107 32.5

Case Crashed 78 23.7 185 56.2

Unspec Tech Prob 66 20.1 251 76.3

Proxy No 8-28 7 2.1 258 78.4

Not Final inTime 2 0.6 260 79.0

5 1.5 265 80.5

No Address/No HU 9 2.7 274 83.3

Vacant HU 29 8.8 303 92.1

314 95.4

Language Barrier 4 1.2 96.7 318

R Unavail 4 1.2 322 97.9

Refusal 7 2.1 329 100.0

Complete – Pers 

28.3

Gate No one 8-28 

Inelig. Data 

No Ans/NeverHome 11 3.3

Sample Analysis

 Original Sample: 329

No Action: 66 

Out of Scope: 38 

 Net Sample: 225 

 Completes: 48

 NIRs: 33

Crash/Tech: 144 

Response Rate: 21.3 percent 

Percent Crash/Tech: 64.0 percent 

Observations

The number of crashed/technical problem cases amounted to 64 percent of the sampled

households. The NIRs only accounted for approximately 15 percent of the sampled households.

200
Table N-4 PSU 862 

Disposition Category Frequency Percent Frequency Percent

No Action 63 20.3 63 20.3

Complete – Pers 57 18.3 120 38.6

Conv Complete 4 1.3 124 39.9

Comp by Paper Sc 40 12.9 164 52.7

Gate No one 8-28 7 2.3 171 55.0

Case Crashed 33 10.6 204 65.6

Unspec Tech Prob 45 14.5 249 80.1

Proxy No 8-28 19 6.1 268 86.2

Inelig. Data 4 1.3 272 87.5

Demolished 1 0.3 273 87.8

Business 1 0.3 274 88.1

No Address/No HU 3 1.0 277 89.1

Vacant HU 25 8.0 302 97.1

No Ans/NeverHome 4 1.3 306 98.4

Refusal 5 1.6 311 100.0

Sample Analysis

 Original Sample: 311

No Action: 63 

Out of Scope: 30 

 Net Sample: 218 

 Completes: 127

 NIRs: 13

Crash/Tech: 78 

Response Rate: 58.3 percent 

Percent crash/tech: 35.8 percent 

Observations

Judging by how few NIRs there were in this PSU, the crashed cases (just over one-third of the 

sampled households) caused the biggest problem. If we had been able to resurrect the crashed/technical

problem cases, we probably would have had a much higher response rate. 

No Housing Units 

There were a total of 1,580 cases that were coded 83 (No Housing Unit). This is an average of 

7.9 per PSU. In our research, we attempted to look for clusters of these that would identify a major

problem with the listing. We identified three potential problems:

x One year’s time between the listing and the screening, which might explain why certain

neighborhoods had changed; 

x Demolished houses coded as “no housing unit” because the interviewer had no way of knowing that 

there were ever houses there; and 

x “Listing errors,” such as back doors to apartments being listed as separate units. 

Conclusions

201
The large number of crashed/technical problems could have contributed to the “hole in the age 

distribution” problem, although this study reveals no definitive information about issue. Nationwide there

were, in total, 1,318 crashed cases. By subtracting 357 cases (the total from the four lowest response rate 

PSUs), we are left with 961 across the remaining 196 PSUs, an average of 4.9 per PSU. 

As far as the “no housing unit” cases, it is highly unlikely that decentralized occurrences of these 

cases had much effect on the hole in the age distribution. 

202
Table N-5 Location of Largest Numbers of Crashed/Technical Problems and “No HU” Cases 

PSU SEG BLK

83: No 

HU

72:

Crash

73: Tech 

Problem COMMENTS

708 001 110 12 FI: U of VT was replacing dormitories; these buildings used for student housing 738 016 101 9 No explanation738 023 201 10 Field manager does not recall this number of crashed cases. 739 001 111 10 May have been college housing, in transition between listing date and screening 739 003 502B 9 May have been college housing, in transition between listing date and screening 739 014 142 26 Trailer park had to be relisted 742 006 710 10 Mostly shells of houses; no roofs; no windows; homeless men on street 742 007 705 14 Field manager has no notes in her log that any cases crashed in this segment 742 008 410 11 Field manager says only 1 case crashed here; does not recall details 778 008 601 8 Field manager says locked gate community; could get in only at certain times of day792 003 110 8 Field manager deceased; no information available 792 024 303 11 Field manager deceased; no information available 796 002 208 9 14 1 Field manager describes technical and personnel difficulties. Technical: difficulties with the software interface and796 003 101 9 Performance of laptops in use by field staff. Often 796 003 401 16 Unable to transmit cases electronically, had to transmit manually796 010 117 13 Suggest we talk further with this field manager 821 003 512 10 See notes above for this field manager. 821 004 204 11 See notes above for this field manager. 821 004 205 1 7 See notes above for this field manager. 821 005 110 20 See notes above for this field manager. 821 008 208 12 See notes above for this field manager. 821 008 209 9 See notes above for this field manager. 821 009 401 17 See notes above for this field manager. 826 005 316 12 Wooded rural area; hunting; vacation cabins; trailer park w/vacant lots; unsafe 854 009 202 13 High producing FI was given new Think Pad. When loaded, @ 200 cases crashed. 854 009 203 14 Field manager is SURE majority of cases eventually resolved with more conventional disposition. 854 009 204 10 FIELD MANAGER is SURE majority of cases eventually resolved with more conventional disposition. 

Each PSU-SEGMENT-BLOCK combination containing a group of at least 8 cases with a disposition of 72 (case crashed/data unavailable) and/or 73 (Data unavailable/Unspecified technical 

problem OR a disposition of 83 (No such address, no housing unit) is listed. An attempt was made to contact the field manager who supervised the NLSY97 screening/sampling task for that location and discuss reasons for these dispositions. 

203
Appendix O 

Report on NLSY97 Debriefing Interviews 

204
In 1997 NORC screened approximately 90,000 U.S. households to find eligible youths for 

NLSY97/PAY97 in 1997. After the screening was completed it was found that the number of people in 

the eligible age range was considerably lower than expected. Debriefing interviews were conducted with 

a sample of field interviewers (FIs) to explore how two factors may have contributed to this shortfall in 

the sample. The first factor is the technical difficulties that FIs experienced with the laptops provided for

screening. Computers were reported to have “crashed” often during screening interviews, perhaps most

often in households with eligible youths, when the screening process was the lengthiest. This would have 

provided opportunities for eligible youths to be omitted from the sample. The second factor that may

have contributed to the shortfall in the sample is the informants’ knowledge of the age range of eligibility

for NLSY97/PAY97. A brochure about NLSY97 that was sent to selected households stated explicitly

that youths ages 12 through 17 were eligible for the study. In addition, FIs were informed of the eligible 

age range for both NLSY97 and PAY97. Thus, informants may have learned in advance of the screening

the ages of eligibility, allowing them to avoid participation in the study by misreporting on the household 

roster. Patterns in the age frequency data, in which the shortfall corresponds well to the exact ages that

are eligible for the study, suggest that informants may have misreported.

This report presents findings from debriefing interviews conducted with selected field 

interviewers who participated in the NLSY97/PAY97 screening in 1997. The debriefing questionnaire 

addresses both the computer problems that FIs encountered and possible misreporting by informants.

Survey Procedures

Questionnaire. A copy of the questionnaire appears in Attachment 1.

Respondents. The respondents were 96 FIs who screened households for NLSY97/PAY97 in 

1997. These FIs were selected at random from 432 FIs who conducted least 10 screening interviews and 

were still on NORCs list of active interviewers as of October 12, 1998.22 The 96 FIs worked on 24.0 

percent of the 65,530 completed screening interviews and 24.7 percent of the 90,924 total screeners (that 

is, including non-interview reports and other cases).

The FIs were asked about the number of years they had been interviewing for NORC and whether 

they had any prior experience with CAPI projects (Questions 11 and 12). FIs ranged from one year to 20

years of employment at NORC. Of the 96 FIs, 68.8 percent reported having worked on CAPI projects 

before the NLSY97 screening. 

Procedure. Six Field Managers (FMs) who served on NLSY97 Round 1 were recruited to 

conduct debriefing interviews with the sampled FIs. Each FM conducted sixteen interviews by telephone.

Interviews were conducted between October 21 and 31, 1998.

22Data on FIs who worked on NLSY and the final disposition codes of the screeners they worked on is 

based on lists created at the close of the field period in September/October of 1997. Since these data were 

subsequently checked and further screeners were completed in 1998, the lists used to create the sample

frame may contain some omissions and inaccuracies. 

205
Main Findings 

 Computer problems. Seventeen FIs stated that they conducted only paper-and-pencil (PAPI) 

screeners (Question 1). Of the 79 FIs who did use the computer to screen, 32 (40.5 percent) reported

experiencing problems with their computers (Question 2). As Table O-1 shows, computer problems

prevented the completion of a screening interview in progress five or fewer times for 30 of the FIs; one FI 

reported experiencing problems on 20 occasions and one other FI reported 100 such problems (Question 

3). Without comparison data from other CAPI studies it is not possible to judge how unusual this is. 

Most of the FIs who experienced problems (n=26, 78.8 percent) reported that the computer crashes 

occurred at least once at a household where they thought there were eligible youth (Question 4).

Although it is assumed that FIs later attempted to complete screening for all such crashed cases, it is not 

clear what their success rate was in securing cooperation again upon returning to the household.

FIs were asked to give a brief description of their computer problems (Question 5). These openended descriptions were then categorized (see Table O-2). As Table O-2 indicates, the most commonly

reported problems were that the computer simply “crashed “ or “froze” during the screening interview.

As Table O-3 shows, FIs without prior CAPI experience were somewhat more likely to experience

problems. Of those FIs who did CAPI screening (n=77), the 49 FIs with prior CAPI experience had 

fewer problems with their computers (32.7 percent with problems vs. 67.3 percent without) than the 28 

who did not have prior CAPI experience (57.1 percent with problems vs. 42.9 percent without problems).

Misreporting by the informant. Households that wanted to avoid participating in the 

NLSY97/PAY97 studies could do so in a number of ways. They could refuse to participate in a screening 

interview, or, if they did participate and were selected for the main study, they could refuse to do the main

interview. However, since refusing to be interviewed can lead to subsequent visits by interviewers 

attempting to secure cooperation, an alternate way of refusing to participate is to misreport household 

information. Informants could have discovered the eligible age range through several sources. The

NLSY97 brochure mailed to households stated the exact ages of eligibility for this study. Neighbors who 

were already screened could supply the relevant information. In addition, the FIs themselves could have 

told informants. In the debriefing interview FIs were asked what they told informants about ages of 

eligibility for the NLSY97 study prior to the actual screening interview (Question 9). FI’s open-ended 

responses to this question were coded into categories (see Table O-4). Thirty-eight FIs (39.6 percent) 

indicated that they told responses about the ages of eligibility; twenty-six of these stated specific ages in

their responses to the debriefing items. It appears from this finding that many FIs did tell informants the 

age range of interest. It is possible to speculate that some informants tailored their responses to avoid 

participating in the survey. However, no data are available that can indicate how often this may have

happened.

If informants were misreporting, it was not often obvious to the FIs. According to FI’s 

judgments, informants seemed cooperative and truthful in their responses to the screener. On a 1 to 5

scale, with 1 representing “very easy” and 5 representing “very difficult”, the FIs mean rating of the 

difficulty of gaining cooperation was 2.6 (Table O-5). Although respondents could have learned some

information about the eligible ages for the NLSY97 and used this information to misreport household 

members, the FIs did not feel that this was the case. FIs judged that informants rarely read the advance 

letter and brochure (which stated the eligible ages for NLSY97) and most felt that informants were 

reporting accurately on the household roster (Tables O-6 and O-7). FIs were also asked to list the most

common questions or concerns that informants expressed about NLSY97. As can be seen in Table O-8, 

FIs were asked typical questions about confidentiality, background of the study, and how long the 

interview would take.

The types of questions informants asked were not unusual and did not suggest any particular reasons why

206
they would try to avoid participating in NLSY97. 

Conclusions

Although many FIs reported experiencing problems with their laptop computers, the frequency of

these problems was not high. Therefore, computer difficulties alone cannot explain the missing youth in 

the sample. Many FIs did tell informants about the age ranges of eligibility, possibly allowing for some

misreporting on the household roster. However, most FIs did not get the impression that misreporting

happened very often. Rather, most FIs thought informants were cooperative and truthful in completing

the screener. 

Table O-1 Number of Times a Computer Problem Prevented 

Completion of a Screening Interview 

Number of times computer 

problem experienced..... Number of FIs reporting 

1 9

2 7

3 7

4 5

5 2

20 1

100 1 

Table O-2 Types of Computer Problems Experienced by FIs 

Computer problem Number of reports 

Computer crashed, screen locked or frozen 28

Computer slow 5

Battery dead 3

Cannot access case or case not spawned 5

Problem with skips 2

Other technical problem 6

Other non-technical problem 8

Note: This table includes 57 problems reported by 36 FIs. 

207
Table O-3 Computer Problems in NLSY97 and Prior CAPI Experience 

NLSY97 computer problems? 

Problems No problems Totals

Yes 16 33 (49)

No 16 12 (28)

 Totals (32) (45) (77)

Prior CAPI 

experience?

n

(%)

Table O-4 What FIs Told Informants About Who is Eligible to Participate in NLSY97 

Followed protocol or script 22

(22.9%)

Scientific sample, computer selects Rs 11

(11.5%)

Eligible ages 

38

(39.6%)

Other

8

(8.3%)

FI does not remember

17

(17.7%)

Table O-5 Number of FIs Giving Each Rating of Difficulty in Gaining Cooperation 

Rating 1

very easy 

2345

very difficult 

n

(%)

7

(7.3%)

40

(41.7%)

40

(41.7%)

6

(6.3%)

3

(3.1%)

mean rating of 2.6 

n = 96 

208
Table O-6 FIs’ Ratings on How Often Informants Read Advance Letter and Brochure 

Rating 1

very often 

2345

not often 

at all 

(%)

4 7

(7.4%)

22

(23.4%)

30

(31.9%)

N 31

(4.3%) (33.0%)

mean rating of 3.8 

n = 94 

Table O-7 FIs’ Ratings on How Often Informants Did Not Report Accurately on Household Roster 

Rating 1

very often 

2345

not often 

at all 

1

(1.0%)

2

(2.1%)

4

(4.2%)

32

(33.3%)

57

(59.4%)

mean rating of 4.4 

n = 96 

Table O-8 FI Reports of Questions and Concerns Expressed by NLSY97 Informants 

Question/concern Number of reports 

Confidentiality 45

How were we selected? 21

What kinds of questions will be asked? 10

Who is conducting study and how will this information be used? 31

How long will this take? 29

Why should I do this? 6

Is this voluntary? 10

Payment 1

Military/FBI/police/IRS 6

Government 5

General distrust 6

Miscellaneous 25

Note: This table includes 195 questions and concerns reported by 86 FIs. 

209
Attachment 1 

NLSY97 Interviewer Debriefing Questionnaire 

October 1998 

This debriefing interview conducted by:

Date: / /

FI Name _______________________________________________________ _

Phone # _________________________________________________________ 

Respondent # (Also write respondent number on top of next page.) 

WHEN YOU HAVE COMPLETED THIS INTERVIEW PLEASE DETACH AND KEEP THIS PAGE.

RETURN REMAINING QUESTIONNAIRE PAGES TO.... 

LISA LEE

55 EAST MONROE, 48TH FLOOR 

CHICAGO, IL 60603 

THANK YOU! 

210
211

Respondent # _______

(copy from face page) 

NLSY97 Field Interviewer Debriefing Questionnaire 

October 1998 

We are interested in learning about field interviewers’ experiences in conducting screening interviews for 

the NLSY97 in 1997. Therefore we are asking a number of interviewers to complete this debriefing 

questionnaire. Your participation is voluntary, but we hope you will take the time to provide us with your 

feedback. The information you provide will allow us to improve our procedures for future studies. 

Participants will be identified by number only, not by name, so your name will not be associated with any 

of your answers. 

1. Were you a PAPI-only screener? 

Yes........................................... 1 (GO TO Q. 6) 

No ............................................ 2 (GO TO Q. 2) 

2. Some interviewers had difficulty with the computers provided for completing screening interviews. 

We are interested in learning about times when a computer problem made it necessary to discontinue 

an interview and return to the household at a later time to complete it. Did this ever happen to you? 

Yes........................................... 1 (GO TO Q.3) 

No ............................................ 2 (GO TO Q. 6) 

3. About how many times did this happen? 

 Times 

4. Did this ever happen during a screening interview at a household where you thought there were 

young people eligible for the study? 

Yes........................................... 1 

No ............................................ 2 
212

4. ______________________________________________________________________ 

Very easy Very difficult 

 1 2 3 4 5 

5. Please describe briefly the problems you experienced with your computer. 

1. ______________________________________________________________________ 

2. ______________________________________________________________________

3. ______________________________________________________________________ 

6. A. In general, how easy or difficult was it to get informants’ cooperation in completing the 

screener? Please give a rating from 1 to 5, with 1 being “very easy” and 5 being “very difficult.” 

B. What makes you give this answer? ______________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

7. How often did informants seem to have read the advance letter and brochure that NORC sent? 

Please give a rating from 1 to 5, with 1 being “very often” and 5 being “not often at all.” 

Very often Not often at all 

 1 2 3 4 5 
213

 1 2 3 4 5 

8. A. How often did you feel that informants were not reporting accurately on the household roster? 

Please give a rating from 1 to 5, with 1 being “very often” and 5 being “not often at all.” 

 Very often Not often at all 

B. What makes you give this answer? ______________________________________________

______________________________________________________________________________

______________________________________________________________________________

 ______________________________________________________________________________ 

9. Prior to the actual interview, what did you tell informants about who was eligible to 

participate in the NLSY97 study? 

 ______________________________________________________________________________ 

 ______________________________________________________________________________ 

 ______________________________________________________________________________ 

 ______________________________________________________________________________ 
214

10. What were the three most common questions or concerns informants had about NLSY97? 

LIST EACH QUESTION BELOW AND ASK:

How did you address this question/concern? 

1. Question: _______________________________________________________________

 _______________________________________________________________________

Answer: ________________________________________________________________

_______________________________________________________________________

2. Question: _______________________________________________________________

 _______________________________________________________________________

Answer: ________________________________________________________________

 _______________________________________________________________________ 

3. Question: _______________________________________________________________

 _______________________________________________________________________

 Answer: ________________________________________________________________

 _______________________________________________________________________ 

11. How many years have you been an interviewer for NORC? 

 Years 

12. Had you worked on any CAPI projects before the NLSY97 screening? 

 Yes...................................................... 1 

No ...................................................... 2 

This is the end of the interview. Thank you for your help!
215

APPENDIX P 

Analysis of Interview Validation
216

Project Summary 

In view of the NLSY97 Round I “observed” undercount, an analysis of the validation interviews 

was completed. The two main objectives of the analyses were: (1) to evaluate the performance of the 

interviewers using the respondents’ comments; and (2) to analyze the validation process of NORC’s field 

staff for the NLSY97 data collection efforts. This report outlines the data collection efforts, describes the 

analyses performed on the data gathered, and discusses the results.

Data Collection 

At the end of January 1997, household screeners and interviews were fielded for NLSY97, and 

the fieldwork continued until November of 1997. The validation process of the screeners and interviews 

began in March of 1997. NLSY97 validation specifications included: (1) validation of one person only in 

each selected household; (2) validation of 0.5 percent of all screeners completed in households that did 

not contain any eligible sample members; (3) validation of 4.0 percent of parent completions per 

interviewer (each interviewer needed a minimum of one complete parent case validated); and (4) 

validation of 5 percent of youth completions per interviewer (each interviewer needed a minimum of two 

completed youth cases validated). 

Initially the field managers (FMs) conducted validation interviews at their own discretion; in May 

of 1997 the validation process was formalized and carried out by a group of NORC interviewers. 

According to the validation report, a total of 1,390 validation interviews were conducted, including 476 

screeners, 518 youth, and 396 parent interviews. The final count of the validation interviews that were 

electronically entered and used for analysis was 1,296, including 347 screeners and 949 youth and parent 

interviews combined. The difference between 1,390 and 1,296 (94 interviews) can be explained by the 

fact that some of the initial validation interviews conducted by the FMs themselves may not have been 

entered in the formal validation questionnaires. 

Data Analysis 

Among the 1,296 validation interviews for which there was an explicit validation questionnaire, 

347 were screeners and 949 were parent and youth interviews. For these cases, the parent and youth 

interviews are not distinguished from each other as it is not possible to do so from the data available in 

the validation questionnaires alone. A link file exists in the archives which would require additional work 

to retrieve and process. 

Validation interviews were conducted on cases which were originally interviewed between 

January and October of 1997, while the validation interviews were conducted between March and 

November of 1997. Overall, the validation status of the 1,296 cases is as follows: 90.2 percent (1,169) 

were successfully validated, 7.0 percent (91) did not remember, 1.5 percent (20) were refusals, 0.2 

percent (2) were break-offs, and 1.1 percent (14) were blank. For the ineligible screeners, only 69.7 

percent (242) were successfully validated, 24.8 percent (86) did not remember, 4.6 percent (16) refused, 

0.6 percent (2) were break-offs, and 0.3 percent (1) were undetermined. The successful validation status 

for the parent and youth interviews was an overwhelming 97.7 percent (927), 0.5 percent (5) did not 
217

 Over four-fifths (1,076) of the respondents reported that the initial interview had occurred in 

person, 3.5 percent (46) by telephone, and 13.1 percent (170) did not respond to this question. The 

majority of those who did not answer the question regarding the mode of the interview were screener 

cases (154). Among the parent and youth interviews, 93.9 percent (891) reported having completed the 

initial interview in person. 

Overall, the validation interview analysis revealed one major lapse on the part of the field staff in 

not conducting and completing the validation interviews within a set time period in order to prevent 

problems with memory recall on part of the respondents. The ineligible household screener consisted 

only of a few questions regarding the composition of the household, and that brief meeting could easily 

be forgotten by the respondents. In addition, identical questionnaires were used to validate both the 

screener and the youth and parent interviews, and thus do not validate any particular responses to any of 

these components. A separate validation interview questionnaire should have been designed to ask the 

respondent to provide information regarding the age composition of the household; this could then have 

been matched to the original response during the screener. 

remember, 0.4 percent (4) were refusals, and 1.4 percent (13) remained undetermined. (A paper and 

pencil follow-up was conducted for all unsuccessful validations, but it is possible that the final disposition 

may not have been entered in the computer system.) 

A cross tabulation of interview dates and validation interview dates showed that, for the ineligible 

household screeners, and the youth and parent interviews, the majority of the validation interviews had 

occurred at least one month after the original interview. A separate analysis, consisting only of those 105 

ineligible household screener cases which were not successfully validated, revealed that only 15.2 percent 

(17) of the respondents were validated within two months of the original interview, and 28.5 percent (30) 

of the respondents were validated at least six months after the original interview. 

In the comments section, 338 respondents out of 1,296 for the validation study made comments 

regarding the interviewers’ mannerisms. In total, 21 percent (272) reported in the affirmative that the 

interviewer was polite and courteous, 3 percent (41) made comments which were categorized as negative, 

2 percent (25) made comments which were categorized as neutral, and 74 percent (958) made no 

comments. The majority of the respondents to the youth and parent interview who made negative 

comments complained about the length of the interview and/or of not receiving the promised 

compensation for the youth test. Ninety-five percent (898) of the parent and youth respondents reported 

getting paid for the interviews. The payment varied between five and ninety-five dollars. 

Discussion
218

Type Successful 

Conclusion

The analyses of the validation interviews failed to reveal any poor conduct on part of the 

interviewers, as most of the comments of the respondents were positive and the few negative comments 

were beyond the control of the interviewers. On the other hand, the field staff failed to conduct timely 

validation interviews, especially for the ineligible household screeners, which leave our results somewhat 

inconclusive.

Table P-1 NLSY97 Round 1 Validation Interviews 

Failed Other Total

Screener 57.8%

(275)

20.2% 

(96)

22% 

(105)

100% 

(476)

Youth 98.6% 

(511)

0% 

(0)

1.4% 

(7)

100% 

(518)

Parent 97% 

(384)

0.5% 

(2)

2.5% 

(10)

100% 

(396)

All

(1,170)

84.2% 7% 

(98)

8.8% 

(122)

100% 

(1,390)

Source: Based upon the report of validation interviews, not upon the validation forms 

 themselves. 

Ineligible Screeners Only 

Validation Status Percent

Table P-2 NLSY97 Round 1 Validation Interviews 

Number

Successful 242 69.7 

Other than Successful 105 30.3

 Don’t remember 86 24.8 

 Refusal 16 4.6 

 Break off 2 0.6 

 Unknown 1 0.3 

Total 347 100.0
219

Youth or Parent 

Table P-3 NLSY97 Round 1 Validation Interview 

Respondent Comments Regarding the Survey or Experience with Interviewer 

Type of Comment Screener Total

Positive 10% 21% 

(37)

25% 

(235) (272)

Negative 2% 2% 

(6)

4% 

(35) (41)

Neutral 2% 2% 

(7) (18)

2% 

(25)

No comment 86% 

(297)

69% 

(661)

74% 

(958)

Total 100% 

(347)

100% 

(949)

100% 

(1,296)

Table P-4 Elapsed Time Between Screener and Validation Interview for 105 Screeners for 

Which Validation was Other Than Successful 

Month of Number Of Months Between Screener and Validation Interview 

Screener 0 2 3 4 5 6 7 8 1 9

February - 14 - - 1 62 2 3 1

March - 1 1 14 3 1 2 12 - -

April - - 23 5 - 2 3 - -

May - 4 - - 3 3 - - - -

June - 5 - - 4 - - - - -

July - - - - - - - - - -

August 3 - - - 13 - - - -

September 1 - - - - -- - - -

Total 4 7 7 14 13 20 16 8 15 1
APPENDIX Q 

Analysis of Parent Education 

for the NLSY97

220
An analysis of parent education was conducted as part of NORC’s exploration into differences

between NLSY97 youths and the national universe of 12- to 16-year-olds.23 The highest grade attained 

by the mother and father are examined in this appendix, along with a summary of the data set, which 

identifies numbers of youths living in households with one or both parents or living without either parent. 

For comparison, the same analyses are shown for families with resident youths aged 12 to 16 responding 

to the March 1997 Current Population Survey (CPS)24.

Framework for Education Analyses. The analyses that follow were conducted at the youth

level. Highest grade attained was captured for the parents of all eligible NLSY97 youths (9,022 total). 

For the CPS, all 12- to 16-year-old youths were selected for analysis (9,130 total), and the highest grade 

attained by the parents of those youths was extracted. For both the NLSY97 and the CPS, only resident 

parents were used in the analyses; any parents not living with the youths were excluded. 

Education is broken down into seven distinct categories: 8th grade or below, 9th through 11th

grades, 12th grade (which may or may not include a high school diploma), some college (no bachelor’s 

degree), a BA or a BS degree, and graduate level coursework or a higher degree (including master’s

degrees, professional degrees, and doctorate degrees). Youths for whom parents were in ungraded 

schooling, did not know his or her highest grade, or refused to provide education information were left

out of the education comparison; the extent of missingness in the NLSY97 and CPS data is discussed 

later.

Both the NLSY97 and the CPS analyses were conducted on an unweighted basis. Raw counts

and percentages associated with those counts are displayed in each of the tables shown below. 

All of the tables showing comparisons between CPS and NLSY97 display the NLSY97 overall 

and by sample type. Because the supplemental (SU) sample consists only of minority youths, educational 

attainment for parents of this portion of the NLSY97 will differ quite noticeably from the cross-sectional 

(CX) sample, which was designed to be representative of the entire population of 12- to 16-year-olds.

Thus, comparisons between the CPS and NLSY97 should be based mainly on the CX portion of the 

NLSY97 sample.

Differences in Question Wording. While both the CPS and NLSY gather information about 

parents’ highest grade attained, the question wording differs quite widely between the two surveys. The 

CPS asks for educational attainment for adults (persons aged 16 and older) grouped into the following 16

categories: less than first grade; 1st, 2nd, 3rd, or 4th grade; 5th or 6th grade; 7th or 8th grade; 9th grade; 10th

grade; 11th grade; 12th grade (no diploma); high school graduate (diploma or equivalent); some college but 

no degree; associate degree in college (occupation/vocation program); associate degree in college 

(academic program); bachelor degree; master degree; professional school degree (e.g., MD, DDS, LLB,

JD); and doctorate degree (e.g., PhD, EdD). The NLSY grouped its parental education variable into 24 

categories based on year of schooling: did not attend; pre-kindergarten; kindergarten; 1st grade; 2nd grade; 

221

23 Final updating of the NLSY97 Round 1 disposition codes was done in November 1999, in light of all evidence on 

eligibility collected to that point in time, including certain evidence collected during Round 2 operations. Because

the tabulations cited in this appendix reflect an earlier generation of the Round 1 disposition codes, there may be 

small discrepancies between counts cited here and the final counts cited in the main body of this NLSY97 Technical

Sampling Report. 24 The CPS Hispanic supplement was not included in these analyses.
3rd grade; 4th grade; ... ; 9th grade; 10th grade; 11th grade; 12th grade; 1st year of college; 2nd year of 

college; ... ; 7th year of college; 8th year of college; and ungraded. These variable value differences

between the two surveys necessitated the creation of a final grade variable, which combined and matched

grade levels from the CPS and NLSY as comparably as possible. Table Q-1 helps to illustrate these 

classifications by showing which of the original categories from the two surveys is included in each of the

final categories. 

Table Q-1 CPS and NLSY97 Components of Final Grade Categories 

Final Grade Category CPS Components NLSY97 Components

8th Grade or Below Less than first grade through 7

grade

th or 8th Did not attend through 8th grade 

9th through 11th Grade 9th grade through 11th grade 9th grade through 11th grade 

12th Grade/High School Diploma 12th grade (no diploma), h.s. graduate 12th grade 

Some College, no BA/BS Some college but no degree through

associate degree (academic program)

1 ear of college through 3

year of college 

st

 y rd

BA/BS Bachelor’s degree 4th year of college 

Graduate Coursework/

Higher Degree 

Master’s degree, professional degree, 

doctorate degree

5th year of college through 8th

year of college 

This matching of CPS and NLSY grade components is far from perfect, but it is probably the best one can 

do based upon the component categories presented in the CPS and NLSY questions. 

Comparison of the Data Sets. Table Q-2 displays the CPS and NLSY97 data sets, broken down 

by the parent(s) with whom the youths reside. Included among the categories are youths living with both 

parents, youths living with their father only, youths living with their mother only, and youths living with 

neither parent. As shown here, exactly the same percentage of CPS and NLSY CX youths (68.4 percent) 

were in a household with both parents. Similar patterns exist for other living situations. For the CPS, 4.5 

percent of youths live with their father only, compared to 3.7 percent for the NLSY CX sample, and 22.7 

percent of CPS youths lived with their mother only, versus 24.2 percent for NLSY CX youths. Both data 

sets had a small percentage of youths living with neither parent (4.4 percent for CPS, 3.7 percent for 

NLSY).

Notice that less than half (46.6 percent) of NLSY SU youths live with both parents, more than 20 

percent fewer than for the CPS and NLSY CX samples. Further, a much larger percentage live with their 

mother only (42.9 percent for NLSY SU, compared to 24.2 percent for NLSY CX) and with no resident 

parents at all (7.3 percent, versus 3.7 percent for NLSY CX). This illustrates the vast differences between 

the CX and SU sample composition, and their potential effects on other comparisons.

222
Table Q-2 Youths by Presence of Parent—Comparison of Data Sets 

CPS NLSY--CX NLSY--SU NLSY--Total

Data Set Breakout N % N % N % N%

Youth living with both parents 6,242 68.4 4,630 1,050 46.6 5,680 63.0

Youths living with father only 412 4.5 251 3.7 72 3.2 323 3.6

Youths living with mother only 2,076 22.7 1,640 24.2 966 42.9 2,606 28.9

Youths with no resident parents 400 4.4 249 3.7 164 7.3 413 4.6

TOTAL 9,130 100.0 6,770 100.0 2,252 100.0 9,022 100.0

68.4

Missing Data. Table Q-2 shows that approximately 4 percent of youths in both the CPS and

NLSY97 CX samples do not live with either parent. For those youths, no parent education data will be

available, and they will be left out of the analyses completely. Table Q-3 displays the level of missing

data for the highest grade attained variables in the CPS and NLSY97 data sets given that a mother or 

father is present in the youth’s household. In other words, for the youths with resident parents, the 

amount of missing data for each parent education variable will be presented. For the CPS, missing data

signifies cases for which a variable value was imputed. The NLSY97 groups three categories as missing:

parents who indicated ungraded schooling as the highest grade attained, those who did not know his or 

her highest grade, and those who refused to provide education information. As shown in Table Q-3, the

CPS had less than one percent missing in each of the education variables analyzed here, while the

NLSY97 CX sample is missing between 3.3 and 3.9 percent of the data for these variables.

Table Q-3 Extent of Missingness of the Data 

CPS NLSY--CX NLSY--SU NLSY--Total

Variable N% N% N% N%

Father’s Highest Grade Attained 52 0.8 189 3.9 114 10.2 303 5.0

Mother’s Highest Grade Attained 56 0.7 206 3.3 85 4.2 291 3.5

In the analyses that follow, imputed values are included for the CPS. In the case of the NLSY97, 

the percentages displayed are based on complete data only. Therefore, for the NLSY97, we have

implicitly assumed that the data were missing at random. In other words, we attribute the grade

distribution of the completed cases to the missing cases. 

Analysis of Highest Grade Attained. Table Q-4 compares the CPS to the NLSY97 based on 

father’s highest grade attained. Again, comparisons should be made mainly between the CPS and the 

NLSY97 CX samples. As seen below, the CPS youths have slightly fewer fathers with an education 

below twelfth grade (13.4 percent, compared to 16.8 percent for NLSY97), the same percentage with a 

twelfth grade education (33.4 percent for CPS, 33.3 percent for NLSY97), and slightly more with at least 

some college (53.1 percent for CPS compared to 49.8 percent for NLSY97). However, category by

category, the differences are quite small. Also, it is interesting to note that, while fewer NLSY97 youths

had fathers with college-level coursework at or below a bachelor’s degree, more of their fathers had 

graduate-level coursework or a higher degree. 

Table Q-4 Father’s Highest Grade Attained 

223
CPS NLSY--CX NLSY--SU NLSY--Total

Highest Grade Attained N% N% N% N %

8th Grade or below 388 5.8 288 6.1 237 23.5 525 9.2

9th through 11th Grade 507 657 7.6 503 10.7 154 15.3 11.5

12th Grade / High School Diploma 2,224 33.4 1,563 33.3 350 34.7 1,913 33.6

Some College, no BA / BS 1,764 26.5 1,053 22.4 171 17.0 1,224 21.5

BA / BS 1,052 15.8 700 14.9 70 6.9 770 13.5

Graduate Coursework / Higher Degree 719 10.8 585 12.5 26 2.6 611 10.7

TOTAL 6,654 100.0 4,692 100.0 1,008 100.0 5,700 100.0

Table Q-5 shows the same comparison for mother’s highest grade attained. As seen here, similar

patterns exist for mothers as did for fathers. Again, while fewer of the CPS youths had mothers with less 

than a twelfth grade education compared to the NLSY CX sample (13.4 percent for CPS versus 17.6 

percent for NLSY), and more of the CPS youths had mothers with a twelfth grade education through a 

bachelor’s degree (81.2 percent for CPS compared to 74.7 for NLSY), the individual category differences 

are modest. And, once again, the NLSY had more youths with mothers who went beyond a bachelor’s 

degree (7.7 percent for NLSY, compared to 5.3 percent for CPS). 

NLSY--CX NLSY--SU

Table Q-5 Mother’s Highest Grade Attained 

CPS NLSY--Total

Highest Grade Attained N% N% N% N %

8th Grade or below 461 5.5 5.6 18.3 691 8.6

9th through 11th Grade 659 7.9 729 12.0 416 21.5 1,145 14.3

12th Grade / High School Diploma 3,137 37.7 2,194 36.2 672 34.8 2,866 35.8

Some College, no BA / BS 2,407 28.9 1,534 25.3 335 17.3 1,869 23.4

1,214 14.6 801 13.2 115 6.0 916 11.5

Graduate Coursework / Higher Degree 440 5.3 468 7.7 40 2.1 508 6.4

TOTAL 8,318 100.0 6,064 100.0 1,931 100.0 7,995 100.0

338 353

BA / BS 

The biggest differences for both father’s and mother’s highest grade attained occur at the 9th

through 11th grade and some college, no BA/BS levels. While the definitions for 9th through 11th grades 

are identical for the CPS and the NLSY, there are distinct differences for the college years. The CPS 

categorizes persons with any amount of college below a bachelor’s degree (including associate’s degrees)

into the some college category, while the NLSY, which only collects year of school but not degree, only

includes persons with one, two, or three years of college in this category. This anomaly carries over into 

the later college categories, also. For instance, the CPS places only persons who attain a bachelor’s

degree into the BA/BS category, while the NLSY places all persons with four years of college into this

group. Thus, any of these parents who attended college for four years but did not obtain a bachelor’s

degree at all would be incorrectly included in the “BA/BS” category. Furthermore, those parents who did 

224
receive a bachelor’s degree, but did not obtain it until his or her fifth year of college or later, should be 

included in this category but would not be. The same applies to the final category, graduate coursework 

or higher degree. The CPS only includes those persons who have actually attained a master’s,

professional, or doctorate degree into this category. For NLSY, all persons whose highest grade attained 

was five or more years of college were included here. In other words, a person who had completed a

bachelor’s degree after six years would be incorrectly included here, as would someone who attended 

college for five years, for example, and did not receive any degree at all. Therefore, the higher percentage 

of NLSY youths with parents who attained more than a bachelor’s education may be explained by these 

definitional differences. Rather than being placed in this category, perhaps the surplus would be spread 

across the previous two college-level categories, helping to minimize the discrepancies there. However, 

despite the definitional differences that do exist, the grade distributions presented here are surprisingly

similar.

While we could produce tests of equivalence of the CPS and NLSY97 sample distributions, we 

would not believe such tests to be appropriate because of the definitional differences between the surveys

and their implications discussed above. Nevertheless, we performed a simple t-test to compare the means

of the weighted distributions of father’s and mother’s highest grade attained for the two samples. The

fathers of NLSY97 youths have an average highest grade attained of 3.5 out of the 6 total categories 

(somewhere between 12th grade and some college, no BA), while fathers of CPS youths have a mean

education of 3.7 on the 6-point scale. Likewise, mothers of NLSY97 youths attained an average highest

grade of about 3.3 out of 6, compared to 3.5 out of 6 for the mothers of CPS youths. The t-values

associated with the differences in means are 4.9 and 5.5, respectively, which are both significant at the

0.01 level; still, the variances of these distributions were found to be not significantly different. However, 

besides the definitional and procedural differences between the surveys discussed earlier, the criteria for a 

formal t-test, including simple random sampling, are not met here. At this writing we have not yet

estimated the design effects for the NLSY97 sample, but are certain that they will be greater than one. 

Borrowing the design effect for the full NLSY79 sample of youths (DEFF approximately equal to 2), the 

results of the t-tests would be smaller, but would still remain significant. Finally, we must note that these

samples, when weighted, are enormous, and with any sample large enough many differences appear to be 

significant. Taken as a whole, these limitations further convince us of the inadequacy and 

inappropriateness of this type of this test. 

Summary. Given sampling variability and definitional and procedural differences 

between CPS and NLSY parent grade distributions, the comparisons are reassuring. Our analyses do not 

prove that there is no difference between NLSY and CPS youths with respect to parents’ educational 

attainment. Yet, in our view, these data offer no conclusive evidence of important differences. On

balance, the analyses reveal little or no evidence of nonrepresentativeness of the NLSY sample.

225
APPENDIX R 

Two Comparisons of Expected

vs. Observed Youth Counts 

226
In this appendix, we compare actual NLSY97 youth eligible counts with estimated expected 

counts in two ways.25 First, we compare these counts at the sample (CX vs. SU) level, paying particular 

attention to the two sampling strata (HIGH vs. LOW-LOW). Second, we compare these counts at the

PSU level, looking for patterns and relationships in the shortfalls.

In order to make these comparisons, we needed to estimate the expected counts of NLSY97 youth

eligibles that we would find at the segment level. These expected segment counts are described in this 

paragraph. For each and every selected segment, we obtained 1990 Census counts of the number of 

youths under age 18 in three race/ethnicity categories: Hispanic, non-Hispanic black, and total. We could 

then calculate the number of non-Hispanic nonblacks under age 18 (Total – Hispanic – non-Hispanic 

black). We now needed to get counts for 12- to16-year-olds from these 0- to 17-year-old counts. Even 

though more complicated methods (e.g. using the latest age distributions from the Current Population

Survey) could have been used, we used the simplest. We assumed a uniform distribution of youths

between the ages of 0-17, which meant a multiplication adjustment of 5/18. We then adjusted the counts 

for the segment sampling rate for HUs in each segment. For example, if we selected half of the listed

housing units, we would expect to find half of that segment’s eligible 12- to 16-year-olds. One final 

adjustment is needed, though. We need a screening adjustment because we are not going to find youths

in households where a screener is not completed. Therefore, we need to multiply the counts of youths by

the screening rate. During the sampling process, we assumed a 91 percent screener completion rate 

(based on previous surveys), but for this appendix, we used the actual screener completion rate of 94

percent. Here is a summary of how we obtained the expected segment counts (counts below are 1990

Census counts of youths under age 18): 

x Hispanic 12-16: (Hispanic 0-17) + .94 + (5/18) + (segment sampling rate for HUs) 

x Non-Hispanic black 12-16: (Non-Hispanic black 0-17) + .94 + (5/18) + (segment sampling rate for 

HUs)

x Non-Hispanic nonblack 12-16: (Total – Hispanic – non-Hispanic black) + .94 + (5/18) + (segment

sampling rate for HUs) 

These expected counts are compared to the actual counts of 12- to 16-year-old youth eligibles found

during screening. 

227

25 Final updating of the NLSY97, Round 1 disposition codes was done in November 1999, in light of all evidence 

on eligibility collected to that point in time, including certain evidence collected during Round 2 operations. 

Because the tabulations cited in this appendix reflect an earlier generation of the Round 1 disposition codes, there 

may be small discrepancies between counts cited here and the final counts cited in the main body of this NLSY97

Technical Sampling Report.
Comparison of Two Samples (CX vs. SU), Focusing on Two Strata for SU Sample

As described in Chapter 3, during the selection of the housing units for the SU sample, selected 

segments were divided into two strata: “high in Hispanic and/or black youths,” or “low in Hispanic 

youths, and low in black youths.” These two strata shall be referred to as HIGH and LOW-LOW for the 

remainder of this appendix. Some of the sampling strategy used for NLSY97 was based upon the

expected segment counts of eligible youths described above. For example, a segment was classified as

HIGH or LOW-LOW based upon these (and the housing unit) counts. One concern is how out-of-date 

the 1990 Census counts are by the time of the 1997 screening operation for NLSY97. It is certain that the 

correlation between 1990 Census counts and actual 1997 counts would be less than 1. This would

indicate that in 1997, the HIGH segments would not be quite as high in minority youths as we would 

expect, and the LOW-LOW segments would not be as low in minority youths as we would expect (the 

regression effect). In other words, there would be movement of minority youths from HIGH segments to 

LOW-LOW segments. One hypothesis that this analysis tries to examine is whether this regression effect

can explain any part of the shortfalls. If the HIGH and LOW-LOW segments were sampled at the same

rate, finding lower counts of minority youths than expected in the HIGH segments, and higher counts

than expected in the LOW-LOW segments might balance out. However, since the HIGH segments were 

sampled at ten times the rate of the LOW-LOW segments, the expected shortfalls in minority youths in 

the HIGH segments should be about ten times as large as the expected “longfalls” in minority youths in 

the LOW-LOW segments.

So, the below analysis compares the actual and expected counts of minority youths in the HIGH 

and LOW-LOW segments. With very little additional effort, these counts, as well as actual and expected 

counts of non-minority youths, are also analyzed for the CX sample.

Table R-1 below shows the (estimated) expected counts of youths aged 12-16. It should be noted 

that the counts of non-Hispanic, nonblack eligibles expected in the SU sample is zero (0) because nonHispanic, nonblack youths are ineligible for NLSY97 in the SU sample. However, the expected number

of non-Hispanic, non-black 12- to 16-year-olds is shown in Table R-1 in order to show how different the 

race/ethnicity mix is between the two strata of SU segments. Table R-2 below shows the actual counts of 

eligible youths aged 12-16 that were screened. 

228
Table R-1 Expected Counts of Youths Aged 12-16 to be Screened 

Sample Hispanic

Non-Hispanic,

black

Non-Hispanic,

nonblack Total

CX 1,164 1,545 7,704 10,413

HIGH 1,431 2,000 1,153 4,584

SU LOW-LOW 60 49 859 968

TOTAL 1,491 2,049 2,012 5,552

TOTAL 2,655 9,716 15,965 2,654

Table R-2 Actual Counts of Youths Aged 12-16 Screened 

Sample Hispanic

Non-Hispanic,

black

Non-Hispanic,

nonblack*

Total

Eligibles

CX 1,026 1,175 5,134 7,335

HIGH 1,055 1,330 0 2,385

SU 47 0 88

TOTAL 1,102 1,371 0 2,478

TOTAL 2,128 2,546 5,134 9,808

LOW-LOW 41

*Non-Hispanic nonblacks were ineligible in the SU sample, and were not counted during the screening 

process. Therefore, these numbers are shown as zero in Table R-2. 

Comparing the numbers in the above tables can be made easier by looking at the percentage of 

expected youths that were actually screened. These percentages are shown in Table R-3, which is made

trickier by the SU non-Hispanic nonblacks in Table R-1. Since non-Hispanic nonblacks are ineligible in 

the SU, they are removed from the denominator in Table R-3. 

Table R-3 Youths Aged 12-16 Actually Screened as a Percent of Expected Youths 

Sample Hispanic

Non-Hispanic,

black

Non-Hispanic,

nonblack

Total

Eligibles

CX 88.00% 76.11% 66.71% 70.5%

HIGH 73.64% 66.81% N/A 69.7%

SU 78.92% 84.32% N/A 81.4%

TOTAL 73.85% 67.22% N/A 70.0%

TOTAL 80.05% 71.06% 66.71% 70.4%

LOW-LOW

Table R-3 shows that both the CX and SU samples screened about 70 percent as many youths as

were expected. However, in the CX sample, the shortfall was concentrated among non-Hispanic 

nonblacks, and, to a lesser extent, among non-Hispanic blacks. In the SU sample, of course, non-Hispanic 

nonblacks were ineligible. Overall, there was less of a shortfall in the SU sample among Hispanic youths

(73.85 percent vs. 67.22 percent for non-Hispanic blacks). 

Table R-3 also allows us to compare the two sampling strata in the SU sample. Recall that 

229
because of the time between the 1990 Census and NLSY97, we hypothesized that we would screen fewer 

minority youths than we expect in the HIGH segments, but more minority youths than we expect in the 

LOW-LOW segments. In fact, there were shortfalls in both sampling strata, but the shortfall in the LOWLOW segments (81.35 percent) was much smaller than in the HIGH segments (69.66 percent). 

We now make some complicated calculations to estimate the shortfall (number of youths) caused

by using the seven year-old Census data to draw the NLSY97 sample. We start by calculating the

expected and observed percentages of minority youths in segments that would be classified as HIGH.

Table R-1 shows that we expected to screen 3,431 (1,431 + 2,000) minority youths in the HIGH

segments. If we had sampled LOW-LOW segments at the same rate (we sampled them at one-tenth the 

rate), we would have expected 1,090 ([60 + 49] * 10) minority youths in the LOW-LOW segments. This 

gives an estimate of 3,431/4,521 = 75.9 percent of the 1990 Census minority youths being in HIGH

segments. A similar calculation using numbers from Table R-2 gives an estimate of (1,336 + 

1,054)/(1,336 + 1,054 + 10 * [41 + 47]) = 73.1 percent of 1997 minority youths being in segments that

would be classified as HIGH using 1990 Census data. So, if there had been no other shortfall, we would

have still expected to find only (73.09/75.89) * 3,431 = 3,304 (127 fewer) minority youths in HIGH 

segments, but would have expected to find (1-.7309/1-.7589) * 109 = 122 (13 more) minority youths in

LOW-LOW segments. Overall, this is an expected shortfall of 114 minority youths in the SU sample.

The total shortfall in the SU sample was (1,491 + 2,049 – 1,377 – 1,101 = ) 1,062. Therefore, the simple

fact that sampling for NLSY97 was done using seven year-old information explains an estimated

114/1,062 = 10.7 percent of the SU shortfall. Of course, it should be noted that this does not explain why

there were fewer 12- to 23-year-olds screened than youths of the age groups outside this range, which is

the most troubling aspect regarding the shortfalls. 

The analyses in this section could be done using a weight for each segment count of expected and 

actually screened youths. However, the results would change very little. This is because the proper 

weights would be the housing unit base weights for each segment, which are almost uniform within each 

of the CX, SU HIGH, and SU LOW-LOW cells of segments.

PSU-Level Analysis 

We now look at PSU-level shortfalls, trying to discover any patterns that might be present. We 

look at the CX and SU PSUs separately because the natures of the shortfalls are different. In the SU 

sample, non-Hispanic, nonblack 12- to 16-year-olds are ineligible for NLSY97. Therefore, any shortfalls

in the SU sample are necessarily among Hispanics and non-Hispanic blacks. In the CX sample, the 

shortfalls are most pronounced among non-Hispanic nonblacks. As we did in the HIGH versus LOWLOW analysis, we examine the number of youths aged 12-16 actually screened as a percent of expected 

youths. Table R-4 below shows some percentile statistics of these PSU-level percentage variables. 

230
Table R-4 shows that fewer youths were screened than were expected in just over 75 percent of

the CX PSUs (Percentile 75 = 99.23 percent), as compared to just under 75 percent of the SU PSUs 

(Percentile 75 = 103.76 percent). Means are not provided because they are heavily increased by the 

largest (extreme) percentages. A better picture of a “typical” PSU is given by the medians. However, 

one odd result is that the SU median percentages for black and Hispanic youths are both much lower than 

the SU median percentage for all youths. This (along with the maximum percentages) indicates that these

percentages are highly variable for blacks and Hispanics. However, as shown by the histograms in 

Figures R-1 through R-3, the non-minority and all youth percentages have a roughly bell-shaped curve. 

Histograms are not shown for the percentages for black and Hispanic youths because the variability is too 

large.

The main reason for showing the three histograms in Figures R-1 through R-3 is to indicate that

the percentages do follow a regular pattern when there are enough expected youth cases to form a stable

ratio of actual youths screened over the number of youths expected to be screened. In particular, none of 

the three histograms shows any outliers that might indicate a problem, except possibly the zero in Graph 

R-3. This is PSU 703, in which the number of expected minority youths was less than 0.5. Three

particular PSUs that might be of interest in Graph R-3 are the three PSUs with a particularly low screener 

response rate. These three PSUs, with their percentages of expected youths screened, are PSU 764 (56.43 

percent), PSU 795 (65.26 percent), and PSU 862 (67.51 percent). None of these three PSUs is among the

ten lowest (see below) percentages of expected youths screened in the SU sample.

231
232

Table R-4. Some PSU-Level Percentile Statistics on Percentage Variables (Percentage of Expected Youths Screened)

.00% .00% 24.52% 87.64% 117.25% 288.56% 1338.76%.00% .00% 74.89% 125.72% 242.86% 867.88% 1413.49%

12.29% 42.24% 68.79% 81.49% 93.57% 121.94% 140.29%44.44% 55.88% 76.37% 87.43% 99.23% 120.90% 154.07%

.00% .00% 44.77% 70.79% 95.49% 157.19% 281.93%.00% .00% .00% 75.32% 117.61% 291.09% 2920.64%

 n/a n/a n/a n/a n/a n/a n/a

.00% 34.83% 64.44% 87.54% 103.76% 131.50% 160.05%

Pct. of Expected Blacks ScreenedPct. of Expected Hispanics ScreenedPct. of Expected Non-Minorities ScreenedPct. of Expected Youths Screened

CX

Pct. of Expected Blacks ScreenedPct. of Expected Hispanics ScreenedPct. of Expected Non-Minorities ScreenedPct. of Expected Youths Screened

SU

Minimum

Percentile

05

Percentile

25 Median

Percentile

75

Percentile

95 Maximum

Figure R-1 Histogram of CX Non-Minority Youth Percentages 

0 50 100 150

0 5 10 15

Non-Minority Youths Age 12-16 Actually Screened as a Percent of Expected Youths

Number of CX PSUs
Figure R-2 Histogram of CX All Youth Percentages 

0 50 100 150

0 5 10 15

Youths Age 12-16 Actually Screened as a Percent of Expected Youths

Number of CX PSUs

Figure R-3 Histogram of SU All Youth Percentages

0 50 100 150

0 2 4 6 8 10

Youths Age 12-16 Actually Screened as a Percent of Expected Youths

Number of SU PSUs

233
We now categorize each PSU in terms of whether the number of youths actually screened

was as many as were expected. Table R-5 divides up the 200 PSUs into the following four 

categories:

1) Zero Expected – The expected count of youths aged 12-16 to be screened was less than 0.5, 

2) Zero Actual – The expected count was greater than 0.5, but none were screened, 

3) Under 100% - The actual count of youths screened was less than the count expected, and 

4) At least 100% - The actual count of youths screened was at least as many as were expected. 

Table R-5 divides up the CX and SU PSUs separately, and for all four different percentage

variables (counts of Hispanic, non-Hispanic black, non-Hispanic nonblack, and total youths aged 

12-16 actually screened as a percentage of expected youths).

Table R-5 Classification of Four PSU-Level Percentage Variables 

Zero Expected Zero Actual Under 100% Over 100%

11 5 22 62

Non-Hispanic black 13 11 36 40

Non-Hispanic nonblack 0 0 87 13

CX

All Youths 12-16 0 0 78 22

Hispanic 25 6 35 34

Non-Hispanic black 8 9 62 21

Non-Hispanic nonblack n/a n/a n/a n/a

SU

All Youths 12-16 1 0 68

Hispanic

31

In the CX sample, there are eleven PSUs that expected no Hispanic 12- to 16-year-olds

and thirteen different PSUs that expected no non-Hispanic, black 12- to 16-year-olds. However, 

there were no CX PSUs that expected zero non-Hispanic, nonblack 12- to 16-year-olds. One of 

the PSUs that didn’t expect any Hispanic eligibles actually screened two in (this PSU is still 

included in the “Zero Expected” column). There were an additional five CX PSUs that didn’t 

screen any Hispanic eligibles, but only one Hispanic eligible was expected in each of these PSUs. 

Of the eleven CX PSUs that expected at least one non-Hispanic black, but didn’t screen any in,

one PSU expected around three, three expected around two, and the other seven expected only

one.

Focusing now on the last two columns for the CX, 87 of the PSUs found fewer nonHispanic, nonblack eligibles than were expected. Meanwhile, more than half of the PSUs (62) 

found more eligible Hispanic youths than were expected. This seems surprising given that Table 

R-3 shows that over the entire CX sample, the number of Hispanic youths screened was only

91.33 percent as many as were expected. Also, discounting the zero expected and zero actual 

cells, there are more PSUs (40) that screened more non-Hispanic, black eligibles than were

expected than PSUs (36) that screened fewer. It seems clear that the shortfalls that lead to only

22 CX PSUs screening as many 12- to 16-year-olds as were expected (78 falling short) were 

driven by the shortfalls in non-Hispanic, nonblack youths.

In the SU sample, there are twenty-five PSUs that expected no Hispanic 12- to 16-yearolds and eight PSUs that expected no non-Hispanic, black 12- to 16-year-olds. In fact, PSU 703, 

chosen to represent counties with very few minority youths, expected zero minority 12- to 16-

year-olds. One of the PSUs that didn’t expect any Hispanic eligibles actually screened in one. 

234
There were an additional six SU PSUs that didn’t screen any Hispanic eligibles, but only one

Hispanic eligible was expected in five of these PSUs; PSU 823 actually expected seven. Of the

nine SU PSUs that expected at least one non-Hispanic black, but didn’t screen any in, two PSUs

expected around four, two expected around three, one expected around two, and the other four 

expected only one. 

We now examine the PSUs with the largest shortfalls in order to see if there are any

patterns shared by these PSUs. We examine the 10 PSUs for each of the CX and SU samples

with the lowest counts of youths (of all race/ethnicities) aged 12-16 actually screened as a percent 

of expected youths. Table R-6 shows the expected counts of youths aged 12-16 to be screened. 

Table R-6 Expected Youths Aged 12-16 to be Screened for 10 “Lowest” CX and SU PSUs 

Hispanic

Non-Hispanic,

black

Non-Hispanic,

non-black Total Eligibles

PSU 857 6 10 78 95

PSU 717 23 36 75 134

PSU 884 0 30 64 95

PSU 807 7 11 38 55

PSU 860 4 8 42 54

PSU 868 36 15 16 68

PSU 729 0 11 44 55

PSU 806 0 2 64 66

PSU 899 1 29 42 73

CX

PSU 804 1 0 55 56

PSU 866 1 16 26 43

PSU 896 3 0 23 26

PSU 870 0 9 35 44

24 7 45

PSU 702 3 7 60 71

PSU 882 0 18 5 23

PSU 900 28 4 23 55

PSU 790 0 31 12 44

PSU 786 23 21 21 65

SU

PSU 863 26 0 51 77

PSU 813 13

We have studied the locations of these twenty PSUs and have concluded that they do not 

follow a strong geographic pattern. Eight of the ten lowest CX PSUs are urban, two of which are 

certainty PSUs. Eight of the ten lowest SU PSUs are also urban, three of which are certainty

PSUs. Louisiana does have one rural parish in each sample among the ten lowest, and three of the 

ten lowest CX PSUs are in Tennessee. Table R-7 shows the actual counts of youths aged 12-16

screened.

235
Table R-7 Actual Youths Aged 12-16 Screened for 10 “Lowest” CX and SU PSUs 

Hispanic

Non-Hispanic,

black

Non-Hispanic,

nonblack

Total

Eligibles

PSU 857 2 2 38 42

PSU 717 23 15 28 66

PSU 884 1 21 27 49

PSU 807 4 12 14 30

PSU 860 5 4 21 30

PSU 868 27 9 2 38

PSU 729 1 2 30 33

PSU 806 0 2 38 40

PSU 899 0 22 22 44

CX

PSU 804 1 0 34 35

PSU 866 1 2 n/a 3

1 0 n/a 1

PSU 870 0 3 n/a 3

PSU 813 2 11 n/a 13

PSU 702 2 2 n/a 4

PSU 882 0 7 n/a 7

PSU 900 14 0 n/a 14

PSU 790 0 14 n/a 14

PSU 786 9 11 n/a 20

SU

PSU 863 12 0 n/a 12

PSU 896 

It should be noted that of the zeroes shown in Table R-7, only one is a significant part of

the overall PSU shortfall. The expected number of non-Hispanic blacks in PSU 900 was four.

However, even if four non-Hispanic blacks had been screened, they would not have missed this

list of ten lowest by much. For all of the other zeroes shown, the corresponding expected count

was zero (i.e., less than 0.5) or one. Finally, Table R-8 shows the counts of youths aged 12-16 

actually screened as a percentage of expected youths.

236
Table R-8 Actual Youths Aged 12-16 Screened as a Percentage

of Expected for 10 “Lowest” CX and SU PSUs 

Hispanic

Non-Hispanic,

black

Non-Hispanic,

non-black

Total

Eligibles

PSU 857 34.15% 19.26% 48.55% 44.4%

PSU 717 100.42% 41.92% 37.36% 49.4%

733.27% 69.70% 41.90% 51.7%

PSU 807 60.95% 112.42% 37.13% 54.6%

PSU 860 119.41% 52.80% 50.07% 55.9%

60.2%

59.48%

74.71%

CX

PSU 804 0.00% 144.34%

PSU 866 

30.9%

n/a

27.10%

0.00% 39.30% n/a 38.6%

PSU 900 49.60% 0.00% n/a 43.1%

PSU 790 0.00% 44.46% 44.1% n/a

PSU 786 38.35% 53.65% n/a 45.5%

PSU 863 45.84% n/a 45.8%

PSU 884 

PSU 868 74.13% 60.25% 12.29% 56.2%

PSU 729 287.49% 18.59% 68.58%

PSU 806 n/a 90.04% 60.5%

PSU 899 0.00% 52.76% 60.6%

61.34% 62.0%

91.44% 12.72% n/a 17.8%

PSU 896 30.55% 0.00% n/a 29.8%

PSU 870 0.00% 32.13%

PSU 813 14.89% 45.71%

PSU 702 62.95% n/a 37.9%

PSU 882 

SU

n/a

34.7%

n/a

It is clear that the percentages given in Table R-8 are lower among the SU sample, but this simply

reflects the larger variability in the SU sample caused by smaller numbers of eligibles expected. 

Among the CX PSUs, most of the small overall percentages are driven by a low percentage

among non-Hispanic nonblacks. However, PSU 717 also had a large shortfall among nonHispanic blacks, and PSU 868 only expected 16 non-Hispanic nonblacks (compared to 36 

Hispanics and 15 non-Hispanic blacks). Also, PSU 857 had only four minority youths screened

(sixteen were expected). Among the SU PSUs, there doesn’t seem to be any pattern in the 

shortfalls. Five of the lowest ten PSUs were low in non-Hispanic blacks, three were low in 

Hispanics, and two (PSUs 813 and 786) were low in both. 

To conclude our PSU-level analysis, we try to look for relationships between the 

shortfalls and the screener response rate and two PSU-level socio-economic variables (also 

studied in Appendix F). Figures R-4 (CX) and R-5 (SU) below look at the relationship between 

the PSU counts of youths screened as a percentage of youths expected and the PSU screener

response rate. These figures show a positive relationship in the CX sample, but no relationship in

the SU. In the CX sample, the shortfalls become greater as the PSU screener response rate 

decreases. This is not very surprising because the expected numbers were calculated under the 

assumption that the screener response rates would be equal (94 percent) for all segments (and 

thus, PSUs). Unlike the analyses with socio-economic variables, we do not perform a simple

regression analysis for screener response rate because we believe that any effect that is shown

here is an artifact of the methodology used to calculate the expected counts of youths to be 

screened. It should be mentioned that Figure R-5 does exclude five PSUs with a screener 

237
response rate of less than 80 percent. These excluded points fit the pattern shown, though (no

relationship with the percentage of expected youths screened). 

Figure R-4. Relationship of PSU Shortfalls to PSU Screener Response Rates – CX

PSU Screener Response Rate

80% 85% 90% 95% 100%

Pct. of Expected Youths Screened

160%

150%

140%

130%

120%

110%

100%

90%

80%

70%

60%

50%

40%

Figure R-5. Relationship of PSU Shortfalls to PSU Screener Response Rates – SU

PSU Screener Response Rate

80.0% 85.0% 90.0% 95.0% 100.0%

Pc

t.

of

Ex

pe

cte

d

Yo

uth

s

Sc

re

en

ed

200%

150%

100%

50%

0%

We next examine the possibility of a relationship between the percentage of expected 

238
eligibles actually screened in and socio-economic variables. Figures R-6 (CX) and R-7 (SU) 

below look at the relationship between the PSU counts of youths screened as a percentage of 

youths expected and the PSU percentage of adults over 25 with a bachelor’s degree, as taken 

from the 1990 Census. The two graphs show opposite trends! In the CX sample, PSUs with 

more college graduates have greater shortfalls, while in the SU sample, PSUs with fewer college 

graduates have greater shortfalls. The two regression lines are: 

YCX = 103.6% - .83 * XCX

YSU = 75.6% + .48 * XSU

where Y = Actual counts of youths age 12-16 screened as a percentage of youths expected 

and X= Percentage of adults at least 25 years old with a bachelor’s degree. 

The slope is significant for the CX (p=.002), but insignificant for the SU (p=.202). 

Comparing the predictions these lines make for a PSU with a low percentage of bachelor’s

degrees (say, 10 percent) with the prediction for a PSU with a high percentage of bachelor’s

degrees (say, 30 percent) gives an idea of the strength of these relationships. With a percentage 

college graduates of 30, the CX line predicts that 95.3 percent of the eligibles expected would be 

screened, while the SU line predicts that 80.4 percent of the eligibles expected would be screened.

With a 30% percentage of college graduates, the CX line predicts that 78.7 percent of the

eligibles expected would be screened, while the SU line predicts that 90.0 percent of the eligibles 

expected would be screened. There seems to be some evidence here that in the CX, PSUs with

more college graduates have larger shortfalls. However, in the SU, PSUs with more college 

graduates have smaller shortfalls, even if this trend is not significant. 

The third and final relationship we examine is between the percentage of expected 

eligibles actually screened in and income. Figures R-8 (CX) and R-9 (SU) below look at the

relationship between the PSU counts of youths screened as a percentage of youths expected and 

the PSU median family income, also taken from the 1990 Census. These figures show a positive 

relationship in the CX, but no relationship in the SU. In the CX sample, the shortfalls become

greater as the PSU median family income increases. A simple regression line is shown in each of

the figures. For the CX sample, the regression line is: 

YCX = 115.9% - .82 * XCX

where YCX = Actual counts of youths aged 12-16 screened as a percentage of youths expected,

and XCX= Median family income, in thousands of dollars ($1000s). 

239
Figure R-6 Relationship of PSU Shortfalls to PSU Education – CX 

Percent Bachelors Degree or Higher

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%

Pct. of Expected Youths Screened

160%

150%

140%

130%

120%

110%

100%

90%

80%

70%

60%

50%

40%

Figure R-7 Relationship of PSU Shortfalls to PSU Education – SU 

Percent Bachelors Degree or Higher

0% 10% 20% 30% 40% 50% 60%

Pct. of Expected Youths Screened

200%

150%

100%

50%

0%

240
Figure R-8 Relationship of PSU Shortfalls to PSU Median Family Income – CX 

Median FamiIy Income, 1990 Census

$10000 $20000 $30000 $40000 $50000 $60000

Pct. of Expected Youths Screened

160%

150%

140%

130%

120%

110%

100%

90%

80%

70%

60%

50%

40%

Figure R-9 Relationship of PSU Shortfalls to PSU Median Family Income – SU 

Median FamiIy Income, 1990 Census

$10000 $20000 $30000 $40000 $50000 $60000 $70000

Pct. of Expected Youths Screened

200%

150%

100%

50%

0%

241
Comparing the predictions this line makes for a PSU with a low median family income

(say, $25,000) with the prediction for a PSU with a high median family income (say, $45,000) 

again gives an idea of the strength of this relationship. With a median family income of $25,000, 

this line predicts that 95.4 percent of the eligibles expected would be screened. With a median

family income of $45,000, this line predicts that 79.0 percent of the eligibles expected would be 

screened. We do not show the regression equation for the SU sample because the regression line 

is almost completely flat, and the slope is not significantly different from zero. 

Conclusions

In the first analysis, comparing the HIGH and LOW-LOW segments, we showed that the 

shortfalls in the CX were deepest among non-Hispanic nonblacks, and that in the CX, more nonHispanic blacks were actually screened in than were expected. In the SU sample, the shortfalls 

were deeper among Hispanic youths than among non-Hispanic, black youths. Also, the shortfall 

in LOW-LOW segments was much smaller than in HIGH segments, as we would expect because 

of the seven-year difference between the 1990 Census data used for sampling and the 1997

screening operation. In fact, we estimated that around 12 percent of the SU shortfall can be 

explained by this seven-year gap. 

In the second analysis, at the PSU level, we showed through histograms that the actual 

counts of youths screened in as a percentage of expected youths followed a roughly normal

distribution when the expected counts were large enough to form a stable ratio. In particular, the 

percentages for minorities were not stable, but the overall and non-minority percentages were 

stable. These histograms also showed a lack of outliers, or problem PSUs.

An examination of the 10 CX and SU PSUs with lowest percentage of expected youths

screened did not show any geographical pattern. These PSUs re-iterated the point that CX 

shortfalls were largely among non-Hispanic nonblack youths, but didn’t show any patterns for the 

SU sample. We also showed that the SU shortfalls, at the PSU level, were unrelated to the 

screener response rate and median family income. There was a trend for PSUs with fewer college

graduates to have higher shortfalls, but the slope of the regression line was not significantly

different from zero. In the CX sample, higher PSU shortfalls were significantly related to lower 

PSU screener response rates, higher rates of college graduates (opposite to the trend in the SU 

sample), and higher median family incomes. Therefore, there is some evidence that CX (but not 

SU) shortfalls might be higher among more affluent PSUs.

In the overall picture, however, the most worrying aspect of NLSY97/PAY97 is the fact

that many fewer 12- to 23-year-olds were in the screening sample than persons under the age of 

12 and persons over the age of 23 (the “age hole”). The analyses in this appendix are directed at 

looking for patterns in the actual shortfalls of eligible youths, rather than giving any insight into

the “age hole” itself. Finally, all of the analyses in this appendix depend upon the expected 

counts of eligible youths; in particular, the assumptions made in calculating them.

242
APPENDIX S 

Adjusted Screener Weights

Used for Age Distribution Analysis 

243
In this appendix, we describe the adjusted screener weights that were used in Section 5.3 

to compare the distribution of all persons 35 years old or younger from NLSY97 screeners to the 

March, 1997 Current Population Survey (CPS). The motivation for these adjusted screener

weights (referred to below as age-distribution weights) is that there are two types of screeners for 

which we do not know the age distribution but we do know whether or not there are any ageeligible NLSY97 youths present: 

a) proxy/gatekeeper cases – we know that there are no age-eligible NLSY97 youths

present, and 

b) “lost” screeners – there are 32 screeners for which we know that there are ageeligible NLSY97 youths present, but the screener data (and therefore the age

distribution) is unavailable. 

These screeners have positive screener weights but are missing the age distribution. We treat 

these screeners as missing data, and add a separate step to this weighting procedure: adjustment

for age-distribution “nonresponse.”

The age-distribution weights generally follow the first four steps that are described for 

youth weights in Section 4. We now proceed through these steps, describing the modifications

made:

Step 1. Base weight with no truncation. The base weight (W ) for the k-th housing unit 

in the screening sample is the inverse of the probability of selecting the unit:

1

W k

k

1

1 S , 

where S k denotes the relevant inclusion probability. In Chapter 4, we described how we 

truncated the base weights in the supplemental (SU) sample, as well as the missed-housing unit 

adjustment. For the purposes of the age-distribution weights, no truncation was performed.

Step 2a. Adjustment for screener nonresponse. The next step is to adjust the screener 

base weights for nonresponse to the screening interviews. The nonresponse-adjusted weight

(W ) is the base weight (W ) inflated by the inverse of the weighted response rate within an

adjustment cell: 

2 1

W

W

W k W

j

j E

j

j S

2 k

1

1

 1





¦

¦

D

D

Where ED is the set of eligible units within the D-th cell and is the set of screener respondents 

in the D-th cell. For the youth weights, we adjusted weights within segments. We used the same

approach for these age-distribution weights. This step distributes the W weight of the screener 

noninterviews across the screener completes within the same segment (or PSU if the segment

SD

1

244
contains fewer than 25 screener completes). There were no deviations in this step from the

methodology described in Chapter 4. 

Step 2b. Adjustment for special disposition codes 71, 74, and 77. The purpose of

these weights is to analyze the age distributions from the household screener data. However, after 

step 2a, some households have positive weights but no age distributions. We do have 

information, however, on whether or not these households have any youths aged 12-23. 

Households with disposition codes 71 and 74 (gatekeepers and proxies) are known to have 

reported no youths aged 12-23, but the household may have younger or older persons. 

Households with disposition code 77 (screener complete, but data unavailable) are known to have 

reported youths aged 12-23, but the report is missing the number of younger or older persons. 

Therefore, we can split all screened households into four groups, based on whether we have an 

age distribution and whether there are any reported youths aged 12-23. Table S-1 will be helpful 

in describing these four groups (the number of cases in each group are in parentheses). 

Table S-1 The Four Groups Used in Step 2b of the Age-Distribution Weighting 

Screener Completes,

No Youths Reported 

Screener Completes,

Some Youths Reported 

Age distribution Disposition codes 6X (53,101) Disposition codes 6X (13,046) 

No age distribution Disposition code 71 and 74 (9,231) Disposition code 77 (32) 

The cases with age distributions all have a disposition code in the sixties (6X). Some of these 

cases have youths aged 12-23, and some do not. Households with a disposition code of 71 or 74 

are more like those with an age distribution but no youths aged 12-23, while households with a

disposition code of 77 are more like those with an age distribution that do have at least one youth

aged 12-23.

Step 2b distributes the W weight of the screened households without an age distribution

across the households with age distributions within the same cell, where the cell is determined by

whether or not youths have been reported. In this sense, step 2b is an extra nonresponse weight

adjustment where completes and incompletes are determined by whether an age distribution is 

available, and the only two cells given different adjustments are determined by whether any

youths have been reported by that household. We could have used different cells for differing

numbers of youths reported, but it is undesirable to split the 32 cases (households) with the 

disposition code of 77 any further. 

2

The adjustment factor for Step 2b is 1.155 in the “no youths reported” cell, and 1.006 in

the “some youths reported” cell. This reflects the fact that very few of the households with 

reported youths do not have an age distribution, while the proxies and gatekeepers make up a 

sizable proportion of the households with no youths reported.

245
Step 3. Adjustment for subsampling ETP youths. This step is not applicable.

W W 3 2 k k .

Step 4. Combine the CX and SU samples using precision weights. At this point, the

sum of the weights in each sample is approximately the total number of U.S. housing units. 

However, we would like the sum of the weights for both samples combined to approximate the

total number of U.S. housing units. Therefore, as for the youth weights, we need to use precision 

weighting to combine the CX and SU samples.

In creating the youth (NLSY, ETP, and STP) weights, the CX and SU samples were 

combined using youth demographics (race/ethnicity, sex, and either age or grade). However, we 

desire to combine households, so we can only use household characteristics. The only data we 

have for all households is geographic. Therefore, the most logical cells for precision weighting 

would be Census region or division. We chose division as the smallest practical cell (state would

be the next smallest). Since the SU sample targets minorities, it consists largely of major cities 

and overrepresents the South and West. This will result in greater weight for the SU sample in 

the South and West, and relatively smaller weight elsewhere. The Os are shown in Table S-2 

below.

Table S-2 Os Used for Precision Weighting for Age-Distribution Weights

Division CX (O SU (1-O

New England .93 .07

Middle Atlantic .92 .08

East North Central .93 .07

West North Central .91 .09

South Atlantic .89 .11

East South Central .94 .06

West South Central .77 .23

Mountain .84 .16

Pacific .83 .17

W W i CX

W i SU

k k

K

4 3

1 3

 

  

 ,

(  ,

O

O

E

E

as in Chapter 4, where E signifies the Census division domain. W then is the age-distribution 

weight.

4k

246
APPENDIX T 

 Urban/Rural Age Distributions

247
In this appendix, we examine the issue of undercoverage of youths aged 12-23 in the 

NLSY97 in more detail by looking at the age distributions of different rural/urban groups. We 

classify the households in the screener into different groups based on their rural/urban status, and 

compare their age distributions to see if there are significant differences in patterns. We attempt

to identify whether there are specific groups (e.g. rural, urban, suburban) where the problem of 

undercoverage of youths aged 12-23 is greater than others. 

Classification of NLSY97 households into urban/rural categories: All the households 

in the screener, including those obtained by the half-open interval procedure, were matched to the 

census block in which they reside. Since PSUs could contain rural, urban and suburban areas, we 

have chosen to classify households into rural/urban categories at the block level instead of at a 

PSU level. The Bureau of the Census defines urban as “comprising all territory, population, and 

housing units located in urbanized areas and in places of 2,500 or more inhabitants outside 

urbanized areas.”26 An urbanized area (UA) is a continuously built-up area with a population of 

50,000 or more. It comprises one or more places called central places and the surrounding 

densely settled areas known as the urban fringe. Territory, population, and housing units not 

classified as urban are considered to be rural. The Census classifies all the census blocks as either 

rural or urban 27. Since the urban category is a very broad one, which encompasses a variety of

housing environments such as urban centers, suburbs, small cities, and towns, we have further 

classified urban areas into three categories:

(1) Central Places of Urbanized Areas -- these correspond to urban centers of urbanized

areas.

(2) Urban Fringe-- parts of urbanized areas which are not parts of central places. This

 category corresponds closely to the concept of a suburb in terms of population size,

 proximity and economic dependence on the urban center. 

(3) Urban areas outside urbanized areas--these are small cities and towns with a

minimum population of 2,500.

Blocks were classified into these categories based on place description codes from the 1990 

Census.28

Analysis of age distributions. In Chapter 5, we had identified the undercount for ages 

12-23 in the NLSY97 age distributions for the entire sample. This was done by comparing the 

number of persons in each age category for the weighted NLSY97 and Current Population Survey

(CPS) age distributions. Tables T-1 and T-2 show the unweighted and weighted29 counts of 

persons aged 0-35 for the NLSY97 screening sample in each of the following categories: total

(entire sample), central places of urbanized areas, urban fringes, urban areas outside urbanized

areas, all urban, and rural areas. We calculate the percentage of people in each age category out 

of all the individuals of ages 0-3530 using the weighted age distributions shown in Table T-2. This 

is done for the overall CPS31 as well as for different rural/urban groups in the NLSY97 and 

permits us to compare the different groups which are of different sizes. Age distributions

26 See Rural and Urban Classifications, www.census.gov 

27 1990 Census, Block Statistics, CD90-1B files were used for the rural/urban classification. 28 For details of place description codes see the data dictionary for the 1990 Census 1B files for the 

variable PLACEDC.

29 Weighting was done using special screener weights for age distributions. 30 The percentage of x year olds in each group = (number of x year olds/number of 0-35 year olds)*100 

31 Data on CPS age distributions for these rural/urban/suburban categories were not available.

248
constructed using these percentages are shown in Figures T-1 to T-5. In each case the NLSY97 

subgroup is compared to the overall CPS. 

When we compare the urban and rural age distributions (Figures T1 and T-2) we find in

both cases that there are spikes at ages 11 and 24 and a hole in the age distribution in the 12-23 

age range. The urban percentages gradually decline between ages 12 and 23, while the decline in 

the case of the rural age distribution is only slight at age 12, and there is a sharper decline from

age 15 to 23. The gap between the NLSY97 rural age distribution and the CPS is slight in the 12-

16 age range (NLSY97 eligible youths) and largest towards the end of the interval around ages 20 

to 23. 

Figure T-1 Weighted Percentages of Persons,

NLSY97 Urban Blocks and CPS

0.00

1.00

2.00

3.00

4.00

5.00

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35

"CPS total"

"NLSY97 urban"

Figure T-2 Weighted Percentages of Persons,

NLSY97 Rural Blocks and CPS

0.00

1.00

2.00

3.00

4.00

5.00

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35

"CPS total"

"NLSY97 rural"

When we look closely at the different subgroups within the urban category, we find that 

all of them have a hole in the age distribution in the 12-23 age range. However we observe certain

differences in patterns. Urban centers (central places of urbanized areas) have age distributions 

249
(Figure T-3) in which percentages in each age category decline sharply around age 12 and then 

remain relatively steady without showing a sharp decline in the 20-23 age range. The size of the

gap between the NLSY97 and the CPS is larger for the younger youths and smaller for older

youths. For the urban fringe (Figure T-4) the proportion of youths falls gradually in the 12-23 age 

range till it reaches the 22-23 age group, which is its lowest point. The gap between CPS and 

NLSY97 remains more or less constant throughout. Urban areas outside urbanized areas (Figure

T-5) have age distributions with spikes at 11 and 13, followed by a small, gradual decline in the 

14-23 age range and a sharper decline in the 20-23 age range.

Figure T-3 Weighted Percentages of Persons,

NLSY97 Urban Central Place Blocks and CPS

0.00

1.00

2.00

3.00

4.00

5.00

1 4 7 10 13 16 19 22 25 28 31 34

"CPS total"

"NLSY97 urban central place"

Figure T-4 Weighted Percentages of Persons,

NLSY97 Urban Fringe Blocks and CPS

0.00

1.00

2.00

3.00

4.00

5.00

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35

"CPS total"

"NLSY97 urban fringe"

250
Figure T-5 Weighted Percentages of Persons,

NLSY97 Urban, Outside UA Blocks and CPS

0.00

1.00

2.00

3.00

4.00

5.00

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35

"CPS total"

"NLSY97 urbanoutside UA"

Summary

A comparison of the NLSY97 age distributions of various groups based on whether they

are in urban centers, suburbs, other urban areas or rural areas reveals both similarities and 

differences between groups. All the rural/urban groups have a drop in the number of people in the 

12-23 age group, with spikes at ages 11 and 24, which supports the view that people may have 

misrepresented the number of youths in the NLSY97-eligible range in the screener. The shape of 

the age distributions do show some variation between groups. The urban age distribution shows a

gradual drop in the percentages of people in each age group in the 12-23 age range, while the 

rural age distribution shows a sharper drop, particularly for older youths. In urban centers, the 

proportion of persons in each age category shows a decline at age 12 and remains quite steady

around that level until age 23. Both suburban areas and urban areas outside urbanized areas show 

a gradual decline in percentages between ages 12 and 23, with the latter showing a more

noticeable depression at the 20-23 range.

When we compare the NLSY97 to the CPS age distributions, there appear to be some

differences in the undercount based on urban/rural status. The undercount is larger for the older 

youths in the rural subsample, while it is more evenly distributed over the 12-23 interval in the

urban subsample. Within the urban category, the urban centers show a larger gap in the younger

age group, while in suburban areas the gap is uniform across the 12-23 range. The sub-sample

comprising small cities and towns shows evidence of having a larger gap towards the end of the

12-23 range.

We are primarily interested in the shortfall of 12- to 16-year olds, since this is the group 

that was eligible for NLSY97. From a comparison of rural and urban age distributions, it appears

that the 12-16 group is undercounted in urban rather than in rural areas. Within the urban areas, 

all categories (urban centers, urban fringes and urban areas outside urbanized areas) have a

shortfall in the 12-16 age range, so we cannot attribute the undercount of youths to one particular 

type of urban area. 

251
We should interpret these results cautiously for two reasons. The first is that the results 

from both the CPS and the NLSY97 are subject to sampling error. The second is that we are using 

the overall CPS as a benchmark for comparisons with the NLSY97 for various rural/urban 

categories. CPS age distributions could vary across rural/urban groups, just as the NLSY97 does, 

in which case observed differences between the CPS and NLSY97 could be due to unobserved

differences in the benchmark age distributions rather than differences in the undercount.

Thus, our analysis of rural/urban age distributions does suggest that there are some

differences in the shapes of the distributions and undercounts of NLSY97 youths for different 

types of areas. However, there is no compelling evidence which would allow us to attribute the 

undercount to one particular type of rural/urban category.

252
Table T-1 Unweighted NLSY97 Age Distributions for Urban/Rural Groups 

Age Unweighted NLSY97 Counts by Urban/Rural Groups 

Total (Rural

and Urban) 

Central Place of

Urbanized Area 

Urban Fringe of 

Urbanized Area 

Urban area 

outside

Urbanized

Area

Total

Urban

Total

 Rural

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

0-35

2799

924

834

236

200

2166

1732

1972

1955

337

406

430

448

1855

2236

2367

2409

2416

2557

2586

2599

2555

2351

2459

2519

2152

2169

2250

2258

2293

1908

1838

1736

1667

1676

1460

1432

2290

2429

2300

2260

2403

2229

2243

2515

2506

2515

2922

81159

684

833

875

889

906

968

937

938

925

827

849

862

696

715

765

723

762

639

668

629

633

681

585

594

976

998

920

979

879

1002

852

912

881

896

1005

29807

607

755

732

773

776

846

855

864

834

753

798

826

719

692

738

758

749

623

594

550

560

514

488

507

717

756

733

694

777

677

894

680

827

807

1012

26319

150

208

215

213

223

228

222

229

213

214

249

187

222

171

190

208

162

159

166

154

144

129

107

191

199

208

217

225

270

200

226

247

234

257

7273

1441

1796

1843

1877

1895

2037

2020

2024

1988

1793

1861

1937

1602

1629

1674

1671

1719

1424

1421

1345

1347

1339

1202

1208

1884

1953

1861

1818

1973

1781

1937

2274

63399

414

440

524

532

521

520

566

575

567

558

598

582

550

540

576

587

574

484

417

391

320

258

224

476

439

442

633

511

543

551

578

648

17760

253
Table T-2 Weighted NLSY97 Age Distributions for Urban/Rural Groups 

Age Weighted NLSY97 Counts by Urban/Rural Groups 

Total

(Rural

and Urban) 

Central

Place of 

Urbanized

Area

Urban

Fringe of 

Urbanized

Area

Urban

outside

Urbanized

Area

Total

Urban

Total

 Rural

6

9

11

12

13

28

3168310.3

1816832.1

3250871.9

4199563

793196.8

784878.5

1252777.3

1119918.6

659058.4

38172173.8

334631.9

275166.8

278227.9

182337.4

135996.2

318473.1

9864411.2

1913078.6

2666585.2

2577752.0

895114.9

899505.8

875311.6

834477.3

382453.0

730723.3

0

1

2

3

4

5

7

8

10

14

15

16

17

18

19

20

21

22

23

24

25

26

27

29

30

31

32

33

34

35

0-35

2577942.4

3030546.7

3303989.6

3372036.5

3279069.3

3484256.5

3498752.5

3537802.1

3409837.6

3288936.9

3404902.3

2669284.1

2697544.3

2740386.6

2726194.2

2735635.1

2372761.3

2211489.2

2149833.7

2079189.4

2164810

1840284.5

3176756.6

3484870.6

3303172.9

3388165

3173183.6

3279151.6

3557179.3

3611873.1

3563318.6

4205061.4

109753794

942508.9

997411.3

1038285.7

1015887.8

1124976.8

1043523.4

997838.7

1033530.6

916472.1

942132.2

948501.4

699864.0

715034.1

682389.1

752407.0

614459.4

678731.7

663318.8

682696.0

779232.6

648688.6

666475.5

1156853.0

1170724.8

1130640.7

1183015.8

1097494.6

1296088.7

1065376.3

1096069.2

1112707.8

1064149.7

1221751.1

34010089.9

891727.9

1073595.1

1164132.4

1123965.7

1241171.9

1255291.4

1314468.7

1166363.5

1089028.5

1165878.7

1212747.5

943516.5

920322.4

959233.7

993285.0

971111.2

833778.8

765850.7

741793.4

757931.0

724035.9

684615.1

1081201.1

1117207.3

1093945.1

1102752.6

1192717.3

1018209.6

1409662.7

1081843.9

1276855.2

1287326.0

1241092.0

1496538.9

228154.0

288344.0

337868.3

302729.5

295707.5

300436.5

310948.2

325988.9

278958.2

316522.1

230317.7

189487.7

216042.8

219633.3

198441.0

188720.3

180287.5

172626.6

150084.4

269606.8

298394.0

313082.1

289032.2

317321.7

326756.1

432546.1

295297.0

324562.0

337427.2

404254.59

2350771.5

2408874.7

2505147.6

2435561.1

2609762.9

2638296.3

2673988.2

2284458.8

2383177.7

2477771.0

1873698.1

1913584.4

1933600.0

1891716.9

1943151.4

1646679.1

1626919.8

1593832.6

1620914.5

1675895.0

1457831.5

1487086.8

2507660.8

2668378.6

2522425.6

2693054.9

2442460.3

3138297.5

2442517.2

2697486.3

2737461.0

2623714.8

3122544.4

82046675.0

664863.7

679774.8

866888.9

843508.2

817671.3

888989.5

883851.6

905759.2

927131.3

795586.0

783959.9

806786.6

792483.7

726082.2

584569.4

556001.2

458274.9

488915.0

329745.4

669095.7

816492.0

725420.9

728446.3

695110.1

1061265.6

836634.5

859692.9

874412.1

939603.8

1082516.9

27707119.3

254