National Center for Health Statistics

Assessing Linkage Eligibility Bias in the National 

Health Interview Survey (NHIS)

Jonathan Aram, Crescent B. Martin, Lisa B. Mirel

Statistician (Health)

2021 FCSM Research and Policy Conference

November 3rd, 2021
Overview

This presentation will cover 6 topics:

1. Background

2. Linkage eligibility bias as defined in the NHIS

3. Motivatioミ aミd マethods for さliミkage eligiHility Hiasざ assessマeミt

4. The extent of linkage eligibility bias in the NHIS

5. The relationship between eligibility rates and bias

6. Mitigating linkage eligibility bias through weight adjustment
1. Background

▪ National Center for Health Statistics (NCHS)

– Natioミ’s priミIipal health statistiIs ageミIy

– One of 13 federal statistical agencies

– Mission: To provide timely, relevant 

statistical information that can be used to 

guide actions and policies to improve the 

health of the American people

2
1. Background

National Health Interview Survey (NHIS)

A nationally representative, cross-sectional household 

interview survey that serves as an important source of

information on the health of the civilian 

noninstitutionalized population of the United States
1. Background

▪ Linking data is a powerful mechanism to 

provide policy relevant information in 

an efficient way

▪ NCHS currently links data from its 

surveys to administrative data sources 

using personally identifiable 

information (PII)
1. Background

▪ Value added by linkage

– Additional variables

– Longitudinal outcome information

– Reduced participant burden

– Improved ascertainment of difficult-to-remember information

▪ Additional considerations when analyzing linked data

– Potential reduction in sample size

– Potential for bias in estimates
1. Background

Sample Frame

Sample

Response

Linkage Consent

Linked

Non-Response

Linkage 

Refusal

Not Linked

Figure Hased oミ AAPOR ProIeediミgs, さNoミ-Consent Error, Non-Response Error, and Measurement Error: Assessing the 

Overall Quality of Liミked “urvey aミd Adマiミistrative Dataざ Hy Joseph Sakshaug
2. Linkage eligibility bias as defined in the NHIS

Prior to 2007 2007 forward

Asked for SSN9

Refused:

Not Linkage 

Eligible

Doミ’t 

know/missing:

Linkage Eligible

Provided: 

Linkage Eligible

Asked for SSN4

Refused/doミ’t 

know/missing

Provided: 

Linkage Eligible

Asked for 

consent to link 

without SSN4

Refused: Not 

Linkage Eligible

Agreed: 

Linkage Eligible
3. Motivatioミ aミd マethods for さliミkage eligibility 

biasざ assessマeミt

Linkage Eligibility, NHIS Sample Adults 

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Year

Linkage Eligible Not Linkage Eligible

Linkage 

eligible

All 

Sample 

Adults
3. Motivatioミ aミd マethods for さliミkage eligibility 

biasざ assessマeミt 

▪ The fact that not everyone is linkage eligible could have implications for 

inference when using the linked data

▪ Survey years were grouped to reflect changes in sample design and in 

linkage eligibility criteria

▪ Bias was measured by comparing estimates from the full sample and the 

linkage-eligible sample

– Demographic variables

– Selected health conditions

▪ Survey weights were adjusted for linkage eligibility 

▪ Bias was reassessed using adjusted weights

4. The extent of linkage eligibility bias in NHIS

Percent Relative Bias, Select Sociodemographic Variables, 

2000-2018

-11

-6

-1

4

9

2000-2003 2004-2006 2007-2009 2010-2013 2014-2016 2017-2018

% Relative Bias

Survey Year Group

Age 45-64 Female Hispanic Bachelor's or Higher
4. The extent of linkage eligibility bias in NHIS

Percent Relative Bias, Prevalence of Select Health 

Outcomes, 2000-2018

-11

-6

-1

4

9

2000-2003 2004-2006 2007-2009 2010-2013 2014-2016 2017-2018

% Relative Bias

Survey Year Group

Diabetes Obesity Smoking Office Visit Past Year
5. The relationship between eligibility rates and 

linkage eligibility bias

0

20

40

60

80

100

-11

-6

-1

4

9

2000-2003 2004-2006 2007-2009 2010-2013 2014-2016 2017-2018

% Eligible for Linkage

% Relative Bias

Survey Year Group

Diabetes

Hypertension

Smoking

Office Visit

Past Year

6. Mitigating linkage eligibility bias through weight 

adjustments

Percent Relative Bias, Select Sociodemographic Variables, 

2000-2018

-11

-6

-1

4

9 Age 45 - 64

-11

-6

-1

4

9

Female

-11

-6

-1

4

9

Hispanic

-11

-6

-1

4

9

Bachelor’s or Higher
6. Mitigating linkage eligibility bias through weight 

adjustments

Percent Relative Bias, Prevalence of Select Health 

Outcomes, 2000-2018

-11

-6

-1

4

9

Diabetes

-11

-6

-1

4

9

Obesity

-11

-6

-1

4

9

Smoking

-11

-6

-1

4

9

Office Visit Past Year
Conclusions

▪ The fact that not everyone is linkage eligible could have implications for 

inference when using the linked data

▪ Changes in data collection and consent procedures resulted in higher 

linkage eligibility over time

▪ For many variables, the largest magnitude of bias coincided with low 

linkage eligibility rates

▪ For the variables used to adjust survey weights, bias was reduced to zero

▪ Analysts should consider estimating bias when selecting survey years for 

analysis and adjusting weights when bias is detected
References

Aram J, Zhang C, Golden C, et al. Assessing Linkage Eligibility Bias in the 

National Health Interview Survey. Vital and health statistics Series 2, Data 

evaluation and methods research. 2021(186):1-28.

Sakshaug, J. W. and M. Huber (2015). "An Evaluation of Panel Nonresponse 

and Linkage Consent Bias in a Survey of Employees in Germany." Journal of 

Survey Statistics and Methodology 4(1): 71-93.
More Information

NCHS Data Linkage Program:

datalinkage@cdc.gov

www.cdc.gov/nchs/data-linkage

Jonathan Aram: JAram@cdc.gov