Weighting with the Redesigned National Health 

Interview Survey

Federal Committee on Statistical Methodology Research and Policy Conference 

 September 21, 2020 

James M. Dahlhamer, Matthew D. Bramlett, & Jonaki Bose 

National Center for Health Statistics 

Ronaldo Iachan & Matt Jans 

ICF
National Health Interview Survey (NHIS)

▪ Agency: Department of Health and Human Services (DHHS); Centers for Disease Control 

and Prevention (CDC); National Center for Health Statistics 

▪ Purpose: To monitor the health of the US population through the collection and analysis 

of data on a broad range of health topics 

▪ Sample: Complex sample of the civilian noninstitutionalized US population 

▪ Mode: In-person interviews -- follow-up by telephone if needed 

▪ Data collection: Continuous by Census field representatives 

▪ Target sample size: 27,000 sample adults & 9,000 sample children 

▪ Content redesign start: January 2019

2
2019 NHIS Questionnaire Redesign

▪ Goals of the redesign 

• Reduce the overall length of the interview 

• Improve the relevancy of covered health topics 

• Focus on leading causes of morbidity and mortality, targets of health promotion 

initiatives, and risk and protective factors 

• Harmonize overlapping content with other federal heath surveys 

• Reduce respondent burden 

▪ Eliminated family interview; now a short household roster followed by 

selection of sample adult (18+) and sample child (<18) 

• Sample adult and sample child are now only people we collect detailed health 

information on

3
Time for a New Weighting Approach?

▪ Questionnaire redesign provided opportunity to reconsider weighting 

approach, especially how we adjust for nonresponse (NR)

• Not updated in previous ~20 years 

• Declines in NHIS household response rates (~90% to ~65%)

• Likely break some trends due to content changes 

▪ Other developments 

• Causes of survey NR have likely changed over time 

• Statistical methods and computing power have improved 

• Increasing availability of auxiliary data

4
Final household base weight

Household NR adjustment 

(segment-level adjustment)

Adult probability of selection adjustment

Adult level NR adjustment 

(segment-level adjustment)

Interim sample adult weight 

Final sample adult weight

Post-stratification to population control totals 

based on age, sex, race/ethnicity

Child probability of selection adjustment

Child level NR adjustment 

(segment-level adjustment)

Interim sample child weight

Final sample child weight

Post-stratification to population control totals 

based on age, sex, race/ethnicity

Post-stratification to population control totals 

based on age, sex, race/ethnicity

Final person weight

 Simplified Look at Past Weighting Approach

5
ICF Contract

▪ Explored various models of survey response incorporating contextual data (county-, 

Census tract-level measures) and paradata related to both response and key health 

measures: 

• Logistic regression 

• Multilevel logistic regression 

• Random forest prediction 

• Least absolute shrinkage & selection operator (LASSO) machine learning

▪ Explored raking versus traditional post-stratification 

• Old post stratification variables: age, sex, race/ethnicity 

• Raking: age, sex, race/ethnicity plus: 

• Education, employment status, MSA status and/or Census division

6
Final household base weight

Household NR adjustment 

(based on multi-level logistic regression)

Adult probability of selection adjustment

Adult level NR adjustment 

(based on multi-level logistic regression)

Interim sample adult weight 

Final sample adult weight

Raking (age, sex, race/ethnicity, education, 

MSA status, Census Division

Child probability of selection adjustment

Child level NR adjustment 

(based on multi-level logistic regression)

Interim sample child weight

Final sample child weight

Raking (age, sex, race/ethnicity, MSA status, 

Census Division

 Simplified Look at New Weighting Approach

7
Modeling Survey Response

FCSM 2020

8
Screening Variables for Inclusion in Models of Response (1)

▪ Step 1: Explore associations between auxiliary variables and response 

• Response defined as response (1) vs. nonresponse (0) 

• Variables significantly associated with response (p < .10) move to the next step 

• This process performed for household, sample adult, and sample child response 

▪ Step2: Explore associations between auxiliary variables (screened in from step 1) 

and a set of key health indicators (KHIs) 

• Variables significantly associated with KHIs (p < .10) in at least 3 of 5 health domains for the adult 

and 2 of 3 health domains for the child move to the modeling stage

9
Auxiliary Variables: Data Sources

▪ Contact History Instrument (CHI) 

• Data collected on every contact attempt 

• Variables summarized to household/respondent level 

• Example: Whether or not householder(s) expressed time constraints 

▪ Neighborhood Observation Instrument (NOI) 

• Data collected from first observation of sample unit 

• Example: Does the sample unit have any indication that the residents are smokers? 

▪ Census Planning Database 

• Decennial Census 2010, American Community Survey 5-year estimates (2013-17) 

• Census tract-level measures 

• Example: Percentage of ACS population that is 65 years old or over 

▪ Area Health Resource File (AHRF) 

• Contains more than 6,000 variables related to health care access 

• County-level measures 

• Example: Number of medical doctors per 100,000 county residents 10
Adult Key Health Indicators

Health care service use Health insurance coverage

 ER visit, past year Public coverage

 Doctor visit, past year Private Coverage

 Flu vaccination, past year Uninsured

 Mental health counseling/therapy, past year

Health care access Health status

 Usual source of care Functional disability

 Skipped doses of prescription meds to save money Asthma episode, past year

Health behaviors Hypertension

 Current smoker Excellent/very good health

 Obese 

11
Child Key Health Indicators

Health care service use Health status

 ER visit, past year Disability

 Doctor visit, past year Asthma episode, past year

 Flu vaccination, past year Excellent/very good health

 Mental health counseling/therapy, past year Current ADD/ADHD

 Stressful life events indicator

Health care access

 Usual source of care

 Public coverage

 Private Coverage

 Uninsured

12
Screening Variables for Inclusion in Models of Response (2)

▪ Step 1: Explore associations between auxiliary variables and response 

• Response defined as response (1) vs. nonresponse (0) 

• Variables significantly associated with response (p < .10) move to the next step 

• This process performed for household, sample adult, and sample child response 

▪ Step2: Explore associations between auxiliary variables (screened in from step 1) 

and a set of key health indicators (KHIs) 

• Variables significantly associated with KHIs (p < .10) in at least 3 of 5 health domains for the adult 

and 2 of 3 health domains for the child move to the modeling stage 

▪ ~60 of the 213 variables explored made it to the modeling stage

• Household (n=48) 

• Adult (n=48) 

• Child (n=26)

13
Modeling Survey Response

▪ Step 1: Estimate a standard binomial logistic regression of response 

(response vs. nonresponse) 

• Includes variables retained from prior screening steps 

• Backward elimination (p < .10) 

• Variables retained in this procedure are carried forward to a multi-level logistic regression of 

response 

▪ Step 2: Estimate a multi-level logistic regression of response (response vs. 

nonresponse) 

• Random effects for Census tract 

• Fixed effects for variables retained from Step 1 

• Output predicted probabilities (response propensities) 14
Forming and Applying Nonresponse Adjustments 

FCSM 2020

15
Household Nonresponse Adjustment

▪ Step 1: Order the response propensities from low to high and group them 

into quintiles 

▪ Step 2: Form the NR adjustment factor by taking the inverse of the median 

response propensity within each propensity quintile: 1 / median RP 

▪ Step 3: Multiply the final household base weight by the NR adjustment 

factor

16
Nonresponse Adjustment Factors: Household, 2019

Response Propensity Quintiles

1 (low) 2 3 4 5 (high)

3.44 1.92 1.40 1.23 1.11

Nonresponse adjustment factor formed by taking the inverse of the median response propensity within each 

quintile.

17
Sample Adult and Sample Child Nonresponse Adjustments

▪ The NR-adjusted household weight is the starting point for the sample adult and sample 

child weight 

▪ Step 1: Multiply the NR-adjusted household weight by inverse of adult (child) probability 

of selection 

• This becomes the adult (child) base weight 

▪ Step 2: Group response propensities from the final adult (child) response model into 

quintiles 

▪ Step 3: Form the NR adjustment factor by taking the inverse of the median response 

propensity within each propensity quintile: 1 / median RP 

▪ Step 4: Multiply the adult (child) base weight by the NR adjustment factor

18
Nonresponse Adjustment Factors: Sample Adult and Sample Child, 2019

Response Propensity Quintiles

1 (low) 2 3 4 5 (high)

Sample adult 1.25 1.12 1.07 1.05 1.03

Sample child 1.24 1.12 1.09 1.05 1.04

Nonresponse adjustment factor formed by taking the inverse of the median response propensity within each quintile.

19
Raking 

FCSM 2020

20
Raking (1)

▪ Adjusting sample weights so that marginal totals of the adjusted weights on 

specified characteristics (e.g., age, sex, race/ethnicity) agree with 

corresponding totals for the population 

▪ Most often used to reduce biases from nonresponse and noncoverage 

▪ Iterative process by which all variables are considered in turn. For each 

variable, weights are adjusted to align the survey marginals with population 

marginals

21
Raking (2)

▪ More flexible than post-stratification (PS) 

• PS requires control totals for ALL cells of a cross-classification 

• PS can spread the sample too thinly over large number of adjustment cells 

▪ Thus, raking can typically handle more variables than PS

22
Raking Variables

▪ Various combinations of the following variables were explored: age, sex, race/ethnicity, 

employment status (adult only), education (adult only), MSA status, and Census 

division 

• Final sample adult raking dimensions: 

• Age by sex (18 cells) 

• Age by race and ethnicity (26 cells) 

• Education (4 cells) 

• MSA status by Census division (23 cells) 

• Final sample child raking dimensions: 

• Age by sex (10 cells) 

• Age by race and ethnicity (15 cells) 

• MSA status (3 cells) 

• Census division (9 cells) 23
Final Weights 

FCSM 2020

24
Sample Adult Weight: 2019

N Min.

1st

Quartile Median Mean

3rd

Quartile Max. CV (%) DEFF

Old approach 31,997 644 4,214 6,634 7,842 9,815 121,990 68 1.46

New approach 31,997 629 3,958 6,254 7,842 9,351 154,456 80 1.64

CV = coefficient of variation 

DEFF = design effect

25
Capping the Household Nonresponse Adjustment

▪ To constrain variance, we explored capping the household NR 

adjustment at: 

• 3.0 

• 2.5 

• 2.0 

▪ Trade-off: While capping will reduce the variance of the weights, it will 

likely result in an increase in NR bias

26
Capping Nonresponse Adjustment Factors: Household, 2019

Response Propensity Quintiles

1 (low) 2 3 4 5 (high)

Uncapped 3.44 1.92 1.40 1.23 1.11

Cap at 3.0 3.00 1.92 1.40 1.23 1.11

Cap at 2.5 2.50 1.92 1.40 1.23 1.11

Cap at 2.0 2.00 1.92 1.40 1.23 1.11

Nonresponse adjustment factor formed by taking the inverse of the median response propensity within each quintile.

27
Sample Adult Weight: 2019

N Min.

1st

Quartile Median Mean

3rd

Quartile Max. CV (%) DEFF

Past approach 31,997 644 4,214 6,634 7,842 9,815 121,990 68 1.46

New approach 

(uncapped) 31,997 629 3,958 6,254 7,842 9,351 154,456 80 1.64

New, cap 3.0 31,997 640 4,027 6,357 7,842 9,455 139,764 76 1.58

New, cap 2.5* 31,997 652 4,107 6,460 7,842 9,551 121,705 73 1.53

New, cap 2.0 31,997 665 4,189 6,527 7,842 9,657 102,032 70 1.49

CV = coefficient of variation 

DEFF = design effect 

*Weight included on 2019 sample adult datafile

28
How Did We Decide?

▪ Relative to the uncapped weight, looked at mean squared error (MSE) 

for 15 adult KHIs (and 13 child KHIs) across the different capping levels 

▪ For all 3 cap levels we saw a reduction in MSE relative to the uncapped 

weight 

• Any increases in bias were outpaced by reductions in variance

▪ Overall, capping at 2.5 provided the best bias-variance tradeoff relative 

to the uncapped weight 

29
Impact on Estimates

FCSM 2020

30
Comparison of Select Adult Estimates Using the Old and New Sample Adult 

Weight: 2019 

Base Weight Old Weight New Weight

Outcome % % SE % SE |New – Old|

Uninsured 8.6 10.2 0.24 11.0 0.27 0.8

Doctor visit, past year 86.7 85.3 0.26 84.9 0.27 0.4

Flu vaccination 50.6 47.7 0.38 46.8 0.39 0.9

Functional disability 10.5 8.7 0.19 9.0 0.21 0.4

Excellent/very good health 56.3 58.9 0.39 57.4 0.40 1.5

Obese 32.2 31.6 0.34 32.0 0.36 0.4

Current smoker 13.8 13.3 0.24 14.0 0.26 0.7

31
Comparison of Select Child Estimates Using the Old and New Sample Child 

Weight: 2019 

Base Weight Old Weight New Weight

Outcome % % SE % SE |New – Old|

Uninsured 4.2 4.3 0.30 4.3 0.30 0.1

ER visit, past year 17.9 18.2 0.52 18.0 0.53 0.2

Flu vaccination, past year 51.9 52.1 0.67 51.7 0.68 0.4

Asthma episode, past year 3.3 3.3 0.23 3.3 0.24 0.0

Excellent/very good health 87.8 87.6 0.45 87.5 0.46 0.1

Stressful life event, ever 17.2 16.6 0.49 16.0 0.47 0.6

Current ADD/ADHD 8.1 7.3 0.33 7.1 0.32 0.2
Summary

▪ Moved from geography-based NR adjustments to adjustments within 

response propensity classes based on response models using rich 

auxiliary data 

• Old approach was static and could not adjust for changing patterns of 

nonresponse over time 

• New approach will update response models annually, ensuring NR adjustments 

continue to perform well 

▪ Moved from weight calibration via post-stratification to raking 

• Raking is more flexible and allows additional dimensions (e.g., education, MSA 

status, Census division)

33
Thank you! 

Jim Dahlhamer 

jdahlhamer@cdc.gov 

▪ Coming to the NHIS website in late September: 

 Bramlett MD, Dahlhamer JM, Bose J, Blumberg SJ. 2020. New procedures for nonresponse 

adjustment to the 2019 National Health Interview Survey Sampling Weights. National Center for 

Health Statistics, Division of Health Interview Statistics.

34