Addressing Nonresponse Bias in Food Security Measures Using 

Weighting Adjustments

Jonathan Eggleston (U.S. Census Bureau), Matthew P. Rabbitt

(Economic Research Service), David C. Ribar (Georgia State University), 

Alisha Coleman-Jensen (Economic Research Service)

FCSM, Session B-2

Thursday, October 26, 2023

1

Any opinions and conclusions expressed herein are those of the authors and do not represent the views of the U.S. Census Bureau. The Census 

Bureau has ensured appropriate access and use of confidential data and has reviewed these results for disclosure avoidance protection (Project 

7531994: CBDRB-FY23-0524).
Survey Non

-Response

• Response rates for the CPS and other 

surveys plummeted during COVID-19 

and have generally fallen

• CPS module response rates have also 

decreased

• Non-response may lead to non- representative samples and affect 

estimates of food insecurity and 

other outcomes

• Data available within the CPS to 

adjust for non-response are limited

2
3

60.0

65.0

70.0

75.0

80.0

85.0

90.0

95.0

100.0

Jan-11

May-11

Sep-11

Jan-12

May-12

Sep-12

Jan-13

May-13

Sep-13

Jan-14

May-14

Sep-14

Jan-15

May-15

Sep-15

Jan-16

May-16

Sep-16

Jan-17

May-17

Sep-17

Jan-18

May-18

Sep-18

Jan-19

May-19

Sep-19

Jan-20

May-20

Sep-20

Jan-21

CPS Response Rates

Source: BLS
CPS Food Security Supplement

• Focus on CPS Food Security 

Supplement

• Sponsored by the Economic 

Research Service-USDA and fielded 

by the Census Bureau as an annual 

supplement to the December CPS

• Source for federal statistics on 

household food security in U.S. 

• Food security: access at all times to 

enough food for an active, healthy 

life for all household members

4
Non-response in the Food Security Supplement

• Recent research on the CPS Annual Social and Economic 

Supplement finds lower income people are less likely to respond 

(Rothbaum & Bee, 2020)

• Differential non-response affects estimates of poverty rates and 

other income distribution statistics

• Not fully addressed by standard weighting procedures

• Because of the relationship between income and food security, 

• Differential non-response likely occurs in the Food Security Supplement

• And may affect estimates of food security

5
This paper

• Applies the methodology developed in research on the basic monthly 

CPS to improve the weighting correction for non-response

• Utilizes data from multiple administrative data sources including IRS 

1040 and 1099 data, SSA benefit data, earnings data from the 

Longitudinal Employer-Household Dynamics, state public assistance 

records, and other sources

• Develops and calibrates new weights from these sources

• Applies the weights and compares estimates based on the standard 

weighting methodology and this new methodology

6
Weighting Overview

7
What Do Survey Weights Do?

• Simple overview: weights increase or decrease the “importance” of 

individual respondents to make the responding sample look more 

like the target population

• For example, if older individuals are more likely to respond to a 

survey than younger individuals, we would give older individuals 

lower weight values and younger people higher weight values to 

make the sample (hopefully!) more representative

8
Overview of CPS-FSS’s Weighting Algorithm

1. Household Noninterview Adjustment

• Using microdata on both respondent and nonrespondent households, 

distribute the weights of nonrespondents to the respondent households with 

similar characteristics

2. Second Stage Adjustment

• Adjust CPS-FSS weighted counts of age, race, Hispanic origin, and sex to 

independent Census population estimates at national and state levels

• Also adjust CPS-FSS metro status and income statistics to same measures 

calculated on the Basic Monthly CPS

9
Overview of CPS-FSS’s Weighting Algorithm

• Current Household Noninterview Adjustment

• Adjustment based only on geography (state and metropolitan 

status)

• Concern: this may not fully account for economic characteristics 

that influence response

• Calibration Step

• While had adjustment for income, target based on CPS basic 

respondents, which may be nonrepresentative 

10
Administrative Data

• Add the following administrative data to CPS-FSS’s weighting algorithm

• IRS 1040 and 1099

• SSA program benefit data

• Demographic data from 2010 Census and SSA

• Industry data from the Census Business Register

• Third-party home value data

• Quarterly earnings data from the Longitudinal Employer-Household Dynamics (LEHD) 

program

• State SNAP/TANF/WIC Data

• Have these data not only for many respondents, but also for CPS-FSS and CPS 

Basic nonrespondents

11
Modifications to Weighting Algorithm

• Replace geography-based noninterview adjustment with one based on IRS 

microdata and other administrative data

• Create new cells with CART (Classification and Regression Tree)

• Run model of household food security on administrative data in order to create cells that 

have a good correlation with our key outcome of interest

• Estimate model on respondents. Apply model output to both respondents and 

nonrespondents to create noninterview adjustment cells.

• Add administrative data to calibration step as well

• Use same CART model. Target is predicted probabilities of food security status, 

where the administrative data are the inputs for creating the predicted probabilities

12
Strength and Weaknesses of Administrative 

Data

• Administrative data includes information that should be highly related to food 

security

• IRS Income

• SNAP and WIC receipt

• Nevertheless, we don’t actually observe food security in the administrative data

• If there are additional factors correlated with both food security and response 

even after controlling for these observables, some nonresponse bias will remain

• E.g. Don’t observe expenditures. Differences in expenditures decisions for a given level 

of income and SNAP benefit amounts could affect food security, but could also be 

correlated with behavioral differences that influence whether someone responds to a 

survey

13
Results-Household Food Insecurity

14
15

Percent Food Insecure 

(Low or Very Low 

Security) Production 

Estimate

Change in Food 

Insecurity Estimates 

with Using 

Administrative Data

2019 2019 2020

All households 10.54% 0.20% 0.36%

With children < 18 years old 13.65% 0.41% 0.49%

____With children < 6 years old 14.47% 0.20% 0.31%

____Married-couple families 7.54% 0.17% 0.43%

____Female head, no spouse 28.73% 0.95% 0.28%

____Male head, no spouse 15.37% -0.42% 0.29%

With no children < 18 years 9.27% 0.10% 0.31%

____More than one adult 6.72% 0.13% 0.24%

____Women living alone 13.05% 0.30% 0.40%

____Men living alone 12.84% -0.37% 0.29%

With elderly 7.22% 0.24% 0.36%

____Elderly living alone 8.70% 0.32% 0.46%

Source: 2019 and 2020 CPS-FSS + Administrative Data
16

Percent Food Insecure 

(Low or Very Low 

Security) Production 

Estimate

Change in Food 

Insecurity Estimates 

with Using 

Administrative Data

2019 2019 2020

White, non-Hispanic 7.93% 0.22% 0.32%

Black, non-Hispanic 19.07% 0.25% 0.46%

Hispanic 15.63% 0.17% 0.45%

Other, non-Hispanic 9.47% -0.01% 0.21%

Under 1.00 Poverty Line 34.86% -0.28% 0.19%

Under 1.30 Poverty Line 33.02% -0.31% -0.01%

Under 1.85 Poverty Line 27.65% -0.33% -0.05%

1.85 and over Poverty Line 5.08% 0.04% 0.13%

Income unknown 8.38% -0.08% 0.32%

Northeast 9.60% 0.30% 0.32%

Midwest 10.53% 0.17% 0.39%

South 11.19% 0.25% 0.33%

West 10.16% 0.06% 0.42%

Source: 2019 and 2020 CPS-FSS + Administrative Data
Results-Child Food Insecurity

17
18

Percent Food Insecure 

(Low or Very Low Security) 

Production Estimate

Change in Food 

Insecurity Estimates with 

Using Administrative 

Data

2019 2019 2020

All households 7.04% 0.36% 0.12%

____With children < 6 years old 6.65% 0.42% -0.09%

____Married-couple families 3.40% 0.12% 0.08%

____Female head, no spouse 16.89% 1.01% 0.03%

____Male head, no spouse 7.02% -0.46% -0.09%

White, non-Hispanic 5.25% 0.18% 0.14%

Black, non-Hispanic 13.54% 0.98% 0.19%

Hispanic 8.28% 0.49% -0.03%

Other, non-Hispanic 5.35% 0.22% 0.01%

Under 1.00 20.91% 0.61% -0.32%

Under 1.30 19.88% 0.33% -0.18%

Under 1.85 17.23% 0.13% -0.14%

1.85 and over 2.36% 0.09% 0.17%

Income unknown 5.53% 0.22% 0.31%

Northeast 7.35% 0.38% -0.01%

Midwest 7.38% 0.10% -0.08%

South 6.77% 0.48% 0.15%

West 6.96% 0.36% 0.32%

Source: 2019 and 2020 

CPS-FSS + Administrative 

Data
Conclusion

• Adding administrative data to CPS-FSS’s weighting algorithm results 

in a modest change in food security estimates

• Shifts largely from change in estimates of the income distribution

• Change in estimates larger in 2020 compared to 2019 for household food 

security 

• Pattern reversed/not as consistent for child food security

19
My Contact Information

Jonathan Eggleston

Senior Economist

Survey Improvement Technical Lead

Survey and Economic Research Group

Center for Economic Studies

U.S. Census Bureau

Office: 301.763.2357 

jonathan.s.eggleston@census.gov

20