Analysis of the September 2020 Current Population 

Survey Food Security Supplement Split Panel Test

Alisha Coleman-Jensen and Matthew P. Rabbitt

USDA Economic Research Service

Federal Committee on Statistical Methodology Research & Policy Conference

October 25, 2022

"The findings and conclusions in this presentation are those of the authors and should not be construed to 

represent any official USDA or U.S. Government determination or policy. This research was supported by the U.S. 

Department of Agriculture, Economic Research Service."
2

Food Security Definitions

• Food Security: Access at all times to enough food for an active, healthy 

life for all household members.

• Food Insecurity: Unable, at some time during the year, to provide 

adequate food for one or more household members because of a lack of 

resources.

– Very Low Food Security: Normal eating patterns of some household 

members were disrupted at times during the year and their food 

intake reduced because they could not afford enough food. 
3

The importance of measuring and monitoring 

household food security

• USDA has been collecting data on household food security for more 

than 25 years

• Food security data used for multiple purposes: 

– Assess well-being including as objective in Healthy People

– Research to assess effectiveness of policies and programs

– Research to understand determinants and outcomes of food security

• Food security measurement modules used in many data collections
4

Current Population Survey-Food Security 

Supplement (CPS-FSS) Overview

• The CPS-FSS is sponsored by ERS and 

conducted by the Census Bureau

• The CPS-FSS is the source for annual 

Federal statistics on food insecurity in 

U.S. Households

– The data are the basis for the annual USDA 

report Household Food Security in the 

United States 

– For all reports in the series, see: 

https://www.ers.usda.gov/topics/foodnutrition-assistance/food-security-in-theu-s/history-background/#annual
5

The importance of looking forward

• ERS has been making investments to ensure the high quality of the food 

security measure and data

– Approaching 25 years of food security data collection was a time to take stock, 

look ahead and plan

• What instrument/survey issues need to be addressed and updated?

– Worked with the U.S. Census Bureau to: 

• review the entire CPS-FSS instrument

• consider revisions for all sections

• conduct cognitive testing of revisions

• implement split-panel test to assess impact of modifications to instrument
6

Cognitive testing 

• Review of questionnaire and cognitive interviewing of modified survey 

items completed by Census Bureau in 2019

– Kephart, Kathleen, Jonathan Katz, Matthew Virgile, Rodney Terry, and Jessica 

Holzberg, 2021, “Cognitive Testing Results for the Current Population Survey 

Food Security Supplement,” Working Paper Number Rsm2021-06, November 12, 

https://www.census.gov/library/working-papers/2021/adrm/rsm2021-06.html. 

• Split-panel test of modified survey items in September 2020 CPS to 

assess performance of revised questionnaire. 
7

CPS-FSS instrument updates:

Food spending and food security

• Modifications to the CPS-FSS Test Instrument:

• Food Spending: Updated to reflect changes in terminology, the retail 

environment, and technology. For example, the new wording asks 

about online food purchases and purchases at farmers markets. 

• Food Security: The section was moved earlier in the survey instrument. 

A few child items were modified to standardize the resource constraint 

to “there wasn’t enough money for food.” Also, the lead-in to one of 

the screening variables (HES9) was changed due to moving sections in 

the questionnaire. 
8

CPS-FSS instrument updates:

Federal and community nutrition assistance

• Modifications to the CPS-FSS Test Instrument:

• Federal Nutrition Assistance: The section was moved after the food security 

section. The lead-in to the first question (HESP1) in the section changed due 

to moving the section. Questions on school meals were modified slightly to 

refer to “reduced-price” meals instead of “reduced-cost.” A new question was 

added on receipt of free or reduced-price afterschool meals and snacks. 

• Community Food and Nutrition Assistance: New questions ask about receipt 

of free groceries and free meals recommended by Feeding America, based 

on their expertise in the charitable feeding sector and their own testing of the 

questions. The section no longer asks separate questions about free meals 

received by the elderly but incudes those programs in the more general 

questions about receipt of free meals. 
9

Data

• In a split panel test, a portion of the survey sample receives one questionnaire, 

and the other portion of the sample receives an alternate questionnaire. 

• CPS months-in-sample groups 2, 3, 4, 6, 7, and 8 were eligible to be interviewed 

in the September 2020 split panel. Did not include CPS months-in-sample groups 

1 and 5 (one-quarter of the total sample) because they would be eligible for 

interview in the regular December 2020 CPS-FSS data collection. 

• In the September 2020 CPS 35,084 households were eligible for the split panel.

– 17,588 households were assigned to the original (standard) instrument and 17,496 

households were assigned to the test instrument 

– 13,160 completed interviews (74.8% response) for the standard instrument and 13,132 

completed interviews (75.1% response) for the test instrument

• Each split panel is weighted separately to be representative of the U.S. 

population. 
10

Results: Food Spending

• Most differences in usual food spending between the split panels are not statistically 

significant. 

• The results suggest that the modified questions functioned at least as well as the 

standard instrument and show possible improvement, with slightly lower 

nonresponse. 

• For both the Standard and Test Instruments total reported median weekly food 

spending was $130

• For both the Standard and Test Instruments median usual food spending per person 

was about $60

• For the Standard Instrument 6.8 percent of unweighted respondents did not answer 

the questions about food spending or reported zero usual spending. For the Test 

Instrument 6.3 percent of unweighted respondents had missing or zero food 

spending. 
11

Results preview: Food security measurement

• Psychometric analyses using Conditional 

Maximum Likelihood (CML) Rasch 

measurement model:

– All the food security items that comprise the 

adult and child food security scales captured 

similar levels of the severity of food hardship 

in the standard and test instruments, 

suggesting they would produce comparable 

measures of adult food security.

– Given the current findings and past findings 

regarding the stability of the food security 

scale to similar minor modifications, we 

expect that the proposed updates to the CPS- FSS instrument will continue to produce 

comparable estimates to previous years.

• Modest differences in prevalence are not 

due to differences in how respondents 

interpreted the food security module.

– Differences in the prevalence of food 

insecurity between the test instrument and 

standard instrument may be due, in part, to 

sampling error that has a greater impact on 

the smaller samples in the split panel and 

was potentially more problematic during the 

COVID-19 pandemic when response rates 

were lower. There may have been more 

sampling error during the pandemic that 

resulted in nonresponse bias. Evidence from 

Census Bureau analysis suggests that 

nonresponse during the pandemic was more 

strongly associated with income than in prior 

years of the CPS data collections (Rothbaum

and Bee 2021). 
12

Households by food security status and instrument version, September and December 2020

Panel A: All Households

Food Secure All food insecure Low food security Very low food security

Category Percent

September Standard Instrument 90.3 9.7*+ 6.2 3.5*+

September Test Instrument 89.3 10.7 6.7 4.0

December 2020 (standard instrument) 89.5 10.5 6.6 3.9

Panel B: Households with Children

Food secure Food insecure HH with food-insecure 

children

HH with very low food security 

among children

Category Percent

September Standard Instrument 87.2 12.8*+ 6.2+ 0.5

September Test Instrument 85.4 14.6 7.1 0.8

December 2020 (standard instrument) 85.2 14.8 7.6 0.8

* Difference between the September standard instrument and September test instrument is statistically significant with 90-percent confidence (t > 1.645).

+ Difference between the September standard instrument and December standard instrument is statistically significant with 90-percent confidence (t > 1.645). 

1 Totals exclude households for which food security status is unknown because household respondents did not give a valid response to any of the questions in the food 

security scale. For Panel A these exclusions representations 0.2 percent of all households for each sample.

2 For households with children shown in Panel B these exclusions represent: September 2020 Standard Instrument 0.1 percent, September 2020 Test Instrument 0.3 

percent, and December 2020 CPS-FSS 0.3 percent of all households with children.

Source: USDA, Economic Research Service using data from U.S. Census Bureau September 2020 and December 2020 Current Population Survey, Food Security 

Supplement.
13

Percent of households affirming items in the food security scale, Standard and Test Instrument, Sept. 2020

Scale item Standard Test

Household items:

Worried food would run out before (I/we) got money to buy more 14.0 14.7

Food bought didn't last and (I/we) didn't have money to get more 10.9 11.6

Couldn't afford to eat balanced meals 10.4 11.8 *

Adult items:

Adult(s) cut size of meals or skipped meals 5.7 6.3 *

Respondent ate less than felt he/she should 5.7 6.5 *

Adult(s) cut size or skipped meals in 3 or more months 4.1 4.7 *

Respondent hungry but didn't eat because couldn't afford 3.0 3.2

Respondent lost weight 2.0 2.2

Adult(s) did not eat for whole day 1.1 1.1

Adult(s) did not eat for whole day in 3 or more months 0.8 0.8

Child items:

Relied on few kinds of low-cost food to feed child(ren) 11.6 11.8

Couldn't feed child(ren) balanced meals 6.6 7.4

Child(ren) were not eating enough (Standard: ...we just couldn’t afford enough food 

vs. Test: ...there wasn’t enough money for food) 3.0 2.9

Cut size of child(ren)'s meals 1.5 1.6

Child(ren) were hungry (Standard: ...but you just couldn’t afford more food vs. Test:

...there wasn’t enough money for food) 0.6 1.0

Child(ren) skipped meals 0.4 0.6

Child(ren) skipped meals in 3 or more months 0.2 0.4

Child(ren) did not eat for whole day 0.1 0.1

* Difference between the 

September standard 

instrument and September 

test instrument is 

statistically significant 

with 90-percent confidence 

(t > 1.645).

Households not responding 

to an item are omitted from 

the calculations of 

percentages for that item. 

Household without 

children are omitted from 

the calculation of childreferenced items. 

Source: USDA, Economic 

Research Service using 

data from U.S. Census 

Bureau September 2020 

Current Population Survey 

Food Security Supplement.
14

Results: Food security items

• Examine items in food security module: 

• Lower percentage of households in the 

September standard instrument affirmed 

the food security items when compared to 

households in the September test 

instrument, with statistically significant 

differences marked with an asterisk. 

• Child items with changes in wording 

between the test and standard instrument 

were not statistically significantly different. 

• Psychometric analyses using Conditional 

Maximum Likelihood (CML) Rasch 

measurement model:

• Rasch model severity parameters capture 

the relative severity (in terms of food 

hardship) for each food security question. 

• Any differences in the estimates of these 

severity parameters for households 

administered the standard and test 

instruments may be indications of 

respondents perceiving the food security 

questions differently or reflect differences 

in the way they experience food hardships. 
15

Figure 1. Comparisons of the adult food security scale question severity 

parameters, 2020 Sept. CPS-FSS test versus standard instrument

worried

fnotlast

balmeal

cutskip

eatless cutskipf

hungry

losewt

whlday whldayf

2

3

4

5

6

7

8

9

2 3 4 5 6 7 8 9

Item severity, test instrument

Item severity, standard instrument

Test instrument Equal Severity

Note. Question severity parameters 

estimated from separate CML Rasch 

measurement models for households 

administered the standard and test 

instruments in the 2020 September 

CPS-FSS.

Source: USDA, Economic Research 

Service using data from U.S. Census 

Bureau September 2020 Current 

Population Survey Food Security 

Supplement.
16

Figure 1. Comparisons of the child food security scale question severity 

parameters, 2020 Sept. CPS-FSS test versus standard instrument

Note. Question severity parameters 

estimated from separate CML Rasch 

measurement models for households 

administered the standard and test 

instruments in the 2020 September 

CPS-FSS.

Source: USDA, Economic Research 

Service using data from U.S. Census 

Bureau September 2020 Current 

Population Survey Food Security 

Supplement.

chfewfd

chbal

chenuf

chcut

chskip

chskipf

chhungry

chwhlday

2

3

4

5

6

7

8

9

10

11

12

13

2 3 4 5 6 7 8 9 10 11 12 13

Severity parameter, test instrument

Severity parameter, standard instrument

Test instrument Equal Severity
17

Results: Community nutrition assistance

• There is no expectation that these items are comparable given the changes in 

wording. 

• For both the question about free groceries and free meals in the test 

supplement, the reported receipt is higher than for the questions about food 

pantries and soup kitchens in the standard instrument. 

– This is consistent with our expectation for the test items. 

– The language for the test items is more inclusive in several ways, including asking if 

“anyone in your household” received assistance rather than just “adults,” referring to 

“free groceries” rather than only “emergency food,” asking about “home-delivered 

meals”, and by asking about any “other place that helps with free food/meals.”

• Based on expert review, cognitive testing, and the results of the test instrument 

the revised questions on community nutrition assistance appear to be an 

improvement and function well.
18

Use of community nutrition assistance1, by standard and test instrument, September 2020

Standard Instrument Test Instrument

Food Pantries Soup Kitchens Free groceries Free meals

1,000 Percent 1,000 Percent 1,000 Percent 1,000 Percent

All households2 7,584 5.9 597 0.5 8,888 6.9 2,705 2.1

Households by food security status3:

Food secure 3,618 3.1 224 0.2 4,355 3.8 1,448 1.3

Food insecure 3,966 31.8 374 3.0 4,533 33.4 1,257 9.3

Low food security 2,174 27.3 206 2.6 2,598 30.5 579 6.8

Very low food security 1,793 39.9 167 3.7 1,934 38.4 678 13.5

1 Questions about receipt of community nutrition assistance are not comparable between the standard and test instrument. Percentages between the two 

instruments are not tested for statistical significance, because the wording of the items changed enough that there is no expectation of comparability. 

2 Households with missing data are excluded. September 2020 Standard Instrument: 0.6 percent of all households had missing data on food pantries, and 0.6 

percent of all households had missing data on soup kitchens. September 2020 Test Instrument: 0.8 percent of all households had missing data on free 

groceries, and 0.9 percent of all households had missing data on free meals.

3 Totals also exclude households for which food security status is unknown because household respondents did not give a valid response to any of the 

questions in the food security scale. 

Source: USDA, Economic Research Service using data from U.S. Census Bureau September 2020 Current Population Survey, Food Security Supplement. 
19

Conclusions

• The revised items functioned as well as the original items, and in some 

cases better. 

• Received OMB approval for full implementation in the CPS-FSS in 

December 2022: 

https://www.reginfo.gov/public/do/PRAViewICR?ref_nbr=202201-0536-

005

• We expect food security estimates to be comparable to be past years. 

• Questions: 

– Alisha Coleman-Jensen alisha.coleman-Jensen@usda.gov

– Matthew Rabbitt matthew.rabbitt@usda.gov
20

Like, Share, & Follow ERS 

www.ers.usda.gov @USDA_ERS linkedin.com/company/

usda-economic-research-service

www.ers.usda.gov/

data-products/charts-of-note

Subscribe to Weekly E-mail Notifications: www.ers.usda.gov/subscribe

Learn About Careers at ERS: www.ers.usda.gov/about-ers/careers-at-ers