Providing Time to Find Income Records--

Does it Help? Some Evidence from the 

National Household Food Acquisition 

and Purchase Survey (FoodAPS)

Consumer Expenditure Survey

Survey Methods Symposium

July 12, 2016

John A. Kirlin, USDA/ERS
2

2

Disclaimer

• The analyses and opinions expressed 

today are the author’s own and do not 

represent those of the ERS or the USDA.
3

3

Plan for Presentation

• Describe FoodAPS, its Income 

Worksheet, and purpose of worksheet

• Present results on who did and did not 

use the worksheet

• Present results on relationship between 

using the worksheet and data quality
4

4

Key Characteristics of FoodAPS

• National Household Food Acquisition and 

Purchase Survey - USDA

• Oversampled households participating in 

the Supplemental Nutrition Assistance 

Program (SNAP) and other low-income 

households

• Nationally representative, N ≈ 5,000

• Fielded from April 2012 to January 2013
5

5

Key Characteristics (cont)

• Measured all purchases and other food 

acquisitions reported by all household 

members for 7 days 

• Primary respondent (PR) identified as the 

person “who does most of the grocery 

shopping or who is the main meal 

planner”
6

PR calls 

to report

PR calls to 

report

Mon Tues Wed Thu Fri Sat Sun Mon Tue

0 1 2 3 4 5 6 7 8

PR calls to 

report

Initial Interview and 

training

Members track food acquisitions 

in booklets and with scanner

Final Interview

Screener

Overview of planned data collection week

6

Feedback Survey
7

7

Collecting Income Data

• Household screener asked about total 

household income

• Final Interview collected detailed 

information on monthly income for each 

household member aged 16 and over
8

8

Income Worksheet

• At Initial Interview, interviewer left an 

Income Worksheet for PR to fill out 

before Final Interview.

• PR could assemble income records for all 

members at his or her convenience

• Purpose was to improve accuracy of 

income data to be collected and to 

reduce respondent burden.
9

9

Income Worksheet
10

10

Income Worksheet – page 1
11

Income Worksheet Instructions

Use the table below to list the name of each person 

living in your household who receives income from any 

source. Next to each person’s name, list 

each type of income received last month 

and how often it was received. List the 

gross income from work. This is not the 

same as take-home pay. See the definitions at 

the bottom of the page for the items to include in each 

category.
12

Income Grid
13

Six income types

• Earnings (gross) from work

• Unemployment compensation

• Welfare, child support, alimony

• Retirement and disability income

• Investment income

• All other income
14

Use of Worksheet

• 60 % of households completed the 

worksheet, in whole or in part.

• Worksheet was not collected, but 

respondent could refer to it during the 

interview.

• We cannot differentiate between “in whole 

or in part”
15

Who used the worksheet?

Logit model covariates

• Age (PR)

• Sex (PR)

• Married (PR)

• Race (PR)

• Ethnicity (PR)

• College education 

(PR)

• Metropolitan area

• Rural area

• Assets > $3,000

• Gave consent for 

data matching

• Own home

• Household size
16

Percent completing worksheet

58%

46%

65% 63%

74%

62%

56% 55%

30%

40%

50%

60%

70%

80%

Differences are statistically significant

As indicated Opposite

* = p <0.10

** = p <0.05

*** = p <0.01
17

Percent completing worksheet

63% 65% 64%

56% 58% 57%

30%

40%

50%

60%

70%

As indicated Opposite

Differences not statistically significant
18

Percent completing worksheet

60%

57%

63% 61%

53%

61% 60% 59%

30%

40%

50%

60%

70%

As indicated Opposite

Differences not statistically significant
19

Percent completing worksheet

57.4%

64.4%

56.5%

41.4%

0%

20%

40%

60%

80%

1 (Ref) 2-3 4-6 * 7 plus **

Household Size

Differences statistically significant as shown
20

Hypothesized outcomes

• Households that fill out the Income 

Worksheet will:

• Have fewer missing data (needing 

imputation)

• Be less likely to “correct” reported 

income data

• Provide more accurate data (gross vs. 

net earnings, fewer “outliers”)
21

Measured Outcomes

• Any missing income data at individual 

level, by income type?

• Any missing income data at household 

level?

• Any need to “correct” reported income?

• Any presence of “outliers”?
22

Income “corrections”

• After all income questions asked, 

interviewer showed complete grid to PR 

on laptop screen and asked if there were 

any errors or omissions

• If yes, then errors corrected and new 

income amounts added

• Mostly new income amounts
23

Income “outliers”

• Generally, in top 1% of distribution

• For income types with few positive values, 

outliers identified subjectively
24

Hypothesized outcomes - caveat

• Relation between use of worksheet and 

measured outcomes is probably 

endogenous. 

• Those more likely to fill out worksheet 

may be more likely to give accurate and 

complete information anyway.

• Cannot infer causality
25

Outcome variables

Variable N

Percent 

(weighted)

Households with missing income data 853 16.0

Individuals with:

Missing/imputed earnings 339 7.0

Missing/imputed UI income 338 6.8

Missing/imputed transfer income 344 7.0

Missing/imputed retirement/disability income 549 10.0

Missing/imputed investment income 361 7.8

Missing/imputed other income 329 7.0

Corrected income data 94 2.1

Income outliers 51 2.2

Gross income reported 1,048 51.8
26

Percent of household records 

with any missing income

10%

25%

0%

10%

20%

30%

Used worksheet Did not use worksheet
27

Percent of individual records 

with missing income, by type

4% 4% 4%

6%

4% 4%

12% 12% 12%

16%

13% 12%

0%

10%

20%

All differences significant at 1% level

Used worksheet Did not use worksheet
28

Percent of individual records 

with any income corrections

1.4%

3.2%

0%

1%

2%

3%

4%

5%

Used worksheet Did not use worksheet

Difference is significant at 1% level
29

Percent of individual records 

with any income outliers

2.4%

2.0%

0%

1%

2%

3%

4% Difference not significant

Used worksheet Did not use worksheet
30

Percent of individual records with 

earnings reported as gross income

60%

37%

0%

10%

20%

30%

40%

50%

60%

70%

Used worksheet Did not use worksheet

Difference is significant
31

Recap – who used the worksheet?

• Primary respondents who were:

• Female

• Married 

• Not African-American

• Living in a non-metropolitan county

• Living in 1-3 person households
32

Recap – how well did it work?

• Primary respondents who used it:

• Had fewer missing income values

• Were less likely to find reporting 

errors during review 

• Were more likely to follow directions 

and report gross rather than net 

earnings

• But, cannot imply causality because of 

endogeneity
33

Thank you!

John A. Kirlin

jkirlin@ers.usda.gov

http://www.ers.usda.gov/data-products/foodaps-nationalhousehold-food-acquisition-and-purchase-survey.aspx
34

Extra slides, if needed
35

35

Definition for Earnings

Earning from Work (before deductions)

Enter the total amount each household member earns from work before taxes 

and other deductions (such as union dues, insurance premiums, and 

contributions to retirement plans). This amount should be listed on your pay 

stub. If a household member is paid in cash, enter that amount. If your income is 

not always the same, list the amount that you normally get. For example, if you 

normally get $1000 each month, but you missed some work last month and only 

got $900, list $1000 per month. If you normally get overtime, include it, but

not if you get it only sometimes.
36

36

Definition for Welfare, Child Support, 

and Alimony

Welfare, Child Support and Alimony

Most welfare comes from the Temporary Assistance for Needy Families (TANF) 

program (which may have a different name in your state) or from General 

Assistance, General Relief, or the GA program. These programs provide financial 

assistance to people with little or no income. In this column also include child 

support payments that are paid by a former husband, wife, or parent of a child 

living in your household; include child support payments paid through a welfare 

agency or court; and include alimony received by a member of your

household. Alimony is court-ordered support paid by one spouse to another 

after they are separated.
37

37

Definition for Retirement and 

Disability Income

Retirement and Disability Income

Enter retirement income from pension payments, retirement account 

withdrawals, and Social Security Retirement Benefit (SSA) payments. SSA is 

received by persons who have worked long enough and had enough Social 

Security deductions taken from their salary to be entitled to payments (these 

payments may be made to the spouse or dependent children of a covered 

worker). Enter disability income from Social Security Disability Benefit (SSDI) 

payments which are received by persons who cannot work because of a medical

condition. Also include: (1) black lung benefits; (2) workers compensation; and 

(3) Supplemental Security Income (SSI).
38

38

Definitions for Investment and All 

Other Income

Investment Income

Include: (1) income from rental properties; (2) interest, dividends and capital 

gains; and (3) trust fund payments.

All Other Income

Include any other income, such as: (1) strike benefits; (2) fuel assistance; (3) 

roomers, lodgers or tenants; (4) educational grants, loans or stipends; 

(5) insurance settlement payments; (6) Veterans Affairs benefits; (7) lottery 

winnings; and (8) employment bonuses.