by

Geoffrey Paulin, Ph.D. 

Senior Economist 

Consumer Expenditure (CE) Survey Program 

CE Survey Methodology Symposium 

July 17, 2012 

Washington, DC 

Experiments in Reducing Measurement Error 

Through Cross‐Survey Imputation: 

 Preliminary Findings for Diary to Interview 

Survey Imputations
Reducing respondent burden 

is an important goal...

...The average quarterly interview is one hour; 10 percent exceed 90 minutes. 2 
However, this must be balanced with 

maintaining high quality of data. 

3 
To achieve this, the CE program is 

investigating the feasibility of imputing 

Diary expenditures to Interview. 

4 
 This presentations includes: 

1. The conceptual framework currently being investigated 

2. Problems encountered or anticipated 

3. A request for comments 5 
At present, there are three basic 

categories of expenditure under 

consideration: 

6 
1. Those for which Interview 

respondents are asked are about 

“usual” weekly/monthly 

expenditures 

2. Apparel 

3. Items collected only in Diary 

(non-prescription drugs, personal 

care) 

7 
1. Respondents asked about “usual” 

weekly/monthly expenditures 

 Food at home 

 Food away from home (except on trips) 

 Alcoholic beverages at home 

 Alcoholic beverages away from home 

(except on trips) 

8 
Reasons for considering: 

 More detailed information is available in the Diary 

Survey than the Interview Survey. Example: Interview 

asks how much was spent in total during a usual week; 

Diary collects specific items each week (e.g., lettuce, 

potatoes, rice). 

 A “usual” question is also asked for food at home in the 

Diary. Preliminary research shows that the answers are 

qualitatively similar to those obtained from the 

Interview, but different from the weekly responses. 

 Confusion caused by the global question (what 

constitutes a “usual” week?) are eliminated in Diary, 

which records actual expenditures as they happen. 

9 
2. Apparel 

Reasons for considering: 

Burden reduction. 

– In 2010, 77 percent of respondents reported 

expenditures for apparel and services (Section 9). In 

these cases: 

• Section 9 accounted on average for 7 percent of total 

interview time (4 minutes), and increased with family size. 

• 25 percent of reporters required more than 5 minutes to 

complete the section; 10 percent required nearly 9 

minutes. 

Many items collected in both surveys are selected 

from the Diary for integrated publications. 

10
3. Items collected only in Diary (nonprescription drugs, personal care) 

 Reasons for considering: 

To improve quality of supplemental poverty 

measures 

Positive externality (side benefit) for public 

use microdata users 

11
The first item considered 

is food at home. 

 Thought to be the simplest item to 

model. 

Nearly 99 percent of all consumer units in 

the Interview Survey reported food at 

home expenditures in 2010. 

Percent reporting weekly expenditures in 

the Diary Survey was presumed to be 

similar. 

12
Until...

13
REALITY CHECK 

 While large, percent reporting each 

week is much lower—about 81 percent 

in 2010. 

 A two-stage approach (stage 1: predict 

probability of purchase this week; stage 

2: predict level of expenditure if 

shopping occurs) is needed. 

14
Still, a general framework 

is constructible. 

 Using Diary data, estimate a binary variable (shopping 

occurred=1, shopping did not=0) and separately model 

level of expenditures for those who shopped. 

 Because there are 13 weeks in a quarter, predict 13 

binary outcomes for each consumer unit in the Interview 

Survey, by applying parameter estimates from the Diary 

model. 

 Similarly, predict level of shopping 13 times, if it occurred. 

 Multiply the binary and level variables. 

 Sum the 13 results, each of which is $0 or positive. The 

outcome is the estimated quarterly expenditure for food 

at home. 

15
REALITY CHECK 

Survey nonresponse plays a role. 

 A sizeable minority (8 to 10 percent each year since 2006) participate 

for only one of the two Diary weeks. 

 In publication, each week (whether or not two are present) is treated 

independently. However: Those who participate in both Diary weeks 

were different than those who participated in only one. 

 83 percent of two-week participants report food at home 

expenditures each week, compared to 74 percent of second-weekonly participants, and 51 percent of first-week-only participants. 

 Even among purchasers, week-one-only participants spend far less 

on average on food at home ($49) than week-two-only participants 

($75) or both-week participants ($85). 

 Conclusion: Not only are one-week only participants different from 

both-week participants, but first-week-only participants are 

different than second-week-only participants! 

16
Additionally, there is no reason to 

presume a priori that the lack of 

reports of purchase are invalid...

...The qualities that cause the 

respondent to participate only one 

week (e.g., too busy) may also 

explain why they did not purchase 

food during the week of 

participation. 

17
Another complication: 

Last one, I promise... 18
Temporary Absence 

 According to Diary Survey methodology, 

even two-week participants can have at 

least one week counted as a valid Diary, 

yet be temporarily absent (out of town, in 

the hospital, or away for another reason). 

 By definition, a temporarily absent 

consumer unit cannot have expenditures 

for food at home. (Apparel, yes. Food at 

home, no.) 

19
Taking into account all these factors, 

the following construct is being tested: 

 Use multinomial logit on Diary data to predict 

participant type (first-week only, second-week 

only, or both weeks) for each Interview family; 

 Predict, 13 times, the probability of temporary 

absence during the week; 

 Using only those present, model probability of 

purchase, and apply to Interview 13 times per 

record; 

 Using only those who purchased food at home, 

model expenditures and apply to Interview 13 

times per record. 

20
Status: 

 A working prototype of the system is producing results for all 

consumer units. It is flexible to allow for different models by 

family type, as the age, income, and other effects for single 

persons is expected to be different than that for married 

couples, families with children of different ages, etc. 

 Results of the prototype are being evaluated for quality 

assessment. 

 Predicted probabilities at each stage are compared to actual 

frequencies. 

 Mean Absolute Deviations (model predicted food 

expenditures compared to actual reported food 

expenditures) are being analyzed. 

 Differences between final estimates (once four stages are 

combined) and reported values are being compared using 

standard univariate and t-tests. 21
Looking ahead: 

22
Apparel 

23
Anticipated questions 

 While a person who is temporarily absent can buy 

apparel while away from home, it cannot be observed 

in the Diary. How frequently does this occur, and what 

is its effect on outcomes? 

 Given that, unlike food at home, many apparel items 

are likely to be purchased at most one time per quarter, 

does the 13-week-by-week estimation still make sense? 

 How many consumer units in Interview will be predicted to 

purchase these items once, twice or more, or not at all each 

quarter? 

 What will the sample size required to model levels of 

expenditures using purchasers only? 

24
General questions 

 Quality assessment: Once acceptable results are 

obtained for Diary participants, how do we 

demonstrate that the imputed values in Interview are 

better than—or at least as good as—those actually 

reported? 

 What are the unintended consequences of replacing 

reported with imputed data? 

 Can/will covariate relationships be preserved (e.g., food 

at home with apparel) 

 If not, what are the implications for the supplemental 

poverty measures, and other important uses of the data? 

25
Technical Questions 

 Should single or multiple imputation be 

used? 

 If multiple imputation is used, what is the 

proper way to use income, which is itself 

multiply imputed? 

 Use average imputed income for each consumer unit the same 

way as a non-imputed variable would be used, generating five 

imputations of the Interview expenditure variable 

 Obtain a regression estimate using the five imputed income 

values; shock; repeat four times. In this way, 25 regressions 

yield 5 imputed expenditure values per consumer unit. 

26
Next Steps: 

 Continue refining food at home model 

and assessing quality of results 

 Receive and incorporate comments and 

suggestions from experts like you(!) 

 Prepare proposal for next phase by 

September 28, 2012 

27
If you have any suggestions, comments, 

or questions of your own...

...The team looks forward to hearing from you.

28
Contact Information 

Geoffrey Paulin, Ph.D. 

Senior Economist 

Consumer Expenditure Survey 

Program 

www.bls.gov/cex 

202-691-5132

Paulin.Geoffrey@bls.gov