Testing the Feasibility of 

Collecting Expenditure Data via 

Individual Mobile and Desktop 

Diaries: Preliminary Results

Ian Elkin, Brandon Kopp, Brett McBride, and 

Lucilla Tan

Consumer Expenditure Survey Division

CE Survey Methods Symposium

July 14, 2015
Presentation Outline

2

I. Individual Diaries Overview

II. Preliminary Results – Sample 

Performance & Demographics

III. Preliminary Results – Paradata

IV. Preliminary Results – Reporting 

Characteristics

V. Summary
I. INDIVIDUAL DIARIES 

OVERVIEW

3
Consumer Expenditure Diary 

(CED) Overview

 The CED currently uses 1 paper diary to collect 

household expenditures for a week (x2) 

 Interviewers “place” the diary in-person and 

conduct up to two more visits 

 The CED gathers more detailed expenditures 

than the Consumer Expenditure Interview Survey

 The Diary is divided into four sections: 

 Food Away From Home 

 Food for Home Consumption 

 Clothing, Shoes, Jewelry and Accessories 

 All Other Products, Services and Expenses 

4
Individual Diary Test Analysis

 Research conducted to-date

 Sample performance: screen-in and placement rates

 Paradata: interviewer monitoring and logins

 Reporting: section entries and interviewer-assisted 

entries

5
Individual Diary Test Analysis –

Future Research

 Further Individual Diaries comparisons by mode:

 Reporting quality: Actual date of diary entry vs. 

purchase date

 Assess duplicate entries within a CU

 Test Versus Production: 

 CU-level response rates

 Patterns of entries by category

 Reporting quality: total recall, level of detail required, 

rounding

 Contact attempt history: effort, reluctance

6
Individual Diaries Background

 The Individual Dairies Feasibility Test (IDFT) was 

designed to inform the operational and data 

quality aspects of collecting expenditures from 

household members using personal electronic 

diaries

 Since 2004, there have been various ongoing 

initiatives to test various forms of instruments 

to address the limitations of one paper diary for 

households with multiple members 

 Two modes of electronic diaries were tested:

 Mobile internet (via smartphone)

 Desktop internet

7
Individual Diaries Logistics

8

 Data Collection Period – Aug - Dec 2014

 Starting Sample Addresses – 1,200 (Philadelphia, Chicago, 

& Los Angeles Regional Offices)

 Targeted sampling on area mobile usage, internet 

penetration, multi-person households, and English 

speaking households

 Developed separate online mobile & desktop applications

 Diary placement occurs once across the 2-week recording 

period

 Interviewers provide username and password to 

Respondents

 No individual-level outcome codes for Ind. Diary (final 

disposition)

 New materials 

 Mobile User Guide

 Desktop User Guide

 YouTube Online Tutorials
Individual Diaries Eligible Cases 

& Assignment Protocol

9

 Eligible Individual Diaries Cases

 English only

 No Multi-CU HHs or Replacement HHs

 Home internet access via PC, tablet, Smartphone

 Assignment Protocol

 If a household screens in as eligible, household 

members are offered modes sequentially

 Mobile mode first and, if they do not have a 

Smartphone, then

 Desktop mode, including tablet 
Individual Diaries Monitoring 

Protocol

10

 Individual Diaries did not provide 

interviewers the opportunity to monitor if 

diary entry occurred during the recording 

period (due to technical/resource 

limitations)

 Interviewer Follow-up Calls

 Reminder Calls (or visits if necessary) on Day 3 

and Day 8 were encouraged for all participating 

households

 Tailored follow-up calls were made to households 

that Census flagged for going three consecutive 

days without logging into the instrument
Individual Diaries Training and 

Materials

11

 Training conducted in-person for one day 

and was a departure from standard CE 

training protocols - focused more heavily on 

protocol changes

 Respondent Materials

 Mobile and Desktop User Guides – Focus on 

logging in and entering expenses

 Three YouTube Online Tutorials

 Getting Started

 What to Enter in the CE Diary

 Entering and Editing Expenses
Individual Diaries Analyses

12

 Focus of the following analyses is on the 

comparison of test conditions to each other 

(mobile versus desktop) and their relative 

performance

 Comparisons to the control group 

(production sample, at the household level) 

are forthcoming
II. PRELIMINARY RESULTS –

SAMPLE PERFORMANCE & 

DEMOGRAPHICS

13
14

Characteristics of Sampled 

Households who Screened In

Households with home internet access (n=471)

Household size (% of n)

DK/REF 1.9

Single persons 20.0

2-3 persons 32.9

4+ persons 45.2

No. e-diary age-eligible CU members 2.1

Combination of e-diaries assigned within CU (% of n) 

Mobile only 49.0

Desktop only 38.6

Mobile & Desktop 9.1

Neither 1.3
15

3

33

38

20

6

2

14

20

43

21

16_<21 21_<35 35_<50 50_<64 >=64

% Prevalence in a mode

Mobile (n=152) Web (n=272)

Assigned Diarists by Age 
16

Assigned Diarists 

by Education Attainment 

4

14

24

59

5

16

32

47

<HS HSgrad Some College College grad

% Prevalence in a mode

Mobile (n=152) Web (n=272)
Assigned Diarists by 

Household Size

17

16 20

19

10

52

55

14

39

MOBILE (NO. DISTINCT 

CUS: 90)

MOBILE & WEB (24) WEB (111)

NUMBER OF HOUSEHOLDS

ASSIGNED MODE TO HH MEMBERS

 CU size: Single 2-3 persons 4+ persons
Individual Diaries Analysis Rate 

Assumptions

18

 Household-Level Screen-In, Diary 

Placement, and Completion

Only assesses household level diary 

placement and completion information 

recorded by the interviewer

Rates computed based on household 

outcome code, placement code, pick-up code, 

and internet screener

 Person-level placement and completion 

outcomes are classified by the outcome 

of the household to which they belong
Individual Diaries Visual 

Response

471/788

CE-Eligible Households (788)

Test-Eligible Households (471)

Placed (437)

Completed Diaries (225)

Full Response

Total Recall Mix
III. PRELIMINARY RESULTS –

PARADATA

20
Log-In Monitoring Protocol and 

Interviewer Compliance

21

 Census monitored placed diaries to determine whether 

respondents were logging in 

 If a respondent went three or more days without logging 

in, Census flagged the household and the assigned 

interviewer was notified to contact the household

Regional Office 

(# of HHs)

Households flagged for 

follow-up

Percent with Follow-up 

(n=17)

Philadelphia RO

(n=203)

45 7%

Chicago RO

(n=247)

76 17%

Los Angeles RO 

(n=338)

60 2%

Total 181 9%
Log-In and Log-In Failures by 

Mode

22

0%

5%

10%

15%

20%

25%

Log-ins

Mobile Web

0%

1%

2%

3%

4%

5%

6%

7%

Days in Diary Period

Log-in Failures
Log-In and Response

23

0%

20%

40%

60%

80%

100%

120%

0 2 4 6 8 10 12 14

Experience (% of Login Attempts Successful)

Number of Days With Unique Login

Relating Login Experience To Expenditure Reports

Mobile Web Size indicates Count of Reported Expenditures

More Unsuccessful

Logins than 

Successful Ones
Assigned Mode versus Log-In 

Mode

24

 The tabulations on number of entries by mode were based 

on assigned mode. 

0

98.2

1.8

1.3

77.0

21.7

NO INFO (MISSING)

NO CHANGE

AT LEAST 1 MODE CHANGE IN LOGIN 

HISTORY

Percent of mode group

Prevalence of mode change at login vs 

assigned mode 

Mobile (n=152) Web (272)
Respondent Log-Ins by 

Operating System

25

Mobile 

Android, 

16.3

Mobile iOS, 

19.6

Mobile Other, 

0.6

Web Linux, 0.6

Web Macintosh 

9.8

Web Windows, 

53.1

Percent distribution of distinct persons who 

logged in by operating system 

(no. persons = 337)
IV. PRELIMINARY RESULTS –

REPORTING 

CHARACTERISTICS

26
Respondent-Level Diary Keeping 

by Mode

27

 Mobile diarists were younger and more likely to have 

completed college

 Desktop diarists were older and more likely to be from 

households with fewer persons

 For both mobile and desktop diarists the largest number of 

entries occur in Food for Home Consumption (FDB)

 Week to Week Entries

 Among desktop diarists the largest decline in average number 

of entries occurred in FDB; however FDB was relatively steady 

for mobile diarists

 Among mobile diarists, the largest week to week decline was in 

All Other Products, Services, and Expenses (OTH)

 With the exceptions noted previously, there did not appear to be 

an association between diary week and the number of entries 

reported in other categories by mode
Entries per Respondent by Mode

28
V. SUMMARY

29
In Summary

 Sample Performance: Diary keeping information 

that is automatically generated allows for 

protocols to be updated more rapidly

 Paradata: Access to up-to-date paradata allows 

for on-demand interviewer intervention 

 Data Quality: Electronic diaries allow for the 

collection of data from some diaries that are not 

successfully picked up

 Reporting Characteristics: Electronic diaries, 

specifically mobile diaries, do an effective job 

targeting groups generally underrepresented in 

CE data

30
Contact Information

31

Ian J. Elkin

elkin.ian@bls.gov