1 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

The Good, The Bad, and The Online Diaries

Ian J. Elkin

Branch of Research and Program Development

2018 CE Survey Methods Symposium

July 17, 2018

Any opinions expressed in this presentation are those of the authors and do 

not constitute policy of the Bureau of Labor Statistics.
2 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Outline

I. Background

II. Web Diary Feasibility Test

III. Individual Diaries 

Feasibility Test

IV. Proof of Concept Test

V. Online Diary Improvement 

Project 

VI. Large Scale Feasibility Test
3 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

I. Background - Consumer Expenditure Diary

Overview (i.e. Single and Searching)

 CE currently uses one paper diary to collect household expenditures 

for a week

 Interviewers “place” the diary in-person and conduct one more visit 

 The Diary gathers more detailed expenditures than the Consumer 

Expenditure Interview Survey

 Single proxy reporter for all HH expenditures

 The Diary is divided into four sections: 

 Food Away From Home 

 Food for Home Consumption 

 Clothing, Shoes, Jewelry and Accessories 

 All Other Products, Services and Expenses 
4 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

II. Web Diary Feasibility Test (i.e. The Meet Cute)

Overview

 Mode: Desktop Diary

 New materials

 Interviewer Talking Points 

 User Guide

 Eligible cases

 English only

 No Multi-CU HHs or Replacement 

HHs

 Single proxy reporter for all HH 

expenditures

 Assignment Protocol

 If a HH screened in as eligible, HH 

was assigned a single diary for the 

entire HH, similar to official 

assignment protocols

Findings

 Higher median expenditures for the 

following sections:

 Clothing, Shoes, Jewelry, and 

Accessories

 Food and Drinks for Home 

Consumption 

 Lower item nonresponse, as defined 

for a diary instrument

 Week-to-week drop-off potentially 

validate shift to one-week collection 

period
5 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

II. Web Diary Feasibility Test
6 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

III. Individual Diaries Feasibility Test (i.e. The Honeymoon 

Period)

Overview

 Modes: Desktop and Mobile

 New materials

 User Guide

 Eligible Cases

 English only

 No Multi-CU HHs or Replacement 

HHs

 Home internet access via PC, 

tablet, Smartphone

 Person-level reporting, main R 

recorded expenditures for ineligible 

Rs and for any HH members not 

participating

 Assignment Protocol

 If a HH screened in as eligible, HH 

members were offered modes 

sequentially

 Mobile first &, if they did not 

possess a Smartphone, then

 Desktop mode

 Absent eligible members were 

assigned mode by main R
7 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

III. Individual Diaries Feasibility Test (i.e. The 

Honeymoon Period)

Findings

 No improvement in household cooperation rates

 Determining the extent of contemporaneous reporting of expenses was 

hampered by data limitations 

 Difficulty distinguishing single proxy respondents versus multimember HHs

 Fewer entries and lower reported expenditure totals

 Positive feedback regarding the ease of recording expenses and the 

security of the data
8 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

III. Individual Diaries Feasibility Test
9 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

IV. Proof of Concept Test (i.e. Honey You’re Great, 

but...)

Overview

 Tested full redesign

 Modes: Desktop, Mobile, & Paper

 Incentivized 

 Person-level reporting, main R 

recorded expenditures for ineligible 

Rs, HH level expenses, and for any HH 

members not participating

 Interviewers had access to a summary 

report of web diaries for their cases

 Interviewers were told to call the main 

R within 2 days of placement, if a R 

hadn’t logged in or if R hadn’t made 

entries

 Assignment Protocol

 Member-level internet access and 

level of use were determined

 Members with internet access were 

offered an online diary or a paper 

diary

 Members without were offered the 

paper diary

 Absent eligible members were 

assigned mode by main R
10 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

IV. Proof of Concept Test (i.e. Honey You’re 

Great, but...)

Findings

 Participation within HHs was much broader for the POC than 

in prior tests

 Respondents generally higher educated and under the age of 

39

 Lower take up rate for the online diary when offered a paper 

option

 Diary expenditures were not significantly different than 

equivalent production amounts

 However, HHs reported lower median expenditure totals and

 Greater numbers of small-value expenditures
11 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

IV. Proof of Concept Test
12 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

V. Online Diary Improvement Project (i.e. The Self 

Reflection Stage)

Findings

 Mobile devices were frequently used, 

but not as expected 

 Most respondents used at home

 Expenses entered via mobile device 

were more timely

 Password the biggest barrier to 

access & mobility 

 Low incidence of data quality issues, 

but still problematic 

Overview

 Two instruments optimized to the R’s 

device (mobile or desktop/laptop) 

and accessed through a single portal 

 Incentivized

 Person-level diary design 

 Developed based on previous CE 

online diaries 

 Robust paradata output 

 Modular design that allows for 

technological enhancements
13 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

V. Online Diary Improvement Project
14 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

VI. Large Scale Feasibility Test (i.e. Does a Large 

Feasibility Test Mean Forever?)

 Large-scale implementation of Online Diary Improvement 

Project diary design 

 Return to HH-level diary in lieu of personal diaries

 Restores two one-week diary design

 Introduction of outlet question for recording business where 

item was purchased/consumed at

 Elimination of conditional incentives with token incentive 

remaining
15 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

VI. Large Scale Feasibility Test

Date & Number 

of Expenditures

List of Expenditures, 

Price Paid, & Edit and 

Delete Buttons

Bar with Filter, 

Search, & Add 

Buttons
16 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

VI. Large Scale Feasibility Test

Outlet 

Question

Expanded 

“Other” 

Categories
17 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

VI. Large Scale Feasibility Test
18 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

VI. Large Scale Feasibility Test
19 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

VI. Large Scale Feasibility Test

 Desktop version will render 

similar to the mobile 

version, but allow for more 

screen real estate

 Screenshots forthcoming
20 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

The Good, The Bad, and The Online Diaries

“You see, in this world 

there’s two kinds of 

surveys, my friend: 

Those that are online 

and those that aren’t. 

This one’s online” –

Blondie
Contact Information

21 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Ian J. Elkin

Senior Economist

Branch of Research and Program Development

The Consumer Expenditure Survey

Elkin.Ian@bls.gov
22 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Double Secret Slide – Wonder Twins Powers 

Activate!