Web Diary Feasibility Test: 

Preliminary Findings 

Ian Elkin, Michelle Freeman, John Gloster, Lauren 

Kirkpatrick, and Brett McBride 

Consumer Expenditure Survey Division/U.S. 

Census Bureau 

July 16, 2013 
Web Diary Team Overview 

2 

Motivation 

To encourage contemporaneous reporting, improve 

accuracy through access across locations and 

household members, provide a platform for individual 

diaries, allow for a multiportal approach, and to 

improve response rates by offering alternative 

response modes to the traditional paper diary. 

Objective

To understand the operational issues regarding 

implementing a web mode for CE and to understand 

the data quality and response rates associated with 

web collection. 
Presentation Outline 

3 

I. Web Diary Logistics 

II. Web Diary Instrument 

III. Web Diary Materials and Training 

IV. Preliminary Results 
I. WEB DIARY LOGISTICS 

4 
Web Diary Logistics 

5 

 Data Collection Period – Jan, Feb, Mar 2013 

 Test Closeout – Apr 30, 2013 

 Starting Sample Size – 1,300 (Nationwide, less 

HI/AK) 

 FR Workload – Approximately 2-3 extra cases/mo. 

per FR 
Web Diary Logistics 

6 

 Separate online WD application (ASD) 

 Debriefing Qs (for both R and FR) 

 Naming conventions developed for: 

 ASD production input files 

 ASD Web Data output / MCS CAPI Data output 

 NPC Web/CAPI data for Visual Basic keying 
Web Diary Logistics 

7 

 All research cases are “double placed” 

 FRs provide username and password to 

Respondent via CAPI instrument 

 New outcome codes for Web Diary (final 

disposition) 

 Test Limitations 

English only 

No Multi-CU HHs or Replacement HHs 

No Reinterview 

 New materials 

FR Talking Points (FR) 

User Guide (FR) 
Web Diary Flow of Data 

8 

 Expenditure Data delivered to DSD from ASD 

 Weekly (every Monday, 12:00am) 

 Cumulative, flat ASCII file 

 Expenditure Data delivered from DSD to NPC for 

keying 

 Weekly, Tuesday afternoon 

 CEPB unduplicates for Case IDs already sent to NPC in prior weeks 

 Recall Data delivered from DSD to NPC 

 Monthly, five working days after closeout 

 Converted to same file structure as Expenditure Data for NPC read-in

 NPC delivers keyed expenditure and recall data to 

DSD

 DSD merges files and converts to SAS datasets per BLS specification 

 DSD conducts parallel data review and releases data to BLS 
II. WEB DIARY INSTRUMENT 

9 
Web Diary CAPI Instrument 

10

 Screener Qs for both research and production 

 HM_NET – “Does this household have internet 

access at home?” 

 EM_ACCS – “Is this through... (PC, tablet, mobile 

phone)?” 

 Username and Password populated on “placement” 

screen 

 “Web” placed as a fill before “Diary” on respective 

CAPI screens. 

 Identical CAPI instrument used for both research and 

production cases 
Web Diary CAPI Instrument 

11

 Additional interviewer instructions serve as cues for 

FRs 

 In order to facilitate the collection of receipts as well 

as recalled data, a recall/receipts tab added for final 

pickup to allow for FR data entry 

 Additional outcome codes created to: 

 Separately identify research vs. production cases 

 Prevent additional programming for ROSCO 
III. WEB DIARY MATERIALS 

AND TRAINING 

12
Materials 

 Materials (both R and FR) 

 Two documents: 

 “Consumer Expenditure Diary Survey User Guide”

 “FR Talking Points”

13
 The Survey User Guide was designed as a reference 

guide for R. 

 The User Guide provided instructions on 

URL for the Web Diary 

Username and Password 

Start date for entering expenses 

Creating a household roster 

Contacting the FR and Regional office with 

questions and other concerns 

14

Consumer Expenditure Diary 

Survey User Guide 
Consumer Expenditure Diary 

Survey User Guide 

15
Consumer Expenditure Diary 

Survey User Guide 

How and where to enter expenses 

– Food and Drink Away From Home 

– Food and Drink for Home Consumption 

– Clothing, Shoes, Jewelry and Accessories 

– All Other Products and Services 

 How to save work and add rows 

 How to submit completed Web Diary expenses 

16
Consumer Expenditure Diary 

Survey User Guide 

17
Consumer Expenditure Diary 

Survey User Guide 

How and where to enter expenses 

– Food and Drink Away From Home 

– Food and Drink for Home Consumption 

– Clothing, Shoes, Jewelry and Accessories 

– All Other Products and Services 

 How to save work and add rows 

 How to submit completed Web Diary expenses 

18
Consumer Expenditure Diary 

Survey User Guide 

19

E 
Consumer Expenditure Diary 

Survey User Guide 

How and where to enter expenses 

– Food and Drink Away From Home 

– Food and Drink for Home Consumption 

– Clothing, Shoes, Jewelry and Accessories 

– All Other Products and Services 

 How to save work and add rows 

 How to submit completed Web Diary 

expenses

20
Consumer Expenditure Diary 

Survey User Guide 

21
Consumer Expenditure Diary 

Survey User Guide 

22
FR Talking Points 

 For the Census FRs, the Web Diary Team created – 

FR Talking Points 

 Similar in nature to the User Guide created for 

respondents 

 The FR Talking Points also reminded FRs to 

emphasize to respondents which expenses not to 

record in the Web Diary 

 The primary goal of the FR Talking Points was to 

provide a resource to FRs to thoroughly explain the 

Web Diary instructions to the respondent 

23
FR Talking Points 

24
Training - Web Diary VTC 

25

 Video teleconference (VTC) was held at the end of 

November 2012 for Regional Office staff 

 HQ virtual presentation to all Regional Offices 

 Background and Overview of WD Feasibility Test 

 Logistics and CAPI Instrument Changes 

 Walk-through of the Web Instrument 

 Question and Answer session 
Training - Web Diary Self-Study 

26

 FR training incorporated into November 2013 

Refresher Training 

 FRs had 4 hours to review materials (Self-study) 

 Introduction and Summary 

 Web instrument data entry 

 Practice interviews 

 Each FR given 

 Unique username and password to log-into diary for 

independent walk-through 

 Respondent User Guide 

 FR Talking Points 
IV. PRELIMINARY RESULTS 

27
Web Diary Preliminary Results 

28

 Help Desk 

 Response Rates 

 Diary Maintenance 

 Training 

 Other Issues 
Web Diary Operational Issues 

29

 No centralized “help desk” for respondents 

 Respondents were instructed to call (800) # as a direct line 

to DSD staff 

– General inquiries or surveys questions handled by 

Regional Office CE staff (or FRs) 

– Other technical issues, comments, or complaints 

forwarded to FSB via phone or email 

– Calls most often related to username/password issues, 

log-in after closeout, etc. 

 DSD routed all non-password-related technical 

issues to ASD 

 Both FSB and DSD maintain Web Diary Issues Database for 

tracking and documentation 
Web Diary Sample Performance 

30

 Response Rate – 45.89% 

 Production Response Rate – 72.11% 

 631 Completed Diaries 

 Ineligible Rate – 48.21% 

 Production Ineligible Rate – 22.63% 

 Screened Out Cases (no internet access/language barriers) – 

321 Cases 

Sample Size (diaries) 1,375 100.00%

Completes (diaries) 631 45.89%

Refusals (diaries) 744 54.11%
Web Diary Respondent 

Compliance 

31

 Web Diary Failure-to-login Cases 

 Defined as a placed diary where respondent failed to log into the instrument 

 Failure-to-login Rate - 28.92%

 Receipt and recall data 

 Average Number of Diary Entries per Week 

 Web Diary (not including recall data) – 28.51 mean diary entries per week

 Production Diary (not including recall data) – 31.60 mean diary entries per 

week 
Web Diary Paradata 

32

 Login Failure Rate – 36.03% 

 Mean Time in Instrument – 11.37 Minutes 

 Device Information 

Personal Computer (PC) 1,305 82.39%

Apple Mac 216 13.68%

Mobile Device 63 3.93%

Total 1,584 100.00%
Web Diary Paradata 

33

 Link Usage 

Itemized Instructions 70 22.01% 

User Guide 65 20.44% 

Instructions 58 18.24% 

About 46 14.47% 

Privacy 32 10.06% 

Burden 25 7.86% 

Other 22 6.92% 

Total 248 100.00% 
Web Diary Debriefing 

34

 R Cooperation 

 Difficult for Rs with little computer experience 

 Initial response was positive; however, R became less diligent in 

the last few days, similar to paper diary 

 “Out-of-sight, Out-of-mind” 

 R Experience 

 FRs report that Rs probably did not use materials 

 Materials needed top use simpler terms for Rs with limited 

computer experience 

 Rs reported misplacing materials 

 More motivational language would allow for easier placement 
Web Diary Debriefing 

35

 R Experience (cont.) 

 Provide flap for receipts in user guide similar to paper diary 

 Official letter provided by Census describing why the R is being 

asked to provide their email address 

 Allow four or five day window after the reporting period ends for the 

R to enter additional expenses that were forgotten during the 

reporting period 

 Rs frustrated that they had to enter purchase date for each 

expense 

 
Web Diary Debriefing 

36

 FR Experience 

 Self-study training inadequate, thought classroom training would 

have been superior 

 Recall and receipts were not handled in an intuitive way 

 Difficult to enter receipts and recall data without having access to 

what was entered by the respondent 

 Send the R reminder emails or text messages 

 Complete a demonstration of the web diary before the FR leaves 

 Let Rs customize their usernames and passwords 
Main Issues 

37

 Issues and expected resolutions 

Lower Response Rate Offer multiple survey modes & improved FR 

training 

Lower Expenditures and 

Failure-to-login

Tailored follow-up through paradata monitoring 

Help Desk Offer dedicated help desk through Census
Input into Individual Diaries 

Field Test 

38

 Targeted Sampling 

 Paradata Monitoring 

 Streamlined and Efficient Design 

 Straightforward R Materials 

 Comprehensive FR Training 

 Additional Follow-up Procedures 

 Receipt/Recall Procedures 
Acronym Definitions 

 WD – Web Diary 

 FR – Field Representative 

 R – Respondent 

 CU – Consumer Unit 

 HH - Household 

 CAPI – Computer-assisted Personal Interviewing 

 MCS – Master Control System 

 FSB – Financial Surveys Branch 

 CEPB – Consumer Expenditure Programming Branch 

 DSD – Demographic Surveys Division 

 ASD – Application Support Division 

 NPC - National Processing Center 

 ROSCO – Regional Office Survey Control 

39
Contact Information 

40

Ian J. Elkin 

elkin.ian@bls.gov