Is a User-Friendly Diary More Effective? December 2006

Nhien To

Lucilla Tan

Division of Consumer Expenditure Surveys

U.S. Bureau of Labor Statistics

2 Massachusetts Avenue, NE, Washington, DC 20212

American Association for Public Opinion Research Annual Conference

Montreal, Canada

May 18-21, 2006

All views expressed in this paper are those of the authors and do not reflect the views or policies 

of the Bureau of Labor Statistics.
2

Is a User-Friendly Diary More Effective?

Nhien To, Lucilla Tan

Bureau of Labor Statistics

I. INTRODUCTION

In theory, redesigning a survey questionnaire to make it user-friendly would be a positive 

change. One would expect that if respondents were asked to complete a survey that is easy to 

use and less burdensome, they would be more likely to respond, and the increased response rate 

would in turn increase data quality. This would be the expected results of a more effective 

questionnaire. However, in practice, surveys are often complicated and multidimensional and 

the Consumer Expenditure Diary Survey (CED) is no exception. This paper compares 

production data collected using a Former Diary survey questionnaire with production data 

collected using a Redesigned “user-friendly” Diary to evaluate whether a user-friendly diary 

really is more effective. After comparing the data, we found that the data collected using the 

Redesigned Diary did not improve on every quality measure, but that it improved on most of 

them, making the new diary design a success. 

A. Background

The CED is one of two nationwide surveys of households used by the Bureau of Labor

Statistics (BLS) to collect detailed information about household expenditures, and is intended to 

collect small, frequently purchased items. Data from the survey are used for economic analysis and 

to provide weights for the Consumer Price Index. Respondents are asked to record their

household's expenses for two consecutive weeks. The weeks are treated as two separate cases. 

Every year expenditure data is collected from 7,500 households, yielding 15,000 completed 

weekly diaries. 

Keeping the diary, a self-administered survey form requiring significant respondent 

effort, is a burdensome task. In 2000 BLS established a team to conduct research leading to the 

design of multiple diary prototypes, with the goal of reducing respondent burden and improving 

data quality. After undergoing several rounds of testing, focus groups, and reviews, the team 

identified a set of user-friendly features (Davis et al., 2002) and narrowed the selection down to a 

single prototype. The prototype was then field tested (Figueroa et al., 2003). Based on the 

results of the field test, the team added cues to the recording pages of the Diary and cognitively 
3

tested it with additional formats of cues (To et al., 2004). The Redesigned Diary was then “fine 

tuned” and implemented in production. 

This paper looks at the effectiveness of the Redesigned Diary (the form that went into 

production in 2005), by comparing its performance with the Former Diary (the version used in 

the production survey from 1993 through 2004). We focus our comparisons on participation 

rates and on measurements resulting directly from format changes, so as to isolate effects 

attributable to form design from time sensitive measurements such as expenditures, which could 

be biased by changes in economic conditions and consumer confidence across the two years. We 

hypothesized that the user-friendly features of the Redesigned Diary would lead to increased 

response rates, and that they will elicit higher-quality data, that is, more complete and detailed 

recording of expenditures.

1

B. User-friendly Features

Simplification of form layout. The CED is organized by the day of the week, and collects 

expenditures separately for each day. In the Former Diary, each day contained 5 major 

expenditure categories, each broken down further into subcategories. For example, Food for 

Home Consumption was further divided into 15 subcategories, such as “Bakery Products” and 

“Poultry.” (see Table 1 for a complete list of the categories.) In order to record a grocery item, a 

respondent first had to locate the major expenditure category and then had to find the correct 

subcategory. The Redesigned Diary has a simpler organization, and collects the four major 

expenditure categories shown on the right side of Table 1. Each diary day consists of 4 pages,

one for each major expenditure category, as opposed to 7 pages per diary day in the Former 

Diary. In place of subcategories in these major expenditure categories, cues (examples) are 

listed at the top of each recording page to serve as a recall and memory aid. In addition, the 

Redesigned Diary shows the days of the week at the top of each page. The interviewer circles 

the correct day (e.g., a respondent might be asked to start reporting on a Wednesday), which 

should help to remind the respondent which day’s expenditures are to be reported on which page. 

(See Appendix A for a comparison of the page layouts for the two Diary versions.) 

 

1

 An underlying assumption of the Consumer Expenditure Surveys is that the surveys are subject to significant recall 

error (e.g., Silberstein, 1989). Therefore, anything that results in a higher level of reporting is “better” and closer to 

the true value.
4

Table 1. Comparison of Main Expenditure Categories (Parts) and Subcategories

Former Diary Redesigned Diary

Part 1. Food Away From Home (MLS)

ß Fast Food, Take-out, Delivery, Concession 

Stands, Buffet and Cafeteria

ß Full-Service Meals, Snacks, Drinks

ß Vending Machines and Mobile Vendors

ß Employer and School Cafeteria

ß Board or Meal Plan Food

ß Catered Affairs

Part 2. Food for Home Consumption (FDB)

ß Flour, Cereal, and Other Grain Products

ß Bakery Products

ß Beef

ß Pork

ß Poultry

ß Other Meats

ß Fish and Seafood

ß Fats, Oils and Dressings

ß Eggs and Diary Products

ß Fruits and Fruit Juices

ß Sugar, Sugar Substitutes and Sweets

ß Vegetables and Vegetable Juices

ß Other Food Items

ß Non-Alcoholic Beverages

ß Alcoholic Beverages

Part 3. Food and Beverages Purchased as Gifts (FDB)

Part 4. Clothing, Shoes and Jewelry (CLO)

ß Casual, Sportswear, Formal

ß Undergarments and Sleepclothes

ß Outdoor, Work, School, Costumes

ß Shoes

ß Sports – Team Clothes and Sport Shoes

ß Jewelry, Accessories, and Sewing Items

ß Clothing Services

Part 5. All Other Purchases and Expenses (OTH)

ß Tobacco and Smoking Supplies

ß Gasoline, Oil, and Additives

ß Medicines, Medical Supplies and Services

ß Personal Care Products and Services

ß Housekeeping Supplies and Services

ß Housewares and Small Household Appliances

ß Home Furnishings, Decorative Items, Linens,

and Major Appliances

ß Home Maintenance, Hardware, Lawn Supplies 

and Services

ß Housing Expenses

ß Entertainment/Amusements and 

Sports/Recreation

ß Transportation Expenses

ß School Expenses

ß All Other Expenses

Part 1. Food and Drinks Away from Home (MLS)

 vendor type subcategories changed to checkboxes

 meal type checkboxes added

 

Part 2. Food and Drinks for Home Consumption

(FDB)

 gift purchases subcategory changed to a 

 checkbox; all other subcategories eliminated

Part 3. Clothing, Shoes, Jewelry, and Accessories

(CLO)

 all subcategories eliminated

Part 4. All Other Products, Services, and Expenses

(OTH)

 all subcategories eliminated
5

Addition of checkboxes to solicit details about purchase. Along with the subcategories, the 

Diary collects additional details about different types of purchases. For Food and Drinks away 

from Home (MLS), these details include the type of meal and type of “vendor.” Both diaries 

request details about packaging for Food and Drinks for Home Consumption (FDB), i.e., 

whether the food is fresh, frozen, etc.), and the gender and age of the expected wearer for 

Clothing, Shoes, Jewelry, and Accessories (CLO). We also ask respondents to identify 

purchases intended as gifts for persons outside of the household in the FDB, CLO, and All Other 

Products and Services (OTH) sections.

As part of the redesign, we added checkboxes for meal type, vendor type, and alcohol 

type to the MLS recording page. In the Former Diary, the meal type and alcohol type were taken 

from the respondents’ written description, and vendor type was collected from the subcategory 

classification. (See Appendix B for a sample MLS recording page.) Checkboxes for gender and 

age were also added to the CLO recording page. Formerly, the respondent had to specify the 

gender-age attribute from a list of coded values. (See Appendix C for a sample CLO recording 

page.)

Clarification of instructions. Instructions, rules, and definitions were simplified and clarified. In 

addition, we added a greater variety of examples, highlighted important recording instructions, and 

introduced a Frequently Asked Questions (FAQs) section. The FAQs and Examples are printed on 

flaps attached to the front and back cover for easy access. The flaps can also serve as bookmarks to 

help respondents keep their place in the diary. 

Reduction in size and change in orientation. The Former Diary measured 8.5” x 14” inches with 

a landscape orientation, and the pages opened from the bottom. The Redesigned Diary is 8.5” x 

11” with a portrait orientation, and the pages open to the side like a traditional booklet. The 

number of pages dropped from 66 to 44. 

Enhancement of the overall look. The Former Diary cover consisted of plain text in a dull green 

color, which was also used on the recording pages. The document had a rather intimidating 

appearance. The Redesigned Diary displays color photographs on the cover and a vibrant blue 

color on the recording pages, resulting in a more visually appealing document.

 
6

II. DATA

This analysis compares data collected in January through December 2005 in the 

Redesigned Diary with data collected over the same months of 2004 in the Former Diary. Based 

on CE placement rules and the cooperation of the respondent, every diary is assigned an outcome 

code stating whether the case is a completed case. There were approximately 14,000 completed 

cases for each type of diary. 2 

III. FINDINGS

A. Participation Rates

Form designers hoped that the smaller, more attractive questionnaire and simplified 

layout would encourage more respondents to participate in the survey. Response and attrition

rates are two indicators of how well they succeeded.

Response Rate

The response rate definition used in this evaluation is the ratio of completed cases to inscope cases and matches the AAPOR response rate definition RR1. The AAPOR definition of 

RR1 is as follows:

(Interview) RR1 = [(Interview + Partial Interview) + (Refusal + Noncontact + Other) 

+ (Unknown Household Occupied + Unknown Other)]

Since there are no partial interviews for the CED, and the field interviewers are trained to code 

refusals, noncontacts, and other cases of unknown eligibility as ‘Type A nonresponse’, RR1 

reduces to the following for CE:

(Interview)

RR1(for CE) =

[ (Interview) + (Type A) ]

Interview in the formula shown above is equivalent to ‘completed cases’ in the CED. 

Completed cases are cases where the household completed the weekly diary or where the 

household was temporarily absent for the week3. In-scope cases are completed cases plus 

 

2

 The U.S. Census Bureau field interviewing staff conducts the Diary survey for BLS.

3

 BLS counts cases that will be “temporarily absent” for the entire data collection period as completed cases for the 

Diary survey. The parallel Consumer Expenditure Quarterly Interview Survey (a second household expenditure 

survey conducted by the CE to obtain large and infrequent expenditure items) collects data for respondents who are 

away from home on trips. Since Diary and Interview data are later merged, this practice avoids double-counting the 

same set of expenditures. 
7

‘Type A’ nonresponse cases (cases where the household refused to do the Diary survey for 

various reasons, or the diary was not placed with the household during the collection period, or 

the returned diary was blank and the majority of items were recalled without receipts4). The 

response rate was computed separately for each Diary week, as well as overall over both weeks. 

The overall response rate for the Redesigned Diary was 71.0 percent, higher than the 68.9 

percent response rate for the Former Diary. This increase was small but statistically significant. 

Attrition Rate

Attrition refers to respondents dropping out of the survey after the first week. Form 

designers hoped that the simplified layout of the Redesigned Diary would make it easier for 

respondents to record their expenditures, encouraging them to continue recording their 

expenditures for a second week. In this evaluation, we looked at the distribution of the pickup 

status of the Week 2 Diary, given that the household was in-scope in Week 1. If the Week 1 

Diary was a completed case but the Week 2 Diary was incomplete, then we have attrition.

Among the in-scope diary keepers, those who completed diaries for both weeks was 66.6 

percent with the Redesigned Diary, up from 64.5 percent with the Former Diary. However, there 

was no significant difference in the proportion of diary keepers who completed diaries in Week 1 

but not in Week 2 (2.6 percent for the Redesigned Diary vs. 2.5 percent for the Former Diary); or 

the proportion who did not participate in Week 1 but completed the diary in Week 2 (4.5 percent 

in the Redesigned Diary compared with 4.4 percent in the Former Diary). That is, the level of 

attrition was essentially unchanged.

Although the Diary form layout was simplified with the Redesigned Diary, keeping the 

diary is still a demanding and burdensome task, as the same amount of detailed information is 

still being asked. While we were hopeful that the response rate and attrition rate would improve, 

we did not expect a large improvement in these rates. We were pleased to see the small overall 

increase in response.

 

4

 If the respondent returns a blank diary at the end of a week, interviewers are trained to ask respondents to recall 

their expenditures for the week. BLS accepts the data if the respondent has receipts for a least half of the recalled 

items; otherwise the diary is considered a Type A nonresponse.
8

B. Data Quality Measures

As used here, data quality refers to the extent to which expenditures are reported with the 

required amount of detail and can be used as reported without any editing. Editing refers to the 

various processes of cleaning the originally recorded diary data because those data were 

misclassified, recorded with inadequate detail, or entered without the requested attributes.

Item misclassification rate

Item or entry misclassification occurs when the respondent writes an entry in the wrong 

major expenditure category of the diary. For example, the respondent wrote “disposable 

diapers” in the All Other Products, Services, and Expenses (OTH) section; the correct major 

expenditure category for the entry is Clothing, Shoes, Jewelry, and Accessories (CLO). The 

entry “disposable diapers” has been misclassified in OTH, and will be moved to CLO during 

data processing.

If the Diary layout has been effectively simplified and the recording instructions 

improved, we expect a lower item misclassification rate in the Redesigned Diary. In fact, the 

proportion of misclassified entries was 2.1 percentage points higher for the Redesigned Diary. 

By major expenditure category, the Redesigned Diary contained higher misclassification rates in 

CLO (8.0 percent, compared with 4.8 percent) and OTH (3.5 percent, compared with 1.5 

percent), but a lower misclassification rate in MLS (0.4 percent, compared with 0.7). The 

majority of misclassified items in CLO were originally entered in OTH. 

Allocation rate

When the description of a recorded expenditure is too general for use – for example, 

‘groceries’ – that expenditure is divided up and allocated among related items during data 

adjustment. The allocation rate for entries in a major expenditure category was computed as the 

ratio of the number of entries needing allocation to the total number of entries in the major 

expenditure category. 

In order to reduce respondent burden, the recording task in the Redesigned Diary was 

simplified by enabling respondents to have a more 'free-form' style of recording, and 

subcategories within a major expenditure category were removed (Appendix A). However, the 

form designers recognized that subcategories were effective in cueing for detail, and that the 

placement of cues at the top of the recording pages in the Redesigned Diary could be somewhat 
9

less effective reminders of the needed detail. Thus, we anticipated higher allocation rates in the 

Redesigned Diary, particularly in the FDB section. This is because we replaced the eliminated 

food item subcategories with examples appearing at the top of the recording page. In contrast, 

we replaced vendor type subcategories in the MLS section with checkboxes in the Redesigned 

Diary. 

Overall, allocation decreased for the Redesigned Diary. The proportion of allocated 

entries was 8.5 percent (vs 11.7 percent for the Former Diary), and the proportion of allocated 

expenditures was 12.1 percent (vs 14.6 percent for the Former Diary). However, the findings by 

major expenditure category are mixed. For FDB in the Redesigned Diary, the proportion of 

allocated entries was 2.1 percentage points higher and the proportion of allocated expenditures 

was 5.8 percentage points higher. This finding probably reflects the effect of removing 

subcategories from the FDB recording pages of the Redesigned Diary. On the other hand, MLS 

in the Redesigned Diary was significantly lower in proportion of allocated entries (5.9 percent)

and allocated expenditures(18.0 percent). (See Tables 2 and 3). The meal type and vendor type 

checkboxes appear to have been effective in eliciting more detail from respondents. The 

proportion of allocated items in CLO was higher in CLO, but the difference was not significant. 

Table 2. Allocation of Completed Diaries: Percent of Entries

Former Diary Redesigned Diary

Record Type Total Entries Percent 

Allocated

Standard 

Error

Total Entries Percent 

Allocated

Standard 

Error

Z score

ALL 500,672 11.7 0.09 498,458 8.5 0.08 -26.2

CLO 17,270 8.8 0.24 20,333 9.4 0.30 1.7

FDB 261,961 7.8 0.07 243,570 9.9 0.10 16.5

MLS 69,551 34.8 0.49 81,168 5.9 0.11 -57.7

OTH 151,890 8.3 0.11 153,387 7.7 0.09 -4.1

Z>1.96 or Z<-1.96 indicates difference is significant at the 5% level. 

Table 3. Allocation of Completed Diaries: Percent of Expenditures

Former Diary Redesigned Diary

Record Type Total 

Expenditures

Percent 

Allocated

Standard 

Error

Total 

Expenditures

Percent 

Allocated

Standard 

Error

Z score

ALL 10,761,144 14.6 0.71 11,275,829 12.1 0.38 -3.2

CLO 499,280 14.6 0.82 523,507 14.3 0.60 -0.3

FDB 1,086,510 19.8 0.33 1,058,280 25.6 0.57 8.8

MLS 678,934 49.6 0.96 687,491 18.0 0.42 -30.1

OTH 8,496,421 11.2 0.87 9,006,552 9.9 0.47 -1.3

Z>1.96 or Z<-1.96 indicates difference is significant at the 5% level. 
10

Item attribute imputation rate

Imputation in the CE Diary is performed only on item attributes, not on expenditures. 

Item attributes of interest collected by the Diary are: 

1. meal type purchased in MLS

2. vendor type (where a meal was purchased) in MLS

3. alcohol attribute in MLS (i.e., whether the meal included alcohol)

4. packaging type for items purchased in FDB

5. gender-age attribute for items in CLO 

6. whether an item was purchased as a gift in FDB, CLO, and OTH. 

1. Meal type attribute in MLS 

In the Former Diary, classification of the type of meal purchased (breakfast, lunch, 

dinner, or snack/other) was dependent on the respondent's recorded description. If the 

respondent failed to specify the meal type in the entry, that detail was lost. In the Redesigned 

Diary, checkboxes for each meal type were provided as a cue to the diary keeper that we wanted 

that detail. We expected the imputation rate for meal type to be lower in the Redesigned Diary.

The meal type checkboxes appear to be effective in the Redesigned Diary. The 

proportion of entries without a usable meal type specification was 4.7 percent in the Redesigned

Diary, down from 33.1 percent in the Former Diary. 

2. Vendor type attribute in MLS 

The Diary requests data on the type of place where a meal was purchased (vendor type). 

In the Former Diary, each of 6 vendor types of interest appeared as a separate subcategory on the 

MLS recording page. The 6 vendor categories were: fast food-take out- delivery-concessionbuffet- cafeteria; full service meals-snacks- drinks, vending machines or mobile vendors;

employer and school cafeterias; board or meal plan food; or catered affairs. To simplify the 

respondent's recording task in the Redesigned Diary, we eliminated the vendor type 

subcategories and replaced them with 4 vendor type checkboxes (fast food /take out /delivery/

concession, full service places, vending machines or mobile vendors, or employer/school 

cafeteria). See Appendix B. In the Former Diary, there was no imputation of the vendor 

attribute because the respondent could only record an entry on the MLS page by writing it in one 

of the vendor subcategories. In the Redesigned Diary, a respondent could make an MLS entry 
11

and omit marking off a vendor type checkbox. Thus, the imputation rates of the vendor attribute 

between the Former Diary and the Redesigned Diary are not comparable. In the Redesigned 

Diary, 4.9% of the MLS entries were missing the vendor attribute. 

3. Alcohol attribute in MLS

In the Former Diary, there was a Yes/No checkbox for respondents to indicate if alcohol 

was purchased with the meal; if the respondent marked the Yes checkbox, the entry was assigned 

an item code which indicates that the meal included an alcohol purchase. (See Appendix B.) In 

the Redesigned Diary, the Yes/No alcohol checkbox was eliminated; instead, we introduced 

checkboxes for diary keepers to indicate which of 3 types of alcohol was purchased (Wine, Beer, 

or Other). If any of these checkboxes was marked, the entry was coded as a report of 'alcohol 

purchased with a meal'. Thus, in the Former Diary, if the Yes/No alcohol purchased checkbox 

was not marked, whether alcohol was purchased with the meal would be imputed. In contrast, 

there is no imputation of whether alcohol was purchased with a meal in the Redesigned Diary,

because if no alcohol type checkbox is marked, the default is that no alcohol was purchased. So 

the imputation rates of reports of alcohol purchased with a meal are not comparable. 

We find about the same proportion in overall reporting of alcohol purchased with meals 

in both diaries: 4.2 percent in the Redesigned Diary compared with 4.3 percent in the Former 

Diary. In addition, we expected the Redesigned Diary to have higher reports of the "alcohol 

purchased with snack/other" meal type than the Former Diary; this was observed in the Field 

Test (Figueroa et al., 2003). Coders assign item codes in the MLS section based on marked 

checkboxes, not on entry descriptions. When only alcohol is purchased, respondents should 

leave the meal type checkbox blank, go on to mark the vendor type and alcohol type checkboxes, 

and record the amount spent on alcohol. However, we suspect respondents might be marking the 

"Snack/Other" meal type checkbox when only alcohol is purchased because of the way the meal 

type checkboxes are placed on the entry line of the recording page. Without explicit instructions, 

it is reasonable for respondents to infer that one of the meal type checkboxes should be marked 

when only alcohol is purchased, and the closest meal type option is 'other'. 

The data suggest that many respondents think the meal type checkbox needs to be 

marked, even if they only purchased alcohol. This problem could be alleviated by adding an 

"alcohol only" checkbox to the set of meal type checkbox options on the MLS recording page.
12

4. Package Type attribute in FDB. Checkboxes indicating an item's package type (fresh, frozen, 

bottled/canned, other) in the 'Food and Drinks for Home Consumption' section were not a new 

feature in the Redesigned Diary; identical package type checkboxes existed in the Former Diary. 

Thus, we did not expect much difference in the imputation rate of an item's package type in the 

FDB expenditure category. However, the imputation rate for package type of FDB items in good 

diaries was 6.6 percent for the Redesigned Diary, higher than 5.7 percent for the Former Diary.

(See Table 4). The difference was significant. Since the checkboxes in both Diaries were 

identical, it is not possible to attribute the increase in imputation rate to any changes in form 

design. One possible explanation may simply be that the data was collected in two different 

years.

Table 4. Percent of FDB Package Type Attribute Imputed

Former Diary Redesigned Diary

Record Type Total 

Entries

Percent 

Imputed

Standard 

Error

Total 

Entries

Percent 

Imputed

Standard 

Error

Z score

FDB 261,961 5.7 0.11 243,570 6.6 0.21 4.1

Z>1.96 indicates difference is significant at the 5% level. 

5. Gender-Age attribute in CLO. 

On the ' Clothing, Shoes, Jewelry, and Accessories ' recording page, we request the 

gender and the age group for which the purchase was intended. In the Former Diary, we asked 

respondents to assign a code to each purchase by selecting from a list of 5 gender-age 

combinations (Male 16 & over, Female 16 & over, Male 2-15, Female 2-15, Under 2). In the 

Redesigned Diary, the code became two sets of checkboxes, one checkbox to indicate gender, 

and another checkbox to indicate age (Under 2 ,2-15, 16 & over). See Appendix C. These age 

and gender attributes are then combined during data processing to form the 5 gender-age 

categories from the Former Diary. 

We expected the use of the checkboxes in the Redesigned Diary to improve the recording 

of gender-age attributes, as it simplifies the recording task. The imputation rate of the CLO 

entries' gender-age attribute was 16.1 percent for the Redesigned Diary, significantly lower than 

23.7 percent for the Former Diary. It appears separate gender and age checkboxes increased the 

reporting of these attributes. 
13

6. The gift attribute in FDB, CLO, and OTH

The gift attribute of an item indicates whether the item was purchased for someone 

outside the household. In the Former Diary, there were separate checkboxes to indicate 'Yes' or 

'No' for CLO and OTH; if neither checkbox was marked, the gift attribute would be flagged as 

missing. In the Redesigned Diary, there is only one check box the respondent marks off to 

indicate that the item was a gift; an unmarked checkbox defaults to a 'No' value, so there is no 

missing value for the gift attribute. Thus, the imputation rate for the gift attribute in the 

Redesigned Diary cannot be compared with its counterpart in the Former Diary. We suspect that 

this may give rise to underreporting of gifts because of the loss of a cue to the respondent. The 

form design assumes 'no' if the gift checkbox is left unmarked.

We see an increase in the proportion of items reported as gifts in FDB in the Redesigned 

Diary (1.2 percent of FDB entries) compared with the Former Diary (0.7 percent of FDB 

entries). However, in CLO and OTH the proportion of entries reported as gifts fell. As noted, in 

these sections, the gift checkbox changed from explicit "Yes/No checkboxes to a single "Yes" if 

the purchase was for someone outside the household. Unlike CLO and OTH, the gift indicator in 

FDB had been a separate subcategory "Part 3 - Food and Beverage Purchased as Gifts" in the 

Former Diary. This difference in representation of the GIFT indicator in FDB from CLO and 

OTH in the Former Diary may partly explain the differential impact of the change to the "Yes" 

only checkbox in the Redesigned Diary. 

In summary, where comparable with the Former Diary, lower imputation rates were 

associated with the introduction of checkboxes to capture the age-sex attribute in CLO, and the 

meal type attribute in MLS in the Redesigned Diary. However, although the package type 

checkboxes in the Redesigned Diary were identical to the package type checkboxes in the 

Former Diary, the package type imputation rate was higher in the Redesigned Diary. We also 

found that the replacement of Yes/No checkboxes with a single Yes checkbox (which if 

unmarked defaults to a ”No” response) in the Redesigned Diary was associated with lower 

reporting rates of the gift attribute in CLO and OTH.

C. Number of Entries

In each major expenditure category, we measure the number of entries as the count of the 

total number of (not unique) items reported in that category. 
14

If the Redesigned Diary was more effective in soliciting participation and recording, all 

other factors being equal, we should find more entries. However, since we are comparing data 

from 2 different time periods (2004 vs. 2005), we must be cautious in attributing differences in 

these measures solely to differences between the Redesigned Diary and the Former Diary. 

Changes in economic conditions could account for some of the differences these measures show. 

Overall, there were fewer entries in the Redesigned Diary, but the difference was not 

significant. (See Table 5). The number of entries reported in CLO, MLS, and OTH was higher 

and FDB lower in the Redesigned Diary; these differences were all statistically significant . The 

smaller number of entries in FDB in the Redesigned Diary may in part reflect the loss of more 

detailed cueing of subcategories in the Former Diary. 

Table 5. Number of Entries Reported by Section of good diaries

Former Diary Redesigned Diary

Record 

Type

Entries Standard 

Error

Entries Standard 

Error

Z score

ALL 500,672 1,523 498,458 1,202 -1.1

MLS 69,551 49 81,168 39 49.3

FDB 261,961 868 243,570 561 -17.8

CLO 17,270 179 20,333 220 40.9

OTH 151,890 455 153,387 397 2.5

Z>1.96 or Z<-1.96 indicates difference is significant at the 5%-level

IV. IMPLICATIONS

Implications from the comparison of the Redesigned Diary and the Former Diary are 

summarized as follows:

A. Survey participation. Survey participation was measured by the response rate and attrition 

rate. Both the response rate and the proportion of respondents who participated in both weeks of 

the survey was about 2 percentage points higher in the Redesigned Diary. The improved 

response rates observed for the Redesigned Diary should be qualified by two factors. First, a 

targeted effort to encourage field interviewers to obtain better response rates for the CE Diary in 

2005. Second, the underlying task of reporting all of a household’s expenses for two weeks 

remained unchanged. However, the increase in survey participation, although small, is 
15

promising and should continue to be watched as future data is collected using the Redesigned 

Diary. 

B. Data Quality. This class of measures is relatively time-insensitive, can be more directly 

attributable to changes in form design, and thus deserves more attention.

• Overall, the entry misclassification rate was higher for the Redesigned Diary. 5 By 

expenditure category, it was higher in CLO and OTH in the Redesigned Diary (by 3.2 

percentage points and 2.0 percentage points, respectively), but lower in MLS (by 0.3 

percentage points). 

• Overall, the allocation rates for entries were lower for the Redesigned Diary. The 

allocation rate in MLS was less than one-fifth the rate for the Redesigned Diary (5.9 

percent, versus 34.8), probably reflecting the introduction of the checkboxes. It was 

higher in FDB (9.9 percent, compared with 7.8 percent for the Former Diary), 

probably because of the removal of subcategories. 

• The imputation rate of the gender-age attribute in CLO was lower in the Redesigned 

Diary (16.1 percent, compared with 23.7 percent for the Former Diary), but it was 

higher for the package type attribute in FDB ( 6.6 percent, compared with 5.7 percent 

for the Former Diary). 

In MLS, the imputation of meal type due to unidentifiable entries dropped 27 percentage 

points in the Redesigned Diary. Only 5 percent of entries were missing vendor type. However, 

the number of reports of alcohol purchased with ‘Snack/Other’ surged 30 percentage points 

above that in the Redesigned Diary. Further analyses revealed that more than 63% of the 'Snack 

with Alcohol purchased' entries were corrected to "Alcohol purchase only" entries in subsequent 

processing in the edit system.6 

C. Number of Entries. This class of measures is relatively time-sensitive, and although reported, 

their interpretation should be made in the light of the two distinct years of data used in the 

evaluation. Overall, the total number of entries fell in the Redesigned Diary (difference not 

 

5

 Entry misclassification is corrected during a later phase of processing, and does not affect final data quality. 

6

 The edit of changing 'Snack with Alcohol purchased' item code to "Alcohol purchase only" item code requires BLS 

analysts to manually review and change the respondent’s actual entry. 
16

statistically significant), with FDB and MLS registering the largest decline in the number of 

entries. However, since the difference is not statistically significant, it is unclear whether the 

decrease can be attributed to the new design. Future data collected using the Redesigned Diary 

should be analyzed to see if there is in fact a trend. 

In summary, the Redesigned Diary seems to have been effective in most areas. We saw a 

small increase in response rates and evidence of some improvement in data quality. The largest 

positive impacts were seen in areas where subcategories were converted to checkboxes. The 

proportion of allocated dollars in the MLS expenditure category and the imputation of meal type 

were reduced by about 30 percentage points in both measures. In addition, the use of gender-age 

checkboxes in the CLO category also appear effective in eliciting the gender-age attribute as its 

imputation rate was lowered. Overall, the number of entries reported in the Redesigned Diary 

increased in all major expenditure categories except in FDB. Researchers thought that by 

eliminating the subcategories and making the recording task simpler, they could encourage 

increased reporting, but it appears that the elimination of the subcategories in FDB may have 

caused a decrease in expenditure reports due to the lack of detailed cueing. The replacement of 

the Yes/No checkboxes with a single Yes only checkbox for collecting gift purchases in CLO and 

OTH may also have a negative impact on reports of gift purchases and should be monitored as 

more data become available. 

V. AREAS FOR FUTURE RESEARCH

The data suggest many respondents may not understand how to record "alcohol only" 

purchases in MLS. We suggest studying the use of an alcohol only checkbox as another meal 

type checkbox option on the MLS recording page, or a separate instruction to handle this 

situation. While the present layout eventually provides the needed information, it has a high 

processing cost (along with the apparent potential for respondent confusion.)

The reporting of gifts in CLO and OTH, and the purchase of alcohol with meals, fell in 

the Redesigned Diary, and these types of reporting coincide with the replacement of the Yes/No

checkboxes with a Yes only checkbox in the Redesigned Diary. We may want to further 

investigate the effect of this aspect of form design. 

We may also want to look into expenditures that respondents regularly enter in the wrong 

section of the diary. For example, most CLO items were misclassified in OTH. We can identify 
17

commonly misclassified CLO items and highlight them in the examples to encourage 

respondents to record them correctly.

The Redesigned Diary represents the culmination of an extensive research program to 

make the Consumer Expenditures Diary easier to use. By comparing the data collected using the 

Former Diary and Redesigned Diary, researchers can analyze how the changes have impacted the 

data and continue to learn what changes work to improve CED data and what changes still 

require further research. 

VI. REFERENCES

Davis, J., Stinson, L., & To, N. (2002). What Does it Really Mean to Be User-Friendly When 

Designing an Expenditure Diary? Paper presented at the annual meeting of the American 

Association of Public Opinion Research.

Figueroa, E., Davis, J., Reyes-Morales, S., To, N., & Tan, L. (2003). Is a User-Friendly Diary 

More Effective? Findings from a Field Test. Paper presented at the ASA Joint Statistical 

Meetings.

Silberstein, Adriana R. (1989). “Recall Effects in the U.S. Consumer Expenditure Survey.” 

Journal of Official Statistics 5 (2), pp. 125-142.

To, N., Figueroa, E., & Tan, L. (2004). The Efficacy of Cues in an Expenditure Diary. Paper 

presented at the annual meeting of the American Association of Public Opinion Research. 
Appendix A: FDB layout

Former Diary (2004)

Redesigned Diary (2005) 
19

Appendix B: MLS layout

Former Diary (2004) 

Redesigned Diary (2005) - with meal type, vendor type, and alcohol type checkboxes
20

Appendix C: CLO layout

Former Diary (2004)

Redesigned Diary (2005)