An Incentives Experiment in the U.S. Consumer Expenditure Quarterly Survey December 2006

David E. McGrath

BAE Systems Information Technology / Bureau of Labor Statistics

Abstract1

Response rates to the Consumer Expenditures Quarterly

(CEQ) Survey fell from 86 percent in 1990 to 76 percent by 

2004. To reduce the current downward trend in CEQ response 

rates, the Bureau of Labor Statistics introduced an incentives 

experiment in November, 2005. The goal was to increase 

response rates by offering respondents a pre-paid monetary 

incentive that is unconditional on response to the survey. We 

mailed debit-card incentives along with the survey’s advance 

letter prior to contacting the potential survey respondent for 

the Wave 1 interview. The experimental design contrasts a 

control group receiving no incentive with groups that receive 

either $20 or $40 debit cards. In this paper, we show the 

design of the incentives experiment, provide preliminary 

results on the effects of the incentives on response rates, and 

discuss our assessment of the debit cards.

Keywords : Incentives, Response rates, Experiment, Debit 

Cards

1. Introduction

1.1 Overview

Response rates to the Consumer Expenditures Quarterly

(CEQ) Survey fell from 86 percent in 1990 to 76 percent

(AAPOR RR1) by 2004. To reduce the current downward 

trend in CEQ response rates, the Bureau of Labor Statistics

(BLS) introduced an incentives experiment beginning in

November, 2005. This paper shows the design of the

incentives experiment, provides preliminary results on the

effects of the incentives on response rates, and discusses our 

assessment of the debit cards. Section 1 introduces the

Consumer Expenditures Survey program and discusses the

general incentives literature. Section 2 describes the design of 

the CEQ incentives experiment, and the specific literature we 

used to guide the design decisions. In Section 3, we present 

preliminary results including the effects of the incentives on 

response rates, possible effects of the incentives on

nonresponse bias, and an assessment of the debit cards,

including respondents’ experiences cashing the cards and the 

rates at which respondents cash the cards. Section 4 shows 

some effects of the incentives on the interview process using 

contact history data. Section 5 provides a brief summary of 

findings to date, and Section 6 discusses future data analysis 

and areas for future research.

 

1

The author thanks Karen Goldenberg, David Swanson, and Moon Jung Cho

of the BLS for helpful comments. Any opinions expressed in this paper are 

those of the author and do not constitute policy of the Bureau of Labor 

Statistics or BAE Systems Information Technology.

1.2 The U.S. Consumer Expenditure Program

The U.S. Consumer Expenditure Survey program provides 

continuous information on the expenditures of American

consumers for use in economic research and in support of 

revisions to the Consumer Price Index. BLS sponsors the 

collection of expenditures in two separate surveys. First, the 

Consumer Expenditures Diary survey is designed to collect 

small, detailed expenditures that would be difficult for

respondents to recall during an interview. The diary is a selfadministered survey where respondents record all household 

expenditures for two one-week periods. Second, the

Consumer Expenditures Quarterly (CEQ) survey is a detailed 

series of five interviews designed to collect less frequent

purchases. Interviewers from the Census Bureau collect the 

data for both surveys. Results from the two surveys are

integrated to create published expenditures estimates. The

research described in this paper uses data from only the CEQ 

survey.

The Census Bureau conducts about 40,000 CEQ interviews 

across the nation each year. Each selected household 2 is 

interviewed five times over one year. The Wave 1 interview 

is primarily a bounding interview, designed to limit

telescoping errors in the Wave 2 interview. Data from the first 

interview do not contribute to expenditures estimates. Waves 

2-5 of the survey ask respondents about detailed expenditures 

for most non-food purchases, such as housing, furniture, 

automobile, and vacation expenses. The survey is

administered by computer assisted personal interview (CAPI), 

either in person or over the telephone (not centralized). An 

average CEQ interview takes approximately one hour to 

complete.

1.3 Incentives Literature

Providing potential survey respondents with incentives

generally increases response rates (Church, 1993; Singer, 

1999). Incentives can be cash or non-cash gifts and are

typically pre -paid to all potential respondents, or promised to 

potential respondents and paid upon compliance with the

survey. Most early research on incentives occurred in mail 

surveys (Church, 1993; Shettle, 1999). However, Singer’s

(2002) meta-analysis on the effects of incentives in

interviewer administered surveys (face-to-face and phone)

showed that the incentive effects illustrated in the mail survey 

literature also apply to interviewer-administered surveys. 

Three important effects are that cash incentives perform better 

than non-cash incentives (e.g., gifts), pre-paid incentives are 

 

2

The CEQ collects data from consumer units, which include people living in 

a household related by blood or marriage, or unrelated people who share 

household expenditures. Each household consists of one or more consumer 

units. For most housing units, the household and consumer unit are the same. 

For this paper, we use the terms interchangeably.
better than promised payment for survey cooperation, and 

response rates improve with increasing amounts of money.

Despite these known effects, government surveys have

traditionally not offered respondents incentives to participate 

in surveys. This has begun to change. Beginning in 1996, the 

Census Bureau conducted a series of incentive experiments on 

the Survey of Income and Program Participation (SIPP). This 

research supports current literature, showing that incentives

work similarly for large, interview-administered government 

surveys. In the 1996 SIPP panel, James (1997) found that a 

$20 pre -paid incentive significantly lowered SIPP 

nonresponse rates in Waves 1-3 compared with a $10 

incentive and a no-incentive control group.3

2. Design of the Incentives Experiment

Because the CEQ is an interviewer administered survey, most 

of the decisions regarding the CEQ incentives experiment are 

based on results from the Singer meta-analysis. For cost 

estimates and expectations related to response rates, we relied 

on results from the SIPP experiments because of the

similarities between the SIPP and CEQ (large, burdensome 

government surveys, face to face mo de, Census interviewers).

The following sections discuss key design decisions we made 

while planning the CEQ incentive experiment.

2.1 Distribute Incentive Only in Wave 1

We made this decision for budget reasons, but the SIPP 

research also provided evidence that distributing incentives 

only in Wave 1 of a panel survey may have lasting effects. 

During the initial SIPP incentives experiment, the Census

Bureau provided incentives only in Wave 1. Expanding on the 

James analysis of the 1996 SIPP experiment, Mack et al.

(1998) reported that the $20 Wave 1 SIPP incentive held 

nonresponse lower through Wave 6, two years out. Based on 

this experience, we decided to distribute the CEQ incentive 

only with the initial interview (Wave 1).

4

 However, if later 

analysis of Waves 4 and 5 CEQ response rates shows a

diminishing incentive effect, we will need to assess whether 

this design is adequate.

2.2 Use a Monetary Incentive (Debit Card)

Singer’s (2002) meta-analysis demonstrates that monetary 

incentives are the preferred incentive. However, the monetary 

incentives tested in these studies are usually cash, not a cash 

substitute such as debit cards. Cash may be a more powerful

incentive than any cash substitute, because the latter requires 

an effort by the respondent to convert into cash. Mitchell 

(2003) reports a study comparing various cash substitutes,

including point of sale (POS) cards, checks, and telephone 

cards, but none of these forms of monetary payment was 

 

3

 In 1996, SIPP issued paper check vouchers. The Census Bureau now has 

methods in place to use the debit cards discussed in this paper.

4

 For the CEQ, one household member responds for the entire household. 

Each household receives only one incentive.

compared with cash. The POS cards had the highest response 

rate, but only 36 percent of households used their cards. Also, 

18 percent of the POS card recipients called with questions or 

complaints related to either the PIN or activation process.5

If respondents don’t use the debit cards, the intended effect of 

the cash incentive may be reduced. A non-trivial percentage 

of card recipients assume the card is a scam, view cashing the 

card as an inconvenience, or have technical problems with the 

card. (See Section 3.5 for respondent’s experiences with the 

debit cards in our incentive experiment.) If respondents don’t 

perceive the debit card as a legitimate gift, they may not

reciprocate by participating in the survey.

Despite these concerns, we selected debit cards for the CEQ 

incentive experiment because the Census Bureau has methods 

in place to use debit cards and has successfully used them on 

other surveys.6 In addition, the cards are less susceptible to 

theft, and they cost less because a large percentage of

respondents never use them.

7

2.3 Use Pre-Paid and Unconditional Incentive

The incentives literature consistently demonstrates that prepaid incentives that are unconditional upon survey response 

most effectively increase response rates (Singer, 2002; Berk, 

1987). To prepay the incentives, we mailed the debit cards 

with the Wave 1 advance letter. We added the following 

paragraph to the current CEQ advance letter:

“Please accept the enclosed Debit card as a token of our 

appreciation for considering this important survey. See the 

enclosed envelope for instructions about using the card, which 

you may use immediately.”

To make the incentive as easy to use as possible, we printed 

the personal identification number (PIN) and the debit card 

amount directly on the debit card.

2.4 Amount of the Incentive

As with all incentive experiments, the goal is to find the least 

expensive incentive that achieves the desired effects, mainly 

increasing response rates. Although academic and commercial survey research centers often provide $1 and $5

incentives, we felt that we needed larger incentives for two

reasons. First, although small incentives generally work well, 

larger incentives work even better (Rodgers, 2002; James,

1997). Second, our decision to use debit cards constrained us 

to incentives in multiples of $20, because many ATM

 

5 Mitchell’s study oversampled households in poverty, which may affect the 

number of problems administering the incentives.

6

 Census has used debit card incentives for the Survey of Income and Program 

Participation (SIPP), National Epidemiologic Survey on Alcohol and Related 

Conditions (NESARC), and the Survey of Program Dynamics (SPD). 

Because current Census Bureau policy does not permit the handling of cash, 

the Census Bureau’s incentive programs have adopted debit cards as their 

standard without comparing their effects with true cash incentives. We are 

unaware of other empirical comparisons of the strengths and weaknesses of 

debit cards in relation to cash. Potential differences between debit cards and 

cash on survey response rates are unknown. 

7

Beckler and Ott (2006) reports cash rates of 39 percent in the Agriculture 

and Resource Management Survey (ARMS), while Marsh et al. (2002) reports 

85 percent cash rates in an ATM card pilot test.
machines only distribute $20 bills. We decided to use

incentive treatment groups of $20 and $40, along with the 

control group. Amounts in excess of $40 were ruled out for 

budget reasons.

We attempted to limit the number of discarded debit cards by 

delivering the incentives using priority mail. Because

enhanced mail procedures are known to improve response

rates (Piskurich, 2001), we split the $0 control group to

contrast the delivery of the survey’s advance letter by priority 

mail versus first class mail (see Table 1). This allows us to 

look at the effect of priority mail on CEQ response rates.

2.5 Sample Size and Design

Incentives primarily affect response rates and only modestly affect 

data quality (Singer, 2002), so we embedded the experiment

within the production survey. We calculated that assigning half of 

the Wave 1 sample to the incentive condition and running the 

incentive experiment for one year would allow response rate

differences of two or more percentage points across groups to be 

statistically sig nificant. The experiment length was later reduced 

from one year to nine months for budget reasons.

We randomly assigned each Wave 1 sample unit to one of the 

four treatment groups shown in Table 1 (highlighted cells). 

Table 1 shows that the experimental design is not fully nested, 

because we decided not to send the debit cards by first class 

mail, hence the two cells with zero households.

Each treatment group has, in expectation, the same proportion 

of households by demographic and geographic variables. The 

experiment ran in all 12 Census regional offices (RO), and 

approximately half of each interviewer’s Wave 1 workload 

consisted of incentive cases.

Table 1. CEQ Incentives -- Experimental Design 

(Actual Household Sample Sizes for Good Addresses)

Incentive Amount Mail Option

$ 0 $ 20 $40

Total

First Class Mail 1,922 

(Control)

0 0 1,922

Priority Mail 1,759 1,838 1,805 5,402

Total 3,681 1,838 1,805 7,324

Note: Table 1 contains Wave 1 housing unit sample sizes; we 

removed vacant and demolished units (Type B and C in 

Census terms) for the data analysis.

2.6 Interviewer Training

To introduce the incentives study and to explain changes in 

field procedures, the Census Bureau developed computer 

based training for the approximately 440 Census interviewers 

that work on the CEQ. In the fall of 2005, the interviewers 

completed the training on their laptop computers from their 

homes. The training took about 30 minutes to complete. 

Although interviewers did not attend special classroom

training prior to the experiment, discussions regarding the

incentives were conducted during annual refresher training in 

the spring of 2006.

2.7 Debit Card Assessment Questions

We wanted to learn about the experiences respondents had 

when receiving and trying to use the debit cards. Respondents

received the debit cards by mail prior to the attempted Wave 1 

interview, and we indicated in the advance letter that the cards 

were for their immediate use. Therefore, we knew some 

percentage of respondents would attempt to use the debit cards 

prior to the interviewer’s first visit. We added the following 

five questions to the incentive version of the Wave 1 CEQ 

interview to learn from these respondents’ experiences.

1) The letter you received should have included a debit card, 

which was a gift from the Census Bureau. Did anyone 

living at this address receive the debit card?

2) The debit card was a gift for you to use. Has anyone tried 

to use the debit card?

3) Is there any reason you haven't tried to use the debit card?

4) Were there any problems using the debit card?

5) What problems did you have using the debit card? 

3. Preliminary Results

3.1 Data

The CEQ incentives experiment includes housing units with 

scheduled Wave 1 interviews between November, 2005 and 

July, 2006. The sample size for the Wave 1 analyses shown in 

this report is the 7,324 households shown in Table 1. Except 

for Section 3.4, all data shown in this report are unweighted.

3.2 Field Issues

Several problems occurred early in the implementation of the 

incentives experiment. In November, one regional office 8

(RO) mailed advance letters with insufficient Priority Mail 

postage to Wave 1 incentive households. The post office left a 

notice with these households saying that they would have to 

come to the post office to pay the extra postage to collect this 

mail. Respondents who paid the additional postage were 

reimbursed by the Census interviewers.

In December, a different RO mistakenly mailed all Wave 1 

households an advance letter indicating an incentive was

included with their letter, but no debit card was included for 

the no-incentive designated part of the sample. To resolve this 

situation, interviewers promised the delivery of a debit card by 

 

8

The Census Bureau’s interviewing staff is managed by 12 regional offices 

(RO), so each error at the RO level affected approximately 1/12 of the 

incentive sample for that month (about 30 cases).
Federal Express at the first household visit. Because these 

errors had a negligible effect on the overall results, these cases 

remain in the analysis datasets used for this paper.

3.3 Effect of the Incentives on Response Rates

As expected, incentives positively impacted CEQ response 

rates. Table 2 shows that response rates increase across the 

four treatment groups in the expected direction, although only 

comparisons with the $40 incentive reach statistical

significance. The $40 incentive produced significantly better 

Wave 1 response rates than the control group (p<.01) and the 

$20 incentive (p< .1), which supports the literature that larger 

incentives work better than smaller incentives (Rodgers, 2002; 

James, 1997). Unexpectedly, the $40 incentive primarily 

affected the noncontact rate as opposed to the refusal rate. 

The noncontact rate for the control group is about 50 percent 

greater than the $40 group (p<.01), compared with about a 10 

percent difference in refusal rates (not significant). It is 

possible that the pre-paid incentive affected non-contact rates 

by influencing the respondents’ decision whether or not to 

answer the door when our interviewer made the initial contact 

attempt.

Table 2. Effect of the Incentive on Wave 1 Response Rates

Rate 

Treatment 

Group n Response 

Rate

Refusal 

Rate

Noncontact 

Rate

Control 

Group

 

1,922

 

77.3

 

13.9

 

8.7

Priority 

Mail

 

1,759

 

78.2

 

13.7

 

8.0

$20 Debit 

Card

 

1,838

 

79.1

 

12.9

 

7.8

$40 Debit 

Card

 

1,805

 

81.3a

 

12.9

 

5.8b

a

 Significantly different from control (p<.01), priority group (p<.05), and $20

incentive (p<.1)9

b Significantly different from control (p<.01), priority group (p<.05), and $20

incentive (p<.05)

Note: Wave 1 cases are from November, 2005 – July, 2006.

Another unexpected finding is that the difference between the 

$20 and $40 incentive (2.2 percentage points) is larger than 

the difference between the control group and the $20 incentive 

(1.8 points). The literature generally shows that providing a 

small incentive creates most of the incentive effect (Trussell 

and Lavrakas, 2004; Brick et al, 2005), and while larger 

incentives are better, the rate of improvement diminishes as 

the incentive increases. A possible reason for our result may 

be that small debit card incentives behave differently than 

small cash incentives. If a person could receive a $1 debit 

card, reading the instructions, remembering the PIN, and 

making a trip to an ATM machine might not be worth the 

value of the gift. Although $20 is not considered a small 

 

9

 At this time, we have not adjusted the statistical tests shown in Tables 2-4

to account for the complex sample design nor the multiple comparisons.

incentive, some households could consider this gift to be more 

trouble than it’s worth.

While the effect of the incentives on Wave 1 response is 

positive, Wave 1 of the CEQ is used primarily for recall 

bounding. Only data from Waves 2-5 contribute to published 

CEQ estimates. Therefore, the success of the incentive is 

contingent on improving response rates in Waves 2-5. Table 3

shows that although we provided the incentives only in Wave 

1, the $40 incentive continued to improve Wave 2 and Wave 3

response rates.

Table 3. Effect of the Incentive on Wave 2 and Wave 3

Response Rates

Rate

Treatment Wave 2 Wave 3

Group

(n)

Response 

Rate (n)

Response 

Rate

Control 

Group

 

1,229

 

75.7

 

591

 

73.8

Priority 

Mail

 

1,155

 

75.7

 

553

 

73.2

$20 Debit 

Card

 

1,196

 

75.6

 

562

 

75.4

$40 Debit 

Card

 

1,160

 

81.5a

 

574

 

78.9b

a

 Significantly different from all other groups (p<.01)

b

Significantly different from control and priority mail groups (p<.05)

Note: Wave 2 cases from February, 2006 – July, 2006 and 

Wave 3 cases from May, 2006 – July, 2006 were available for 

this analysis.

3.4 Nonresponse Bias

Although the incentive increased the CEQ response rate, the 

effect of the incentives on nonresponse bias is less clear. 

Nonrespose bias is a function of both the response rate and the 

differences on survey variables between respondents and

nonrespondents (Groves, 1989). Mean spending and response

rates vary by demographic subgroup in the CEQ (McGrath, 

2005). To reduce nonresponse bias, the incentives need to 

increase response rates by inducing response from the types of 

households often missed by the CEQ. To investigate this 

question, Table 4 compares some demographic distributions of 

CEQ households for the control group, incentive group, and 

population estimates from the Census Bureau.10

 

10 Although Census estimates have known coverage issues, they are 

considered to have better coverage than other sample surveys. For Table 4, 

we consider estimates of geographic and demographic distributions by the 

Census Bureau’s Population Division to be a ‘gold standard’.
Table 4. Effect of the Incentive on Wave 1 Respondent

 Sample Composition

Geographic / 

Demographic 

Groups

No 

Incentive11

(percent)

Incentive12

(percent)

T Test13

(p)

Census 

Estimates14

(percent)

Gender (male) 48.0 47.9 p=.94 49.3

Race15

 White alone 79.2 77.1 p<.05 80.2

 Black 11.1 13.1 p<.05 13.4

 Asian 4.3 3.9 p=.45 4.9

 Other races 5.4 5.8 p=.51 1.5

Hispanic 14.6 14.9 p=.75 14.4

Age

 0-19 29.5 29.3 p=.88 27.6

 20 – 34 18.6 19.2 p=.56 20.6

 35 – 49 22.4 21.0 p=.20 22.4

 50 - 64 17.7 18.2 p=.72 17.0

 65 Plus 11.8 12.3 p=.56 12.4

Household 

Size 16

 1 24.9 26.7 p=.13 26.8

 2 33.1 33.9 p=.54 32.7

 3 17.3 15.2 p<.05 15.9

 4 15.0 13.6 p=.14 14.5

 5 + 9.8 10.7 p<.28 10.1

Notes: Counts of people are weighted by baseweights. 

Percentages shown in this table may not add to 100 percent 

due to rounding.

As Table 4 shows, the incentives only slightly change the

demographic makeup of the CEQ respondents. The incentives 

appear to bring more blacks into the respondent sample

(p<.05). Table 4 shows that because blacks are

underrepresented in the CEQ, the incentive is ma king the 

respondent sample look more like the population that the 

original sample represents for this characteristic, which will 

tend to decrease nonresponse bias. Examining the

comparisons in Table 4, the sample composition of the

incentive group more closely matches the target population 

(census estimates) in two of the three comparisons that reach 

statistical significance (p<.05). Table 4 shows mixed results 

for the effects of the incentives on nonresponse bias, although 

the increase in black and single person households is 

 

11 The no incentive group includes the control groups where advance letters 

were delivered by 1

st class or priority mail. 

12 The incentive group includes the $20 and $40 incentive groups.

13 These T Tests compare the no incentive group (collapsing the control and 

priority mail $0 groups) with the collapsed incentive groups.

14 The July, 2005 Census estimates of Population and Housing come from the 

Census Bureau web page (www.census.gov). Source is Population Division, 

Release Date: May, 2006. (Tables NC-EST2005-0*)

15 For this table, respondents are classified as ‘Black’ if they checked ‘Black 

or African American’ alone or in combination with other races. (We treated 

Asian the same way.) This is consistent with the Census classification. 

‘Other races’ includes people that select ‘Other race’ (72 percent of these 

people), ‘Native American’, ‘Pacific Islander’, or ‘Don’t Know’ when 

answering the CEQ race question. 

16 Census Estimates come from Table 2-9 (pp 68). Household Composition –

Occupied Units. American Housing Survey for the United States: 2005.

considered important because household surveys frequently 

under-represent urban households containing these groups.

3.5 Debit Cards

3.5.1 Debit Card Assessment Questions

1) Did anyone living at this address receive the debit card?

Because we thought priority mail would increase the

percentage of respondents that read the CEQ advance letter, 

we were surprised by responses to this question. Of the 2,875 

respondents that answered, 36 percent indicated that they 

never received a debit card (answered ‘No’). Reasons for this 

could be that we mailed the card to an incorrect address, a 

household member mistakenly (or intentionally) threw the

debit card away (e.g., thought it was junk mail), or another 

household member received and kept the card and the 

respondent was unaware of the card’s existence. We can 

determine from the debit card tracking system that 21 percent 

of households claiming to have never received the debit card 

cashed the original debit card that we sent, but we do not have 

information regarding who cashed the card.

It is impossible to determine from the assessment questions 

why such a high proportion of households reported not

receiving their debit cards. As we discussed earlier, pre-paid 

incentives are the literature-recommended method for

distributing cash incentives, but does this apply to debit cards? 

Prior versions of the CEQ asked respondents whether or not 

they received our advance letter, and about 86 percent of 

Wave 1 respondents answered ‘Yes’. We would have

expected a larger number of respondents to recall receiving a 

debit card than recall a traditional advance letter. Did the 

advance letter sent by priority mail fare worse than our usual 

advance letter? Respondents may casually say ‘yes’ they 

recall seeing an advance letter, even when maybe they don’t 

recall, but when they realize they may have missed a gift they 

might make sure to answer that they haven’t received a debit 

card. Either way, our incentive cannot induce the desired 

effect if the respondent does not receive the gift.

When households said they didn’t receive the debit card, the 

interviewer read the following,

“I'm sorry that you did not receive the debit card. My 

regional office will send you another card as soon as possible. 

If you also receive the original card we sent, please destroy 

it.”

The RO then mailed an additional card by Federal Express to 

households indicating that they did not receive the debit card. 

However, this protocol changes an important design element 

of the study, because the incentive effectively becomes a 

promis ed incentive. It is impossible to compare response rates 

between households that receive pre-paid versus promised 

incentives because no experimental treatment was used in our 

data. We did learn that 98 percent of the households that were 

offered the promised incentive as a result of their response to 

question 1 completed the CEQ interview, but these households

are not a random sample of households. By answering the 

incentive assessment questions, these households have

effectively begun the interview, and are thus cooperative 
households. The uncooperative households, such as

households that prove impossible to contact or immediately 

refuse, never receive the assessment questions, and also never 

received a debit card by the promised distribution method. An 

interesting, but unanswered, question is whether response rates 

would be still higher if we could get the pre-paid incentives to 

a higher proportion of households.

2) The debit card was a gift for you to use. Has anyone tried

 to use the debit card?

About 30 percent of debit card recipients reported attempting 

to cash the debit card prior to the arrival of the Census

interviewer. Data from these households provided early 

feedback into whether respondents had problems with the 

debit cards, and the results vary by population subgroup. 

Households where the reference person was aged 35 or 

younger were twice as likely to report trying to use the card 

prior to the Wave 1 interview as households with reference 

persons 56 or older. Also, Black and Hispanic households 

were 50 percent more likely than White households to use the 

debit card prior to the interview. 

3) Is there any reason you haven't tried to use the debit card?

To speed the interviewing process, we provided interviewers

with 9 closed-end answer choices 17 to this question (plus 

‘Don’t Know’ or ‘Other, specify’).

Table 5 shows that almost half of the households (49 percent) 

had not had a chance to use the debit cards for time-related 

reasons. After time -related reasons, the next most commonly 

selected reason (11 percent) was ‘Other, specify’. We coded 

these write-in reasons, most of which could have been placed 

into the closed ended response alternatives (e.g., time related 

reasons were also the most common write-in, and should have 

fallen into row 1 of Table 5). Of the approximately 150 writein answers, we coded 38 as related to the respondent thinking

the debit cards were illegitimate and another 10 related to the 

respondent throwing the card away as junk mail. Another 

common reason respondents didn’t cash the debit cards is that 

they were waiting to talk with the interviewer (9 percent), 

perhaps to gain additional information to determine the

legitimacy of the gift, or to determine whether the gift 

obligates the respondent to complete the survey.

Some of these same issues could have arisen with true cash 

incentives, while others likely would not. For instance,

recipients of a true cash incentive may still be concerned about 

spending the cash prior to the interview, and wait until they 

can discuss the gift with the interviewer to make sure ‘no 

strings are attached’. However, it seems likely that no 

respondents would wonder how to use the cash and almost no 

 

17 Closed-end response options included 1) Haven’t had time, 2) Not clear 

how to use the debit card, 3) Not clear where to use the debit card 4) Thought 

using the debit card was a commitment to doing the survey, 5) Debit card is a 

waste of Government money, 6. Didn't know card could already be used, 7) 

Another person has the card, 8) Wanted to talk to interviewer about the survey 

first, 9) Lost, misplaced, or threw away debit card, 10) Other – specify, and 

11) Don't Know. We removed response choices that were rarely selected 

from Table 5.

households would lose the cash. Most of these skeptical debit 

card recipients ended up responding to Wave 1 of the CEQ, so 

it remains unclear whether the debit card incentives impact 

CEQ response differently than true cash incentives would 

have.

Table 5. Reasons why Respondents Didn’t Use the Debit

Card Prior to Interview (n = 1,262)

Reason Debit Card not Used Percent

Haven’t had time 49

Not clear how to use the debit card 10

Commitment to doing the survey 5

Didn’t know card could already be used 5

Wanted to talk with interviewer about survey 9

Lost or threw away debit card 5

Other, specify 11

4) Were there any problems using the debit card?

Only 32 respondents (< 1 percent) reported having trouble 

cashing the debit cards.

5) What problems did you have using the debit card? 

Most of these 32 respondents reported problems with the

ATM machines or the debit card PIN.

3.5.2 Debit Card Cash Rates

As discussed in Section 2.2, we were concerned that cash is a 

more effective incentive than any cash substitute, such as a 

debit card. The Census Bureau tracks whether the CEQ debit 

cards were cashed, allowing comparison with other surveys. 

Beckler and Ott (2006) reported that 39 percent of farmers 

cashed their $20 debit cards, and Mitchell (2003) reported a 36

percent debit card cash rate. Table 6 shows the cash rates for 

the CEQ incentives by incentive amount, by whether the 

household responded to the survey, and by selected

demographic characteristics for respondent households.

Table 6. Percentage of CEQ Households that Cash the Debit

Cards (by selected characteristics)

All Households Respondents Only

Characteristic

Debit Card 

Cash Rate 

(percent)

Characteristic

Debit Card 

Cash Rate 

(percent)

OVERALL 63 Black 79

Respondent 75 White 75

Nonrespondent 16 Asian 66

$20 59 Hispanic 66

$40 68 Not Hispanic 76

Aged 66 or Older 69

All Other Ages 76
As Table 6 shows, about 63 percent of eligible households18

cashed the debit cards. These cash rates are higher than many 

surveys using POS or debit cards, most likely because the 

CEQ incentive program reminds the respondent about the 

debit card and provides a replacement card when necessary. 

This protocol increases costs through higher cash rates but 

should maximize the effect of the incentive on response. 

Table 6 also shows that very few nonrespondent households 

cash the cards. Although this saves the survey organization 

money (funds are recoverable), it is unclear whether this is a 

positive outcome. One scenario is that these households 

weren’t going to respond to the survey anyway, they don’t 

cash the cards because it doesn’t seem right, and therefore 

BLS saves money from their low cash rate without adversely

affecting response rates (a win/win scenario). A competing 

scenario is that a subset of these households could have been 

survey respondents if the incentives (debit cards) had the

intended effect. Perhaps some of these households questioned 

the cards legitimacy, or they didn’t understand how to use the 

cards. In this scenario, the debit cards were not cashed and

they did not have the intended effect on response. It is 

impossible to say from our experiment which of these

scenarios more closely represents the truth.

Recipients of $40 debit cards cashed their incentives at much 

higher rates than the $20 group (p < .01). This may support 

the argument that the debit card is a cash substitute that 

requires effort by the respondent to convert to cash. As the 

dollar value increases, it becomes more worthwhile for

respondents to ma ke this effort. This finding may indicate that 

$20 and $40 debit card incentives may not work similarly to 

their cash equivalents.

The ‘Respondents Only’ column of Table 6 shows that 79 

percent of Black households cashed the debit card, compared 

with 75 percent for White households and 66 percent for 

Asian households. Hispanic households cashed the card at a 

66 percent rate, lower than non-Hispanic households at 76

percent. Recall from Section 3.5.1 that more Hispanics than 

Non-Hispanics had attempted to use the debit card prior to the 

Wave 1 interview. Combining these two measures, it appears 

that Hispanic households either cash the debit cards quickly or 

do not cash them at all. One possibility that may explain this 

is a language problem. Although interviewers distribute

Spanish advance letters when encountering language

problems, the original (mailed) advance letter and debit card 

instructions are currently only printed in English, which may 

have reduced the utility of the debit card for Hispanic

households that only speak Spanish.

Prior to the experiment, we also hypothesized that older CEQ 

respondents could have trouble with a relatively new

technology like debit cards. Table 6 shows that households 

where the reference person was aged 66 or older cash the debit 

card at lower rates (69 percent) than households in other age 

groups (76 percent). Also, these older households reported 

 

18 The data in Table 6 exclude vacant and demolished housing units (Type B 

and C in Census Bureau terms). If we include these types of households, the 

debit card cash rate is slightly over 50 percent. Also, because respondents 

have six months to cash the debit cards, the debit card cash rates shown in 

Table 6 are not final, but will be very similar to final rates.

more than double the rates of “Not clear how to use the debit 

card” and triple the rates of “lost card’ when responding to 

debit card assessment question 3.

4. Contact History Data

Prior to the experiment, we hypothesized that the incentives 

might influence respondents to cooperate with the survey

more quickly, thus reducing the number of contacts required 

to complete a case. If true, this would lead to a reduction in 

field costs that could partially offset the cost of the incentives. 

Table 7 shows the average number of contacts per case

recorded via the Census Bureau’s Contact History Instrument 

(CHI). Although the $40 group is statistically different from 

the other groups, we didn’t expect cases assigned the $20 

incentive to require more contacts than the control group. 

Given this, the effect of the incentives on CEQ field costs 

seems inconclusive.

Table 7. Average Numb er of Personal Visit Contacts per CEQ

Case (Wave 1)

Control Priority 

Mail

$20 $40

Number of Contacts 3.17 3.33 3.29 3.06

CEQ interviewers’ record detailed data about each attempted 

contact with CEQ Households. Interviewers report whether 

the respondent showed any concerns related to the survey 

(e.g., Too Busy, Privacy Concerns, etc.). We hypothesized 

that due to the goodwill of the incentive, interviewers would 

receive fewer reports of concerns from incentive households. 

However, the data do not support our expectation. Although 

the incentive had a positive impact on response rates, it does 

not appear that the incentive lessened respondent concerns 

regarding the survey (data not shown).

5. Discussion

Incentives significantly improved Wave 1 CEQ response rates. 

Average Wave 1 response rates were 81.3 percent for the $40 

incentive, compared with 79.1 percent for the $20 incentive,

78.2 percent for $0 by priority mail, and 77.3 percent for the 

control group. Results for all incentive treatment groups were 

in the expected directions. For the $40 incentive, preliminary 

results look encouraging that the response rate gains will 

continue for several waves of the survey.

Few respondents had difficulty using the debit cards; however, 

about 1/3 of all debit card recipients reported not receiving the 

card. It is unclear why this occurred, but some data captured 

during the debit card assessment questions lends support to the 

theory that many respondents did not perceive the debit card 

as a legitimate, unconditional gift.
6. Future Research

The incentives data shown in this paper are preliminary. 

Although we don’t expect the substantive findings relating to 

response rates to change, only complete Wave 1 data were 

available for the paper. We will continue to track the effects 

of the incentives on CEQ response rates through Wave 5. We 

will also analyze the effect of the incentives on expenditures 

data, imputation rates, and the quality of income data as edited 

and weighted files become available.

In February, 2006, we implemented a similar incentives

experiment in the Consumer Expenditures Diary (CED)

survey. We will assess whether the incentive impacts

response rates and data quality to the self-administered diary 

survey similarly to the CEQ.

7. References

American Association for Public Opinion Research (AAPOR). 

2006. Standard Definitions: Final Disposition of Case Codes 

and Outcome Rates for Surveys. Lenexa, KS: AAPOR.

Beckler, Dan and Kathleen Ott. (2006). “Indirect Monetary 

Incentives with a Complex Agricultural Establishment

Survey”. 2006 Proceedings of the American Statistical

Association, Survey Research Methods Section, Seattle, WA.

Berk, Marc, N. Mathiowetz, E. Ward, and A. White (1987). 

“The Effect of Prepaid and Promised Incentives: Results of a 

Controlled Experiment”. Journal of Official Statistics, Vol 3, 

1987, pp. 449-457.

Brick, J. Michael, Montaquila, Jill, Collins Hagedorn, Mary, 

Brock Roth, Shelley, & Chapman, Christopher (2005). 

“Implications for RDD Design from an Incentives

Experiment”. Journal of Official Statistics, Vol 21, 2005, pp. 

571-589.

 

Church, Allan H. 1993. “Estimating the Effect of Incentives 

on Mail Survey Response Rates: A Meta-Analysis.” Public 

Opinion Quarterly 57:62-79.

Groves, Robert M. 1989. Survey Errors and Survey Costs: 

New York: Wiley.

James, Tracy. 1997. “Results of the Wave 1 Incentive

Experiment in the 1996 Survey of Income and Program

Participation.” Proceedings of the Survey Research Section of 

the American Statistical Association.

Mack, Stephen, Vicki Huggins, Donald Keathley, and Mahdi 

Sundukchi (1998). “Do Monetary Incentives Improve

Response Rates in the Survey of Income and Program

Participation?” Proceedings of the Section on Survey

Methodology, American Statistical Association, pp.529-34.

Marsh, Shawn E. R. Stapulonis, and M. Mishkind. “Lessons 

Learned from an ATM Respondent Payment Program.” 

Poster presented at the 2002 American Association of Public 

Opinion Research Conference.

McGrath, David E. (2005). “Comparison of Data Obtained by 

Telephone versus Personal Visit Response in the U.S.

Consumer Expenditures Survey.” 2005 Proceedings of the

Section on Survey Methodology, American Statistical 

Association.

Mitchell, Susan, C. Lamothe-Galtte, and F. Potter. (2003) 

“Survey Response Incentives for a Low-Income Population”. 

Paper presented at the 2003 American Association of Public 

Opinion Research Conference.

Piskurich, Karen, Dawn Nelson and Diane Herz. 

“Maximizing Respondent Contact in the American Time Use 

Survey”. Paper presented at the 2001 American Association of 

Public Opinion Research (AAPOR) Conference. Montreal, 

Canada.

Rodgers, Willard. (2002). “Size of Incentive Effects in a 

Longitudinal Study” 2002 Proceedings of the American 

Statistical Association.

Shettle, Carolyn and Geraldine Mooney. 1999. “Monetary 

Incentives in Government Surveys.” Journal of Official 

Statistics 15:231-50.

Singer, Eleanor, Nancy Gebler, Trivellore Raghunathan, John 

Van Hoewyk, and Katherine McGonagle. (1999). “The Effect 

of Incentives in Interviewer-Mediated Surveys.” Journal of 

Official Statistics Vol. 15 :217-30.

Singer, Eleanor (2002). “The Use of Incentives to Reduce 

Nonresponse in Household Surveys”. Chapter 11 of Survey 

Nonresponse (Groves, Dillman, Eltinge, and Little) New

York: John Wiley & Sons, Inc.

Trussell, N. and Lavrakas, P.J. (2004). “The influence of 

Incremental Increases in Token Cash Incentives on Mail

Survey Response: Is There an Optimal Amount?” Public 

Opinion Quarterly, 64, pp 171-188.