Consumer Expenditure Survey Anthology, 2008 23

Nathan McDermott

Lucilla Tan

The Effect of Refusal 

Conversion on Data 

Quality in the Consumer 

Expenditure Interview 

Survey

Nathan McDermott is an economist formerly 

working in the Branch of Production and Control, Division of Consumer Expenditure Surveys, U.S. Bureau of Labor Statistics.

Lucilla Tan is an economist working in the 

Branch of Research and Program Development, 

Division of Consumer Expenditure Surveys, U.S. 

Bureau of Labor Statistics.

Achieving and maintaining high 

response rates are important 

goals of the Consumer Expenditure (CE) Survey program. However, 

as with other Federal Government surveys, response rates for the Interview 

Survey component of the CE have been 

declining in recent years. Between 1998 

and 2002, the response rate for the Interview Survey was about 79 percent, 

and it fell to about 76 percent between 

2003 and 2006.1 In an attempt to stem 

the decline in response rates and minimize nonresponse bias, guidelines for 

refusal conversion were established. 

Refusal conversion is a process by 

which the interviewer makes additional 

efforts to persuade an initial refuser to 

become a survey participant. In order 

to offset the decline in response rates, 

interviewers have increased their efforts to encourage survey participation. 

This is evidenced by the proportion of 

completed interviews accounted for by 

converted refusals, which rose from 9 

percent in 2003 to 12 percent in 2006. 

The level of effort required to convert an initial refusal raises issues of 

higher-than-anticipated field costs, as 

well as concerns over the quality of 

data provided by such respondents, 

compared with respondents who were 

cooperative throughout the interview 

experience. Converted refusers may 

exert less cognitive effort to respond, 

or interviewers may be more willing 

to accept ‘satisficing’ responses from 

reluctant respondents to obtain a completed interview (Triplett et al., 1996). 

Satisficing occurs when respondents 

exert minimal effort when answering survey questions in order to hurry 

through the interview. 

Findings about the effect of converted refusers on data quality have been 

inconclusive. Burton et al. (2006) cited 

various studies that compared survey 

estimates with and without converted 

refusers. They reported that significant 

differences between these two groups 

were found in fewer than half of the 

survey measures, and some of these 

differences disappeared after controlling for demographics. While some of 

these studies found consistent differences in demographics between converted refusers and respondents, others 

did not.

In this study, we compare measures 

of data quality for the survey estimates 

between converted refusers and other 

respondents in the Interview Survey, 

describe their demographic characteristics, and summarize the nature of 

interviewer contact attempts. By treating the group of converted refusers as 

proxy nonrespondents, we estimate the 

nonresponse bias in major expenditure 

categories. 

1 Response rate calculation is based on The 

American Association of Public Opinion Research 

(AAPOR) Response Rate 1 definition (2006). 
24 Consumer Expenditure Survey Anthology, 2008

Methods

We began with all respondents to the 

survey and divided them into two 

groups, converted refusers, and other 

respondents (henceforth, referred to as 

respondents). We differentiated converted refusers from respondents on 

the basis of the interviewer’s response 

to the question “Was this a converted 

refusal?” at the end of the interview. If 

the interviewer responded, “Yes,” we 

classified the case as a converted refuser; if “No,” the case was classified as a 

respondent. The identification of converted refusers in this analysis is not 

rigorous: For completed interviews, we 

rely only on the interviewer’s response 

to the converted refusal question; for 

noninterviews, there is no question 

asking if the interviewer attempted to 

convert the refusal. Thus, the universe 

of cases subject to refusal conversion 

is unknown. Although there are prescribed criteria for identifying initial 

refusals for possible refusal conversion, there is anecdotal evidence that 

these criteria may not be uniformly applied in the field. 

Since April 2005, interviewers have 

been able to maintain detailed information about contact attempts for each assigned case using the Contact History 

Instrument (CHI). The information 

includes day and time of the contact 

attempt, outcome of the attempt, strategies used to attempt contact, and perceived concerns of respondents.

During the interview, expenditure 

information is collected by describing 

a category of goods or services (for example, home furnishings, healthcare) 

and asking if anyone in the consumer 

unit (CU) either had expenditures in 

the general category or had expenditures for specific items that fall into that 

category. If they did, then the respondents are probed for details about their 

spending, such as a description of the 

product or service, cost, quantity, and 

month purchased. Most information 

about goods and services is collected 

using detailed lists of goods and services, rather than general categories. 

Better data quality is expected from 

respondents who are more willing to 

complete the interview and are thorough in their reporting. Poorer quality data are expected from reluctant 

respondents, as they are more likely 

to give “satisficing” responses. For the 

Interview Survey, the following measures taken together suggest poorer 

data quality:

lower levels of reported expenditures

fewer responses to expenditure 

questions

more “don’t know/refused” responses to the expenditure questions 

more expenditure and income 

reports requiring editing due to 

incomplete reporting 

In addition, interview characteristics, 

such as the following, support the hypothesis of poorer data quality: 

the respondent took less time to 

answer the expenditure questions

there was less use made of the 

information booklet and records 

of purchases to answer questions 

during the interview. 

The information booklet is a document that lists examples of different 

categories of products and services. 

Interviewers are supposed to show it to 

respondents to help respondents understand and recall purchases in different 

expenditure categories. Records refer 

to receipts, bank statements, and other 

documentation of purchases. These 

aids to better reporting may not be used 

or may be used less in interviews where 

the interviewer perceives that the respondent is anxious to hurry through 

the interview.

The category of total expenditures is 

regarded as the most direct measure of 

data quality for the Interview Survey. 

For the data quality measures, we used 

1-sided t-tests to test the alternative hy-

•

•

•

•

•

•

pothesis that converted refusers provide 

poorer quality data than respondents to 

the null hypothesis of no difference between converted refusers and respondents. The alternative hypothesis reflects the belief that converted refusers 

are more likely to exhibit satisficing behavior and provide poorer quality data. 

For comparisons on demographic characteristics, and characteristics of contact attempts and interviews, we tested 

the alternative hypothesis that there is 

a difference between converted refusers and respondents. We used 2-sided 

t-tests on the means of characteristics 

that are continuous variables and the 

Chi-square test of association between 

type of respondent and characteristics 

that are categorical variables. Following these bivariate comparisons, we fitted multiple regressions with controls 

for demographic characteristics and 

interview characteristics to assess the 

effect of converted refusers on reported 

total expenditures. All analyses were 

performed on unweighted data.

We used completed interviews 

with known converted refusal status 

in waves two through five from April 

2005 through September 2006 in our 

analysis.2

 Since the Interview Survey 

is a panel survey, a household can be 

represented multiple times in the analysis data. However, we treated each 

completed interview as an independent 

case. There were 43,395 completed interviews in the sample for this study, 

of which 11.8 percent were converted 

refusers. 

Findings 

Data quality measures

Every measure supported the alternative hypothesis that converted refusers provide poorer quality data than 

respondents. (See table 1.) Converted 

refusers reported lower total expenditures ($10,499 v. $11,302); they answered fewer expenditure questions 

(37.3 v. 46.1 questions); they had a 

2

 The Contact History Instrument was 

introduced for the Interview Survey in April 

2005. The latest data available at the time of 

analysis was September 2006. 
Consumer Expenditure Survey Anthology, 2008 25

Quarterly total expenditures after editing..................... $10,449 $11,302 -854 (-1,104,-604) -5.62 <0.001

Number of expenditure questions answered............... 37.3 46.1 -8.8 (-9.3, -8.4) -30.4 <0.001

Percent of expenditure questions answered 

 “don’t know/refused”2................................................. 6.0 2.7 3.3 (3.0, 3.6) 20.92 <0.001

Percent of complete expenditure reports2 ................... 80.6 85.8 -5.1 (-5.6, -4.7) -23.2 <0.001

Percent of income reports not requiring imputation..... 39.7 55.2 -15.5 (-16.9, -14.1) -21.2 <0.001

higher percentage of “don’t know/refused” responses (6.0 v. 2.7 percent); and 

they were less likely to provide complete 

reports for expenditures (80.6 v. 85.8 percent) and income (39.7 v. 55.2 percent). 

These bivariate comparisons together reflect satisficing by converted 

refusers and indicate poorer quality 

responses from them. However, there 

may be additional characteristics of 

converted refusers, when compared to 

respondents, which could account for 

these differences in the measures of 

data quality. 

Interview characteristics of converted 

refusers and respondents

Contact attempts and interview 

CHI data showed that more effort was 

Table 1. Mean estimates of data quality measures for converted refusers and respondents, Consumer Expenditure Interview 

Survey, April 2005–September 2006

Data quality measure

Converted

refusers

(N = 5,314)

Respondents

(N =

38,081)

Difference

95 percent

confidence

interval

t-stat 1-sided

p-value1

1

 The null hypothesis of no difference between converted refusers 

and respondents was tested against the alternative 1-sided hypothesis 

of poorer quality measures from converted refusers. 

2

 Complete reports are reports that do not require editing. Standard 

t-test for ratio of variable means was used to test this difference. 3

 Standard t-test of proportions was used to test this difference.

required to reach and complete an interview with converted refusers. It was 

more difficult to contact converted refusers than respondents: 39.3 percent 

of contact attempts resulted in noncontact for converted refusers compared 

with 33.3 percent for respondents. (See 

table 2.) A smaller percent of interviews was completed by personal visit, 

the preferred method, for converted 

refusers, compared with respondents 

(about half versus two-thirds). Converted refusal cases are active in the 

field for a longer time period (10.7 v. 

8.1 days for respondents). By day 15 of 

the month, 63.9 percent of respondents 

have completed their interviews, compared with 43.2 percent of converted 

refusers. (See chart 1.) Interviews with 

converted refusers involve less use of 

records (20.1 v. 37.3 percent), less use 

of the information booklet (37.7 v. 51.0 

percent), and less time on the expenditure questions (34.4 v. 42.2 minutes at 

the median of the distributions).

Respondent concerns

In the CHI, interviewers can record 

their perceptions of respondent concerns for each contact made. For each 

case, we summed up the number of 

times a specific concern was cited, 

then computed the average number of 

times each concern was reported for 

respondents, converted refusers, and 

refusers (those eligible respondents 

who refused to participate in the survey). We compared the differences in 

Table 2. Characteristics of Contact Attempts and Interviews, Consumer Expenditure Interview Interview Survey, 

April 2005–September 2006 

Characteristic

Converted 

refusers

(N = 5,314)

Respondents

(N = 

38,081)

Difference

95 percent

confidence 

interval

t-stat 2-sided 

p-value1

Number of contact attempts ........................................... 4.2 3.4 0.88 (0.79, 0.97) 15.9 <0.001

 Percent of noncontacts................................................ 39.3 33.3 6.1 (4.8, 7.3) 9.3 <0.001

Mode of data collection: Percent by personal visit2 ...... 52.1 66.5 -14.3 (-15.7, -12.9) -20.5 <0.001

Number of days between first and last attempt .............. 10.7 8.1 2.57 (2.3, 2.8) 17.9 <0.001

Percent use of records3, 4 ............................................... 20.1 37.3 -17.2 (-18.5, -15.8) -24.6 <0.001

Percent use of information booklet4 .............................. 37.5 51.0 -13.5 (-14.9, -12.0) -18.4 <0.001

Time to complete interview (minutes)............................. 57.1 64.0 -6.9 (-7.7, -6.1) -14.7 <0.001

Time spent on expenditure questions (minutes) 

 25th percentile ............................................................ 23.1 28.8 -5.8

 50th percentile ............................................................ 34.4 42.2 -7.9

 75th percentile ............................................................ 49.0 60.4 -11.3

1

 The null hypothesis of no difference between converted refusers 

and respondents was tested against the alternative 2-sided hypothesis 

of a difference between the two groups. 2

 This was based on the interviewer’s report of the mode, either inperson or by telephone, in which most of the data was collected.

3

 Use of records is originally a 6-level categorical variable; for this 

study, we indicated “record used” when the response was “mostly” 

through “always.” 4

 Standard t-test of proportions was used to test this difference.
26 Consumer Expenditure Survey Anthology, 2008

the reporting rates between converted 

refusers and respondents, and between 

converted refusers and refusers. (See 

charts 2a and 2b.) 

Interviewers reported higher incidences of time-related concerns (too 

busy, interview too time-consuming, 

scheduling difficulty) for converted refusers, compared with respondents and 

refusers. In contrast, higher incidences 

of hostile behavior, lack of interest, 

and anti-Government concerns were 

reported for final refusers compared 

with converted refusers. Differences 

suggest that initial refusals are more 

likely to be converted if the perceived 

respondent concerns are of a temporary, 

circumstantial nature (such as time-related concerns), than if the concerns 

were attitudinal in nature. This finding 

is consistent with Burton et al.’s (2006, 

page 467) analysis of reasons for refusal and interview outcome in subsequent 

waves; they found that “...refusals are 

indeed more likely to be temporary if 

the reason for refusal is situational or 

due to a short-term circumstance.” 

Demographics 

We examined associations between 

categorical characteristics and the type 

of respondent (converted refuser v. respondent) using a Chi-squared test of 

independence. Income was divided 

into five categories: Negative income 

(less than zero) and the four quartiles 

for income greater than or equal to 

zero. Converted refusers differed from 

respondents in household composition, 

in addition to respondent characteristics of age, race, educational attainment, and income before tax. (See table 

3.) Differences were significant statistically, but not substantively. For continuous characteristics, we used the ttest to compare the mean characteristic 

for converted refusers to respondents. 

We found that converted refusers were 

more likely to live in Metropolitan Statistical Areas, and were less likely to be 

homeowners. (See table 4.) Converted 

refusers were not significantly different 

from respondents in income, average 

number of members in the consumer 

unit (CU), average number of members under age 18 and over age 64, and 

number of earners in the CU. 

Regression analysis 

Since the bivariate comparisons showed 

that converted refusers differed from 

respondents in some demographic and 

interview characteristics, we used linear models to examine the effect of 

converted refusers on reported total expenditures, controlling for these characteristics. The first regression model 

includes only demographic characteristics for the respondent and household. 

The second regression model includes 

covariates for interview attributes in 

addition to the demographic covariates. 

Converted refusers spend less time 

answering expenditure questions, are 

less likely to use records, and are more 

likely to have telephone interviews. 

Using the distribution of time spent on 

answering expenditure questions, we 

created time categories of “shorter than 

average” (less than 25th percentile, 

28 minutes), “average” (25th to 75th 

percentile), and “longer than average” 

(greater than 75th percentile, 59.1 minutes).

From the first model with only demographic covariates, the coefficient 

for converted refusals is negative as 

expected, and significant (-606.56, p 

= <.001). (See table 5.) While the coefficients for the positive income percentiles exhibited the expected “staircase” pattern (reported expenditures 

increased with higher income percentiles), the large positive coefficient 

for negative income was unexpected 

(6,135.78, p <0.001). This could be 

due to under-reporting of income, the 

reporting of large short-term losses, 

or reporting by those with low income 

but large savings (such as retirees). Besides the coefficients for income, most 

other demographic coefficients had 

the expected directions and most were 

found to be significant (p <0.05). This 

suggests there are characteristics about 

converted refusers—other than these 

demographic characteristics—that are 

associated with their reporting lower 

expenditures. 

Chart 1. Rate of cases reaching final disposition
Consumer Expenditure Survey Anthology, 2008 27

Chart 2a. Perceived respondent concerns among converted refusers and respondents 

Chart 2b. Perceived respondent concerns among converted refusers and final refusers 

No concerns

Intends to quit survey

Survey is voluntary

Too many interviews

Scheduling difficulty

Privacy concerns

Not interested

Interview too time consuming

Too busy

Not interested

Survey is voluntary

Hangs up/slams door

Hostile behavior

Anti-government

Too many interviews

Interview too time consuming

Scheduling difficulty

No concerns

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

Mean report

Converted refuser Respondent

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

Converted refuser Respondent

Mean report
28 Consumer Expenditure Survey Anthology, 2008

( )

jT

jR

j

Z

Bias Z

ˆ

ˆ ˆ = = 

jT

jTjR

Z

ZZ

,

,,

ˆ

ˆˆ −

respondents to report higher expenditures and income (McGrath 2005). 

The regression results suggest that 

differences in interview characteristics—not differences in demographics—explain converted refusers’ lower 

reporting of expenditures compared to 

respondents. 

Effect of converted refusers on 

expenditure estimates 

The quarterly mean of overall expenditures is lower for converted refusers 

than it is for respondents. One way to 

estimate the effect that data collected 

from converted refusers have on the 

mean expenditure estimates is to treat 

converted refusers as proxy nonrespondents, and compute the bias of 

nonresponse in the respondent mean as 

shown in Groves (2006): 

Here

= mean expenditure estimate for expenditure category j 

from respondent R, 

= mean expenditure estimate for expenditure category j 

from converted refusers, who are 

proxy nonrespondents,

nN = number of converted 

refusers, who are proxy nonresponders, and

nT = total sample (converted 

refusers plus respondents).

The relative nonresponse bias estimate 

for expenditure category j is computed 

as

where

= mean expenditure estimate for expenditure category j 

from the total sample.

•

•

•

•

•

( ) ( jN)

T

N

jR Z

n

n Bias Z

ˆ ˆ Z jR − = ˆ

The second regression model includes interview characteristics as additional covariates. Coefficients of the 

demographic covariates remain significant and with expected signs, as in the 

first regression model. The coefficients 

for the interview characteristics are significant, but the coefficient for converted refusers is no longer significant. On 

the length of time spent answering expenditure questions, the coefficient for 

“greater than 59 minutes spent answering expenditure questions” was significantly large and positive (3,124.71, p = 

<.001), while the coefficient for “shorter than 28 minutes” was significantly 

negative (-1,634.61, p = <.001). This is 

consistent with expectations that more 

expenditures are reported when more 

time is spent on answering expenditure 

questions. The coefficient for “no records used” is negative and significant 

(-444.80, p = <.001). The coefficient 

for “interview by telephone” is positive 

(273.06, p = .002); this is consistent 

with other research which found phone 

Table 3. Frequency distribution of demographic characteristics for converted 

refusers and respondents, Consumer Expenditure Interview Survey, 

April 2005–September 2006 

Characteristic

Converted 

refusers

(N = 5,314)

Respondent

N=

38,081)

Chi-square P-value

CU size.............................................. 7.59 .0553

 1....................................................... 27.1 28.1

 2....................................................... 31.0 31.4

 3-4 ................................................... 31.5 29.7

 5+ .................................................... 10.4 10.9

CU composition ................................. 21.45 .0003

 Husband-wife only......................... 20.8 21.7

 Husband-wife families ................... 30.3 30.5

 Single parent ................................. 7.4 5.9

 Singles........................................... 27.1 28.1

 Other.............................................. 14.4 14.0

CU tenure .......................................... 18.31 .0001

 Owner............................................ 66.1 67.9

 Renter............................................ 33.3 31.0

 Other.............................................. 0.6 1.1

CU area type ..................................... 89.9 <.001

 In MSA........................................... 91.7 87.1

 Outside MSA.................................. 8.3 12.9

Respondent gender........................... .27 .6040

 Male ............................................... 39.5 39.8

 Female............................................ 60.5 60.2

Respondent age ................................ 23.62 <.001

 <21 ................................................. 1.8 2.3

 21-34 .............................................. 22.1 20.7

 35-49 .............................................. 33.4 31.5

 50-64 .............................................. 23.8 25.8

 65+ ................................................. 18.9 19.7

Respondent race ............................... 31.84 <.001

 White .............................................. 79.3 82.3

 Black............................................... 13.6 11.5

 Native American ............................. .6 .5

 Asian............................................... 4.8 4.0

 Pacific Islander............................... 0.3 0.4

 Multi-race........................................ 1.3 1.4

Respondent’s education level ........... 32.71 <.001

 Less than high school..................... 16.6 14.9

 High school graduate...................... 27.6 25.6

 Some college.................................. 29.6 31.3

 College graduate ........................... 26.2 28.2

Income before tax distribution 

 after imputation................................ 15.16 .0044

 Negative income............................ 23 .14

 Less than 25th percentile .............. 24.9 25.0

 25th-50th percentile ...................... 26.2 24.8

 50th-75th percentile....................... 25.5 24.9

 Greater than 75th percentile ......... 23.1 25.2

Z jN

ˆ

Z jR

ˆ

Z jT

ˆ

,

.
Consumer Expenditure Survey Anthology, 2008 29

The variance for relative nonresponse bias does not have a closedform solution. An estimate of the variance for the relative nonresponse bias 

for each expenditure category was calculated using random groups (Wolter 

1985). With the number of random 

groups equal to 10, the variance formula is as follows: 3

where

 is the respondent sample 

mean on expenditure category j

for random group k,

 is the total sample mean 

on expenditure category j for 

random group k

 is the 

relative bias on expenditure category j for random group k, and

 is the average 

of the relative bias on expenditure category j over all 10 random groups.

We used the variance formula provided above to calculate 95 percent 

confidence intervals for the relative 

nonresponse bias for each expenditure 

•

•

•

•

10

( ) ( )

( )11010

~ ˆ

~ var 1

2

,

−

−

=

∑

k=

jjk

jrg

Table 4. Means of demographic characteristics for converted refusers and respondents, Consumer Expenditure Interview Survey, 

April 2005–September 2006 

Number of members.................................................... 2.59 2.55 0.03 (0.00, 0.07) 1.51 .1321

Number of earners....................................................... 1.37 1.35 .02 ( .00, .04) 1.39 .1632

Number of members under age 18.............................. .70 .67 .03 (.00, .05) 1.57 .1164

Number of members over age 64................................ .31 .32 -.01 (-.02, .01) -1.07 .2824

Converted 

refusers

(N = 5,314)

Respondent

(N = 38,081)

95 percent

confidence 

interval

Difference t-stat 2-tailed 

p-value Characteristic of the CU

3

This random group’s variance for the relative 

bias was first implemented in a nonresponse bias 

study for the Interview Survey, CE Nonresponse 

Bias Team (2007a). Nonresponse Bias: Using 

Harder-to-Contact Respondents as Proxies for 

Nonrespondents. 

Z ,, jkR

ˆ

Z ,, jkT

ˆ

jkT

jkTjkR

jk

Z

ZZ

,,

,,,,

, ˆ

ˆˆ ˆ − =

∑=

=

10

1

, ˆ

10

~ 1

k

j jk

category. The general formula for the 

95 percent confidence interval is

 ,

where

is the full 

sample estimate of the relative 

nonresponse bias on expenditure 

category j.

The sign of the relative bias measure 

indicates if converted refusers lowered (positive relative bias) or raised 

(negative relative bias) the expenditure estimate for a category, while the 

magnitude of the relative bias indicates 

the extent to which the expenditure 

estimate is changed by including the 

expenditures of converted refusers in 

computing the mean expenditure estimate. Where the 95 percent confidence 

interval of the relative bias includes 

zero, it indicates that the relative bias 

reflects sampling variability and that 

the expenditure estimate is not subject 

to nonresponse bias. The underlying 

assumption behind these computations 

is that nonresponse is the only source 

of bias.

The estimated relative bias for average quarterly total expenditures is 

0.86 (.64, 1.08) percent. (See table 6.) 

The survey estimate of expenditures on 

reading materials has the largest relative bias (3.6 percent), followed by personal insurance and pensions (3.5 percent), tobacco (2.7 percent), alcoholic 

beverages (2.5 percent), cash contributions (2.0 percent), entertainment (1.9 

percent), apparel (1.3 percent), health 

•

) ~(var ˆ

j 975.0,9 jrg t ∗±

∑=

=

10

1

, ˆ

10

~ 1

k

j jk

care (1.2 percent), and personal care 

(1.2 percent). The 95 percent confidence interval for the estimated relative 

bias for quarterly expenditure estimates 

of transportation, housing, education, 

and food include zero, indicating that 

nonresponse bias is not affecting these 

expenditure estimates. 

Other recent studies of potential nonresponse bias in the Interview Survey 

used other definitions of proxy nonrespondents, and their estimates of relative bias of quarterly total expenditures 

ranged from -0.14 (-1.4, 1.1) percent to 

5.8 (2.0, 9.6) percent (CE Nonresponse 

Team, 2007a, 2007b, Reyes-Morales 

2007). These estimates of potential nonresponse bias are limited by how closely these various proxy nonrespondents 

represent nonrespondents. 

A pattern emerges when the expenditure categories are listed in decreasing magnitude of the relative bias. 

Categories with large relative bias are 

reading materials, personal insurance 

and pensions, tobacco, alcoholic beverages, cash contributions, entertainment, 

apparel, health care, and personal care. 

Expenditure categories with small nonresponse bias (and not statistically different from 0 at the 5 percent level) are 

transportation, educational expenses, 

and housing. The categories with large 

relative bias represent more discretionary types of expenditures, while the 

categories with smaller relative bias 

are more likely to be “necessities.” 

This difference in relative bias between 

expenditure categories may partly be 

explained by recognizing that “necessity” goods and services are less dependent on personal characteristics and 

preferences, and these regular expenditures require less respondent effort to 

,
30 Consumer Expenditure Survey Anthology, 2008

Table 5. Multiple regressions with quarterly total expenditures as the dependent variable, Consumer Expenditure Interview 

Survey, April 2005–September 2006 

Explanatory variable1

Model with only demographic 

characteristics

Model with demographic and interview 

characteristics

Parameter 

estimate

Standard 

error (SE) P-value Parameter 

estimate

Standard 

error (SE) P-value

Intercept............................................................................. 8,500.68 268.50 <.001 8,278.29 273.87 <.001

Converted refuser2

............................................................. -606.56 122.72 <.001 -85.62 121.90 .4825

Respondent characteristic

Male2 ................................................................................. 94.68 84.24 .261 328.21 83.10 <.001

Age

 Less than 21 years old .................................................. -731.06 287.19 .0109 -489.09 281.97 .0828

 21-34 years old.............................................................. -712.18 117.72 <.001 -593.11 115.63 <.001

 50-64 years old.............................................................. 3.80 117.23 .9741 -225.42 115.18 .0503

 Greater than 65 years old.............................................. -34.10 233.21 .8838 -371.45 229.08 .1049

Race ..................................................................................

 Black.............................................................................. -1,275.38 129.94 <.001 -1,032.57 127.77 <.001

 Asian.............................................................................. -1,032.14 206.93 <.001 -604.39 203.35 .003

 Other race...................................................................... -137.19 273.80 .6163 -338.24 268.68 .2081

Education level ..................................................................

 College graduate ........................................................... 1,840.05 109.20 <.001 1,648.74 107.35 <.001

 High school graduate..................................................... -876.37 108.25 <.001 -618.15 106.39 <.001

 Less than high school.................................................... -1,773.17 132.75 <.001 -1,257.12 131.00 <.001

Household characteristic

Age composition--number of members

 Under 18 years.............................................................. 587.80 52.35 <.001 511.54 51.42 <.001

 18-64 years.................................................................... 165.83 89.61 .0642 209.24 87.96 .0174

 Over 64 years................................................................ -383.78 148.01 .0095 -365.10 145.20 .0119

Family composition ...........................................................

 Husband and wife only .................................................. -428.66 155.68 .0059 -422.84 152.81 .0057

 Single parent ................................................................. -1,465.51 213.50 <.001 -1,276.94 209.55 <.001

 Single............................................................................. -2,443.97 195.04 <.001 -2,123.56 191.53 <.001

 Other family type ........................................................... -1,522.71 146.45 <.001 -1,278.00 143.87 <.001

Income before tax distribution3 .........................................

 Negative income............................................................ 6,135.78 1,052.52 <.001 5,729.98 1,032.66 <.001

 25th-50th percentile ...................................................... 1,755.38 123.78 <.001 1,429.96 121.98 <.001

 50th-75th percentile....................................................... 4,184.24 142.15 <.001 3,642.01 140.52 <.001

 Greater than 75th percentile ......................................... 11,619.00 165.14 <.001 10,740.00 163.82 <.001

Number of earners............................................................. -192.75 71.24 .0068 -206.51 69.91 .0031

Renter................................................................................ -1,012.39 102.76 <.001 -584.86 101.71 <.001

Outside of MSA2 ................................................................ -1,475.64 124.54 <.001 -1,431.44 122.45 <.001

Interview characteristic

Interviewed by telephone2 ................................................. 273.06 85.81 .0015

No use of records2 ............................................................. -444.80 88.27 <.001

Time spent on answering expenditure questions 

 Less than 28 minutes ................................................... -1,634.61 100.65 <.001

 Greater than 59 minutes................................................ 3,124.71 101.50 <.001

R-square............................................................................ .3453 .3700 

1

 The reference group is female, 35–49 years old, White, with 

some college education, family composition of husband and wife with 

children, income in the positive first quartile, and an interview time 

between 28 and 59 minutes. 2

 Following are indicator (0, 1) variables: Converted refuser, male, 

renter, outside of MSA, interviewed by telephone, and no use of 

records. 3 Income distribution was broken out into five categories: Negative 

income and positive (greater then or equal to zero) income divided 

into quartiles.

recall and report. Thus, the differences 

between respondents and converted refusers, who are proxies for nonrespondents, are less likely to affect spending 

on necessities.

Conclusion

In summary, we found that converted 

refusers answered fewer expenditure 

questions with valid values, more frequently responded with “don’t know” 

or “refused” answers, reported lower 

overall expenditures, provided less 

complete reporting of expenditures and 

income, and spent less time answering 

expenditure questions than respondents. 

These behaviors are consistent with 

satisficing, and suggest that, when converted refusers agree to do the survey, 

they are more likely to rush through it 

and, thus, to provide poorer quality responses. There was mixed evidence for 

differences in demographics between 

converted refusers and respondents. 
Consumer Expenditure Survey Anthology, 2008 31

Table 6. Effect of converted refusers on expenditure estimates measured by relative bias, Consumer Expenditure Interview 

Survey, April 2005–September 2006

Expenditure category

Base-weighted mean expenditures ($) Relative bias

Converted

refusers

(N = 5,314)

Respondents

(N = 38,081)

All

respondents Percent

95 percent 

confidence 

interval

Quarterly total expenditures ........................................... 10,336.64 11,128.31 11,033.11 0.86 (0.64, 1.08)

Reading materials........................................................... 22.48 31.70 30.59 3.62 (3.03, 4.27)

Personal insurance and pensions .................................. 842.97 1,169.02 1,129.82 3.47 (3.27, 3.67)

Tobacco .......................................................................... 64.78 82.68 80.53 2.67 (1.98, 3.42)

Alcoholic beverages ....................................................... 73.05 91.30 89.10 2.46 (1.96, 2.97)

Cash contributions.......................................................... 369.16 439.80 431.30 1.97 (0.47, 3.64)

Entertainment ................................................................. 481.62 571.86 561.01 1.93 (0.97, 2.79)

Apparel ........................................................................... 282.41 315.45 311.48 1.28 (0.54, 2.03)

Health care ..................................................................... 601.68 668.91 660.83 1.22 (0.67, 1.75)

Personal care ................................................................. 63.85 70.49 69.69 1.15 (0.60, 1.69)

Transportation ................................................................ 2,014.91 2,141.52 2,126.30 0.72 (-0.06, 1.48)

Housing .......................................................................... 3,633.79 3,670.39 3,665.99 0.12 (-0.27, 0.52)

Education........................................................................ 207.31 203.63 204.07 -0.22 (-2.10, 1.82)

Food ............................................................................... 1,520.59 1,472.14 1,477.97 -0.39 (-0.59, -0.21)

Interviewers also perceived converted 

refusers as having more time-related 

concerns than respondents and as being 

less likely to express hostile behavior 

than final refusers. Regression analysis 

provided some evidence that differences in interview characteristics—not demographics—account for the disparity 

in the quality of reporting. This finding 

suggests that interview characteristics 

should be considered when evaluating whether the interview is providing 

good data.

When treating converted refusers 

as proxy nonrespondents, we found a 

small positive nonresponse bias for 

quarterly total expenditures. That is, 

including the expenditure reports of 

converted refusers lowers overall expenditure estimates. Our estimate is 

within the range of estimates of relative 

bias from recent studies which used 

other definitions of proxy nonrespondents. This result raises the broader 

question of whether the cost and effort 

of converting refusal cases is justified.

One possible limitation of this research is the use of the converted refusal 

question in the questionnaire to identify 

converted refusal cases because of possible inconsistencies in how interviewers respond to the question. Another 

possible limitation derives from the 

measures used to evaluate data quality. Since there is no single measure of 

quality, we considered a variety of indicators. We were not able to quantify 

the trade-off between these measures to 

come up with a single “weighted” measure of overall quality.

Further research is needed to examine the impact of converted refusers 

on published expenditure estimates by 

treating them as nonrespondents and 

reweighting the data. Another relevant 

research project would be using the 

panel feature of the Interview Survey 

to study the behavior of converted refusers in later waves; the study could 

measure whether they are more likely 

to become dropouts or intermittent 

respondents, or to exhibit reluctance 

compared to fully cooperative respondents.
32 Consumer Expenditure Survey Anthology, 2008

The American Association for Public 

Opinion Research. Standard Definitions: Final Dispositions of Case 

Codes and Outcome Rates for Surveys. 4th edition. Lenexa,

Kansas: AAPOR, 2006.

Burton, Jonathan, Heather Laurie, and 

Peter Lynn. “The Long-term Effectiveness of Refusal Conversion Procedures on Longitudinal Surveys.” 

Journal of the Royal Statistical Society, Series A, Vol. 169, Issue 3, 

2006: 459–478.

CE Nonresponse Bias Team. Nonresponse Bias: Using Harder-to-Contact Respondents as Proxies for 

Nonrespondents. Internal BLS Report. 2007a.

CE Nonresponse Bias Team. Pattern of 

participation while in sample: interAcknowledgements

References

Steven Bass, Division of Consumer 

Expenditure Surveys, assisted with 

coding to create several key data quality measures; Jeffrey Gonzalez, Office 

of Survey Methods Research, assisted 

with the relative bias and variance computations; and Geoffrey Paulin, Division of Consumer Expenditure Surveys, assisted with the regression models. 

mittent respondents and attritors as 

proxy nonrespondents in the Interview Survey. Internal BLS Report. 

2007b.

Groves, Robert M. (2006). “Nonresponse Rates and Nonresponse 

Bias in Household Surveys.” Public 

Opinion Quarterly Vol. 70, No. 5, 

2006: 646–75.

McGrath, David E. “Comparison of 

Data Obtained by Telephone versus Personal Visit Response in the 

U.S. Consumer Expenditures Survey.” Proc. of the Section on Survey 

Research Methods Section. August 

7-11, 2005. American Statistical Association, 2005.

Reyes-Morales, Sally E. Characteristics of Complete and Intermittent 

Responders in the Consumer Expenditure Quarterly Interview Survey: 2001–2006. Internal BLS Report. 2007.

Triplett, Timothy, Johnny Blair, Teresa 

Hamilton, and Yun Chiao Kang. 

“Initial Cooperators vs. Converted 

Refusers: Are There Response Behaviour Differences?” Proc. of the 

Survey Research Methods Section. 

August 4-8, 1996. American Statistical Association, 1996. 

U.S. Bureau of Labor Statistics. “Chapter 16. Consumer Expenditures and 

Income.” BLS Handbook of Methods. June 2007. Online: Internet. 1 

June 2007. Available: http://www.

bls.gov/opub/hom/pdf/homch16.pdf. 

Wolter, Kirk M. Introduction to Variance Estimation. NY: Springer-Verlag, 1985.