Exploring a Balance Edit Approach in the 

Consumer Expenditure Quarterly 

Interview Survey 

 

Scott Fricker and Brandon Kopp 

Office of Survey Methods Research 

Nhien To 

Branch of Research and Program Development 

October 14, 2011
2

EXECUTIVE SUMMARY

Overview 

The purpose of this study was to examine the application of a balance edit procedure to a 

modified Consumer Expenditure Quarterly Interview Survey (CEQ), and to evaluate its effects on 

indicators of data quality and respondents‟ experience in the survey. Respondents began by 

answering a series of global spending questions about the household that encompassed all of the 

expenditure categories covered by the current CEQ, and provided information about household 

income and changes in assets and liabilities. We examined and presented to respondents their 

resulting income-to-spending ratios for the 1-month reference period. When the ratio deviated 

from perfect income-to-spending balance (i.e., 1.0) by more than 20 percent (15 percent for 

households making less than $30,000), we initiated a balance edit procedure. The first 

component offered respondents the opportunity to review and revise their previous reports. If 

respondents‟ reports remained unbalanced (i.e., if they had ratios that deviated by more than 10 

percent from 1.0), they were asked a brief set of follow-up probes designed to capture 

expenditure and income sources that might have been missed. Following administration of the 

modified CEQ survey with balance edit, we conducted a debriefing session to collect information 

about respondents‟ survey experience, and probe about potential sources of reporting error. 

Key Findings 

Effectiveness of the Balance Edit

Change in Income-to-Spending Ratio (Balance) 

Only four of eighteen participants who received the balance edit in this study obtained acceptable 

income-to-spending ratios by the end of the interview. The balance edit did improve 

respondents‟ income-to-spending ratios - that is, the majority of respondents (13 of 18) moved 

closer to being balanced as a result of the balance edit process. However, despite making relative 

gains in their ratios during the interview, these respondents‟ reports remained notably unbalanced, 

with an average deviation of 42 percent from unity. Most of the movement towards balance was 

the result of respondents (8 of 13) who initially reported higher income than spending but then 

subsequently reported additional expenditures. Only one participant did worse over the course of 
3

the interview, and four individuals essentially remained unchanged (either by making no 

adjustments to their reports, or by switching the sign of their ratio but not decreasing its 

magnitude). 

Levels of Reporting 

During the initial review-and-revise component of the balance edit procedure, nearly two-thirds 

of respondents made changes to their reported spending (with an average of one change per 

respondent), a third made changes to reported income (with an average of two changes per 

respondent), but relatively few revisions were made to reported changes in assets or liabilities 

(average number of changes per respondent: 0). The mean dollar change in expenditures during 

the review and revise phase of the balance edit was $159 (

X

SD

= $429); this represented a mean 

change of 6.2% in respondents‟ initial total reported expenditures. The mean change in income 

was $1,866 (

X

SD

= $2,621); this represented a mean change of 45.3% in respondents‟ initial total 

reported income. During the final phase of the balance edit, in which respondents were probed 

about possible sources of spending and income missed earlier in the questionnaire, 56% of our 

sample reported additional expenditures (overall mean: $188; reporter mean: $503) and 50% 

reported additional income (overall mean: $973; reporter mean: $1,757). 

Factors Affecting Data Quality in a Balance Edit

In subsequent debriefing, respondents identified several factors that they felt affected accuracy. 

Five participants said that they could have reported more accurately if they had been given 

advance notice to record their expenses in some form (e.g. using an excel spreadsheet, using their 

phone, bringing records to reference, writing it down on paper). The size, frequency, and 

saliency of expenditures was mentioned by half of the respondents as contributing to their 

reporting accuracy, with smaller everyday expenses like food and transportation reported as more 

difficult to recall accurately than more stable items like income, mortgage payments, and utilities. 

Respondents‟ household composition – its size and the division of financial responsibilities – also 

played a role. 
4

Respondent Reactions to the Balance Edit

Respondents reactions to the balance edit fell into one of three groups. The first group (n=3) 

initially felt confused by the chart; they didn‟t understand its purpose or what it was supposed to 

represent, and had a hard time comprehending the concept of income-plus-net-assets-andliabilities. A second group of respondents (n=5) understood the purpose of the balance edit but it 

elicited some emotional reaction, often somewhat negative. For example, several respondents 

said that it was somewhat surprising and uncomfortable to be confronted with information that 

showed spending in excess (sometimes far in excess) of their income. Others in this group whose 

reports were unbalanced initially made an inference that they must have done something wrong 

(e.g., “I felt a little stupid.”), or that by being asked to review their previous reports that the 

interviewer mistrusted them in some way (“I was a little frustrated because I knew I was being 

truthful.”). Finally, half of our respondents (n=10) seemed to generally comprehend the 

objective of balance edit, and either have no emotional reaction or be genuinely intrigued by the 

information presented and motivated to resolve reporting discrepancies. Moreover, there did 

appear to be a relationship between these groupings and the magnitude of change in the incometo-spending ratio over the interview and in the final balance status (see page 22). 

Recommendations 

Despite evidence that our balance edit led to some improvements in our data quality measures, we 

cannot on the basis of this study recommend its use for the current CEQ without substantial 

additional testing. The efficacy and appeal of a balance edit will depend largely on the overall 

survey design (e.g., its mode, use of detailed vs. global items, length, etc.) and its analytic 

purposes. Depending on design changes under consideration for the CEQ, further research 

should be done on whether a balance edit is feasible over the phone, with a larger set of detailed 

reports, and with a longer reference period. If CE is interested in pursuing the possibility of 

including a balance edit measure, we recommend that subsequent studies attempt to examine 

measurement error more directly, for example through the use of record validation, or studies of 

within-household reporting consistency/reliability (e.g., comparing reports between spouses). In 

addition, Statistics Canada conducted some analysis on the Survey of Household Spending (SHS)

2008 data and the results suggested that expenses from the unbalanced questionnaires were not 

much different from those of balanced questionnaires (personal communication, May 2011). It 
5

would be worth exploring the specifics of this finding, and Statistics Canada‟s decision to again 

drop the SHS balance edit starting in 2010, in more detail. 
6

1. INTRODUCTION 

1.1 Background 

The U.S. Consumer Expenditure Survey is an ongoing monthly survey conducted by the U.S. Bureau of 

Labor Statistics (BLS) that provides current and continuous information on the buying habits of 

American consumers. The Consumer Expenditure Survey consists of two independent components: 

The Quarterly Interview (CEQ) Survey and the Diary (CED) Survey. For the CEQ, interviewers visit 

sample households five times over the course of thirteen consecutive months. Each interview is 

conducted with a single household respondent who reports for the entire household. The first interview 

establishes cooperation, collects demographic information, and bounds the interview by collecting 

expenditure data for the previous month. This „bounding‟ interview is designed to limit forward 

telescoping, which is the process by which respondents remember and report events or purchases as 

taking place more recently than they actually occurred. The four remaining interviews are administered 

quarterly and ask about expenditures in the 3-month period that just ended. In the second and fifth 

interview, respondents are asked additional detailed questions about household income, assets, and 

liabilities. 

The CEQ survey presents a number of challenges for both interviewers and respondents. The interview 

is long, the questions detailed, and the experience can be perceived as burdensome. In part because of 

these challenges, there is concern that some CEQ data are underreported (e.g., Shields & To, 2005; 

Bosworth, Burtless, & Sabelhaus, 1991). Underreporting has been variously attributed to recall error, to 

panel conditioning, to respondent fatigue, and other causes. 

One approach that has been used in other expenditure surveys to combat response errors like 

underreporting is the use of a data quality control measure known as a „balance edit‟ check. The 

implementation of this measure differs across surveys, but the basic process is one in which respondents 

are given the opportunity to review and revise their reported expenditures, income, and changes in 

assets and liabilities. For example, in the 2009 Survey of Household Spending (SHS) conducted by 

Statistics Canada, households that had expenditures that were out of balance with the reported cash flow 

(i.e., spending that was significantly above or below income plus net assets and liabilities) were probed 

to identify and reconcile possible sources of error. There is evidence from two empirical studies that 
7

the data resulting from use of a balance edit are of higher quality than those collected by the alternative 

methods (Brzozowski & Crossley, 2011; Hurd & Rohwedder, 2010; see Section 2.1 and 2.2). 

1.2 Study Objectives and Design Considerations 

The current CEQ uses built-in range edit checks to flag reports that exceed normative thresholds 

for an expenditure category and consistency edits to flag reports that are inconsistent with data in 

related fields. The purpose of this study was to explore the feasibility of including a balance edit 

check based on a household‟s computed total spending relative to its income, assets, and 

liabilities. Given time and resource constraints within our organization, and because this was our 

first attempt to examine a balance edit measure in the CEQ, we conducted a small-scale lab study 

that addressed three basic areas of inquiry: (1) the effects of a balance edit in identifying and 

correcting reporting errors; (2) respondents‟ reactions to the balance edit process; (3) and the 

factors that impact the quality of respondents‟ reports or their reactions to the reconciliation 

process. 

The study is qualitative in nature and exploratory, and there were a number of design 

considerations that impacted its scope and analytic objectives. One consideration was the 

potential increase in respondent burden that might accompany a balance edit process. The current 

CEQ asks an extensive battery of questions that takes an average of 65 minutes to administer. 

We were concerned that incorporating balance edits at the end of the existing CEQ would 

lengthen an already long and burdensome interview. Moreover, for some households the number 

of reports respondents would need to review could be unduly large, therefore making a balance 

edit process impractical to implement. In addition, in this study we wanted to be able to conduct 

real-time balancing checks and immediately follow-up with respondents during the interview. 

This necessitated the development of an electronic instrument that would record and tally 

respondents‟ expenditure, income, asset, and liability totals, and it was not feasible to do this for 

the full CEQ questionnaire given the project timeline and available resources. 

On the basis of these considerations, and after consulting with CE management, we decided to 

test a balance edit measure using a modified CEQ in which the detailed expenditure questions 

were replaced with a fewer number of global items that asked respondents to report their total 

household spending in each of the CEQ section categories (see Section 3.1 for details). We 
8

acknowledge that the use of global questions and (therefore) a shorter interview are significant 

departures from the current CEQ procedures, that both factors may affect the nature of response 

errors, and that these effects have the potential to interact with the balance edit response process. 

Nevertheless, we view this study as a useful first step in investigating the feasibility of 

implementing a balance edit, and in gathering some initial information about factors that may 

affect its outcome (e.g., frequency of purchase; topic sensitivity; household size and respondent 

knowledge; conceptual clarity; cognitive difficulty; etc.). 

2. PREVIOUS RESEARCH 

Two surveys have implemented some form of a balance edit procedure in an effort to improve the 

quality of expenditure estimates – the Survey of Household Spending (SHS) conducted by 

Statistics Canada, and the American Life Panel (ALP) maintained by RAND Labor and 

Population. We briefly summarize these surveys and their use of this approach below, and 

discuss the studies that have examined the impact of reconciliation procedures on data quality. 

2.1 Survey of Household Spending (SHS) 

The SHS is a face-to-face survey that collects household expenditure and income data for the 

previous calendar year. To help combat recall errors arising from the long recall period, Statistics 

Canada allows respondents to report expenses for smaller and more frequently purchased items 

(e.g., food) on a weekly or monthly basis, and encourages respondents‟ use of records during the 

interview. In addition, the SHS has implemented a balance edit check in which respondents‟

reported expenditures are compared against the sum of reported income and net change of assets 

in the household. When expenditures were more than 20 percent different from reported 

income/assets, the interviewer attempted to collect additional information to bring the two into 

better balance (i.e., 15 percent or less). According to Statistics Canada, most of the changes 

respondents made to their reports during this process were to reported income and assets 

(personal communication, May 2011). SHS households that remained unbalanced were deemed 

unusable and excluded from estimates. 
9

In 2006, SHS data collection moved to computer-assisted personal interviewing (CAPI) and the 

balance edit was not used in the collection phase that year. Instead, balancing was applied at the 

processing phase (with no active involvement or reconciliation by respondents), and this had the 

unexpected consequence of significantly increasing the number of records that were deemed „out 

of balance‟ (from 546 in 2005 to 4,300 in 2006!). This comprised an unacceptably large 

percentage of completed SHS questionnaires for 2006 (29.4%), so a decision was made to 

reinstitute a field balance edit feature for 2007 data collection. To assess the effect of the 

balance edit on data quality, Brzozowski and Crossley (2011) compared SHS data from 2006 (no 

interview-based balance edit procedure) with data from 2005 and 2007. They found no 

differences in income or expenditure reporting across the three years for the top 15 income 

vigtiles. However, respondents in the bottom of the income distribution (lowest 5 vigtiles) underreported income when there was no field balance edit. 

2.2 American Life Panel (ALP) 

The ALP is an Internet panel of approximately 1,500 respondents who are solicited once a month 

to participate in surveys typically taking less than 30 minutes to complete. From June 2009 

through December 2010, a cohort of ALP respondents was asked to complete a monthly 

questionnaire that collected information about household spending in 25 medium- to highfrequency purchase categories and a quarterly questionnaire that collected data on spending in 11 

less frequently purchased categories. Because outliers are a problem in self-administered surveys 

(where there is no interviewer to probe unusual reports), the ALP presented these respondents 

with a „reconciliation‟ screen at the end of each survey that provided a summary table listing the 

individual reported expenditures and the spending total for the household. Respondents then 

were asked to review this information and correct any items, but no automatic edit checks were 

used. Examining data from this panel, Hurd and Rohwedder (2010) found that ALP respondents 

corrected about 2 – 3% of entries per interview wave, and that there were significant reductions in 

item nonresponse and in the frequency and magnitude of outliers due to the reconciliation 

process. There also was good agreement between the total annual spending estimates derived 

from the reconciliation-aided ALP and those from CE over this time period (i.e., ALP spending 

was 96% of CE spending). 
10

2.3 Gaps in Existing Research 

Although the studies by Brzozowski and Crossley (2011) and Hurd and Rohwedder (2010) 

provide evidence that offering respondents an efficient means of reviewing and making 

appropriate changes to their prior responses improves the quality of survey estimates, there is no 

published empirical work that has examined the cognitive underpinnings of this effect. How do 

respondents interpret their task? What changes do they make to their reports? (The SHS work 

suggests that they are more likely to change reported income and assets, but this may be an 

artifact of the types of probes used by Statistics Canada, which tended to focus on missed income 

or sources of financing for larger purchases, not expenditures.) What are respondents‟ reactions 

to being questioned about previous reports, or to seeing their household spending totals? The 

answers to these questions are important for understanding the quality and consequences of 

implementing this kind of procedure, and the present study was designed to begin to fill a gap in 

the literature on these and related issues. 

3. METHODS 

3.1 Design and Procedures 

The study was a small-scale, lab-based test that presented participants with a modified version of 

the CEQ survey and a balance edit procedure for reconciling expenditure-cash flow disparities. 

The test sessions were conducted in the Office of Survey Methods Research (OSMR) laboratory 

in the Bureau of Labor Statistics (BLS). At the start of each session, a researcher explained the 

study‟s purpose and procedures and obtained informed consent from the participants. Study 

participants then took part in a CAPI interview that asked a brief set of demographic questions 

about the household, global expenditure questions for 34 categories (covering all of the CEQ 

section topics), and questions about household income and changes in assets and liabilities for the 

reference month (see Appendix I). All participants were given a modified version of the CEQ 

Information Booklet to refer to throughout the survey. The booklet contained a set of flashcards 

that provided a list of examples of the kinds of expenditures asked about in each category. All 

three authors served as interviewers. 

Expenditures, assets, and liabilities were collected for the preceding month (1-month recall 

period). We used a 1-month recall period rather than a 3-month period (as the CEQ does) or a 
11

flexible reference period (as is done in the SHS) for two reasons. First, we needed to measure the 

change in assets and liabilities over the reference period (e.g., depletions/additions to savings; 

changes to credit card balances) to get a full picture of the household cash flow. We felt that 

asking respondents to recall the relative change in their accounts over a longer recall period 

would be very difficult, especially in a lab setting in which respondents had limited or no access 

to their household records. The second reason was practical – it was not feasible to program and 

administer an electronic collection instrument that allowed flexible recall periods and tailored 

question fills; the resulting database tracking required for the balance edit would have been too 

complex for this study. The drawback of using a 1-month recall period is that it likely misses 

larger and less frequent purchases, and it is possible that these types of purchases could affect 

both the need for reconciliation and participants‟ response processes during reconciliation. We 

attempted to probe for this information during a post-interview respondent debriefing. 

For household income, we allowed respondents to report using a flexible reference period 

(weekly, bi-weekly, monthly, quarterly, or annual) because there were relatively few income 

questions (so programming was manageable), and we felt that respondents would provide more 

accurate information if they could choose their preferred time period. One potential problem with 

this approach was that our unit of measure for the balance edit check was one month. We had to 

aggregate up from weekly and bi-weekly income reports (by multiplying by four or two, 

respectively) or disaggregate quarterly and annual reports (by dividing by three or twelve, 

respectively) in order to get a 1-month income value1

. Income can naturally fluctuate for some 

people over time, however. The extent to which the (dis)aggregated income values differ from 

respondents‟ true income for the reference period impacts the likelihood of triggering the balance 

edit, with potentially more effort focused on „fixing‟ the derived income values. Again, we 

examined the effect of this issue on the survey outcomes and respondents‟ reactions to the survey. 

 

1

 In addition to giving participants the flexibility to choose their preferred recall period for income, they 

could report pre-tax or after-tax income. When pre-tax income was reported, the instrument automatically 

calculated the after-tax value based on current federal, state, and county tax schedules. 
12

3.1.1 Balance Edit Procedure 

Data were recorded in an Excel workbook that calculated the ratio of participants‟ income-tospending over the reference period2

. We calculated the ratio as follows: 

( ) i i i Ai Li ITS ratio I C C S , 

where

i

I

is the (derived) monthly, after-tax household income for household i, CAiand 

CLi

are 

the change in assets and liabilities, respectively, for the household in the reference month, and 

i

S

is the total spending for the household in the month. A ratio less than 1.0 indicates that the 

household spent more than its reported available income (plus net assets and liabilities). A ratio 

greater than 1.0 indicates that the household spent less than its total available income. 

After all of the basic expenditure, income, asset, and liability questions were asked, the 

interviewer showed participants a graph depicting the ratio of their income-to-spending and read 

the following text: 

Thank you very much for your time so far. I'd like to take a look now at 

the overall picture of your household finances last month based on the 

information we've collected from you. This simple chart plots your 

reported household expenditures and your income plus any assets and 

liabilities. Ideally, we'd expect to see that these two figures match up 

pretty closely. [IF RATIO EXCEEDS THRESHOLD, READ:] However, 

sometimes when there is a big difference between these two amounts in a 

given month, it's because we missed some of your HH's expenditures or 

income, or need to make other changes to bring these in line. 

The balance edit check was triggered by one of two income-to-spending ratios. For households 

with at least $30,000 in annual income, the balance edit was triggered if this ratio deviated by 

15% or more from 1.0. For households with annual income below $30,000, the balance edit was 

triggered if the ratio differed by 20 percent or more from 1.0. When respondents‟ income-tospending ratios indicated acceptable balance at this stage of the interview (Phase 1), the 

interviewer terminated the interview and proceeded to the debriefing portion of the study session. 

 

2

 Throughout the remainder of the report we use the phrase „income-to-spending ratio‟ for convenience to 

refer to the „income-plus-net-assets-and-liabilities-to-spending ratio‟ unless otherwise stated. 
13

If households were out of balance (based on the criteria above), respondents were given the 

opportunity to review a summary page of their reported expenditures, income, assets, and 

liabilities (individual reports and summed totals), and to make changes to their earlier reports in 

order to bring their ratio closer to 1.0. We recorded any changes made by respondents during 

this review and revision phase (Phase 2), and then showed them a revised graph of their updated 

household income-to-spending figure. If the household still was unbalanced at that point in the 

interview, the interviewer administered a brief set of probes designed to capture additional 

sources of income and expenditures that might have been missed (Phase 3; see Appendix II for 

the full set of balance edit materials). The balance edit procedure was terminated in Phase 2 or 

Phase 3 when expenditures were within 10 percent of reported income plus net assets and 

liabilities, or when all of the CEQ items and follow-up probes had been administered. 

3.1.2 Debriefing Procedure 

Following administration of the modified CEQ (with any balance edit checks required), 

respondents filled out a short paper-and-pencil questionnaire and then participated in a semistructured debriefing session with the interviewer. The purpose of the debriefing was to assess 

the following topics: 

Respondents‟ general reaction to the survey and the balance edit procedure; 

Respondents‟ perceptions of the accuracy of their reported data; 

Sources of confusion or conceptual difficulty (e.g., global items, reference period, proxy 

reporting); 

Respondents‟ perceptions of survey burden 

Appendix III contains the full debriefing protocol. 

3.2 Participants 

Participants for this study were recruited from an OSMR-maintained database of individuals who 

responded to advertisements for research studies placed in DC-area newspapers. We used a nonprobability based convenience sample but attempted to recruit participants who varied in their 

family size, educational attainment, household income, and employment status3. We interviewed 

 

3

 We achieved reasonable balance on participant education, gender, and family size, but higher-income 

households and employed individuals were over-represented in our sample. We had only two respondents 

with household incomes under $30,000 (sample mean: $67,800) and only three who were unemployed or 

retired. 
14

twenty participants but report findings for only 19; one individual provided insufficient data 

during the survey and debriefing session. All twenty participants were paid $40 for their time. 

Interviews lasted approximately one hour on average – 30 minutes to administer the modified 

CEQ survey and balance edit, and 30 minutes to conduct the participant debriefing session. 

3.3 Data Quality Indicators 

In the next section of the report we present our study findings. We focus on several key results. 

We do not have a benchmark for „true‟ spending and income, so we examine the level of 

expenditure reporting as one commonly accepted measure of data quality. Balance edit check 

measures have been developed primary to reduce the likelihood of response errors due to 

underreporting, and thus more reporting is taken as evidence of better reporting. Another one of 

our primary measures of interest is the change in the household income-to-spending ratio across 

the interview. If the balance edit procedure implemented in this study was effective, we would 

expect to see those ratios improve (i.e., get closer to 1.0). Finally, we explore the qualitative 

responses obtained in our debriefing session to give us some additional insight into the quality of 

the data reported in the interview, the factors that may impact the effectiveness of a balance edit, 

and respondents‟ reactions to their survey experience. 

4. RESULTS 

4.1 Effect of Balance Edit on Income-to-Spending Ratios 

Table 1 shows the distribution of respondents‟ income-to-spending ratios at different phases of 

the survey. The first column shows the ratio criteria that we used to determine degree of balance. 

The second column indicates the number of respondents who fell into each ratio category at the 

end of Phase 1 of the interview (prior to any balance edit procedures). Only one respondent 

obtained an income-to-spending ratio good enough - 0.91 - to obviate the need for a balance edit. 

Two additional respondents obtained ratios of 0.82 and 1.19 in Phase 1, respectively, but the 

balance edit procedure was triggered because they had annual household incomes over $30,000 

(for which „balance‟ was defined as a deviation of 15 percent or less from a ratio of 1.0). The 

remaining 16 respondents were considerably unbalanced at the completion of the basic 
15

questionnaire, with a mean deviation from unity of 43 percent (i.e., reporting 43 percent more 

available household income than reported spending). 

Table 1. Number of Respondents in Each Income-to-Spending Ratio Group by Interview Phase 

Deviation from 

Income-Spending 

Ratio of 1.0

End of Phase 1

(Main questionnaire)

End of Phase 2

(Review & Revise)a

End of Phase 3

(Additional Probing)b

0.0 – 10.0% 1 2 2

10.1 – 15.0% 0 3 3

15.1 – 20.0% 2 3 3

20.1% + 16 10 8

a Includes only respondents who were out of balance at the end of Phase 1 (n=18). 

b

 Includes only respondents who were asked the additional expenditure and income probes (n=16) 

The third and fourth columns of Table 1 give one indication of the extent to which the balance 

edit procedure was effective. Recall that once the balance edit was triggered in this study (using 

the 15% or 20% deviation criterion at Phase 1), a household was deemed „balanced‟ if it achieved 

an income-to-spending ratio between 0.90 and 1.10. Examining the first row of the table, we see 

that two additional respondents achieved balance after being given the chance to review a 

summary of their reports and make revisions (Phase 2, column 3), and two more respondents 

achieved balance after answering the additional expenditure and income probes (Phase 3, column 

4). Thus, a total of 5 respondents out of 19 obtained acceptable income-to-spending ratios; one 

respondent did so without going through the balance edit and four did so only with the help of the 

balance edit. 

The shaded cells in Table 1 show the number of respondents who did not reach balance in any 

phase of the interview by our study criteria. Nevertheless, there is some indication in these cells 

that the balance edit did have a positive impact on respondents‟ income-to-spending ratios in the 

aggregate. For example, the number of respondents with ratios deviating by more than 20 percent 

from 1.0 was cut in half by the end of the balance edit process (from 16 to 8). This effect is even 

more evident when we examine the pattern of changes in income-to-spending ratios for individual 

respondents over the course of their interviews (see Table 2). 
16

Table 2. Distribution of the Direction of Ratio Changes in the Interviewd 

Type of Ratio Change # of Respondents

Moved Closer to 1.0 13

More Income than Spending (ratio > 1.0) 8

More Spending than Income (ratio < 1.0) 3

Below 1.0 to Above 1.0 2

Above 1.0 to Below 1.0 0

Moved Farther from 1.0 1

Flipped 2

No Change 2

d

 Includes only respondents who were administered the balance edit (n=18). 

Table 2 presents the number of respondents whose ratios improved, worsened, stayed the same, 

or simply switched direction but not magnitude over the course of the interview. The majority of 

our respondents (13 of 19) moved closer to being balanced as a result of the balance edit process. 

(However, despite making relative gains in their ratios during the interview, these respondents‟

reports remained notably unbalanced, with an average deviation of 42 percent from unity.) Most 

of the movement towards balance in this group was the result of respondents (8 of 13) who 

initially reported higher income than spending but then subsequently reported additional 

expenditures (these respondents‟ ratios got smaller but remained above 1.0). Five respondents 

initially reported significantly more spending than income and moved closer to balance 

throughout the interview by making small reductions to their reported spending and increases to 

income (e.g., reporting additional wages or forgotten tax refunds). Only one participant did 

worse over the course of the interview, and four individuals essentially remained unchanged 

(either by making no adjustments to their reports, or by flipping the sign of their ratio but not 

decreasing its magnitude). 

4.2 Levels of Reporting 

Table 3 gives a more concrete look at respondents‟ reporting throughout the interview. The 

second column shows the average number of reports and average dollar amounts given by our 

study participants for the different survey topics in the main questionnaire (Phase 1). Consistent 

with our earlier findings, prior to the balance edit check our sample in the aggregate reported 

slightly more income (plus net change in assets and liabilities) than spending. During the initial 
17

review and revise component of the balance edit procedure (Phase 2), 61 percent of respondents 

(n=11) made changes to their reported spending (with an average of one change per respondent), 

33 percent (n = 6) made changes to reported income (with an average of two changes per 

respondent), but relatively few revisions were made to reported changes in assets or liabilities 

(average number of changes per respondent: 0).4

 

Table 3. Reporting Incidence and Level by Topic and Interview Phase 

Phase 1

(Main Questionnaire)

(n=19)

Phase 2

(Review and Revise)

(n=18)

Phase 3

(Additional Probes)

(n=16)

X

Total ($)

X

# Reports

% 

Reporting

Change

X

Change ($)

% 

Reporting

Change

X

Change ($)

Expenditures $4,781 14 61% $92 56% $188

Income $5,196 2 33 $589 50 $973

Change in Assets $67 1 22 $28 n/a n/a

Change in Liabilities $49 0.5 5 -$528 n/a n/a

The mean dollar change figures shown on the right-hand side of the Phase 2 column reflect 

averages across the entire eligible sample (i.e., all respondents who were administered the Phase 

2 balance edit, regardless of whether a change was made or not). Restricting our analyses to only 

those respondents who made revisions in Phase 2, the mean dollar change in expenditures was 

$159 (

X

SD

= $429); this represented a mean change of 6.2% in respondents‟ initial total reported 

expenditures (the overall sample mean – for changers and non-changers – was 3.6% of total 

expenditures). The mean change in income was $1,866 (

X

SD

= $2,621); this represented a mean 

change of 45.3% in respondents‟ initial total reported income (the overall sample mean was 

14.3% of initial total income). During Phase 3, in which respondents were probed about possible 

sources of spending and income missed earlier in the questionnaire (but not new changes in assets 

or liabilities), 56 percent (n=9) of our sample reported additional expenditures (overall mean: 

$188; reporter mean: $503) and 50 percent (n=8) reported additional income (overall mean: $973; 

reporter mean: $1,757). 

 

4

 Only one participant in this study revised her reported change in liabilities, recalling that she had paid off 

a business loan for $9500 during the reference month. 
18

4.3 Debriefing 

Following the completion of the modified CEQ survey, respondents completed a selfadministered questionnaire that asked them to rate how comfortable they felt sharing expenditure, 

income, asset, and liability information during the interview, and how accurate they felt those 

reports had been. The original questions used a 5-point Likert scale – ranging from „very 

uncomfortable‟ to „very comfortable,‟ and „very inaccurate‟ to „very accurate,‟ with a neutral 

„neither‟ middle response option. We collapsed the „very‟ and „somewhat‟ categories for each 

dimension and omit the middle response option data for reporting purposes. Table 4 presents 

those data. 

Table 4. Respondents’ Ratings of Comfort and Accuracy (%) by Reporting Category (n=19) 

Category

Comfort Accuracy

Very or 

Somewhat 

Uncomfortable

Very or 

Somewhat 

Comfortable

Very or 

Somewhat 

Inaccurate

Very or 

Somewhat 

Accurate

Expenditures 5% 65% 5% 80%

Income 30 55 0 85

Change in Assets 30 60 5 85

Change in Liabilities 15 60 0 95

The majority of our study respondents reported feeling „very‟ or „somewhat‟ comfortable 

providing the household financial information asked during the interview, though reporting 

income was more sensitive for some respondents than discussing expenditures or changes in 

assets and liabilities. In response to a follow-up question, several participants acknowledged that 

they felt awkward disclosing income to a stranger, but understood the purpose of the question and 

believed that their responses were important and would be kept confidential. Additionally, 

several participants said that they were comfortable reporting income information for themselves, 

but had been reluctant or unable to do so for other household members. 

Respondents‟ ratings of the accuracy of their reports were very high (80% - 95%). On the one 

hand, this may simply reflect the effects of self-presentation management – the desire to represent 
19

oneself as a diligent and accurate participant in the data collection effort. On the other hand, a 

number of respondents in this study evinced behavior or made explicit comments that indicated 

that they had engaged in effortful and thorough recall and reporting. For example, most 

respondents followed along with the Information Booklet during the interview and said during 

debriefing that it provided definitional clarity to some of the global expenditure categories/items, 

and helped them recall expenditures they would otherwise have forgotten to report. Two 

respondents spontaneously brought out their checkbooks or personal calendars during the 

interview to aid reporting. In addition, the balance edit process itself may have strengthened 

respondents‟ perceptions of accuracy since most respondents made adjustments to their reports 

(recalling forgotten items, revising earlier reports with greater specificity) and saw visual 

evidence that those efforts led to improvements in their household cash flow balance. 

The self-administered questionnaire also included items to assess respondents‟ perception of 

survey length and burden. We asked respondents to estimate how long the interview lasted under 

the assumption that those estimates would exceed actual survey length if respondents felt 

burdened (see, e.g., Block, 1990). The average estimated interview duration was 32.3 minutes, 

about five minutes longer than the actual interview duration (mean 27.2), suggesting some degree 

of respondent burden. However, when asked whether the interview was “too long, too short, or 

about right,” all respondents replied that the survey length was „about right.‟ In addition, we 

asked respondent to rate how burdensome they felt the survey was and again the responses were 

uniformly positive (e.g., “It was not at all burdensome.” “It was great.” “...very interesting and 

easy.”). 

4.3.1 Factors Affecting Accuracy 

In subsequent debriefing, respondents identified several factors that they felt affected accuracy. 

Five participants said that they could have reported more accurately if they had been given 

advance notice to record their expenses in some form (e.g. using an excel spreadsheet, using their 

phone, bringing records to reference, writing it down on paper). The size, frequency, and 

saliency of expenditures was mentioned by half of the respondents as contributing to their 

reporting accuracy, with smaller everyday expenses like food and transportation reported as more 

difficult to recall accurately than more stable items like income, mortgage payments, and utilities. 
20

Respondents‟ household composition – its size and the division of financial responsibilities – also 

played a role. Eleven out of fourteen participants who lived in multi-person households said that 

they had a „good sense‟ of what other people in their house bought and how much they spent, but 

they also identified gaps in that knowledge (e.g., food eaten out by spouses or children; gas 

expenditures; purchases made on other‟s credit cards; etc.). To a lesser extent, respondents made 

similar comments about their knowledge of other household members‟ income, assets, and 

liabilities. In fact, four participants admitted to „forgetting‟ to include some or all of their other 

household members‟ expenditures and income (e.g., “I was just focused on me!”). Two of those 

participants were able to remedy these omissions during the balance edit, but two did not realize 

their mistake until the debriefing discussion. 

We also asked respondents whether the 1-month reference period presented any reporting 

difficulties and if a 3-month reference period would have been easier or more difficult. One selfemployed participant said the 1-month reference period was difficult because her income varied 

considerably from month to month, but none of the other participants reported difficulties with 

the monthly time frame. Respondents‟ views about the efficacy of a longer reference period 

varied by topic. For expenditures, 17 participants said that using a 3-month reference period 

would be more difficult than using a 1-month reference period because of the additional memory 

demands and the fact that some expenses are intermittent and therefore more likely to be 

forgotten. We asked respondents how they would come up with their total household expenses 

for three months. The word “estimate” was used often, and many respondents said that they 

would think of their „typical‟ monthly expenses and multiply by three. A few respondents did say 

that they would think of the expenses for each specific month and try to systematically calculate 

an accurate 3-month figure. For income, two-thirds of our study participants said reporting for a 

3-month period instead of 1-month period would be essentially the same because their income 

was fairly regular and stable. For assets and liabilities, our sample was split: about half preferred 

the 1-month and half preferred the 3-month reference period. The responses depended largely on 

how closely the respondent tracked their accounts and how regular or irregular their account 

activity usually was. 
21

4.3.2 Reactions to the Balance Edit 

We were interested in exploring respondents‟ reactions to various features of the balance edit 

implemented in this study. We began by asking them about the chart they were shown at the end 

of the basic questionnaire which displayed their income-to-spending ratio. Opinions fell into one 

of three groups. The first group (n=3) initially felt confused by the chart; they didn‟t understand 

its purpose or what it was supposed to represent, and had a hard time comprehending the concept 

of income-plus-net-assets-and-liabilities. These respondents indicated that the accompanying 

explanation about the chart provided by the interviewer was helpful in deciphering its meaning, 

or at least in clarifying the overall objective of the balance edit process. 

A second group of respondents (n=5) understood the purpose of the chart and balance edit but it 

elicited some emotional reaction, often somewhat negative. For example, several respondents 

said that it was somewhat surprising and uncomfortable to be confronted with a chart that showed 

spending in excess (sometimes far in excess) of their income. Others in this group whose reports 

were unbalanced initially made an inference that they must have done something wrong (e.g., “I 

felt a little stupid.”), or that by being asked to review their previous reports that the interviewer 

mistrusted them in some way (“I was a little frustrated because I knew I was being truthful.”). In 

contrast, two respondents in this group whose reports were reasonably balanced and showed more 

income than spending expressed satisfaction upon seeing the chart (e.g., “I don’t want to be 

balanced. I want to have more income so I can spend and save more.”). 

Finally, half of our respondents (n=10) seemed to generally comprehend the chart and the 

objective of balance edit, and either have no emotional reaction or be genuinely intrigued by the 

information presented and motivated to resolve reporting discrepancies. The respondents in this 

group were able to clearly articulate the objective of the edit process in their own words and did 

not appear to have any conceptual difficulties or negative emotional reactions. 

Table 5 shows the relationship between these subjective respondent groupings and the balance 

edit outcomes. Given the small and disproportionate sizes of the three groups, caution should be 

used when making inferences based on the results in this table. However, it does appear that 

there is a relationship between these groupings and the magnitude of change in the income-tospending ratio over the interview (column 3) and in the final balance status (column 4). 
22

Table 5. Relationship Between Respondents’ Initial Reactions to Balance Edit and Survey Outcomee

Ratio Moved 

Closer to 1.0

X

Change in 

Ratio

Achieved 

Balance

Group 1 – Initially Confused 

(n=3) 3 -52.3% 0

Group 2 – Emotional Response

(n=5) 3 50.7% 0

Group 3 – Understood/Positive Response

(n=10) 7 -2.0% 4

e

 Includes only those respondents who were administered the balance edit (n=18). 

Respondents who were confused by the chart and balance edit objective (Group 1) 

initially reported greater income than spending. Those who had an emotional reaction 

initially (Group 2) reported greater spending than income. The magnitude of the average 

ratio change was very similar for both groups, and no respondents in either group 

achieved balance. Respondents who understood the chart and edit objective without any 

associated negative emotional reaction (Group 3) made smaller (or at least off-setting) 

changes on average during the balance edit than respondents in the other two groups, and 

all of the balanced households came from this group. 

We also queried respondents on their reactions to Phase 2 of the interview in which we gave 

respondents the chance to look over a summary page of their reports and make any corrections or 

additions they felt were needed. Again, reactions tended to be split. Most respondents did not 

express any substantive opinion about this page – they simply engaged in the review process and 

moved on. Three respondents gave only cursory examination of this page and either did not fully 

understand their task or chose not to exert the effort required to review the information more 

carefully; they made no changes on this screen. A few participants described the review and 

revise process as „daunting‟ or „chastising‟ because it forced them to examine (in front of the 

interviewer) some hard truths about their household finances. And, finally, several other 

respondents said that they really liked the table and task because it prompted them to consider the 
23

relative amounts reflected in each category or because the presentation triggered memories of 

additional items that they then could report. 

Some of the most frequent adjustment and additions made on this page stemmed from 

respondents‟ memories for expenditures related to landmark events (e.g., birthdays, trips, 

Mother‟s Day). Another common change that respondents made stemmed from financial 

activities by other household members. As noted earlier, sometimes this information was 

neglected altogether in respondents‟ answers to the basic questionnaire, other times it was only 

partially reported or respondents simply provided best guesstimates. This page afforded 

respondents the chance to re-focus attention on proxy-related items they may have missed (e.g., 

spouses contributions to retirement accounts), or revise their estimates for other household 

member items in order to achieve better balance. There also were a few instances in which we 

believe that respondents were just simply trying to improve the household balance by making 

seemingly arbitrary adjustments, but this was not common. 

In the final phase of the interview, respondents who remained unbalanced were asked a brief set 

of additional probes about possible sources of expenditures and income. Fifteen of the 16 

participants who received these questions said that they were clear and easy to answer. The two 

most common items that we picked up as a result of these probes were child care expenses and 

tax refunds/tax payments (the reference month in this study was April). 

5. DISCUSSION 

5.1 Summary 

The purpose of this study was to examine the application of a balance edit procedure to a 

modified CEQ, and to evaluate its effects on indicators of data quality and respondents‟

experience in the survey. Respondents began by answering a series of global spending questions 

about the household that encompassed all of the expenditure categories covered by the current 

CEQ, and provided information about household income and changes in assets and liabilities. 

We examined and presented to respondents their resulting income-to-spending ratios for the 1-

month reference period. When the ratio deviated from perfect income-to-spending balance (i.e., 
24

1.0) by more than 20 percent (15 percent for households making less than $30,000), we initiated a 

balance edit procedure. The first component offered respondents the opportunity to review and 

revise their previous reports. If respondents‟ reports remained unbalanced (i.e., if they had ratios 

that deviated by more than 10 percent from 1.0), they were asked a brief set of follow-up probes 

designed to capture expenditure and income sources that might have been missed. Following 

administration of the modified CEQ survey with balance edit, we conducted a debriefing session 

to collect information about respondents‟ survey experience, and probe about potential sources of 

reporting error. 

The findings from this small study shed some light on the challenges and opportunities that a 

balance edit introduces to expenditure surveys. We found that 18 of 19 participants in this study 

provided reports in the initial interview that were sufficiently unbalanced to trigger the balance 

edit. The balance edit procedure improved income-to-spending ratios for 13 participants, but 

only 4 individuals actually achieved balance (i.e., obtained ratios that were within 10 percent of 

unity). Despite the relative improvements in the ratios of the remaining 9 participants, their final 

average deviation from unity remained quite high (42 percent). The debriefing session revealed 

that a sizable minority of individuals in this study either did not understand the purpose in the 

balance edit or had somewhat negative reactions to the process. In addition, we identified a 

number of factors that likely contributed to its effectiveness (e.g., respondent knowledge of other 

household members‟ spending/income; the size, frequency, and salience of the expenditure 

category). 

5.2 Limitations 

Our study was limited by a number of factors that may have impacted the results. First, we used 

a small, convenience sample; a larger, probability-based sample would have strengthened our 

ability to generalize the results and reduced the influence of outliers introduced by individuals 

with very large (small) reports. Second, some of the respondent confusion evidenced in this 

study may have stemmed from our specific implementation of the balance edit – a hybrid 

between the approaches taken in the SHS and ALP. In particular, the language we used to 

introduce the objective of the edit and to describe the chart reflecting the income-to-spending 

ratio likely could be improved based on what we learned in this study. In our roles as 
25

interviewers, we strove to provide respondents additional task clarification as necessary, but 

further refinement and standardization of this feedback might have improved respondent 

understanding. Finally, our decision to use global expenditure questions may have affected the 

incidence of out-of-balance households. Global questions may tend to encourage respondents to 

provide rough estimates for many expenditure categories when other response strategies would be 

more optimal. Their use also may have affected respondents‟ perceptions of the burden of the 

survey and the balance edit process; attempting to conduct our balance edit with the full 

complement of detailed, disaggregated CEQ questions likely would have been far more difficult 

for respondents. 

5.3 Recommendations 

Despite evidence that our balance edit lead to some improvements in our data quality measures, 

we cannot on the basis of this study recommend its use for the current CEQ without substantial 

additional testing. As we suggest elsewhere in this report, the efficacy and appeal of a balance 

edit will depend largely on the overall survey design (e.g., its mode, use of detailed vs. global 

items, length, etc.) and its analytic purposes. Under the existing CEQ protocols, there are a 

number of feasibility concerns with a balance edit implementation. First, there is the additional 

time required to administer the edit. In our study, the balance edit process accounted for nearly a 

third of the total survey duration (about 8 – 10 minutes out of 25 – 30 total minutes, on average). 

The relative administration time of the edit would be reduced in the CEQ compared to our study 

since the production survey uses significantly more detailed questions than our study did. 

However, the complexity of applying this type of edit to such a large number of items likely also 

would significantly increase respondent burden and overall survey length, as well. Second, we 

discovered that our respondents did not always understand what we wanted them to do in the 

balance edit. This made it difficult to stick to a standardized administration protocol, and we 

would have concerns about this lack of standardization in a production environment. Finally, it is 

not a trivial matter to identify all of the potential range edits that would serve as inputs into a 

balance edit. To be optimally effective, we would ideally want to tailor these edits in some way 

(e.g., by household size, income, reporting patterns), and this requires fairly sophisticated use of 

both previously reported data and information provided within the interview itself. 
26

Were a redesigned CEQ to incorporate significant numbers of global expenditure questions 

(which we expect are subject to significant reporting errors) and still be conducted primarily inperson, then some form of a balance edit might worth considering. If, on the other hand, the CEQ 

continues to be a long survey consisting of hundreds of detailed expenditure questions, and/or a 

significant portion of the interviews each month are administered by telephone, then we believe 

that this measure is less attractive given implementation issues and the potential negative impact 

on respondent and interviewer burden. Additionally, we used a 1-month, not a 3-month, 

reference period for this study, and we suspect that the longer time frame would reduce the 

quality of respondents‟ estimates of change in assets and liabilities, in particular. Depending on 

design changes under consideration for the CEQ, further research should be done on whether a 

balance edit is feasible over the phone, with a larger set of detailed reports, and with a longer 

reference period. Finally, our study was designed to examine only relative changes in respondent 

reporting; we had no direct measures of the actual quality of the data collected. If CE is 

interested in pursuing the possibility of including a balance edit measure, we recommend that 

subsequent studies attempt to examine measurement error more directly, for example through the 

use of record validation, or studies of within-household reporting consistency/reliability (e.g., 

comparing reports between spouses). In addition, Statistics Canada conducted some analysis on 

SHS 2008 data and the results suggested that expenses from the unbalanced questionnaires were 

not much different from expenses from balanced questionnaires (personal communication, May 

2011). It would be worth exploring the specifics of this finding, and Statistics Canada‟s decision 

to again drop the SHS balance edit starting in 2010, in more detail. 
27

REFERENCES 

Block, R. A. (1990). “Models of psychological time.” In R. A. Block (Ed.), Cognitive models of

psychological time, pp. 1-35. Hillsdale, NJ: Lawrence Erlbaum Associates. 

Bosworth, B., Burtless, G., and Sabelhaus (1991). “The Decline in Saving: Some 

Microeconomic Evidence.” Brookings Papers on Economic Activity, 1, p. 183-256. 

Brzozowski, M., Crossley, T.F., , (2011). “Measuring the Well-being of the Poor with Income or 

Consumption: A Canadian Perspective.” Canadian Journal of Economics, 44, pp. 88 – 106.. 

Hurd, M. and Rohwedder, S. (2010). “The Effects of the Financial Crisis and the Great 

Recession on American Households.” NBER Working Papers 16407, National Bureau of 

Economic Research. 

Shields, J. and To, N. (2005). “Learning To Say No: Conditioned Underreporting in an 

Expenditure Survey.” Paper presented at the annual meeting of the American Association for 

Public Opinion Association, Fontainebleau Resort, Miami Beach, FL. 
28

APPENDIX I 

CE Balance Edit Study Survey Instrument Items: PHASE 1 

INSTRUCTIONS/Transition: Let's get started. Today, we'll be going through a Consumer 

Expenditure survey. This survey collects information about your household's expenses and 

income, but we'll start briefly by asking you a few questions about your household. Are you 

ready to begin?

HH Characteristics

Which state do you live in? 

How many people live in your household (including yourself)? 

How many of the people in your household are over the age of 18? 

Do you currently have a job? 

INSTRUCTIONS/Transition: Next, I will ask you about expenses your household had or bills 

that you've received. As we start, please understand that we ask the same questions of everybody 

we talk to; I realize some of these questions may not apply to your household. 

Most questions that I will be asking refer to a specific time period. During this interview, the 

time period, unless I state otherwise is for last month - that is, from the 1st day of [REFERNCE 

MONTH] to the [LAST DAY, REFERENCE MONTH]. Please remember to include expenses 

for every member of your household in your answers. 

Expenditures

1. Last month, did you or any member of your household purchase a car or truck? (If so, 

what was the cost?) 

2. ... a refrigerator? [Cost?] 

3. ... a washing machine and/or dryer? [Cost?] 

4. ... a dishwasher? [Cost?] 

5. ... a television? [Cost?] 

6. ... a computer? [Cost?] 

7. ... pay a mortgage for any property or real estate? [Cost?]

8. ... pay property tax? [Cost?] 

9. ... pay rent? [Cost?] 

10. ... renters or homeowner‟s insurance? [Cost?] 

11. ... electricity? [Cost?] 

12. ... water? [Cost?] 

13. ... heating fuel? [Cost?] 

14. ... telephone, cable, or internet? [Cost?] 

15. ... furniture? [Cost?] 

16. ... housekeeping or yard supplies? [Cost?] 

17. ... home repairs and maintenance? [Cost?] 
29

18. ... gasoline? [Cost?] 

19. ... vehicle maintenance? [Cost?] 

20. ... vehicle payments? [Cost?] 

21. ...vehicle insurance? [Cost?] 

22. ... food and beverages (excluding dining out)? [Cost?] 

23. ... dining or drinking out? [Cost?] 

24. ... clothing and apparel? [Cost?] 

25. ... health insurance? [Cost?] 

26. ... prescription or non-prescription medicine? [Cost?] 

27. ...health care services (out-of pocket cost of hospital care, doctor services, lab tests, eye, 

dental, and nursing home care)? [Cost?] 

28. ... medical supplies? [Cost?] 

29. ... trips and vacations? [Cost?] 

30. ... tickets to movies, sporting events, and performing arts? [Cost?] 

31. ... hobbies (including exercise, bicycles, trailers, camping, photography, reading 

materials, etc.)? [Cost?] 

32. ... education (including student loan payments, tuition, room and board, books, and 

supplies)? [Cost?] 

33. ... contributions to religious, educational, charitable, or political organizations? [Cost?] 

34. ... cash or gifts to family and friends outside your household (including alimony and 

child support payments)? [Cost?] 

INSTRUCTIONS/Transition: The next few questions are about income. We know 

people aren‟t used to discussing their income, but please be assured that, like all other 

information you have provided, these answers will be kept strictly confidential. 

You can report weekly, bi-weekly, monthly, or yearly amounts. You can also report 

these amounts with or without taxes. You will be asked to specify which of these 

amounts you chose to report. 

INCOME

1. Last month, did you or any members of your household earn any income from wages, salary, 

commissions, bonuses, or tips (do not include income from self-employment)? 

1a. How much did you earn from these income sources in all jobs? 

1b. Is this amount paid weekly, bi-weekly, monthly, or yearly? 

1c. Is this net (after-tax) or gross (before-tax) income? 

2. Last month, did you earn any income from self-employment? 

2a. How much did you earn? 

2b. Is this amount paid weekly, bi-weekly, monthly, or yearly? 

2c. Is this net (after-tax) or gross (before-tax) income? 

 
30

3. Last month, did you earn any income from interest, dividends, net rental income, or income 

from trusts or estates? 

3a. How much did you earn? 

3b. Is this amount paid weekly, bi-weekly, monthly, or yearly? 

3c. Is this net (after-tax) or gross (before-tax) income? 

 

4. Last month, did you earn any income from Social Security? 

4a. How much did you earn? 

4b. Is this amount paid weekly, bi-weekly, monthly, or yearly? 

4c. Is this net (after-tax) or gross (before-tax) income? 

 

5. Last month, did you earn any income from Supplemental Security (SSI)? 

5a. How much did you earn? 

5b. Is this amount paid weekly, bi-weekly, monthly, or yearly? 

 5c. Is this net (after-tax) or gross (before-tax) income? 

 

6. Last month, did you receive any public assistance or welfare? 

6a. What is the cash value of that assistance? 

6b. Is this amount paid weekly, bi-weekly, monthly, or yearly? 

 6c. Is this net (after-tax) or gross (before-tax) income? 

 

7. Last month, did you earn any income from retirement, survivor, or disability pensions? 

7a. How much did you earn? 

7b. Is this amount paid weekly, bi-weekly, monthly, or yearly? 

 7c. Is this net (after-tax) or gross (before-tax) income? 

8. Last month, did you earn any income from veteran's payments, unemployment compensation, 

child support, or alimony? 

8a. How much did you earn? 

8b. Is this amount paid weekly, bi-weekly, monthly, or yearly? 

8c. Is this net (after-tax) or gross (before-tax) income? 

 

9. Last month, did you earn any income from any other regular source? 

9a. How much did you earn? 

9b. Is this amount paid weekly, bi-weekly, monthly, or yearly? 

 9c. Is this net (after-tax) or gross (before-tax) income? 

 

INSTRUCTIONS/Transition: Next, we will collect information on changes in your 

household finances last month. We'll ask you to estimate whether your household assets 

(such as savings or checking accounts) and liabilities (such as credit card balances and 

other loans you owe) increased or decreased last month and by what amount. 

ASSETS/LIABILITIES

1. Do you or any member(s) of your household have a checking account? 

1a. How many checking accounts do you have? 
31

1b. Did the balance of checking account (1,2,3,4) increase, decrease, or stay about the 

same in (MONTH)? 

1c. How much did checking account (1,2,3,4) (increase/decrease) in (MONTH)? 

 

2. Do you/members of your HH have a savings account? 

2a. How many savings accounts do you have? 

2b. Did the balance of savings account (1,2,3,4) increase, decrease, or stay about the 

same in (MONTH)? 

2c. How much did savings account (1,2,3,4) (increase/decrease) in (MONTH)? 

 

3. Do you have a personal retirement account? 

3a. How many personal retirement accounts do you have? 

3b. Did the balance of retirement account (1,2,3,4) increase, decrease, or stay about the 

same in (MONTH)? 

3c. How much did retirement account (1,2,3,4) (increase/decrease) in (MONTH)?

 

INSTRUCTIONS/Transition: Okay, now we'll talk about credit card balances and other loans 

that your household may have. 

4. Do you/any members of your HH have a credit card? 

4a. How many credit cards do you have? 

4b. Did the balance of credit card (1,2,3,4) increase, decrease, or stay about the same in 

(MONTH)? 

4c. How much did credit card (1,2,3,4) (increase/decrease) in (MONTH)? 

 

5. Do you have any personal loans (including friend and family loans)? 

5a. How many of these loans do you have? 

5b. Did the balance of personal loan (1,2,3,4) increase, decrease, or stay about the same 

in (MONTH)? 

5c. How much did personal loan (1,2,3,4) (increase/decrease) in (MONTH)? 

 
32

APPENDIX II

CE Balance Edit Study Instrument Items: PHASE 2 & 3

INSTRUCTIONS/Transition: Thank you very much for your time so far. I'd like to take a look 

now at the overall picture of your household finances last month based on the information 

we've collected from you. This simple chart plots your reported household expenditures and 

your income plus any assets/liabilities. Ideally, we'd expect to see that these two figures 

match up pretty closely. [IF THEY DO NOT, READ: However, sometimes when there is a big 

difference between these two amounts in a given month, it's because we (missed some of 

your HH's expenditures/income) or need to make other changes to bring these in line. 

$0 

$1,000 

$2,000 

$3,000 

$4,000 

$5,000 

$6,000 

TOTAL Monthly Expenditures TOTAL Monthly Income + Assets & 

Liabilities
33

INSTRUCTIONS/Transition: Here is a summary of your answers. Please take a moment to 

review the entries, using the scroll bar on the right to move up and down through the entire 

list. If you find an item that you would like to change, enter the revised number in the yellow 

cell next to the item you wish to update. 

Category

Reported

Cost

Updated

Cost

Large Expenses

Car $0 $0

Refrigerator $0 $0

Washer/Dryer $0 $0

Dishwasher $0 $0

Television $0 $0

Computer $0 $0

Home and Utilities

Mortgage $0 $0

Property Tax $0 $0

Rent $0 $0

Renter/Homeowner's Insurance $0 $0

Electricity $0 $0

Water $0 $0

Heating Fuel $0 $0

Telephone, Cable, Internet $0 $0

Other Household

Furniture $0 $0

Housekeeping and Yard Supplies $0 $0

Home Repairs and Maintenance $0 $0

Vehicle

Gasoline $0 $0

Vehicle Maintenance $0 $0

Vehicle Finance $0 $0

Vehicle Insurance $0 $0

Food

Food and Beverages $0 $0

Dining/Drinking Out $0 $0

Clothing

Clothing and Apparel $0 $0

Health Related 

Health Insurance $0 $0

Medications $0 $0

Health Care Services $0 $0

Medical Supplies $0 $0

Entertainment

Trips and Vacations $0 $0

Tickets $0 $0

Hobbies $0 $0

Education

Education $0 $0

Miscellaneous

Contributions $0 $0

Cash or Gifts $0 $0

TOTAL EXPENDITURES $0 $0 $0

Expenditures
34

Category

Reported

Income

Updated

Income

Wages $0 $0

Self-Employment $0 $0

Interest/Dividends $0 $0

Social Security $0 $0

Supplemental Security $0 $0

Public Assistance $0 $0

Pensions $0 $0

Veteran's Payments/

Alimony/ Child Support $0 $0

Other $0 $0

TOTAL INCOME $0 $0 $0

Category

Reported

Asset

Updated

Asset

Checking Account(s) $0 $0

Savings Account(s) $0 $0

Retirement Account(s) $0 $0

TOTAL ASSETS $0 $0 $0

Category

Reported

Liability

Updated

Liability

Credit Card(s) $0 $0

Personal Loan(s) $0 $0

TOTAL LIABILITIES $0 $0 $0

Liabilities

Income

Assets

PROBES FOR ADDITIONAL EXPENDITURES AND INCOME 

Expenses

1. Last month, did you have any expenses for personal services (including cost of care for 

elderly and/or children)? 

2. ... other child-related expenses not already mentioned (including toys, gear, and 

equipment)? 

3. ... pay income tax as part of filing your tax return? 

4. ... purchase any vehicles other than a car or truck? 

5. ... purchase any appliances other than a refrigerator, washer/dryer, or dishwasher? 

6. ... any home electronics other than a television or computer? 

7. ... make any other purchases greater than $50 that I have not asked you about? 

Income

1. Last month, did you receive a tax refund? 

2. ... receive a pay advance? 

3. ... receive an inheritance? 

4. ... earn income from the winnings from lotteries, bingos, or casinos? 

5. ... earn income from the sale of a house, condo, or piece of land? 

6. ... earn income from the sale of a car or any other vehicle? 

7. ... earn income from the sale of any furniture or appliances? 
35

8. ... earn income from the sale stocks, bonds, or mutual funds? 

9. ... earn income from gifts? 

10. ... have any other source of income we haven't already asked about? 
36

APPENDIX III

CE Balance Edit Study Debriefing Protocol 

INTERVIEWER SCRIPT AND OUTLINE 

1. Introductions (5 minutes) 

Hi! Thank you for coming in today. 

I am ...... 

[This is my colleague ( ) who will be taking notes for us today] 

Consent form 

Permission to audiotape 

Explanation: 

We are going to be working with some possible new questions for the Consumer 

Expenditure Survey, an ongoing survey that provides a continuous flow of 

information about the buying habits of American consumers. The survey furnishes 

data to the Consumer Price Index or CPI, which is a basic measure of inflation. 

What we are going to do today is go through a series of questions just as if you are 

participating in the actual survey. Once we have completed all of the questions in 

the survey, I will get your feedback to them as well as your thoughts on the overall 

experience participating in the survey. The purpose of today’s session is to help us 

find out more about how people, like you, respond to these questions. We are not 

here to evaluate you, we are looking to improve the questions, so there are no right 

or wrong answers. We are primarily interested in how effective the questions are; 

how easy they are to understand and answer and how well they can capture the 

information that we are interested in collecting more than the information about 

your expenses. All the information you give us will be kept completely confidential, 

and will be used to improve the survey questions. 

Any questions? 

What I’m going to do is read you a question and ask for your answer. Once I have 

asked all of the questions I will ask you 

Let me know at any point if you have a question or comment, or if you need to take 

a break. 

INTRODUCTION TO DEBRIEFING QUESTIONS 

Thank you for completing the survey. We’re going to shift gears now and ask you a few 

questions about your experience in today’s session. We realize that some of the items you 

were asked to answer may be challenging for some people, so we’re hoping to get a better 

understanding of how you answered the questions, and to get any reactions you had to the 

survey. Your responses will be confidential and will be used to improve the quality of the 

Consumer Expenditure Survey. 
37

PARTICIPANT DEBRIEFING QUESTIONNAIRE

1. How comfortable or uncomfortable were you giving information about your expenditures, 

income, assets, and liabilities? Please circle one answer per row.

Very 

Uncomfortable

Somewhat

Uncomfortable

Neither 

Comfortable nor 

Uncomfortable

Somewhat

Comfortable

Very 

Comfortable

Expenditures 1 2 3 4 5

Income 1 2 3 4 5

Assets (e.g., checking 

or savings accounts) 1 2 3 4 5

Liabilities (e.g., credit 

cards or loans) 1 2 3 4 5

1a. If less than five, what could we have done to make you more comfortable answering 

those questions? 

2. How accurate or inaccurate do you think you were able to report your expenditures, income, 

assets, and liabilities? Please circle one answer per row. 

Very 

Inaccurate

Somewhat

Inaccurate

Neither 

Accurate nor 

Inaccurate

Somewhat

Accurate

Very 

Accurate

Expenditures 1 2 3 4 5

Income 1 2 3 4 5

Assets (e.g., checking 

or savings accounts) 1 2 3 4 5

Liabilities (e.g., credit 

cards or loans) 1 2 3 4 5

2a. If less than five, what could we have done to help you report the information more accurately?

3. Were there any items that you felt were particularly easy to report accurately? 

4. Were there any items that you felt were particularly difficult to report accurately? 

5. If you had to describe how burdensome the interview was, what would you say? 

5a. Could you tell me what the term burden means to you? 

6. What do you think about the length of the survey? 
38

6a. How long do you think the survey lasted? 

6b. How long did you expect it to be? 

DEBRIEFING QUESTIONS

1. General Reactions: Thank you. I‟ve got a few additional questions that I‟d like to 

discuss with you now. Let‟s start by getting your general reactions to the survey. What 

was it like for you to complete the survey? 

a. [Additional probes if necessary: were there questions that you found particularly 

difficult or confusing? Did the questionnaire seem to flow smoothly or not? Were 

there any items that you felt were too sensitive or personal?]

2. Comfort: review responses to question 1 and 1a on self-administered questionnaire 

a. Probe about any items that the respondent rated low and get the respondent to 

expand on their answers. 

b. If answers varied among the sections, what made specific sections more or less 

comfortable? 

3. Accuracy: review responses to question 2 and 2a on self-administered questionnaire 

a. Probe about any items that the respondent rated low and get the respondent to 

expand on their answers. 

b. If answers varied among the sections, what made specific sections more or less 

accurate? 

c. How exact did you feel your answers needed to be? 

a. Do you think you estimated for any of your answers? 

i. Which? 

ii. Why? 

b. Felt answers needed to be exact – based on what? 

c. How accurate did you feel your answers were overall? 

4. Information Booklet: Did you think it was helpful to have the information booklet? 

5. Reference Period: You were asked to tell us about your household income and expenses 

for a 1 month period. 

a. If you were asked to report your household expenses for a 3 month period 

instead, do you think it would be easier, the same, or more difficult? 

i. Can you tell us more about that? How would you go about coming up 

with your total household expenses for 3 months? [Additional probes, as 

necessary: For example, would you think about the average monthly 

amount and multiply by 3, or would you think about each month 

separately and report based on each specific month?) 

b. How about household income - if you were asked to report your household 

income for a 3 month period instead, do you think it would be easier, the same, 

or more difficult? 

i. Can you tell us more about that? What would make it [fill]? 
39

c. And how about household assets and liabilities (such as bank accounts, credit 

card balances, and loans) - if you were asked to report your household assets and 

liabilities for a 3 month period instead, do you think it would be easier, the same, 

or more difficult? 

i. Can you tell us more about that? What would make it [fill]? 

[Additional probe, as necessary: Over a 3 month period people clearly purchase more 

things than they do in just a single month. But are there specific types of purchases or 

expenses that you can think of that we might be missing by asking about just one month? 

6. Proxy Issues Reporting Expenditures [for Multi-person CUs]: At the start of the 

interview, we asked you a set of expenditure questions that began “Last month, did you 

or anyone in your household purchase....” – for example, a car, a refrigerator, etc. – and 

we went through about 30 expenditure categories. You indicated earlier [in the selfadministered questionnaire, or in response to a previous verbal debriefing question] that 

you felt that your answer to these question were [very/somewhat/not very/not at all] 

accurate 

a. Do you feel that you have a good sense of what people in your household buy 

and how much they spend? 

i. If not/so, why not/tell me more about that – why/why not? 

b. Are there particular types of expenditures that you‟d be likely to be unaware of, 

or are there particular household members whose expenditures you may not 

know as much about? 

c. Do you think the answers to any of the questions would have been different if 

someone else in your household answered the interview questions? 

7. Proxy Issues Reporting Income [for Multi-person CUs]: In the survey, we asked you a 

series of income-related questions, such as “Last month, did you or any members of your 

household earn any income from wages, salary, commissions, bonuses, or tips (do not 

include income from self-employment)?” And then we asked you how much your HH 

earned from these sources. You indicated earlier [in the self-administered questionnaire, 

or in response to a previous verbal debriefing question] that you felt that your answer to 

these question were [very/somewhat/not very/not at all] accurate 

d. Do you feel that you have a good sense of the earnings of other people in your 

household? 

i. If not/so, why not/tell me more about that. 

e. Do you think the answers to any of the questions would have been different if 

someone else in your household answered the interview questions? 

f. Did you think it would need to consult with other people in your household about 

their income before responding to this survey in order to give a reasonably 

accurate answer to the income questions? 

i. Why or why not? 
40

8. Proxy Issues Reporting Assets and Liabilities [for multi-person CUs]: Okay, we also 

asked you to report your HH‟s assets (such as savings and retirement accounts) and 

liabilities (such as credit card debt and other loan balances). 

a. Do you feel that you have a good sense of the earnings of other people in your 

household? 

ii. If not/so, why not/tell me more about that. 

g. Do you think the answers to any of the questions would have been different if 

someone else in your household answered the interview questions? 

h. Did you think it would need to consult with other people in your household about 

their assets and/or liablities before responding to this survey in order to give a 

reasonably accurate answer to the assets and liabilities questions? 

i. Why or why not? 

9. Reconciliation 

a. At one point in the interview, we generated that chart that showed you your/your 

HH‟s expenses for last month and your income and assets/liabilities next to each 

other [show chart from tab 5 again]. 

i. What was your initial reaction to this graph? 

ii. Did it make sense to you? Was it easy or difficult to understand? 

(How so/why?) 

iii. Could we have done anything else to make it clearer/better? 

b. [For Rs who were out of balance] When I showed you that graph, I read the 

following statement, and I‟d like to get your reaction to it. I said, “Ideally, we 

expect to see the two figures (expenditures and income/assets) match up pretty 

closely. However, sometimes when there is a big difference between these two 

amounts in a given month, it's because we (missed some of your HH's 

expenditures/income) or need to make other changes to bring these in line.”

i. Could you tell me in your own words what you think we are 

trying to say here? 

ii. What is your reaction to this statement? 

iii. Did it make sense to you? Was it easy or difficult to understand? 

(How so/why?) 

iv. Could we have done anything else to make it clearer? 

c. [For R who were out of balance] After we showed you the graph, we presented 

this summary screen [show tab 6] and asked you to review your answers and 

make any changes you felt were appropriate. 

i. Did you have any reactions when we showed you this summary 

table? 

ii. In your own words, could you tell me what you think we were 

asking you to do on this screen? 

iii. [For R who had range edits flagged] One/some of the values 

listed in the table was/were highlighted and I told you that the 

instrument had flagged that/those amount(s) as being potentially 

too high or too low – and then I asked you to please verify that 

it/they was/were correct?

1. What were any reactions to this part of the interview? 
41

2. [For any changes that were made] Can you tell me a bit 

more about how you decided what to change and what 

not to change, and how you arrived at the revised 

amount(s)? 

a. Was this easy or difficult for you, and why? 

d. [For R who were out of balance and made changes on tab 6] What about the 

remainder of the survey? After you made the changes to these items [on tab 6], 

we showed you an updated graph [show tab 8]. 

i. Again, what were your reactions to this graph? (e.g., was it what 

you expected to see or not? Was it helpful to see this revised 

graph or not?) 

e. [For R who were asked additional expenditure/income/assets/liability questions] 

We then tried to get your income and expenditures reports a bit closer by asking 

you a few follow questions. For example, we asked you [Insert example from 

R‟s survey]. 

i. What did you think of these questions? 

1.Were they easy or difficult? Why? 

2.Any other reactions to these follow-up questions? 

10. Do you or anyone in your household keep a budget or records of household income 

and/or expenses? 

Yes Go to 10a

No Go to 11

10a. How well do you think that budget matches your overall finances? 

10b. Are there items (of expenditures or income) we asked about today that you do not 

include in your regular budget? What are those items? 

10c. Are there items (of expenditures or income) we did not ask about today that you do

include in your regular budget? What are those items? 

11. Use of records 

a. On a scale from 1-5 with1 being very unlikely and 5 being very likely, how likely 

is it that you‟d use records to help answer the questions in this survey? 

i. If not „very likely,‟ why not? 

b. [For those who indicated some likely use of records:] If we had conducted this 

interview in your home AND you had wanted to use the records you had 

available 

c. How much more effort would it take for you to gather and use the records (5 pt 

scale)? 

d. How would your use of records affect the accuracy of your responses (5-pt 

scale)? (What types of items might it affect most? – i.e., getting at intersection of 

their perceptions of accuracy for various items and their knowledge of specific 

records ?) 
42

12. If you had been given the choice to complete the survey over the phone or in person, 

which would you choose? Why? 

13. Do you have any other comments or feedback about the interview?