1

2004 Consumer Expenditure Survey 

Field Representative Focus Groups 

Final Report 

September 2004 

Jennifer Shields 

Office of Survey Methods Research 
2

Table of Contents 

FR QUESTIONNAIRE............................................................................................................................ 3

CAPI............................................................................................................................................................. 9

UNDERREPORTING............................................................................................................................ 20

DATA QUALITY ................................................................................................................................... 30

CHI.............................................................................................................................................................. 34

TRAINING ............................................................................................................................................... 35

RECEIPTS................................................................................................................................................. 41

MISCELLANOUS.................................................................................................................................. 44
FR Questionnaire 

3 

Results of 2004 FR Focus Groups 

Over the course of six days in June 2004, twenty-four Field Representatives were 

brought to Census Headquarters. They participated in a series of focus groups about specific 

topics, completed activities, and provided feedback on various CAPI features. The topics 

covered included: the FR Questionnaire from 2004 CEQ Refresher training, CAPI, 

Underreporting, Data Quality, CHI, Training, and Receipts. Each of the discussions about 

each topic was analyzed independently, and summary reports were produced for each. 

FR QUESTIONNAIRE

 After introductions, the first topic covered during the FR focus groups was a review of 

the FR Questionnaire collected during 2004 CED Refresher Training. A select number of 

questions were selected; the results tabulated and reviewed with the focus groups. 

Most difficult features of CAPI

 One of the first questions on the FR questionnaire asked FRs to rate CAPI features in 

terms of their difficulty. 

CAPI Pop-Up Error Messages 

CAPI pop-ups were rated by many FRs as one of the most difficult features of CAPI. 

When asked to explain this result, the focus groups unanimously said that pop-ups are not 

difficult; rather they are slow and time consuming. In each group, FRs pointed out that they 

are always trying to speed up the interview and pop-ups slow it: “when you are in an 

interview, you are on a roll and it stops you....you lose your train of thought and the attention 

of the respondent.” 

 After additional discussion, FRs also pointed out that the pop-ups are repetitive. With 

continuous expenses or in extremely large, poor or wealthy households, FRs have to suppress 

many pop-ups for the same types of expenditures. A FR from New Jersey pointed out that 
FR Questionnaire 

4 

typical prices in area often exceed the predetermined „normal‟ ranges, and she gets very many 

error messages per interviews. Other FRs agreed, saying that the ranges of the error messages 

is too sensitive and should be reviewed. One said that having to verify each „extreme‟ value is 

frustrating for both the FR and the respondent: “makes the respondent feel like a liar.” to 

 Most of the discussion referred to soft error messages, but a few FRs had comments 

about hard errors. One said that she is sometimes trapped by them “it will give you two 

choices on what put in....and neither will work and you‟re trapped!” Another noted that this 

happens often in the construction section, it‟s difficult to move past it even if the input is 

correct. 

 FRs were very eager to offer suggestions to improve the problems with the error 

messages. One suggested including an item level note on the error dialog box itself. This 

would reduce the number of keystrokes or mouse clicks required to deal with an error, and 

would increase the number of notes entered. Another suggested having a way to return to all 

the places an error was suppressed after the interview was concluded. This would allow FRs to 

return to each place an error message was encountered and either enter a note or change the 

entry. 

Navigating on a single screen with multiple questions 

Another CAPI feature that was rated as difficult by FRs on the questionnaire was navigating 

on a single screen with multiple questions. When asked about this, FRs again were unsure 

why this was rated as difficult; they all agreed that this was not a problem. Most went so far as 

to say that they preferred having many questions on a screen as it saves time. After probing, 

they were only able to come up with a few reasons this feature may have been rated as 

difficult. These reasons included: the code to move to the next screen changes from 95 to 99 

depending on the question, and when FRs use the mouse-pad to navigate, the cursor can move 

accidentally to the first question. Other than these, FRs though of no reasons why navigating 

on a single screen with multiple questions is difficult. 
FR Questionnaire 

5 

Backing up and changing answers in a previous section 

 The third CAPI feature that was rated as difficult on the FR questionnaire was backing 

up and changing answers in a previous section. This was the first result discussed that the FRs 

agreed with. They noted that you have to be very familiar with the sections and what 

expenditures go into each of them to be able to successfully jump between sections. One FR 

suggested “there should be a place to put it [the forgotten/incorrect expenditure] and be able to 

enter it later, after you‟ve left the respondent.” Many FRs agreed, saying that to go back to add 

or change an answer is a slow process, and asking the respondent to wait during it is 

frustrating. One FR admitted that she does not always jump back to enter a forgotten expense, 

because it takes too long. A SFR agreed: “I have some interviewers who won‟t take the 

miscellaneous thing at the end because it is too time consuming.” 

 They had some questions for BLS regarding this issue: 

- Why can‟t incorrect items be overwritten instead of deleted (e.g. changing 2 to 

3)? 

- If FRs put forgotten items in notes, would BLS get this information? 

- Are FRs allowed to write expenditures on paper and enter it into the 

instrument after the interview is completed? 

They also had some suggestions to make this CAPI feature easier to use. One group 

discussed the idea of having an option at the end of the interview which would leave the case 

open, allowing them to return and add or change items. This would also allow them to call the 

respondent back if there was a question about an item after the interview. Another group 

suggested adding a „catch all‟ screen that could capture expenses that the respondent reported 

after the interview was completed. This data could then be matched back to the appropriate 

section, without making the FR or the respondent take the time to jump back. 

CAPI‟s impact on data quality

 Another question on the FR questionnaire asked FRs to rate the impact CAPI had on 

data quality. The majority of FRs gave this a neutral rating. During the discussion, FRs gave 
FR Questionnaire 

6 

both sides; giving reasons why CAPI may have had both a positive and a negative on data 

quality. The reasons named for why CAPI improved data quality were: 

- It‟s easier to move through skip patterns 

- Pre-programmed edits and error message 

- use of PC to collect data more attractive to respondents 

- We look a whole lot more professional walking up with PC 

- Harder to take short cuts („cheat‟) than on paper. 

- Reasons named for why CAPI decreased data quality included: 

- Senior respondents are intimidated by the computer 

- outdoor conditions not always conducive to CAPI 

- strict date enforcement on CED, lose cases would have gotten on PAPI 

- One of the SFRs feels the FRs are taking more short cuts in CAPI by simply not 

collecting every item. 

- Have to choose something, even if it‟s not right (i.e. misc. clothing) 

- Having lists (make, model of car) reduced data quality; if the R doesn‟t know, or says 

something not on the list, you have to choose a wrong answer. 

One FR felt that CAPI had no impact on data quality, simply saying, “good interviews take 

longer." It‟s time, not CAPI or PAPI, which dictates data quality. The overall reactions of the 

groups were mixed, with most FRs giving reasons for CAPI both improving and decreasing 

data quality. 

Most Difficult Sections

 Another question from the FR questionnaire that was discussed in the focus group 

was the rating of the most difficult sections in the CEQ. The top three sections were 

reviewed with the focus groups, and comments and suggestions were collected. 

Utilities 

 FRs were not at all surprised that utilities was the section most often rated as most 

difficult. They were able to easily name reasons why this was so: 
FR Questionnaire 

7 

- Many respondents are on budget billing so an expenditure for each month doesn‟t 

need to be collected. 

- Have to spend so much time navigating through that when it could be put on a 

spreadsheet 

- You have put in the name of the utility company every time and that takes a while 

- The placement of the section is bad because you just get the interview started and 

you are bogged down. It starts the interview off badly. 

- The number of times you have to ask about business expenses 

- Utilities takes too long to collect 

- The hesitation by the computer between fields is too long 

- You have to go utility by utility instead of month by month. This makes the task 

harder for the respondent, and takes longer. 

- Not all the comments FRs had about the utilities section were negative. They had 

many suggestions on how it can be improved: 

- Prefill the electric company, and just ask the respondent “last time your electric 

company was ___. Is it still the same?” 

- Have the instrument verify 3 month bills, instead of asking for the time period 

each time

- Have a catch all business question for all utilities 

- Customize lists of utility companies by region and have list 

- Use questions that have yes or no responses to minimize typing 

- Allow for continuous expenses, or a budget plan option 

- Allow data to carry forward as a default, or have a repeat key 

- Shorten pause between questions 

- Prefill company name 

They felt that there was a lot of potential for improvement in the utility section, and seemed 

excited by the idea of making it faster and easier to use. 
FR Questionnaire 

8 

Clothing 

The section that was rated the second most difficult was clothing. Again, FRs were not 

surprised by this result. Unlike utilities, most of the explanations FRs gave for this result were 

focused on the respondent, rather than the instrument or the FR. All groups agreed that 

clothing is a difficult section because it is a very hard type of expenditure for a respondent to 

recall. They were able to name many specific instances where respondents have difficulty with 

this section: “especially if they have kids, when its time t go back to school”, and “if there are 

more than two people in the household, they can‟t name every item.” 

FRs also said that in addition to recall issues, the clothing section takes a long time to 

complete “you can spend half an hour on just clothes.” They said that having to break down 

clothing purchases by month takes a long time. FRs said that requiring two lines of data for 

two pairs of pants purchased in different months is very time consuming. 

As with other features, FRs had suggestions to improve the clothing section. They 

suggested that the month breakdown be eliminated. They also noted that diapers should have a 

continuous option, since CUs with this expense have it regularly. They suggested that the 

reference period be shortened, since respondents can‟t recall three months of clothing 

purchases. 

In addition to suggestions, FRs had several questions about clothing for BLS: 

- What quantity information is required, number of packages or number of pieces 

- If a receipt shows 1 item "diapers - $9", how many items should be recorded as 

purchased? 

- Is it important to CPI to get socks versus pants, or if a total clothing amount would 

work. 

FRs pointed out that it‟s important to get their questions answered because they use such 

information to convince respondents to take the time to provide the required information: 

“if we had some ammunition to convince them that these answers are important. But we

don‟t know what the value of it is!” 
FR Questionnaire 

9 

Trips and Vacations 

 The final section rated as difficult on the FR questionnaire was trips and 

vacations. FRs initially were surprised, noting how much easier the section is on CAPI, 

“the skip pattern is there, so I don‟t have to figure out where to go next.” After additional 

thought, FRs said that, like clothing, the trips section is difficult because respondents 

have a difficult time remembering all the associated expenses. They also pointed out that 

the questions are confusing in this section, “you have to repeat the questions to the 

respondent. They are too wordy and repetitive and the respondent doesn‟t always 

understand what you‟re talking about.” As with most things in the CEQ, FRs also felt 

this section was too long. 

 There was only one specific suggestion for this section, in addition to the general 

one of shortening the questions. The suggestion was to change the wording of 

“convenience stores.‟ FRs felt that this term caused respondents to miss a lot of 

miscellaneous expenditures at gas stations, which could add up to a large amount for 

some trips. 

 In conclusion, the focus groups provided some interesting insight about the 

answers given to the FR questionnaire in the spring. This type of follow up activity 

allows respondents to give anecdotal evidence for why they chose specific answers. It 

also allows them to participate in discussions with other respondents and bounce ideas off 

each other. FRs were able to think about the results found from the survey, and provide 

additional information about the findings. 

CAPI

In June 2004, three focus groups were conducted with CE FRs. One of the topics 

discussed during the day and a half sessions was CAPI. The four topics focused on were 

jumping, intro screens, notes and deleting. Many suggestions for improvement came out of 

this session, as did much interesting information. Observer notes and transcripts were 
CAPI 

10

combined and analyzed. Main themes are described below and specific suggestions are 

included. 

Jumping

Why They Jump 

During the CAPI portion of the focus groups, FRs discussed jumping between 

sections. All FRs agreed that the respondent is the cause for most jumping behavior, that very 

few jumps are FR initiated. Reasons that respondents cause FRs to jump include: 

- Remembering something from a previous section 

- Having a list or receipt and wanting to go in order 

- Other CU members are needed, or arrive 

- Remember details and want to make sure answer was correct 

Only two FR reasons for jumping were brought up: with certain respondents FRs might skip 

the utilities section because it is too time consuming, and if the respondent is reluctant, FRs 

might jump around the interview to collect the most important information. 

How often they Jump 

 FRs referred to respondents again when asked how often they find themselves 

jumping. They said it varies greatly by respondent, but that generally jumping is fairly rare. 

One FR noted that some respondents make you jump around a lot, especially those with large 

families or more hectic households that rely on other CU members for additional information. 

Why they DON’T Jump 

When discussing jumping, a non-anticipated topic arose in all three groups – why FRs 

do not jump. They were able to identify a lengthy list of reasons that they don‟t jump: 
CAPI 

11

- Are used to going in order, if jumped wouldn‟t know where they were 

- Because respondents are using the information book 

- It‟s too difficult to skip 

- It‟s difficult to remember what sections items go in 

- Too time consuming 

- The programmed order is how the interview is designed to be done 

- Don‟t know how to get back to main path 

- Not confident enough to jump, don‟t want to get stuck in a loop 

These reasons are consistent with the comments indicating that FRs do not often jump within 

an interview. 

Where They Jump 

 When asked where jumping occurred most often, most FRs did not have an easy 

answer for this question. FRs reiterated that jumping depends on respondent behavior, which 

varies greatly from respondent to respondent, and is difficult to generalize. Of the FRs that 

were able to answer this question, these sections were named, for given reasons: 

- Cars, respondents may have to consult with other CU members 

- Clothing, respondents may not know expenses for whole CU 

- Household appliances, often remembered after the fact 

- Medical expenses, often remembered after the fact 

- Income, “get it before I lose them” 

Across groups, there was no one section identified as a consistent jumping point. FRs insisted 

jumping is caused by respondent, and can occur for many reasons at many places. 

Using Transitions when Jumping 

Since jumping causes the FR to administer the interview out of the designed order, for 

whatever reason, they were asked how they handle the transitions between sections. FRs 

varied greatly on this topic, some said they always read the transition introduction (i.e. “now 
CAPI 

12

I‟m going to ask you some questions about...”) statements verbatim, others said that it 

depended on the situation. After a bit of discussion, FRs had mentioned a few situations when 

they do not do this: 

- If jumping within a section 

- If not going to do a whole section 

- If doing recall or adding something to a section 

- If jumping backwards 

- If respondent is using information booklet as an aid 

- If returning to main path after jump 

No consensus across FRs was drawn about the use of transition statements when jumping, an 

as with other jumping issues, FRs agreed that often it depends on the respondent. 

How they Jump 

 Most FRs said that they used the tabs at the top of their screen to jump between 

sections, while only a few cited F4 as their preferred method. One of the FRs who said she 

used tabs, said that for situations when she doesn‟t know the section number for a specific 

purchase, she‟ll use F4, while another uses her mouse and waits for the section label to appear 

in that situation. Two FRs noted that they only used the tabs because they didn‟t realize there 

was any other method. 

Jumping Problems 

 The majority of problems with jumping overlapped with the reasons FRs did not 

jump. The consistent theme across all groups and problems was that FRs were not familiar or 

comfortable enough jumping. They were afraid they would get stuck in a loop, or would not 

be able to get back to where they jumped from. One FR summed it up very well “Being able 

to jump is a great feature, but you need to be very familiar with it to jump successfully.” 
CAPI 

13

Suggestions for Jumping 

 FRs had only a few suggestions to improve jumping. One simple one was to put F4 on 

the template so it was easy to remember. Another was to make the tabs smaller so they all fit 

on one screen and scrolling wasn‟t necessary. Finally, one FR suggesting adding a „catch all‟ 

screen where forgotten items could be collected, and jumping wouldn‟t be necessary. 

Intro Screens

Transition Statements 

FRs were asked about the three purposes of intro screens, to provide a transition from 

one section to the next, to provide space to display inventoried data in inventoried sections, 

and to provide space for all “pre-chart” information for the section from the last interview. 

Negative Feedback: Regarding the transition statements on the intro screens, most FRs 

had negative comments. Most criticism focused on the repetitive nature of the statements, FRs 

felt that the transition and the first question had the same purpose and the transitions are 

unnecessary. One FR commented “They‟re repetitive. First we say „now we are asking about 

vehicle rental.‟ Then we ask „have you had any expenses for vehicle rental.‟ It‟s just not 

necessary.” Another said she found the intro question embarrassing: “it‟s like me saying, in a 

minute I‟m going to ask you if you want to go to lunch. Then I wait a minute and say „do you 

want to go to lunch.‟ 

Other criticism varied: FRs felt that respondents are smart, they catch on and don‟t 

need the lead in; the transitions are too time consuming and “the ones that serve a purpose are 

too lengthy;” seasoned FRs don‟t need them, they know what is coming next. 
CAPI 

14

Positive Feedback: Some FRs did have positive feedback on the transition statements. 

These included: 

- The survey is complicated, transitions are important 

- They show you where you are in the survey 

- Helpful for new FRs 

- Lead-in to owned/rented properties makes sense because of order of sections. 

These comments showed that FRs understood the advantages of having transition statements 

on the intro screens. 

Reading Verbatim 

 When asked if FRs read the transitions verbatim, FRs began the discussion by 

reporting that they most often did (Note: focus groups were conducted and observed by 

Census and BLS staff, which may have biased results). However, once an FR mentioned that 

she didn‟t read the statements verbatim when a respondent was hostile or impatient, others 

chimed in with reasons they don‟t, which were the same as the negative feedback about them. 

One FR commented that the transition statements “make you sound stupid” because they end 

up being repetitive with the following question. 

Suggestions 

 As always, FRs were asked if they had any suggestions on how to improve the 

transition statements. Many FRs said that the transitions statements could be removed from 

the interview completely or at least from the second through fifth interviews. They also 

suggested: 

- Removing transition statement from some sections where not necessary 

- Placing one sentence transition on same page as question 

- Abbreviate transition statements 

- Make them into a title: „the next section is‟ 

- Making the lead in match the first question for baby clothes and diapers 
CAPI 

15

- Include jewelry or baby accessory expenditures in transitions 

- Improve the transition for section 9 

One FR summarized many of the comments on transition statements by saying “if they are 

going to shorten the interview, this is where to do it.” 

Inventoried Data

 The overall feedback about the intro screens containing inventoried data was neutral. 

Most FRs weren‟t able to distinguish between them and precharts, but indicated they liked 

having both. One FR wanted to be able to change information on the screen, particularly 

insurance company; while another said that having the term „sample unit‟ in the inventoried 

data wasn‟t desirable. When discussing the design of these data, one FR said “I think it‟s 

great” and others agreed it is well designed. Asked specifically about increasing the amount of 

data shown on these screens, FRs indicated that although they would like as much data as 

possible available to them, they would prefer not to scroll. One FR commented “I would 

rather see the earlier months dropped if you can‟t fit everything,” and most in the group 

agreed. 

Precharts 

 After demonstrating the difference between intro screens containing prechart 

information and those with inventoried data, FRs commented on precharts. The overall 

attitude towards them was very positive; they recognize the importance of not duplicating data 

and feel that precharts allow them to avoid this. They also noted that precharts help them 

know what a respondent may be forgetting, by knowing what bills they‟ve had in the past, 

they can tell what are missing. FR conversation then moved to the global prechart, noting that 

having the relevant information on the intro screens “just saves you time of having to go 

back.” It should be noted however, that there was at least one FR who didn‟t know how to get 

to the global prechart. 

 FRs were asked if inconsistency would be ok, if they would mind having some 

precharts eliminated from some sections. The groups were unanimous in their positive 
CAPI 

16

response – they wouldn‟t mind inconsistency. FRs noted that precharts are not necessary in all 

sections, that eliminating them might save time, and any redundancy in the instrument should 

be removed. 

 As with all other topics, FRs were asked if they had any suggestions for improvement 

to precharts. The suggestions were: 

- Show what utilities are combined instead of just showing “C” 

- Have more precharts 

o Groceries 

o Rental equivalence 

- Have precharts available on following screen 

Overall, FRs were positive about the precharts, and agreed that they were well designed and 

useful. 

Notes

When they Write Notes 

An initial discussion on notes revealed differences between FRs. Some indicated they 

always write notes, and some never do. All FRs were asked to clarify when they did and did 

not write notes. However, upon further discussion, almost all FRs were able to name situations 

they wrote notes, including: 

- when error requires clarification

- if there is a problem getting out of an item

- when the instrument can‟t handle the situation (i.e. employee benefits package)

- when forced to choose an option that doesn‟t fit well (i.e. when repairs are done on 

a rental unit and not reimbursed)

- when adding insurance company name

- when need to change an answer but the instrument blocks you

- when a situation requires clarification and an error message doesn‟t occur
CAPI 

17

- When there‟s a discrepancy or a problem. 

One FR noted that during refresher training she was told to write a note every time 

there was an error, another said “we were told BLS loves notes so I just clarify and put a lot”. 

Other FRs do not remember learning this during training; “I don‟t do that, my FRs don‟t do 

that, I wasn‟t trained to that.” 

Notes versus Descriptions 

 When discussion the difference between putting additional information in a note on in 

the description field, FRs consistently said it‟s easier to write in the description. Another 

advantage of putting information in the description is that FRs get that information back, while 

they do not get to see their item level notes again. Even though they noted that space is 

limited in the description field, and they sometimes have to abbreviate, they consistently said 

they use description field in lieu of notes when possible. 

Case versus Item Level Notes 

 The difference between case level and item level notes was described by FRs. They 

said that case level notes are used only when the information would be useful for the next 

interview. Information such as sleeping patterns and personal information that should be 

remembered are included in case level notes. Item level notes are used when the information is 

related to a specific expenditure, not the respondent. Things such as „amount verified with 

respondent‟ and further explanation about unusual expenses are included as item level notes. 

Although FRs were relatively clear about the difference between the two types of notes, there 

was confusion about which types of notes are useful to BLS. “We need to know what they 

want” one FR said, during a discussion about how much detail they should be including in 

their notes. 

Problems with Notes 

 The most commonly cited problem with entering notes is that once a case is closed, 

FRs are locked out and cannot enter any additional notes. They said they find themselves in a 
CAPI 

18

situation, often with a reluctant respondent, where they‟re rushing to finish the interview, and 

don‟t add notes, but would like to once they‟re out of the house and have more time. They 

also mentioned the situation where respondents remember purchases once the interview is 

finished, without being able to get back into the case, this is data that is lost. They also 

commented that writing notes further slows down the interview, especially given the number 

of keystrokes required to do so. One group discussed the uncertainty of having to back up to 

put an item level note with the field that caused the problem, or if it could appear anywhere 

with the item. 

Suggestions for Notes 

 FRs suggested that there should be a way to write a note on the error popup screen, 

with FRs agreeing with the statement given by one: “then I would do it more consistently.” 

FRs also suggested that they “should be able to go back into a case until it‟s transmitted” to 

add notes. 

Deleting

Why they Delete 

When asked why they delete, FRs gave two categories of response – FR error and 

respondent error. Included in FR errors were: 

- Typos 

- Invalid value 

- Accidentally made wrong keystroke 

- System is so slow, get ahead of it when trying to keep up with respondent 

For respondent errors, FRs included: 

- Respondent realizes purchase is before a reference period 

- Report a purchase and then realize it was a gift 

- Paid combined bill but named them all 

- Respondent made mistake. 
CAPI 

19

Most deletions occur when a mistake has been made, either by the FR or the respondent, and 

no other reasons were discussed during any of the focus group sessions. 

How Often they Delete 

 FRs seemed to be broken into two groups when asked how often they delete. One 

group said that it rarely happens “not often, one time out of twenty-five interviews”, while the 

other said it happens often “two to three times in each case.” A discussion about the frequency 

of deletion revealed no insight as to the reason for this difference, nor were the groups able to 

arrive at a consensus. 

Where they Delete 

When asked where deleting occurs most often, no single section jumped out to FRs. 

The initial response to the question was consistent with many others: “It depends on the 

respondent.” When probed further, FRs named utilities, clothing and vehicles as sections 

where they may delete more often. 

 Suggestions for Deleting 

 FRs had many suggestions to improve the deleting process. Suggestions included: 

- Be able to delete wherever you are, don‟t have to go to beginning of item 

- Be able to delete a single field 

- Be able to delete an entire vehicle 

- Put an “X” on the left, highlight it and it deletes the whole line – just like payroll 

- Have a single keystroke to delete a whole line 

- Delete a single bill, or be able to change the number of bills 

While FRs said that deleting generally wasn‟t difficult, some said that because they don‟t 

delete often, it is sometimes difficult to remember how to do so. 
CAPI 

20

General Suggestions

As anticipated, when the topic of CAPI issues was brought up, FRs had suggestions on how to 

improve the instrument. While none of them related to the topics focused on, they are useful 

suggestions and should be considered: 

- Improve section 4 

- Make more like a spreadsheet 

- Make business expense a main pre-question 

- Lump months together under one bill 

- Make months radio buttons and mark all that apply 

- Prefill utility company 

- Make diapers a continuous expense 

- Add an intro screen at the end of the interview “We‟ll be coming back in three 

months, can we set up an appointment now?” 

- Make range edits more realistic 

- Vary ranges based on income, locality 

UNDERREPORTING

When asked to discuss the underreporting problem in CE, FRs had a lot to say. They 

identified possible respondent, situational and instrument causes for underreporting, as well as 

reviewing the relationship between underreporting and CE materials. The difference between 

phone and personal interviews was also discussed during this session. 

Materials

Without prompting, FRs discussed the info book and file folder during the 

underreporting session. They felt that both these materials related to underreporting, and had 

interesting comments about each. 
Underreporting 

21

Info boo 

In all groups, the information booklet was a main topic. One FR went so far as to say 

“the number one reason respondents underreport is they don‟t want to take the time to look at 

the info book.” Other FRs agreed that the info book played an important role in the amount of 

expenditures reported. They noted that using the info book can speed up an interview, and it 

helps respondents associate items with the correct question. Not all comments about the info 

book were positive however, FRs reported some problems with it: 

- Not sure if respondents pay attention to the books 

- Respondents sometimes get lost in the books 

- Reluctant respondents find the books intimidating 

- Some respondents are illiterate and can‟t use the books 

- Some respondents scan pages and move too quickly. 

They also noted that not all FRs correctly coordinate the use of the information book, page 

numbers are now always specified, and the respondent is not always in the right place. 

File Folder 

In addition to the info book, FRs in two groups brought up the file folder given to 

respondents to store their records. They said that few respondents use this folder, although 

those that do tend to pack it full, “they want to show you every little thing.” The consensus 

was that respondents may appreciate the gift of the folder, but typically do not use it to 

improve their reporting. 

Respondent Causes

Many of the causes of underreporting FRs thought of were specific to the respondent. They 

included privacy issues, embarrassment, learning to say no, forgetting, and reluctance. 
Underreporting 

22

Privacy 

FRs brought up the respondents‟ concern for privacy as a reason for underreporting a 

number of times. They commented that respondents don‟t want to reveal certain expenditures, 

“they‟ll say why do you want to know.” FRs noted that respondents in wealthy areas don‟t 

always provide all expenditures because of privacy considerations, nor do respondents on 

assistance. They have noticed that elderly respondents are having more and more concerns 

about identity theft; while others are concerned about the amount they spend impacting their 

disability benefits. Many FRs commented that respondents don‟t seem to report their entire 

income, possibly because of „under the table‟ sources, and also pointed out that credit liability 

is often not given because of privacy issues. 

Embarrassment 

Related to privacy, but a distinct respondent concern is embarrassment. FRs said that 

they often feel respondents are not reporting certain purchases because they are embarrassed 

to name them. They named lottery tickets or gambling expenses and alcohol as falling into this 

category. One FR also mentioned that respondents may not report purchases made at a thrift 

store because they are embarrassed. 

Learn to say no 

One interesting discussion that occurred during the underreporting session regarded 

the idea that respondents „learn to say no.‟ All three groups of FRs brought this idea up, and 

seemed to feel strongly that it is a phenomena that does occur. One FR described the situation: 

“Respondents learn that if they answer „yes‟ to the main question, more questions follow, so 

they have learned to say „no‟ to hurry the interview.” FRs said this tended to happen after the 

first or second interview, but noted that not every respondent changes their behavior. One FR 

said that it is mostly reluctant respondents who „learn to say no.‟ 
Underreporting 

23

Reluctant Respondents 

FRs noted that not all respondents have a problem with underreporting. Reluctant 

respondents were brought up a number of times as being much more likely to not give all 

expenses. As one FR put it, “if you talk a reluctant household into participating, they give the 

bare minimum, torn between civic responsibilities and personal issues, and give a half an 

interview.” All groups agreed that respondents who had to be convinced to participate were 

more likely to underreport, and „learn to say no‟ than others. 

Forgetting 

The cause of underreporting mentioned most frequently was forgetting. FRs believe 

that many things respondents purchase they do not remember while they‟re completing the 

interview, and can‟t report them. FRs said that respondents rely mainly on recall rather than 

records, and have a hard time with the 3-month reference period. 

Situational Causes

Not all causes of underreporting FRs discussed were related to respondent features, some dealt 

with specific situations, such as gifts, proxy reporting and out of scope purchases. They also 

noted that FRs themselves may have an impact on the amount of underreporting that occurs. 

Gift 

FRs said that another area where underreporting occurs is gifts. They said that 

“respondents think we don‟t want gifts,” that only purchases made for the household are to be 

collected in the interview. 

Proxy 

Another situation that FRs mentioned as contributing to underreporting is proxy 

reporting. They noted that most respondents miss expenditures made by other household 

members. Specifically they said that teenagers and adult children living at home do not report 

their expenses to others, and so they are often missed. They also noted that spouses are not 
Underreporting 

24

always familiar with the expenses of the other, “one knows furniture, and the other knows 

motor vehicles.” FRs noted that although questions are worded „you or other members of your 

consumer unit‟, respondents still primarily focus on themselves. 

Out of Scope 

FRs also believe respondents underreport purchases is that they believe the expenses 

are out of the scope of the interview. Specific examples of this were mentioned, “respondents 

may not consider [reporting] their daily coffee, they‟ll say it‟s „nothing to speak of.‟ Other 

purchases cited were newspapers, hair accessories, school supplies and pay phone calls. FRs 

said that although they tell respondents every expenses is wanted, the generality of the 

questions and examples might be giving respondents a different idea. 

FR Behavior 

In the focus groups, both FRs and SFRs were present, allowing for interesting insights 

about the impact of FR behavior on underreporting to be revealed. It was noted that although 

interviewers are trained to read every question exactly as worded, often FRs hand the info 

book to the respondent and ask them to read the page themselves. This may cause more 

underreporting than if each question was read as intended. They also commented that just as 

respondents can get distracted, FRs can as well, and when distracted, they are not able probe 

as they should. 

 Although the groups were able to identify FR behaviors that may increase 

underreporting, they also pointed out behaviors that may reduce underreporting, such as: 

- Probing for spouse‟s expenditures 

- Giving common examples after a question (i.e. naming Starbucks after 

restaurants) 

- Asking specifically about lunches eaten out 
Underreporting 

25

These behaviors may reduce underreporting, but as one group pointed out, only experienced 

FRs know to probe for these things; inexperienced FRs may not take advantage of these 

opportunities to probe. 

Interview Cause

In addition to respondent and situational causes, FRs identified a cause of 

underreporting stemming from the interview, the wording of the questions. 

Question Wording 

A number of times, FRs mentioned that underreporting may be occurring because of 

how the questions are worded. They noted that respondents don‟t always match up the 

question with the appropriate expenditures, so purchases are missed. Specifically they said that 

these items were problems: 

- Charitable donations, missing things like „boys club‟ donations or church 

donations 

- Retirement plans, missing 401K and stock investments 

- Accounting services, missing tax preparations 

- Clothing, missing diapers 

FRs agreed that respondents are not listening to the question at times, “because they are 

constantly flipping the booklet, or they get distracted by what‟s going on in their homes.” This 

lack of attention may cause them to miss purchases because they are not obvious given the 

wording of the question. 

Sections

Hypothesized Sections 

When asked which sections FRs thought had the most problems with underreporting, 

these were the expense types they named: 
Underreporting 

26

- Trips 

- Charitable donations 

- Lottery tickets 

- Clothing 

- Home décor 

- Cosmetics 

- Clothing 

- Insurance 

- ATM fees 

- Automatic deductions 

For each area, FRs thought that forgetting was the reason that respondents did not report all 

their expenditures. 

Furniture 

Analysis (Tucker, Biemer and Meekins, 2003) has shown that furniture is one of the 

CEQ with the most underreporting, and FRs were asked to hypothesize about why this might 

be. The first reaction was surprise; FRs were overwhelmingly surprised that furniture was 

underreported, they said it‟s usually well documented, respondents tend to remember big 

ticket items, and FRs notice new pieces of furniture and can ask about them specifically. 

When probed, they were able to come up with several reasons respondents might not be 

reporting all furniture expenses: 

- Respondents and FRs not using the information book 

- FRs not probing for purchases 

- Respondents don‟t consider paying for things on „lay-away‟ an expense 

- Respondents may not consider everything that falls under furniture 

- FRs may not be reading all categories, just asking „have you bought any furniture‟ 

Primarily though, most FRs felt that furniture wasn‟t one of the most underreported sections, 

and were surprised by this result. 
Underreporting 

27

Motor Vehicles 

Analysis also revealed that motor vehicles is an underreported section. FRs were not 

as surprised by this result, though they didn‟t think it had a major underreporting problem. 

They were only able to think of two possible reasons that motor vehicles were underreported: 

“when you‟re interviewing a woman, she may not know what her husband has done” and that 

the respondent might not know names of what was done on their car. 

Clothing 

This section was named as one of the sections FRs believed there was a lot of 

underreporting, as well as being among the top of the sections identified through analysis as. 

When discussing reasons for this however, FRs were only able to come up with forgetting. 

They thought that respondents have a hard time remembering how much money the spent on 

clothing. After further discussion, they also suggested that proxy reporting might be a problem 

in this section, respondents may not know of all clothing expenditures other CU members 

have. 

Other Personal Business 

Although FRs were initially confused by the types of purchases that fall into this 

section, once they reviewed a list, they were not surprised that these purchases are often 

underreported. Across all groups, the main discussion point was ATM fees, with FRs noting 

that “some respondents don‟t even know their ATM fees.” FRs also noted that these types of 

expenditures shouldn‟t be in section 19, Miscellaneous, “Accounting doesn‟t go with funerals 

and flowers.” 

Misc. 

FRs mentioned several specific cases where they suspected underreporting occurs, for 

a variety of reasons. These comments included: 

- “Any answer on interest is probably way off” 
Underreporting 

28

- Respondents often underestimate the amount they pay in tolls, or combine it with 

parking expenses and miss a lot of money 

- Respondents give expenses for dry cleaning and alterations together 

- Respondents (and some FRs) don‟t know where expenses for passports should go 

- Shipping expenses are not thought of in current question “need to add FedEx or 

something” 

- Minor housing expenses aren‟t thought of; “respondents just don‟t think of housing 

expenses like fertilizer...they think of larger expenses such as lawn services” 

Phone Interviews

Although not necessarily related to underreporting, during this session, FRs were asked to 

discuss phone interviews, and compare them with this done in person. 

Phone Frequency 

FRs were first asked how often they conduct CEQ interviews over the telephone. The 

answers varied greatly, ranging from “almost never” and “twice in my life” to “almost half” 

and “about 75% of conversion cases.” They agreed that phone interviews happen rarely for 

first interviews, and noted that they only suggest phone interviews when the first contact is 

good, “if I‟m on a roll, I‟ll try it.” Others noted that this may set a dangerous precedent, if a 

respondent does one interview on the phone, they may be less likely to be willing to do a 

personal interview in the future. 

Personal Visit 

FRs were able to name a few advantages of conducting the CEQ interview in person: 

can use the information booklet, it‟s easier to rush a person on the phone, in person you can 

take more time, and you can get better quality data in person. One SFR said “FRs who do half 

of their interviews over the phone are not good interviewers” she noted that you could tell a 

difference in the data between interviews done in person and those done over the phone. 

Another FR was also against phone interviews, saying that she only uses them when necessary 

to convert reluctant respondents. 
Underreporting 

29

Phone Advantages 

A few FRs strongly disagreed with these ideas, arguing that it is possible to obtain 

quality data over the phone, one FR insisted “I‟m good on the phone, I know I am.” FRs 

pointed out other advantages to phone interviews: 

- Phone interviews are often the only way to get a respondent to agree to participate 

- Respondents are more comfortable with FRs not in their homes 

- FRs can drop off info booklets before a phone interview, and pick it up after 

- phone data is better than no data 

- Can do longer interviews on the phone, “I once did a 2 hour phone interview” 

- Respondents may not feel as in a hurry on the phone, and may take more time to 

get and review bills 

- For cases that are a long distance away, phone interviews are more realistic 

FRs in favor of phone interviews did not feel the loss of the info booklet impacted data quality 

negatively. They said that often phone interviews are used for 2nd and subsequent interviews, 

and respondents already know what‟s expected. They also repeatedly stated that often the only 

way to convert a reluctant respondent is by offering a phone interview and “phone data is 

better than nothing” 

Suggestions

FRs had some suggestions to improve specific areas on the CEQ: 

- It is difficult for respondents to estimate the amount they spend on meals out, 

additional questions might help 

- Might get more credit liability by asking for a range rather than an exact amount 

- Respondents don‟t know how much taxes are taken out of their paycheck without 

a paycheck in front of them 

- If Respondents could report average weekly expenditures (for items like gas and 

restaurant bills), that would be easier and they‟d underreport less 

- Inform FRs how data is used, they are unsure how useful is half an interview 
Underreporting 

30

- Utilize high school and college students, they are more enthusiastic about survey 

participation 

- Give respondents a notebook to record Christmas gifts

DATA QUALITY

One of the few sessions specifically about the CE Diary during the FR Focus 

Groups focused on data quality, specifically the review process FRs go through when 

picking up a diary. This session revealed much interesting information, as well as provided 

insight about actual field behaviors. 

When the topic was first brought up, FRs reacted strongly to the idea that the CED 

has a problem with data quality. They all agreed that there are issues with data quality; one 

FR said “I hate the fact that the CPI is based on the diary because of the quality of the data.” 

Specifically, FRs named the following concerns about CED data quality: 

- Recall questions don‟t include enough (i.e. things paid for by check) 

- Respondents forget deductions from pay checks 

- Respondents put in more effort for the week 1 diary, the data from week 2 isn‟t 

as good 

- Not all FRs ask all questions, some questions are skipped to save time and 

“refused” is put as an answer 

- When a respondent gives you receipts only, you miss a lot of data. 

Most FR concerns focused on underreporting, rather than the quality of the expenditures 

reported by the respondent. 

 The FRs were then asked to think about the diary review process during week 1 and 

week 2 pickup. At first, they were not sure what facilitators were referring to when they said 

„review process.‟ FRs initially thought of recall questions, and only after further 

explanation, did they begin to discuss the actual review process. Throughout the session, 
Data Quality 

31

only a few comments were made in support of the process. Comments supporting the 

review process included the non-descriptive nature of clothing receipts, without review the 

data isn‟t complete; the respondent can learn from the review process and “if it‟s week 1, 

you [the FR] don‟t want them [the respondent] to make the same errors. 

The majority of FRs seemed to think that reviewing the diaries was a good practice 

in theory, but in reality it is not practical. FRs repeatedly expressed the idea that completing 

the diary is a big burden on respondents, and they are hesitant to increase burden in any way. 

As one FR put it, “we‟ve bothered them enough,” and they don‟t want to ask them to do 

further work by reviewing their entries and correcting their mistakes. Other FRs commented 

that it is often awkward to correct a respondent, they have asked for a favor, and telling the 

respondent they did it wrong is not appropriate. One FR said “if I can get it corrected, great. 

If not, oh well, I‟m not going to correct anyone.” Other FRs noted that diaries are often left 

at the door for FRs, there is no FR-respondent contact for diary pickup, and review is 

impossible. These reasons were heard across all groups, and almost all FRs agreed with the 

idea that most diaries do not get reviewed. 

FRs were then asked what types of errors they commonly find when they are able to 

review a diary. Most FRs said that they typically don‟t find many errors in diaries, but those 

that are found included: 

- Entries without amounts 

- Entries in the wrong section 

- Missing age or gender information on clothing 

When FRs are able to complete a diary review, they agreed it didn‟t take a long time. One 

FR said “it‟s quick, less than 15 minutes,” while another said that if done in the presence of 

the respondent, it takes about 5 minutes. They agreed it varied greatly depending on the 

respondent and their expenditures. FRs did not agree on how often they were able to review 

diaries however. Estimates ranged from 25% to 80%, with no consensus reached. 

 The process by which FRs review a diary was discussed at length. All agreed that 

the respondent dictated how the review was done. If a respondent is not reluctant, and seems 
Data Quality 

32

interested in the process the FR will review the diary with the respondent. With the 

respondent, FRs admitted they do not typically go through the diary „page by page,‟ “I don‟t 

want them [the respondent] to think that I‟m picking apart what they just worked on.” 

Instead they said they flip through the diary, just checking for blank pages or obvious 

mistakes. 

FRs said that most of the diary review occurs after picking up the diary, not in the 

presence of respondents. They said they prefer to correct errors at home, as so not to further 

burden the respondent. When reviewing a diary at home, FRs said they “don‟t hesitate to 

call” the respondent to clarify an entry or make a correction. One FR said that she notes the 

question she has about specific entries and asks the respondent about them during week 2 

pickup. 

After the discussion, FRs were given copies of actual diary pages with errors on 

them. Their reactions were varied “I‟ve never had a diary like this” while another said “I 

have.” FRs were able to identify almost all of the errors in the diary, and said that for many 

of them they would call the respondent to get the additional information needed. This was 

not true across all errors, FRs agreed. One group stated they wouldn‟t call back for a small 

detail, like the type of milk, while they would for an illegible entry. One FR said she 

prioritizes by the amount of money associated with the error, if it‟s a large purchase; she 

takes the time to contact the respondent. 

The review exercise revealed some confusion about FRs as to what constituted an 

error. One FR admitted “I don‟t think I‟m looking close enough, I don‟t know what needs to 

be there.” FRs disagreed whether meal type (i.e. lunch, dinner) was necessary in Food Away 

from Home. Some FRs said that this had been addressed during the most recent refresher 

training, while others claimed they had never been told this was necessary. Not all FRs new 

how to handle multiple purchases of the same item; some FRs said they list each 

individually (i.e. cereal, cereal, cereal), while others said they would simply write „3 boxes 

of cereal.‟ In general, FRs had differing ideas about the review process. One FR claimed “I 

didn‟t know we were supposed to go in and critique or change what they put.” Another said 
Data Quality 

33

“we were told what they wrote is what is there, I can move it or add it, but not change or 

remove it.” 

Related to the review process is the CED recall questions. All FRs agreed that they 

ask the recall questions when they are able to pick up the diary in the presence of the 

respondent. Some did admit however, that they often ask them without reviewing the diary, 

because it is too time consuming to locate specific items in the diary. They expressed 

concern about underreporting for these questions, “they [respondents] just say „no, no no.‟” 

When discussing Total Recall, a few FRs expressed some confusion. One FR said 

she didn‟t know what to do with total recall when she got a blank diary and a pile of 

receipts. Another said “I‟m learning a lot! I have been doing total recall for each CU, but 

maybe I‟m not supposed to.” Others were confused about how total recall works when the 

diary is picked up a day late; they didn‟t understand why it‟s a Type B if more than 50% of 

the diary is recalled without receipts. 

As with the other topics covered during the focus groups, FRs had suggestions for 

improving the diary and review process: 

- Reassign cases more efficiently 

- Reword “for not in CU” 

- Put tabs for each page 

- In manual, list problems you have to call respondent about, and those that you 

can leave as is 

- Add pocket or sleeve in diary for receipts 

- Increase amount of time given to complete a diary case 

- Clarify: does it matter where in the diary an entry is written? 

Some of these suggestions (pocket and clarification) will be addressed by the redesigned 

diary. Others however, particularly the one about adding information to the manual about 

the required elements for entries should be considered. 
CHI 

34

CHI

One of the main reactions across all groups was how CHI overlaps with notes. Some 

FRs said that CHI may shorten notes; since most of their contact information is captured in it, 

they won‟t record the information twice. One FR said “I don‟t like writing notes, this will 

make it easier, ”while another commented“ I‟ll never use notes again!” CHI was thought to 

have the potential to standardize the notes process, and FRs liked that their supervisors and 

SFRs could see them in Case Management, without opening the case like you have to do to 

read notes. 

Some FRs showed some slight confusion when reviewing the instrument. On the 

contact strategies screen, FRs didn‟t know what response option to choose if they didn‟t do 

anything, if they sent a letter, if they found out the house was vacant or if they found out when 

to come back. FRs were also initially unclear that they can change the date and time in CHI, 

and on the reluctant respondent screen, they took a while to figure out what to put for a nonreluctant respondent. The partial interview screen also caused some confusion in one of the 

sessions, FRs were unsure why a contact where an appointment had been scheduled should be 

classified as a partial interview. The discrepancy between the case status definition of partial 

interview and the CHI definition was unclear, and FRs suggested changing the wording. One 

group was also initially confused about the relationship between their case and CHI. They 

thought that when entering information into CHI it would change the status of their case – for 

example entering that a phone call to a household would change the status of the case to open 

in case management. 

FRs brought up many advantages of CHI during their discussions. One FR claimed 

“it‟s like your own personal secretary!” Other FRs discussed the benefits of being able to get 

the history of a case when they receive it from other FRs, and noted that it will make follow up 

and refusal conversion easier: “It‟s a wonderful blessing for future FRs.” Another advantage 

FRs saw in CHI is that supervisors and SFRs will be able to see how much effort is put in on a 

case. One FR commented that even when she‟s not meeting her percentages, her supervisor 

can see how hard she‟s working. Another noted that CHI “justifies our job” and provides 

proof that they‟ve been making attempts nights and weekends. In addition to showing others 
CHI 

35

what effort has been done on a case, CHI will allow FRs to review their attempts and 

determine when they should attempt next. 

FRs were allowed the opportunity to express concerns they had about CHI, and they 

came up with only a few. The primary concern was the burden the instrument may add. Many 

FRs across all sessions talked about how long it takes to open and close a CE case, and 

described how often they contact a CU without opening a case. FRs feared that to record each 

contact attempt they‟d have to wait for the case to open, enter the contact information, and 

close the case, a time-consuming process. FRs were also concerned about the number of 

contacts they make, and having to record each one; one FR said “I can call a case 15 or 20 

times” and she was concerned about how long it would take to enter that information in CHI. 

One FR noted that they already have to put the numbers of times they call a household in the 

instrument, so this information is redundant. 

Overall, CHI was very well received by FRs, both by those who have used it on other 

surveys, and those who have not. They liked the number of keystrokes required, the speed the 

information could be entered, the completeness of the response alternatives and the overall 

flow of the instrument. 

TRAINING

In June 2004, three focus groups were conducted with CE FRs. One of the topics 

discussed during the day and a half sessions was training. Many suggestions for improvement 

came out of this session, as did much interesting information. Observer notes and transcripts 

were combined and analyzed. 

Peer Contact

 Every FR said, or agreed with the idea, that one of the best parts of trainings was the 

opportunity to meet, talk with and learn from other FRs. One FR said that the trainings gave 

them “a chance to be with your peers, because our work is so solitary.” Another noted that 
Training 

36

after trainings she “feel[s] back in a group and remember[s] how important the survey really 

is.” In addition to this increased feeling of being part of a group, FRs noted that being in 

training with other FRs, they were able to get ideas from their peers, learn how other people 

do things, and learn things unofficially from people who are also in the field. Things that were 

mentioned as being learned this way included: how to respond to reluctant respondents, how 

to escape a virus scan, how to handle computer hibernation and about the night differential. 

Many FRs commented on the advantages of working with experienced FRs. They felt that 

new FRs are not given enough supervision in the field early on, and without it is difficult to 

feel confident that they‟re doing the interview correctly. FRs also wanted the opportunity to 

observe their SFRs conduct interviews; one said “I‟ve never seen my SFR talk to a 

respondent, and she‟s amazing at converting refusals, I‟d like to see that.” They commented 

that they‟d learn a lot from getting to watch SFRs conduct interviews and would like this 

opportunity. Although one Regional Office provides newly hired FRs a Pre-training 

observation (PTO) which the FR thought was a good practice, none of the other FRs in that 

session had heard of such an observation. An SFR who was participating gave the other side 

of the story, noting “I don‟t like to have my FRs observe me because I don‟t want them to say 

I could never do that.” This reaction was rare and not shared by all SFRs present. 

New Hires

 Although the FRs participating in the focus groups tended to have a lot of experience 

with CE, many of their training comments related to new hires. One FR noted that classroom 

training doesn‟t approximate field work: “I have done more interviews on the porch trying to 

balance everything” but noted that all practice cases in training were done sitting down at a 

table. Another FR suggested that new FRs aren‟t given the “whole picture of the consumer 

expenditure survey” and this is important as their field work. Across groups it was noted that 

smaller classes were more effective and CE tends to “hire all these new FRs and make the 

groups [class size] too big.” A last related idea was that new FRs learn a lot by being in 

trainings with experienced FRs. In addition to being exposed to training materials, they often 

learn many tips and tricks from FRs that they would have had to figure out on their own 
Training 

37

otherwise. A few FRs suggested and liked the idea of shortening initial training, and bringing 

new FRs back a few months later for additional training. 

Training Length

 The groups were not as consistent in their feedback on the length of trainings. One FR 

brought the topic up by saying “it [CEQ refresher training] was a day and a half, and I don‟t 

think it needed to be, a day would have been sufficient.” Other FRs felt this way, saying that 

having day and a half trainings were inconvenient and unnecessary. One FR suggested that 

trainings last only one day, but be held more frequently. Shorter, more frequent, trainings 

could target specific problems and would be useful to new and experienced FRs alike. Not all 

FRs agreed that trainings were too long, one said that “sometimes there is not enough 

classroom training” given all there is to learn for the CE. Another FR noted that she had 

“never gone to a training that didn‟t get out early,” and that trainers often rush through, or 

skip, materials. A few FRs across groups commented that during trainings, there needs to be a 

time to discuss questions and not just go through scenarios. 

Training Cases

 A topic that came up multiple times in each group was the need for more realistic 

training cases. As one FR put it, currently, “everything is prefect in the classroom” and FRs 

“need more realistic problems that are more difficult.” This discussion was primarily focused 

around the CAPI based portion of trainings, the scenarios presented in class, as well as the 

paired practice situations. “Training cases don‟t reflect what we find in the field,” said one FR. 

The groups commented that once they‟re in the field, it is not the time to learn how to handle 

complicated situations, yet that‟s often where they first encounter them. One FR said that in 

other trainings, they are given time to try to „trick‟ the computer, and this was very helpful. 

Another FR suggested that they have the chance to bring an actual situation they‟ve 

encountered in the field to class, and deal with it as a group. Things that were suggested to be 

added to the training were: troubleshooting computer problems, navigation, multiple 
Training 

38

households, multiple CUs, vacation homes with additional utilities. Overall, FRs feelings 

seem to be summed up in this comment by one “training is lacking real life.” 

Self Studies 

 FRs were eager to discuss specific aspects of CE trainings, and self studies were often 

mentioned. The overall opinion was favorable on self studies, FRs said they liked getting to 

work at their own pace, and have additional time to practice CE cases. One FR went so far as 

to say “I learn more in self studies than I do at trainings.” Although FRs had positive feedback 

on self studies they did offer several ways to improve them. One suggestion was to include 

more troubleshooting concepts, giving FRs the time to work on these issues at home. Another 

suggestion was to increase the frequency of the self studies. This suggestion came up across 

all groups, and FRs were energized when discussing the possibility of having a regular CE self 

study, similar to those received by SIPP. They said these self studies gave them time to review 

the manual and “keep you sharp.” Two of the three groups also recommended moving the 

self studies from paper to another form, such as audio or computer based training (CBT); they 

said that it was often difficult to work with both the computer and the paper training. Related 

to this, one discussion focused on the problem they often find with the self study: “how many 

times have you gone through the paper training and there‟s something left out. You have to 

stop what you‟re doing and try to figure out where you are and what‟s missing.” 

When relating the self studies to their classroom training, a few FRs said that having 

the same material twice is repetitive, while others said that the more ways you can hear 

something the more you‟ll get from it. A few SFRs noted that they don‟t know how to handle 

FRs who come to trainings without completing their self studies. This is a problem when one 

of the reported advantages of the self studies is that “gets everyone on the same page.” Since 

FRs get paid for the time taken to complete a self study, one possible solution to this would be 

to track who doesn‟t come to training with a completed self study, and make sure they did not 

charge for that time. 
Training 

39

Walk-Throughs

 Another aspect of CE training that was discussed was walkthroughs. FRs 

overwhelmingly said that they are tedious and too long, making it hard to concentrate and 

often “people become glazed over.” The exception to this opinion related to new FRs, who, 

the groups agreed, benefit from as much practice going through the interview as possible. FRs 

commented that it should be possible to get to the point without having to go through the 

whole interview, and suggested that trainers allow time for questions and discussions during 

the walkthroughs instead of rushing through the scenarios. 

Paired Practice

 Paired practice was another aspect of training brought up across all three groups. 

While a few FRs said that they are tedious and unnecessary, more noted that any time they get 

to hear another FR go through an interview is helpful. Working with other FRs while going 

through an interview reminds them “how it should be done.” FRs also noted how helpful this 

type of exercise is for new FRs, and that everyone benefits from practice, noting that during 

training, “we spend a lot of time crossing off milk and putting skim milk” and this time could 

be better used. To improve paired practice, FRs suggested specifically pairing up new and 

experienced FRs and making the scenarios more complicated. 

FR Manual

 Although not specifically training related, the FR manual was discussed during this 

section. FRs felt that the time spent reviewing the manual was useful, however they “are not 

able to look at the manual during an interview, [they] have to refer to it later.” Many FRs said 

that they are often unfamiliar with the manual since they don‟t get paid to sit and read it at 

home. Although one FR said “[I] love the new manual, less mistakes, more information about 

the survey than the old one”, the majority of FRs had the opposite opinion. One FR said “[it] 

doesn‟t help me, it‟s worthless. I‟ve gone to it many times and have never gotten an answer.” 

A few FRs said that they call their SFR instead of looking in the manual, and most said they 

would like it to be more organized and complete. 
Training 

40

 In conclusion, the training section of the FR focus group was one of the most energetic 

and interesting of the day and a half session. FRs felt strongly about the training, and had 

many suggestions for improvement. A comment all FRs in the group agreed with might 

summarize the overall conclusion: “training is money well spent.” FRs seem to enjoy and 

learn a lot from training and related materials, but there is still much room for improvement. 

Specific Suggestions

As with other sessions, FRs had many suggestions to improve CE training: 

- Make trainings a day instead of a day and a half 

- Have trainings more frequently 

- Have new FRs attend new hire training, and then return for additional training (of more 

complex topics and to address their questions) after a few months in the field 

- Have more interaction between experienced and inexperienced FRs during training 

- Allow for more SFR observations 

- Allow FRs to observer their SFRs 

- Incorporate Pre Training Observations across all Regional Offices 

- Make CAPI training cases more realistic. Add: 

o troubleshooting computer glitches 

o navigation 

o multiple households, multiple CUs 

o vacation homes with additional utilities 

- Let FRs try to trick instrument during training 

- Let FRs bring cases from field to training and deal with as a group 

- Give SFRs regular assignments so they stay familiar with the survey 

- Keep CPI fact sheets current 

- Don‟t have supervisors who don‟t have CE field experience conduct training 

- Keep training classes small 

- Don‟t have CRAFT training for just new hires – they need to see how experienced FRs 

handle reluctant respondents 

- Increase frequency of self studies (monthly, format like SIPP) 
Training 

41

- Move to Computer based training (CBT) 

- Add audio based training 

- Give trainers a way to handle FRs who attend trainings without completing their self 

studies 

- Program training cases so you don‟t have to through the whole case to „get to the point‟ 

- Make manual more organized and useful 

- Have manual organized question by question

RECEIPTS

The response to a general „how do you feel about using receipts for the CED?‟ 

revealed that the overwhelming majority of opinions were positive. Only a few negatives were 

given, and these were only provided after prompting. The negative comments included the 

amount of work required of the FR when receipts are used, and that respondents can become 

dependent on receipts and not fill in purchases they do not have receipts for. The positive 

comments were consistent across groups and included: 

- Entries are more thorough with receipts 

- Receipts reduce burden and increase respondent cooperation 

- Receipts reduce underreporting and increase accuracy and quality 

- Receipts are „invaluable‟ 

- Asking for receipts emphasizes you want exact amount 

- Receipts are the “only way to get quality data” 

The discussion revealed that FRs felt strongly that receipts were a powerful tool in not only 

getting respondents to participate in the survey, but also in getting quality data. 

 The group then discussed what, if any, direction their Regional Office (RO) gave them 

about receipts. The trend seemed to be that no specific policy was in place regarding receipts; 
Receipts 

42

only one FR mentioned that her RO told her to collect receipts, however, she was not told how 

to use them. FRs noted that they didn‟t recall receipts being covered in trainings recently, but 

some remembered initial training referring to receipts as a last resort. They commented that 

the preferred way is still to have the respondent fill out the diary themselves, though this is 

happening less often and receipts should be covered in trainings now that it‟s a bigger issue. A 

few FRs agreed with this idea, and after thinking back to earlier trainings they recalled being 

“trained you‟re not supposed to do it that way [have respondents only give you receipts].” 

 A theme that came up multiple times in each group is the usefulness of receipts in 

persuading reluctant respondents. Many FRs quickly commented that they offer the use of 

receipts when they have a reluctant respondent, because it is an easier task and requires less 

commitment from the respondent. As one FR said “if you don‟t offer to use receipts, you‟re 

more likely to lose the respondent.” The groups also noted that for respondents who were 

initially reluctant, using receipts is often the only way to get quality data, since they are 

unlikely to give the time and effort required to complete a diary. 

When asked how often FRs use receipts, the answers varied greatly. Some FRs said 

that at least half of their respondents give them some kind of receipt. Others said that at least 

80% give them a grocery receipt, so they don‟t have to enter all the purchases, while still 

others said that only their elderly respondents provide receipts, and even only about half of 

them do. The disparity in frequency may be occurring in the field, however it should be noted 

that FRs may have been less likely to report high numbers as a result of the setting and 

observers. 

In addition to varying in how often they use receipts, FRs also varied in the way they 

handled receipts. Some FRs said they transcribe the receipts before leaving the respondents. 

That way they do not have to worry about unclear purchases or returning the receipts. Other 

FRs say they that they take the receipts with them when they leave the respondent and either 

destroy or return them, depending on the respondent‟s request. One FR said she didn‟t take 

receipts, rather she called the respondent every night to get the purchases. Another FR said she 

asks for check registers in addition to receipts to ensure that all payments are included. There 

were no consistent trends in how FRs used receipts. 
Receipts 

43

To gain a better understanding of what types of receipts FRs encounter in the field, 

they were given copies of actual receipts and asked to record them into blank production 

diaries. They then participated in a discussion about specific receipts, and the task in general. 

The first reaction all FRs had to the task was that the receipts are not typical receipts. One FR 

said “what I‟m getting handed is way user-friendly,” while another noted “if I had this, I 

wouldn‟t put it in.” Others agreed that if the receipts they typically received were this unclear, 

they wouldn‟t advocate using them as strongly, however they noted that they know the stores 

in their area and are familiar with the receipts. A few FRs disagreed about where to put 

purchases they couldn‟t classify. Some said that it should go under „other,‟ because recording 

the dollar amount even in the wrong section was better than not recording it at all, while others 

said an entry shouldn‟t be recorded unless it was in the correct section. 

When discussing unclear receipts, FRs had many methods of dealing with them. Most 

FRs said that they have no reservations about calling respondents and asking for clarification 

when they can‟t interpret a receipt. A few FRs said they call the store for clarification, while 

others said they simply make their best guess. If possible, FRs said they try to review receipts 

before leaving the respondent, and ask any questions at that time; however they note this is an 

ideal case and doesn‟t always happen. 

Clothing receipts cause unique problems, because even when the receipt is clear, 

without information from the respondent, it is often impossible to determine who the clothing 

was purchased for. A few FRs said that they tells their respondents to let her know when 

they‟ve given her clothing receipts and she gets the additional information before leaving. 

As with all other topics, FRs had many suggestions to improve the use of receipts, and 

the diary in general: 

- Emphasize the idea of using a teenage „diary commando,‟ a member of the 

household who is tasked with collecting expenditures from other CU members 

- Have a machine which can read UPC codes/titles 

- Incorporate the use of receipts into training 

- Have a info package for the diary to help FRs „get in the door‟ 
Receipts 

44

- Have a different diary for the elderly or disabled, the font is too small and hard to 

read 

During the session, a few general diary issues were raised, and should be noted. One 

FR said she was never clear on placement dates, and another said she had to refer to the 

manual anytime she had to do total recall. These issues can be addressed in training. 

MISCELLANOUS

Before closing the focus groups, a series of small miscellaneous topics were covered, 

and FRs were given an open forum to share their concerns and suggestions. 

2005 CAPI Changes

FRs were shown mock ups of the upcoming change to the clothing section. They had 

many positive comments about being able to combine items and people in the clothing 

section. They said that it will make the clothing questions easier for respondents, and will 

make the section go faster. FRs also said that there will be less combined codes with this 

option. The questions and suggestions they had about the change included: 

- Can you combine multiple types of items? 

- Will the name of the person for whom the clothing still appear on the 

prechart? 

- Alphabetize the list of clothing items 

- Allow „school clothes‟ as an option 

- Move baby clothes to the clothing section 

- Move costumes to the clothing section 

Top FR Concerns

During the spring refresher training, FRs participated in an activity which resulted in a 

list of top FR concerns. The lists from each regional office were combined, and the top ten 

concerns were reviewed during the focus groups. During the discussion, FRs suggested a 

variety of things to alleviate these concerns: 
Miscellaneous 

45

- Provide incentives: anything with the census seal on it 

- Have the website on a magnet to give to respondents 

- Create a video for FRs or respondents explaining the survey 

- Create materials from BLS showing the respondents how their answers help 

- Distribute the “what‟s in it for you” handout to all regional offices 

- Put more current data on promotional materials. 

- Include a handout with bullets of what the survey is in the advance letter 

- Provide all FRs with posters to advertise data collection work in their area. 

- Increase advertising of Census Bureau and the on-going surveys 

FRs felt that these changes would make their jobs easier, by increasing response rates. 

Open Discussion

During the final hour of the focus group, FRs covered a wide range of topics; 

providing suggestions and input about many CE features. 

CAPI Suggestions 

As expected, FRs had many suggestions to improve the CAPI instrument: 

- Alphabetize lists 

- Pre-fill vehicle information 

- Put codes in computer for SUV, truck, van, etc. and use them for „description 

of a car‟ 

- Add an option to be able to delete a vehicle because a CU member moved and 

took vehicle with them 

- Add a list of what services are considered vehicle maintenance versus vehicle 

services 

- Need to be able to see HH Roster by using function key 

- Move business expenses question to beginning or end of each section and only 

ask once 
Miscellaneous 

46

- Move „purchased for someone outside your CU‟ question to beginning or end 

of each section and only ask once 

- Add continuous expense for diapers 

- Have respondent phone number on case management screen 

- Add function key to repeat entries (e.g. rent payments) 

- Increase computer speed 

- Auto fill utility company names with prechart information, and give FRs the 

option to edit if necessary 

- Add option to select „budget plan‟ in utilities 

These were general suggestions FRs had, without much prior thought, or access to an 

instrument. Additional CAPI suggestions were made during the CAPI portion of the focus 

groups. 

Wording Suggestions 

FRs had specific suggestions on questions in the CEQ that needed improvement in 

their wording. The pointed out that the wording „received interest‟ was unclear to both FRs 

and respondents, they thought it meant that interest was accumulated, but were unsure. They 

also were unclear on the difference between maintenance and alterations, and claimed that the 

majority of the time they simply guess which category to put an expense under. They 

suggested that the „outside the CU‟ phrase be eliminated from every item; instead ask it as a 

separate section, such as „now I am going to ask for expenses you have had for someone 

outside your CU.‟ They also felt that repeating the reference period on each screen was 

unnecessary and irritating to the respondent, and could be reduced to a periodic reminder 

throughout the interview. 
Miscellaneous 

47

Materials Suggestions 

In addition to the suggestions for additional materials to improve response rates 

mentioned above, FRs had many comments on how the materials they have available to them 

could be improved: 

- The page numbers on the information booklet are too small 

- Alphabetize lists 

- Laminate the pages of the info book because some respondents are wary of 

touching them for health reasons 

- “We need a cool video on interviewing techniques” 

- Create a video available on the laptop that shows the CU how the data is used 

and explains confidentiality issues. 

- Don‟t say that the interview lasts 90 minutes. FRs said normal interviews last 

about 45 minutes 

- Stress confidential nature of data collected 

- Add that someone from the survey will be contacting the respondent 

- Allow all FRs to mail out own letters, ensuring the respondent will receive 

them shortly before the FR visits 

- Remove mention of voluntary participation 

- Add information to help respondents prepare for the types of questions that 

will be asked, especially income 

- Add that the FR will return in three months 

- Remove statement “...will call on you” as it gives impression of a phone 

interview 

Miscellaneous Suggestion 

Finally, there was a suggestion from an FRs that do not fit into the above categories. 

She said that the re-interview program needs to be redone, as it is not effective as it is. 
Miscellaneous 

48

The comments and suggestions from FRs heard during this session of the focus group 

provided a lot of interesting ideas and perspectives that should be followed up on. FRs are in 

the field everyday, and have first hand knowledge of the CE areas that can use improvement.