COGNITIVE ISSUES ASSOCIATED WITH 

COLLECTING CASH CONTRIBUTIONS IN 

 THE CONSUMER EXPENDITURE SURVEY 

 SYLVIA K. FISHER 

WESTAT, INC.

FisherS1@westat.comFor more information: (301) 610-4826 

JOHN BOSLEY 

BUREAU OF LABOR STATISTICS

 PAPER PRESENTED AT THE ANNUAL MEETING OF THE AMERICAN ASSO FOR PUBLIC OPINION RESEARCH, ST. LOUIS, MO, MAY 16, 1998. 
BACKGROUND AND INTRODUCTION TO PROBLEM

Currently, the U.S. Bureau of Labor Statistics (BLS) collects cash contribsections of the Consumer Expenditure Survey, which is a survey interview household on a quarterly basis. The BLS wanted to identify a more effective cash contributions, because the reports of cash contributions during the thrperiod collected from respondents appeared both statistically and intuitively tfor Cash Contributions that were lower than should have been anticipated. A team was appointed to identify proposed rewording of the Cash Contributioteam, the Redesign Task Force, developed an alternative version of the Cash was designed to improve reporting accuracy and minimize respondent budescribes the results of cognitive testing procedures conducted to evaluate tcognitive testing of the proposed revision of the Cash Contributions quesinformation about many of the cognitive issues associated with collecting fiof this type. In addition, the final version of the Cash Contributions that wacognitive testing procedures is provided, as well. 
COGNITIVE TESTING ACTIVITIES 

Pretesting: Issues Associated with Cash Contributions Questions

• Cognitive interviews were conducted using probes that queried respon

issues associated with collecting information about cash contributions; debriefing type of protocol was administered. The goal of pretesting was to

• identify all major issues and/or problem areas associated with resp

generate accurate responses to the protocol; 

• identify any issues that could be eliminated from later cognitive testing 

not salient; and 

• refine and standardize the cognitive protocol to be administered thro

of testing. 

• Pretest results were used to develop a modified version of the Cash Con

designed to address questions raised during the pretesting phase. 

Round I Testing: Redesign of Cash Contributions Questions

• The purpose of Round I testing was to address general issues and/or probl

during the Pretesting phase. New stimuli and protocols were developeidentified during the pretesting phase. 

• More intensive and narrow probing during cognitive interviews 'keyed in' o

interpreted important concepts such as "cash contributions," and "money." 
Round I Testing: Redesign of Cash Contributions Questions (continued)

• Round I testing evaluated whether the revised wording of the question

accurate reporting of expenses and greater recall of information 

• Round I testing compared respondents' performance on the revised ve

Contributions with the current version of Cash Contribution questions ucollection around the country. 

• Cognitive interviews were conducted using primarily respondent debriefing 

research questions included: • Did the two-stem approach to wording the question function effectively? • Did the revised wording of the Cash Contributions question(s) confus

were respondents able to recall their Cash Contributions more accurately

Round II Testing: Feasibility of Revised Cash Contributions Questions

• Round II testing focused directly on problems of response identified duri

Questioning and probing were geared to identify any fine distinctions in resthe effectiveness of the revised Cash Contributions questions. 

• Round II functioned as a type of "feasibility" test of the revised questions 

results of Round I testing, providing a preliminary test of their "usefulness,

and "accuracy." 

The outcome of Round II testing was to generate a final version of Cash Conthat could be incorporated within future versions of the Consumer Expenditur
Revised Cash Contributions Questions to be Tested

The proposed rewording of the cash contribution questions to be tested were aSince the first of (month, 3 months ago) have you (or any members of your CU1. paid any of the following for support of someone outside your CU... 

a. alimony b. child support c. cash to college students living away from home 

2. given any gifts of cash to... 

a. other persons not in your CU b. educational institutions c. political organizations d. churches and other religious organizations, excluding parochial schoe. charities and all other organizations 

3 given any stocks, bonds, or mutual funds to persons or organizations ou your Consumer Unit (i.e., - yourself and any other person(s) with whom  an economically interdependent relationship)? 
Total Subject Pool Characteristics

• Thirty-nine (n=39) respondents participated and were compensated $25 per • Twenty-seven (n=27) females, twelve (n=12) males • Twenty-six (n=26) whites, twelve (n=12) African-Americans, one Asian/PaPretest Subject Pool Characteristics

• Nine (n=9) respondents participated and were compensated $25 per hour • Six (n=6) females, three (n=3) males • Seven (n=7) whites, two (n=2) African-Americans Round I Subject Pool Characteristics

• Fifteen (n=15) respondents participated and were compensated $25 per hou• Thirteen (n=13) females, two (n=2) males • Eight (n=8) whites, six (n=6) African-Americans, one Asian-American Round II Subject Pool Characteristics

• Fifteen (n=15) respondents participated and were compensated $25 per hou• Eight (n=8) females, seven (n=7) males • Eleven (n=11) whites, four (n=4) African-Americans 
TABLE 1 

Pretest Issues (Response Problems) Associated with 

the Collection of Cash Contributions and Solutions 

Identified After Two Rounds of Cognitive Testing 

Pretest Issue #1:

• The wording "paid any of the following for support of someone outside 

awkward and slightly confusing for a number of respondents. 

• Respondents suggested revising the wording to clarify the purpose of this q

Solution Issue #1:

• Although all of the respondents were unable to generate their own revisio

they all preferred the interviewer’s suggestion revision "paid any of the support someone outside of your CU." 

• The revised wording "paid any of the following to help support someon

CU," presented no problems for respondents, even when specifically queriproblems in the statement. 

• Respondents generated at least as many reports with equal accuracy as the 

- there was no loss of data and cognitive confusion was minimized by the re
TABLE 2 

Pretest Issues (Response Problems) Associated with th

Collection of Cash Contributions and Solutions Identif

After Two Rounds of Cognitive Testing [continued] 

Pretest Issue #2: • Based upon focused querying, the majority of respondents agreed the u

"alimony" and "child support" had the potential to offend some respondentthemselves had no objection to these terms, some reported they knew would have been upset by these terms, because these individuals were perin situations involving alimony and/or child custody. 

• Some respondents reported they thought male respondents would take par

the use of these terms. 

Solution Issue #2: • The categories "cash to college students living away from home," "chi

"alimony," were re-ordered so that the two potentially controversial termand “alimony” were presented as later options. 

• Re-ordering categories did not alter recall of reported events for any of 

Respondents favored the revised placement of the items to the earlier versio
TABLE 3 

Pretest Issues (Response Problems) Associated with th

Collection of Cash Contributions and Solutions Identif

After Two Rounds of Cognitive Testing [continued] 

Pretest Issue #3: • As anticipated, the use of the term "gifts" (see Section 22F) presented 

majority of respondents, because they generally imposed their own defin"gifts" when responding to this portion of the question. 

• All of the respondents favored dropping the use of the term "gifts," a

absence of the term did not detract from the desired meaning of this portion 

Solution Issue #3: • Elimination of the term "gifts" did not appear to inhibit recall of "giving" 

of the fifteen respondents. Respondents did well simply being probed amoney..." and then being provided with the list of cash alternatives. 
TABLE 4 

Pretest Issues (Response Problems) Associated with th

Collection of Cash Contributions and Solutions Identif

After Two Rounds of Cognitive Testing [continued] 

Pretest Issue #4: • Although the term "cash" was not problematic for respondents, per se, in

indicated that when asked about "cash" only, nearly half of the respondencontributions of checks and/or money orders. 

• Respondents did not automatically think of alternative forms of cash, u

specifically probed about these alternative forms. 

Solution Issue #4:• Including the terms "checks, money orders, or credit cards" to the origin

only included "cash" resulted in improved reporting of events (i.e., respondlarger number of cash contributions than they had previously reportrespondents to more carefully consider their cash contributions prior to resp

• Some respondents reported they never would have reported "money o

cards," because they did not think of them as an alternative form of cash. Iadditional terms improved respondents' recall of "giving" events and genepresumably more complete, estimates of expenditures. 
TABLE 5 

Pretest Issues (Response Problems) Associated with th

Collection of Cash Contributions and Solutions Identif

After Two Rounds of Cognitive Testing [continued] 

Pretest Issue #5: • The use of the phrase "other persons not in your CU" was particularly tro

majority of the respondents. Its placement at the top of the list of ca

encourage respondents to consider all other types of cash contributions beducational, and political, etc. 

• Thus, many respondents provided a list of responses that should have b

latter categories (e.g., the religious, educational, and political categories)many respondents, the preponderant category of non-household contribthose contributions made to organizations, not individuals. 

Solution Issue #5:• The revised placement of the phrase "other persons not in your CU" at the 

categories was a substantive improvement; however, some respondents stiltheir family and/or friends when responding. They had to be reminded a the category included cash contributions made to strangers.
TABLE 6 

Pretest Issues (Response Problems) Associated with th

Collection of Cash Contributions and Solutions Identif

After Two Rounds of Cognitive Testing [continued] 

Pretest Issue #6: • The use of the phrase "churches and other religious organizations, exc

schools," did not offend any of the respondents, but a few mentioned that i

to use more inclusive language instead of only "churches," pointing out thahouses of worship commonly found in non-Christian religious organizatioinclusive and appropriate. 

• Some respondents thought that a government survey should be more repr

general American population. When queried explicitly about the addition "temple," "synagogue," and "mosque," all of the respondents reported they and inclusion of these terms within the question. 

Solution Issue #6:• The use of the phrase "religious organizations, including "churches, temple

but not including parochial schools?" was a definite improvement over t"churches and other religious organizations, excluding parochial schoorespondents reported being offended by any aspect of this phrase.
TABLE 7 

Pretest Issues (Response Problems) Associated with th

Collection of Cash Contributions and Solutions Identif

After Two Rounds of Cognitive Testing [continued] 

Pretest Issue #7: • The phrase "religious organization" was compared to the terms "religious g

of worship." Some respondents did not understand the term "house of w

unable to evaluate its usefulness and appropriateness. Other respondents 

"religious organization" to "religious group," because they perceived thinclusive and more formal than "religious group." 

• A few respondents reported the term "religious organization" would enco

range of religious groupings, such as religious charitable groups, houses religious day schools/camps. Thus, all types of religious groups coregardless of their degree of formality, their size, or their functions and goa

Solution Issue #7:• The use of the phrase "religious organization" appeared to generate 

respondents, even when specifically probed about this term. Respondents favored the term "religious organization," which was perceived to be the mmost clearly understood of the three terms.
TABLE 8(a) 

Pretest Issues (Response Problems) Associated with th

Collection of Cash Contributions and Solutions Identif

After Two Rounds of Cognitive Testing [continued] 

Pretest Issue #8:

• The majority of respondents agreed it was difficult for them to recall 

contributions for a three-month span, once they understood that they wetypes of cash contributions, including: 

• formal (such as payroll deductions for a recognized charitable organiza• informal (giving a small cash donation for an office birthday party). 

• Low-incidence and irregular events generated the most difficulty for respo

high-incidence and regular giving events, such as weekly or monthly tithes to a religious organization were the easiest for respondents to recall. 

• When the question was more inclusive and provided more examples of idio

respondents were able to recall a larger number of "giving events." Whand specific examples were provided to respondents, reports of "givrespondents increased substantially. 
TABLE 8(b) 

Pretest Issues (Response Problems) Associated with th

Collection of Cash Contributions and Solutions Identif

After Two Rounds of Cognitive Testing [continued] 

Pretest Issue #8 [Continued]:

• Typical respondent comments as a result of the provision of more example

the question included: "Oh, I didn't include the time I gave money for the that had had a fire in their house," or "Yeah, we had a retirement party two guy at the office," indicating they would not have normally recalled thesmore extensive cueing directed at such irregular low-incidence events. 

Solution Issue #8:

• Problems in recall of respondents' cash contributions for a three-month 

eliminated completely by these wording modifications. 

• However, there was some amelioration of recall burden, because the que

directive and specific about the recall task the respondent was to accomplish
TABLE 9(a) 

Pretest Issues (Response Problems) Associated with th

Collection of Cash Contributions and Solutions Identif

After Two Rounds of Cognitive Testing [continued] 

Pretest Issue #9:

• Some respondents were unable to recall the exact stem for the Ca

question(s), "Since the first of (month, 3 months ago) have you (or any mCU)" throughout the length of the Cash Contributions question(s). Sincneeds to remember: 

• the length of the reference period while responding to all of the Ca

question(s); and 

• to respond for him/herself and all other members of the Consumer Uni

• Respondents would benefit from additional cueing to remind them of 

including all reports Contributions questions. 

• Respondents appeared more likely to recall the length of the reference per

they had to report cash contributions made by other members of their CU. 

• After direct probing, some respondents generated additional reports of co

CU members. 
TABLE 9(b) 

Pretest Issues (Response Problems) Associated with th

Collection of Cash Contributions and Solutions Identif

After Two Rounds of Cognitive Testing [continued] 

Solution Issue #9:

• To facilitate respondents' ability to recall the exact stem for the Ca

question(s), "Since the first of (month, 3 months ago) have you (or any m

CU)" throughout the length of the Cash Contributions question(s), a bristem was introduced at two points throughout the Cash Contributions quest

• This abbreviated stem, "Have you (or any members of your CU):" was inc

places within the Cash Contributions question(s): 

• before the second question, "2. given any money, such as cash, check

or credit cards to benefit:" and 

• before the third question, "3. given any stocks, bonds, or mutual fun

organizations outside of your CU?." 

• This brief stem enhanced respondents' ability to recall the three-month 

When probed, respondents remembered they were responding for a threeven when provided with no additional cues; thus, the stem served to between question (1), (2), and (3), and made clarified the meaning of these 
Final Recommended Version of Cash Contributions Questions

Since the first of (month, 3 months ago) have you (or any members of your CU1. paid any of the following to help support someone outside of your CU: 

a. cash to college students living away from home? b. child support? c. alimony? [this option placed in the last position] Have you (or any members of your CU): 

1. given any money, such as cash, checks, money orders, or credit cards to b

a. educational institutions? b. political organizations? c. religious organizations, including "churches, temples, and 

including parochial schools? 

d. charities and all other organizations? e. any and all other persons not in your CU (such as friends, co-w

 homeless persons)? [this option placed in the last position] 

Have you (or any members of your CU): 

3. given any stocks, bonds, or mutual funds to persons or organizations outs
Conclusions

The results of cognitive testing resulted in the development of a revised vContributions questions for the Consumer Expenditure Survey, which is desig

 accomplish the following goals: • Resolve many problems associated with recall of low-incidence and irregu

yet minimizes respondent burden by providing specific "giving" instanexamples jog respondent's recall for extraneous and irregular "giving events

• Minimize the potential for negative associations for some respondents by 

and "child support" later in the list provided for the first question. 

• Increase additional "giving instances" likely to be recalled by the res

irregular check-writing events, by specifying equivalent forms for the term 

• Include several religious organizations which causes the question to be 

more representative of the varied respondents to the CE Interview (winegative consequence from respondents). 

• Include two additional abbreviated forms of cues to remind the respond

Contributions made by all members of their Consumer Unit (CU). 

• Overall, the revised version of the Cash Contributions questions has be

sensitive to respondent concerns about important social and personal issuand alimony, and should improve: 

• respondents' ability to recall their cash contributions; and • the accuracy of their reports.