IMPROVING COMPREHENSION AND RECALL IN THE CONSUMER 

EXPENDITURE INTERVIEW SURVEY: DISCREPANCIES IN COMPREHENSION AND RECALL AS A SOURCE OF NONSAMPLING ERROR * 

Leslie A. Miller, and Theodore Downes-Le Guin, Bureau of Labor Statistics 

Leslie A. Miller, 600 E. Str. NW., Rm. 5217 Washington, D.C. 20212 

The current paper describes research focusing on the Consumer 

Expenditure Interview Survey. Over the years, data analysis has 

indicated that this survey may be underestimating expenditure levels 

(Gieseman,1987). This, combined with the fact that researchers 

increasingly depend on surveys as a fundamental source of data 

(Anderson, Kasper, and Frankel, 1979)leads us to investigate ways 

to improve our data collection methods. The Bureau of Labor 

Statistics (BLS) has begun research to develop and test 

improvements in questionnaire wording and design for the Consumer 

Expenditure Interview Survey Questionnaire. Specifically, the 

research discussed below was designed to employ both cognitive 

laboratory research and survey methodology to evaluate and 

recommend a new collection format for the Health and Medical 

Expenditures (Section 15) of the Consumer Expenditure Interview 

Survey Questionnaire. 

We begin with a discussion of the Consumer Expenditure 

Survey, including the current format of the Health and Medical 

Expenditures Section. Baseline research in comprehension and 

recall is outlined, followed by the goals of the current research. The 

current research includes two phases: laboratory research on the 

cognitive approaches respondents use in answering questions, and a 

feasibility field test of the revised instrument to be conducted in fall of 

1989. The results of the laboratory research are presented and a 

discussion of how these results and cognitive theory proved useful 

and supportive in redesigning the instrument follows. 

THE CONSUMER EXPENDITURE SURVEY 

Description of the Consumer Expenditure Survey 

The Consumer Expenditure Survey program produces an 

ongoing and comprehensive flow of data on the purchasing habits of 

American consumers. The principal purpose of this survey is to 

provide data on the importance of different commodities in the 

Consumer's budget. The survey also reveals spending patterns for 

households with varying characteristics and has been a component 

in constructing "standard budgets" for various socio-economic groups 

(Pearl, 1975). The data also are used for economic research and 

analysis (U.S. Department of Labor, 1984). Data collection for this 

survey is carried out by the Bureau of the Census under contract to 

the Bureau of Labor Statistics. 

The continuing Consumer Expenditure Survey began in 1980 

utilizing two separate samples: (1) a Quarterly Interview Survey 

consisting of a rotating panel of approximately 5000 respondents who 

are interviewed once a quarter for five consecutive quarters with the 

first quarter of data used only for bounding purposes; and (2) a Diary 

Survey in which approximately 5000 respondents keep a daily log of 

expenditures for two consecutive one-week periods (BLS Handbook 

of Methods, 1988). Although both the Interview Survey and the Diary 

Survey collect consumer expenditure information, the approach of 

these surveys is different. The Interview Survey consists of 22 

sections collecting information on expenditures respondents can be 

expected to recall over a period of three months or longer. This 

includes fairly large expenditures such as rent, health and medical 

expenses and educational expenses. The Diary Survey, on the other 

hand, is designed to collect detailed data on frequently purchased 

smaller items such as food and beverages, tobacco, non-prescription 

drugs, housekeeping supplies, and personal care services and 

products. Diary survey items are generally less likely to be recalled 

by respondents over the three month recall period .used in the 

Interview Survey. 

Status of Section 15 Prior to Research 

Section 15 of the Consumer Expenditure Interview Survey 

concentrates on Medical and Health Expenditures. As stated above, 

the purpose of this section is to obtain information about medical 

expenses paid for by the Consumer Unit (CU) and any 

reimbursement for these expenses in order to determine net out-ofpocket medical expenses for a sample CU. A Consumer Unit is 

defined as a person or group of persons in a household related by 

either blood, marriage, adoption, or who, as a person or as a group, 

are independent of all other persons in the household for payment of 

their major expenses. Section 15 also covers medical expenses paid 

by the Consumer Unit for care of persons outside the Consumer Unit. 

Section 15 is divided into three parts. 

1. Part A contains screening questions to determine if the CU 

has paid or been reimbursed for medical or health care, services 

or items during the reference period. 

2. Part B obtains detailed information about medical 

payments. 

3. Part C obtains detailed information about medical 

reimbursements. 

An information booklet accompanies the interview survey and is 

intended to serve as a cueing device for respondents. 

STATEMENT OF PROBLEM 

Collecting accurate information from people regarding their 

health payments and reimbursements has been acknowledged by 

researchers as a difficult task (Anderson, Kasper, and Frankel, 

1979). This task difficulty is reflected in reports from various sources 

which offer recommendations for improving respondent recall and 

comprehension in the Interview Survey. In January of 1987, the 

Bureau of Labor Statistics organized a Questionnaire Design 

Advisory Conference to explore the contributions cognitive 

psychology could offer to survey methodology (Bienias, Dippo and 

Palmisano, 1987). As a result of the Advisory Conference, BLS 

contracted with the Research Triangle Institute (RTI) for preliminary 

cognitive research on certain sections of the Consumer Expenditure 

Survey (Lessler, 1988, 1989). From the outset Section 15 appeared 

to pose a number of problems for respondents; these problems were 

investigated by RTI and discussed in memoranda and interviewer 

intercoms from the United States Bureau of the Census (Green, 

1988; Lessler 1988, 1989). 

Report from the Research Triangle Institute (RTI), memoranda 

and intercoms 

Research conducted by RTI included: (1) detailed examination 

of the response task (forms appraisal); (2) gathering information from 

CE interviewers; (3) detailed observations of the response process 

(response protocols); and (4) examination of alternate methods of 

assessing respondent comprehension. 1 Major issues raised by 

RTI's research and Census memoranda and intercoms include: 

• Cues and markers. Currently the questions do not assist 

respondents in placing events in time by taking advantage of 

personal landmarks or cues respondents might use to anchor 

different reference periods. For example, Section 15 asks both 

for the month of each payment/reimbursement and the month 

and year in which the corresponding medical care was received. 

Respondents may not understand that these are separate 

questions. 

502 
• Difficult question phrasing. Many questions use 

technical terms that are specific to the questionnaire or do not 

have a widely shared definition among respondents. Response 

categories may not correspond to respondents' methods of 

classifying expenditures. Other questions include "hidden 

instructions" in the interviewer manual; these instructions modify 

the question meaning slightly but are not read to the respondent. 

In addition, RTrs conversations with interviewers indicated that 

questions may be awkward in length and unnatural wording. 

The problems above served as the basis for identifying 

section-specific comprehension and recall tasks to be tested in 

the BLS cognitive laboratory. The goals of the current research 

are to: 

• Inform respondents fully about the task to be completed by 

improving contextual cues and comprehension through rewording 

and reordering questions, clarifying technical terms, removing 

hidden instructions and improving the Information Booklet. 

• Improve recall by changing the task requirements and 

interview conditions to better fit respondent capabilities, 

encouraging the use of records and other recall aids for use 

between and during interviews. 

• Evaluate other structural and grammatical changes. 

LABORATORY RESEARCH 

The current research involved two phases. Phase I included 

group and personal interviews focusing on comprehension and recall 

problems, concluding with the design of a new data collection form. 

Phase II (to be completed in the fall of 1989) consists of a feasibility 

field test of the new data collection form. 

Subjects 

Sixty respondents were recruited (via radio advertisements and 

flyers posted in the Washington metropolitan area) to participate in 

laboratory research. All respondents were selected with an attempt 

to balance characteristics such as age, race, gender and income. 

Respondents were reimbursed $15 for time and travel expenses. 

General Procedure 

Participants in the laboratory research were informed of the 

nature of the research and were asked to complete a screening 

questionnaire to obtain demographic information. In addition, 

respondents signed a consent form. Upon arriving at the laboratory, 

all participants in group interviews were asked to self-administer 

several written questions pertaining to concepts used in Section 15 

and recall strategies for expenses. Participants were then asked to 

discuss as a group their interpretations of several concepts and 

definitions and how they felt they would best recall information about 

medical expenditures. The moderator used outlines to stimulate and 

guide discussion; each participant was encouraged to express 

his/her individual responses in addition to group discourse. A limited 

number of personal interviews were conducted using the same 

questionnaire and discussion outline. 

Data Analysis 

Laboratory material was qualitatively interpreted. Written 

responses were transcribed verbatim and grouped into broader 

categories. Verbal responses were reviewed on video tape and 

major ideas were documented. All data were then analyzed using 

contingency tables. 

Qualitative Research Caveat 

The research reported in this paper employed focus groups and 

personal interviews both as primary means of data collection and as 

supplements to a large-scale sample survey. The verbal protocols 

503 

and the concepts analyzed from the self administered questions 

served as a basis for revising the Interview Survey instrument and as 

a precursor to a quantitative feasibility field test. 

From a social science viewpoint, focused group discussions can 

stand on their own as data collection method or serve to supplement 

both quantitative and other qualitative methods. Some researchers 

have argued that focus groups are a good tool for preliminary or 

exploratory research, but require subsequent quantitative research 

for validation (Morgan, 1988). More recent use of focus groups in the 

social sciences has shown, however, that well-conducted qualitative 

research need not always act as a supplement to quantitative 

studies. Focus groups have the advantages of providing relatively 

economical, easily-gathered information using a larger group of 

participants than most qualitative methods (Kreugar, 1988). Our 

research issues, which focus on detailed internal processes people 

use in forming answers to survey questions, would be very difficult to 

investigate in a sample survey. Providing that focus groups are not 

interpreted as probability samples, and participants' behavior and 

attitudes are not presented as projectable to a larger population, the 

technique proves extremely useful in gathering detailed information 

about respondent behavior. 

RESULTS OF LABORATORY RESEARCH 

The majority of the respondents had similar difficulties with 

question wording, term definition and recalling expenditures and 

reimbursements. The following breaks down the most commonly 

observed difficulties into problems of comprehension and problems of 

recall, followed by changes made to the questionnaire to address 

these problems. 

Comprehension 

Qualitative observations from the laboratory interviews confirm 

the lack of effort many respondents make at recalling past events 

and answering questions within the desired definition and temporal 

boundaries. Offering respondents an introductory statement at the 

beginning of the section may motivate respondents to consider the 

question more carefully, and is consistent with the Cannell et al.'s 

recommendation (1987) that researchers help respondents to 

understand their reporting role in the interview. In addition, 

increasing question length is thought to elicit higher frequency and 

quality reporting (Laurent, 1972). Thus, the introductory sentence 

recommended was, "Now I am going to ask you some questions 

about medical payments and reimbursements. We will begin with 

your payments." 

In addition, the laboratory research addressed a number of 

specific comprehension problems common in Section 15's question 

wording and response categories. 

Poorly Defined or Technical Terms 

Laboratory research revealed that Section 15 uses technical 

terms with a variety of possible interpretations (Miller, 1988), a 

common issue in complex survey instruments. Before revision, the 

questionnaire asked respondents, "Have you (or any members of 

your CU) made any payments for any of the following medical 

services? Include all payments, even those for persons who are not 

CU members." In the case of reimbursements, respondents were 

asked, "Have you (or any members of your CU) received any 

reimbursements from insurance companies or others for any medical 

services?" Respondents also were asked whether they made any 

payments or received any reimbursements for physician's services 

and services by a practitioner other than a physician. When asked to 

define the terms payment, reimbursement, physician and practitioner 

in the context of these questions, participants provided a variety of 

sometimes conflicting definitions and expressed the need for 

clarification. Confusion about these terms suggests that survey 

respondents' most salient definitions may not correspond at all to the 

definitions intended by BLS. For payments, the majority of laboratory 

participants applied rather abstract definitions, such as money issued 

in the form of a check, money paid out-of-pocket and anything paid 
out for medical services. When probed, several respondents 

wondered if payment includes only out-of-pocket expenses or related 

medical expenses such as health insurance premiums and credit 

transactions. 

Participants' definitions of reimbursement correspond more 

closely to the meaning intended by BLS. Most participants defined 

reimbursements as money sent by an insurance company to cover 

all or part of a medical expense, not including insurance payments 

made directly to the hospital. However, several respondents 

qualified their statements in a manner not consistent with the sectionspecific definition. Several vague definitions and questions -- money 

sent to me by an insurance company, refunds from a doctor's office 

for overpayment, not sure how to report a credit from a provider -- 

suggest that reimbursements, like payments, are ambiguously 

defined in respondents' minds. A similar difficulty arises when 

respondents try to distinguish physicians and practitioners. For 

example, several participants insisted that, although chiropractors do 

not hold M.D.s they are physicians because they perform a medical 

service. Others classified chiropractors as practitioners because 

they do not have M.D.s. Nearly half of the participants agreed that a 

medical professional must have an M.D. to qualify as a physician. 

On the other hand, when asked to define practitioner, most 

participants said they didn't know to what the term referred. 

Evidently respondents categorize medical professionals differently 

based on personal associations and unstandardized cues. The 

degree to which these definitions differ from the meanings intended 

by BLS throws doubt on the validity of measures based on these 

terms. Because the Consumer Expenditure Interview Survey must 

be understandable to respondents who represent all levels of 

cognitive sophistication, the solution used in rewording the section 

was to simplify terms and provide less ambiguous definitions so that 

all respondents interpret questions similarly (Dillman, 1978; Jabine et 

al., 1984). 

Events vs. Expenditures and Expenditures vs. Medical 

Expenditures 

As we have seen above, several questions in Section 15 have a 

tendency to prompt respondent misunderstanding; over the course of 

the interview this misunderstanding increases respondent burden 

and decreases respondents' desire to pay close attention to the task 

(Lessler, 1988, 1989; Miller, 1988). Lack of attention is evidenced by 

reports of irrelevant or inappropriate information in response to 

payment and reimbursement questions. Many laboratory 

participants, when asked if they had made payments or received 

reimbursements for medical services, tended to explain why they 

consulted a provider and what services they received rather than the 

germane payment or reimbursement. The original question wording 

may have induced this effect by emphasizing services over 

payments. Before revision the questionnaire asked, "Have you (or 

any members of your CU) made any payments for any of the 

following medical services?" followed by a cueing list of services 

presented in the Information Booklet. A slight rewording reverses the 

emphasis: "Since the first of (MONTH, 3 MONTHS AGO) has 

anyone in your CU made any payments for...?" followed by a similar 

cueing list. 

In addition, when asked for the definition of payment in the 

context of Section 15, laboratory participants frequently reported nonmedical expenditures such credit payments (car loans and bank 

cards) and monthly bills. Just as the intended emphasis on 

payments/reimbursements can become lost in respondents' attention 

to services, emphasis on medical payments/reimbursements can 

become confused with other types of expenses discussed during the 

interview. In addition to emphasizing payments over services, the 

rewording presented above compels respondents to focus on the 

appropriate type of payments by cueing them with categories of 

specific services presented in the Information Booklet. 

Similarly, a later sequence of questions reads, "What care or 

service was received and who received it?" and, "Was the person a 

CU member?" The questions do not signal whether respondents 

previously have referred to items, care or services; in instances in 

which respondents have mentioned medical items, and the question 

provides a faulty cue. Rewording the question to read, "What was 

the care (service, item) and who received the care (service, item)?" 

should result in better respondent understanding. 

Other 

Before revision, Section 15 listed Services by a Practitioner Other 

than a Physician under Other Medical Care Services. This 

categorization poses problems since respondents are unable to 

define non-physician medical practitioners, as discussed above. For 

the field test, services by non-physician providers have been - 

grouped with physician services to form a new category of Physician 

and Other Medical Professionals Services. This categorization serves 

three purposes. First, Physician Services is removed as a subset of 

In-Patient Hospital Care. Second, practitioner is changed to Other 

Medical Professionals to reduce ambiguity. Finally, all services by 

medical care providers are collected in the same expenditure 

category, decreasing the chance of respondents reporting all 

provider expenditures as Physician Services. 

Recall 

Order of Recall 

Currently, Section 15 is structured to collect payments made for 

certain expenditure categories, then reimbursements for these 

categories. This payment-then-reimbursement structure is repeated 

on every page until all expenditure categories are exhausted. Is this 

the best way to collect medical expenditure information? Data from 

laboratory research suggest otherwise. Many participants said they 

would prefer to report payments and reimbursements in an 

unstructured format. Given a choice of reporting structures, 

however, about an equal number of participants said they prefer 

reporting specific payments followed by specific reimbursements 

(38), and a structure in which all payments are reported before all 

reimbursements (39). This is in contrast to a lesser number of 

respondents preferring to report using the current structure. An 

educational difference is evident with respect to reporting preference: 

more participants who had attained a higher education level (BA or 

above) said they prefer the more structured approach of reporting 

payments followed by any reimbursement received for that payment. 

Less educated participants tended to prefer reporting all payments 

followed by all reimbursements. Whatever the reporting structure, 

however, respondents said they prefer to report first by CU member 

and month of payment/reimbursement. 

Placing events in time 

Most laboratory participants reported they would not use any 

temporal markers or cues to help them recall payments and 

reimbursements during the three month reference period. As 

mentioned above, however, participants say they would use the CU 

member for whom the payment/reimbursement was made and the 

month the payment/reimbursement was made/received to aid in 

reporting. Thus CU member and month are serving as mnemonic 

cues. Other research has pointed to memorable events and 

temporal landmarks as increasing respondent self-reports and 

reducing false-positive reports (telescoping) (Moss and Goldstein, 

1979; Loftus 1983). While the Interview Survey does not provide 

respondents formal opportunities to use personal landmarks, the 

laboratory research suggests that many respondents attempt to 

create their own landmarks by remembering the three-month history 

of specific CU members. We can speculate that respondents may 

associate the reference period dates with personal events, such as a 

CU member's birthday. 

FEASIBILITY FIELD TEST 

Subjects 

Two hundred twenty-two sample addresses will be selected 

504 
using a non-probability sample of households from four Census 

Primary Sampling Units (PSUs). 

Procedure 

Thirteen Senior Field Representatives (SFRs) and thirty-one 

interviewers will conduct field interviews of revised Section 15. The 

interviews will be conducted according to standard Census practice, 

with changes in collection procedures for the revised sections only (2 

other sections are being tested also). SFRs and interviewers will be 

apprised of revisions and thoroughly trained in revised collection 

procedures. 

Debriefing 

In order to obtain additional criteria for comparing the 

effectiveness of the experimental instrument, a set of debriefing 

questions will be given to interviewers to administer to the 

respondents. In addition, interviewers themselves will complete a 

debriefing at the conclusion of each interview to collect their 

evaluations of respondents' reactions to the revised questions. The 

purpose of the debriefing instruments is: (1) To determine whether 

the scope of the survey and definitions of specific terms has been 

adequately communicated to respondents and (2) to help evaluate 

whether the instrument is more successful in prompting recall of the 

sorts of incidents which respondents are likely to shield or forget. 

Data Analysis 

Approximately 40 field interviews will be audio taped and 

transcribed. All transcribed tapes will be coded for both respondent 

behavior (question answering) and interviewer behavior (question 

reading). Debriefing data will be analyzed along with coding results 

to provide an overall assessment of the effectiveness of changes in 

the survey instrument in improving respondent comprehension and 

recall. 

COMPREHENSION AND RECALL: DISCUSSION OF 

LABORATORY WORK AND THEORY 

In an attempt to increase comprehension and recall, and 

ultimately to increase the accuracy of survey responses, research 

and literature from sociological and psychological sciences aided us 

in redesigning the Health and Medical Expenditures Section of the 

Consumer Expenditure Interview Survey. With the current research 

discussed and the results presented, what remains is a review of the 

applicability of theory to the research findings. 

Survey respondents' task of answering questions can be difficult, 

especially when it involves retrieving factual and retrospective 

information rather than present attitudes. Survey researchers have 

acknowledged this by paying more attention to the mental processes 

that take place in answering questions (Bradburn et al., 1987; 

Cannell et al., 1981). After all, the nature of these processes 

ultimately determines the amount and accuracy of reported data, 

which is to say the very utility of the survey results (Bradburn et al., 

1987). The researcher's job is to encourage respondents to answer 

questions accurately and with minimum cognitive burden. To do this 

the researcher must structure questions to mimic the mental 

processes respondents use when answering and retrieving 

information outside of the survey context. Results from cognitive 

psychology can be useful in understanding the question-answering 

process, particularly for comprehension and recall, and help us meet 

this objective. 

Most issues related to comprehension deal with respondent 

understanding of the task and respondent meaning assignment to a 

survey question (Torangeau, 1984). With respect to ambiguities in 

terminology, the aim should be to reduce abstract or ambiguous 

terminology to account for individual and cultural variations in the 

understanding of terms (Jabine et al., 1984). Ambiguity arises not 

only with abstract concepts but with seemingly concrete terms -- in 

our case, for instance, physicians and payments. Reduction of these 

ambiguities lends more credence to the assumption that all 

respondents have understood questions similarly (standardized 

stimuli) and that the questions being asked are interpreted as the 

researcher intended (validity). 

Methods of reducing ambiguities and clarifying terms include the 

use of commonly-understood vocabulary, providing examples of 

complex terms and explicitly defining unusual terms. The choice is 

dictated by preference, convenience and the constraints of the 

survey project. Providing examples as a form of definition is a typical 

procedure in survey design (Jabine et al., 1984; Tucker, Vitrano, 

Miller, and Doddy, 1989) but involves the risk of influencing 

respondents' answers. In a separate BLS research project, for 

instance, Diary Survey respondents were observed to report 

products used as examples on the diary pages with greater 

frequency than similar products that were not used as examples. 

Researchers have yet to demonstrate whether examples should 

consist of items that are typical, atypical or bounding to the response 

category. 

In the Health and Medical Expenditures Section, laboratory 

research demonstrated that several key words -- physician, 

practitioner, payment and reimbursement -- had non-universal 

definitions. Due to the length of the Consumer Expenditure Interview 

Survey,2 several types of medical providers were collapsed under 

the categorical term physician, with examples of types of providers to 

clarify the category's meaning. In the case of the terms payment and 

reimbursement, the questions were reworded to define the terms 

within the context of the survey rather than allowing respondents to 

use the more common, broad definitions. 

As a new perspective on a perennial issue, researchers have 

examined the literature of cognitive psychology for methods to 

improve the frequency and quality of retrospective reporting. Several 

methods suggested for reducing the impact of memory decay on the 

reporting of events include: 

1. Reducing the reference period (Cannell and Fowler, 1965 

in Fowler, 1988). 

2. Increasing the number of questions about events to allow 

more time for respondents to recall and form answers (Madow, 

1963 in Fowler, 1988) 

3. Giving respondents a chance to reconsider their answers 

by mailing a verification of responses or requesting the same 

information more than once in the survey. 

4. Using a bounding or reinterview procedure to give 

respondents an anchoring point and to sensitize them to the 

types of information that will be requested. 

5. Asking respondents to keep a diary to report events 

(Sudman and Bradbum, 1974 in Fowler, 1988). 

The use of such methods usually is limited by the requirements 

of, and constraints upon, a particular survey. The Consumer 

Expenditure Interview Survey uses some methods for increasing 

recall, such as bounding interviews, but is prevented from using 

others by reference period requirements and limits on interview 

length. What happens when the researcher is unable to maximize 

respondents' recall abilities through conventional means? Cognitive 

theory provides a number of alternatives. As we describe below, 

theories of false negative/false positive reports and scripts and 

schemata proved useful in reordering the collection sequence of 

payments and reimbursements in Section 15; judgmental heuristics 

and part set cueing provided us with information about stimulating 

recall with landmark events and examples. 

False negative/false positive reports provide an illustration of the 

different types of distortions possible when respondents attempt to 

answer questions, including failure to report information due to failure 

of retrieval, carelessness or unwillingness of the respondent to make 

the necessary retrieval efforts and falsely reporting events outside of 

the specified reference period (telescoping). We can increase the 

power of retrieval by: (1) providing an introductory sentence to 

505 
explain the purpose of the data being collected; (2) restructuring the 

interview to better suit respondents' scripts and (3) providing recall 

cues and markers so respondents use the reference period. Using 

an introductory sentence in Section 15, re-ordering the collection 

sequence of payments and reimbursements and presenting month 

and CU member cues may decrease the likelihood of false reports. 

Theories of scripts and schemata served as the basis for the 

revision of the collection sequence of payments and reimbursements. 

Many psychological researchers agree that the storage, retrieval and 

interpretation of past experiences is managed by higher-level 

knowledge structures, variously named scripts and schemata. 

Schank and Abelson (1977) have conducted extensive research into 

mnemonic scripts which they believe serve to guide steriotypical 

sequences of action. Their research and that of others (Bower, Black 

and Turner, 1979) indicates that scripts/schemata exist primarily to 

encode the prevalent sequence of events or the most typical 

information in a specific or general situation; subsequent recall is 

influenced by these schemata (Anderson, 1985). Thus, in some 

surveys, schema theories can serve as a valuable bases for 

estimating missing information and for correcting inaccurate 

information (Anderson, 1985). For example, in the context of a 

survey question about past dental appointments, respondents may 

refer to a script for their visits to the dentist. The "dentist script," 

when properly evoked by the question, prompts respondents to 

answer according to their stereotypical dental appointment behavior 

rather than by retrieving memories of actual behawor. In an oft-cited 

example, respondents frequently report going to the dentist twice a 

year because most dentists recommend twice-yearly checkups -- 

even though respondents may actually go annually or less often. 

While use of a script in this instance has a negative influence on 

response accuracy, awareness that the script exists for a majority of 

respondents allows the researcher to predict the error and to design 

the question in such a way to discourage reliance on a stereotypical 

script. For example by providing reasons for visiting the dentist as a 

cue. 

In the case of the current survey, our intent was to increase 

reporting of payments and reimbursements for medical expenses 

(which ultimately begin with recall of the visit itself) by increasing 

recall abilities of the respondent. Given the finding that most lab 

participants considered reimbursements as a subsequent event 

contingent on (usually previous) payments, our judgment was to 

reorder the collection sequence to collect all reimbursements after 

payments. In this case, we did not intend to employ respondents' 

scripts for specific visits to health professionals as an aid for 

collecting payments and reimbursements, but to take advantage of 

their apparent schema for the payments/reimbursement sequence. 

Judgmental heuristics lends further support to the notion that 

reporting can be improved by encouraging respondents to exercise 

their own retrieval methods. According this research, people use 

different strategies to estimate past behaviors and events (Jabine et 

al., 1984). Some respondents anchor estimates with average 

frequencies ("1 usually go to the dentist twice a year...") and then 

adjust up or down for the particular question ("...but this year I went 

three times because of a cavity"). For these respondents, reporting 

may be improved by providing anchoring information. The difficulty 

arises in identifying which strategies or heuristics (if any) the majority 

of respondents will use in answering a question. A more thorough 

understanding of the process by which questions are answered 

should yield more accurate estimates. For the Interview Survey, 

laboratory research provided us with information as to the retrieval 

methods preferred by most respondents; these methods were used 

in developing cues such as an optional calender to report medical 

payments and reimbursements by CU member or by date.3 

Finally, part set cueing theory suggests ways to increase the 

amount of information reported by survey respondents. Part set 

cueing relies on some members of a category as cues to maximize 

the amount of information reported for all members of the category. 

As with examples in complex questions, it is not yet clear whether the 

representative members should be common examples or atypical, 

boundary cases. Part set cueing, like all the cognitive theories 

above, holds great promise for increasing comprehension and recall 

in the survey context. 

Theories from cognitive psychology go beyond a simple stimulusresponse description of the process respondents use in answering 

survey questions. By incorporating these theories into laboratory 

research conducted for the Health and Medical Expenditures Section 

of the Consumer Expenditure Interview Survey, we were able to 

examine in-depth the reasons respondents may misunderstand 

concepts and may be unable to recall expenditures. This information 

aided in redesign of the data collection instrument to better conform 

to respondents' "natural" comprehension and recall processes. The 

results should be more and better data for BLS, and a less 

burdensome task for respondents. 

References 

Anderson, K. J., and Frankel, M. R. (1979), Total Survey Error, 

San Francisco: Jossey-Bass. 

Anderson, J.R. (1985), Cognitive Psychology and Its Implications 

(2nd ed.), New York: W.H. Freeman and Co. 

Bienias, J., Dippo, C., and 

Palmisano, M. (1987), Questionnaire Design: Report on the 1987 

BLS Advisory Conference, Washington, DC: U.S. Department of 

Labor, Bureau of Labor Statistics. 

Bower, G. H., Black, J. B., and Turner, T. J. (1979), "Scripts in 

Memory for Text," Cognitive Psychology, 11,177-220. 

Bradburn, N., Rips, L. J., and Shevell, S. K. (1987), "Answering 

Questions: The Impact of Memory and Interference on Surveys," 

Science, 236, 157-161. 

Cannell, C. F., Miller, P. V., and Oksenberg, L. (1981), "Research on 

Interviewing Techniques," in Leinhardt (Ed.), Sociological 

Methodology, 389-433, San Francisco: Jossey-Bass. 

Cannell, C. F., Groves, R. M.,Magilavy, L. J., Mathiowetz, N. and 

Miller, P. V. (1987), "An Experimental Comparison of Telephone and 

Personal Health Surveys," Vital and Health Statistics, Ser. 2, No. 

106, Washington, DC: U.S. Government Printing Office. 

Dillman, D. A. (1978), Mail and Telephone Surveys: The Total 

Design Method, New York: Wiley and Sons. 

Dippo, C. S. (1989). The Use of Cognitive Laboratory Techniques 

For Investigating Memory Retrieval Errors in Retrospective Surveys. 

Unpublished manuscript. 

Fowler, F. J., Jr. (1988), Survey Research Methods, Vol. 1, Newbury 

Park, CA: Sage Publications. 

Geiseman, R. W. (1987). The Consumer Expenditure Survey: 

Quality Control by Comparative Analysis. Monthly Labor Review, 

March, pp. 8-14. 

Green, C. S. (Sept. 8, 1988), Suggestions for the SORAR and CE 

Surveys, United States Bureau of the Census, Internal Memorandum 

to E. Wilson Burdorff, Washington, DC 

Jabine, T. B., Straf, M. L., Tanur, J. M., and Tourangeau, R. (Eds.) 

(1984), Cognitive Aspects of Survey Methodology: Building a Bridge 

Between Disciplines: Washington, DC: National Academy Press. 

Kreugar, R. A. (1988), Focus Groups: A Practical Guide for Applied 

Research, Newbury Park, CA: Sage Publications. 

Laurent, A. (1972), "Effects of Question Length on Reporting 

Behavior in the Survey Interview," Journal of the American Statistical 

Association, 67(338), 298-305. 

506 
Lessler, Judith (1988), Consumer Expenditure Interview Survey 

Preliminary Cognitive Laboratory Studies, Report to Bureau of Labor 

Statistics, Research Triangle Park, North Carolina: Research 

Triangle Institute. 

Lessler, Judith (1989), Recall Strategies for Questions from the 

Consumer Expenditure Survey, Paper prepared for the AAAS Annual 

Meeting, Research Triangle Park, North Carolina: Research Triangle 

Institute. 

Loftus, E. (1983) "Since the Eruption of Mt. St. Helens, Has Anyone 

Beaten You Up? Improving the Accuracy of Retrospective Reports 

with Landmark Events," Memory and Cognition, 11(2), 114-120. 

Miller, L. A. (December 21, 1988), Census, CPI and CE Definitions of 

Physician and Practitioner, Bureau of Labor Statistics Internal 

Memorandum to P. L. Hsen, Washington, DC 

Miller, L. A. (December 22, 1988), Results for the First Part of Phase 

I- Consumer Expenditure Interview Survey, Section 15 (Medical 

Expenditures), Bureau of Labor Statistics Internal Memorandum to P. 

L. Hsen, Washington, DC 

Morgan, D. L. (1988), Focus Groups as Qualitative Research 

(Qualitative Research Methods, Series 16), Newbury Park, CA: 

Sage Publications. 

Pearl, R. B. (April 21, 1975), Methodological Analysis of 1972-1973 

U.S. Expenditure Survey Data: First Phase Report, Unpublished 

Research Report, U.S. Bureau of the Census, Washington, DC. 

Schank, R. C., and Abelson, R. (1977), Scripts, Plans, Goals and 

Understanding, Hillsdale, NJ: Lawrence Erlbaum Associates. 

Tourangeau, R. (1984). "Cognitive Sciences and Survey Methods." 

in Cognitive Aspects of Survey Methodology: Building a Bridge 

Between Disciplines. T. Jabine, et al., Washington, DC: National 

Acedemy Press. 

Tucker, C., Vitrano, F., Miller, L., and Doddy, J., (May 1989), 

Cognitive Issues and Research on the Consumer Expenditure Diary 

Survey, Paper presented at the annual conference of the American 

Association for Public Opinion Research, St. Petersburg, Florida. 

U.S. Department of Labor, Bureau of Labor Statistics (1984), 

Consumer Expenditure Survey: Interview Survey, Bulletin 2267, 

Washington, DC: U.S. Government Printing Office. 

U.S. Department of Labor, Bureau of Labor Statistics (April 1988). 

BLS Handbook of Methods (Bulletin 2285). Washington, DC: U.S. 

Government Printing Office. 

*The views expressed are attributable to the authors and do not 

necessarily reflect those of the Bureau of Labor Statistics 

Reviewed by Paul L. Hsen and Maria P. Fracasso 

1RTl's research involved many sections of the Consumer 

Expenditure Interview Survey. However, only those issues viewed by 

the present authors to be relevant to Section 15 are included. 

2The Interview Survey averages two hours in length 

3Unfortunately the calendar was not adopted as part of the Interview 

Survey 

507