aD

19

to be of concern.

Other Recommendations

Perhaps the greatest potential for improvement in data quality relates to
the within-household selection of the respondent. Intuitively, I believe that
potential respondents should be restricted to either a male or female head of
household. Uncles, aunts, nephews, ete. should not be eligible respondents.
Once the respondent is selected, efforts should be made to ensure that this
individual is the target respondent for-.future interviews. If the target
respondent cannot be reached after three callbacks, then a co-head-of household
could be interviewed. This may be the easiest way to improve response quality
at a relatively low cost.

To test whether or not such procedures could lead to improved reporting,
family members could be interviewed individually and then the consistency of
responses compared across reports. Efforts should be made to conduct such
tests in communities in which unemployment is very high. Hopefully, the
cooperation of major employers in the area could be obtained in order to gauge
the accuracy of some of the responses. Such research may help to determine if
there are characteristics of a “most responsible respondent" for which
screening questions could be developed.

It seems obvious that in conducting both small field experiments and
laboratory research, the subjects most likely to have problems with the
questions should be recruited. As suggested above, this argues for testing in
communities with significant unemployment, and where there are a minimum number
of companies that dominate employment and are willing to permit record checks
to aid in validity assessment. Some research should also be conducted with

“typical, employed respondents, but it seems that the questions and concepts


13

members. Unless the interviewers are incredibly motivated to supply the best
possible data and well-trained, it seems easy to imagine that such callbacks
may not always be made,

Research into the use of an initial screening interview may prove
beneficial. Screening questions might provide information concerning the
importance of having more than one respondent available during the interview
and the chances of arranging a time when more than one CU member could respond.
Screeners also may suggest which CU member would be in a position to provide
the highest quality data. Also, it would provide an opportunity to estimate
the length of the CE for a given CU and allow the respondent to choose whether
s/he would rather complete the interview in one or two sessions.

Changes in which member of the CU responds to the CES across quarters
probably adverseiy affects estimates. Efforts should be made to identify the
respondent in the CU capable of providing the highest quality reports and
efforts should be made to conduct each of the five interviews with this single
respondent.

‘ Ideas for possible improvements may be generated through conducting
surveys with CES interviewers. Information may be obtained on their attitudes

and opinions about the survey and perceived problems with specific questions.

aN

cae Specific question wording problems

There are a number of questions which are probably difficuit for
respondents to answer because of failure to understand the question or failure
to retrieve the information needed to accurately respond. Many of these
questions appear appropriate for laboratory research. A few examples of
questions which are candidates for lab research are listed below.

Section 1, Q6 - When the question is asked in this manner, can people


lead to refusals to participate. (One may argue that perhaps the reports
obtained from those that would refuse are of such poor quality that an initial
refusal is not too great a loss.) Probably an "intermediate" manipulation of
commitment (in which respondent motivation is increased somewhat without
pushing so hard that refusals result) is most desirable. A positive aspect of
the commitment is that very few resources are needed to implement such a
change. Small field tests could be conducted in which the commitment
manipulation is compared to a control group in which the manipulation is not
used. Results across groups may be compared on the accuracy/completeness of
reporting, the percentage of respondents using recall to complete the diary,
the number of initial refusals to participate and mortality rates, and general
attitudes and opinions about the diary.

There may be some benefit to setting up a specific time each day for
completing the diary. This may allow the interviewer to show concern and
empathy for the schedule of the respondent which may increase the respondent's
motivation to comply with the task. In addition, it would reinforce the idea
that it is important to complete the diary each day.

Concern was expressed about how to motivate respondents to accurately
complete the diary during the second week. The interviewer certainly should
assess and provide appropriate feedback pertaining to the performance during
the first week. The reasons why it is important to obtain accurate information
on a daily basis should be reiterated. The initial signed commitment may be
repeated and initialed by the interviewer and the respondent. There seems to
be concern that the interviewers place too little emphasis on the diary and it
seems likely that this feeling may be communicated to the respondent when the

diary is picked up after the first week. .It seems imperative that the


20
of greatest concern pertain to the unemployed, and thus effort should be made
to obtain a mix of subjects that could provide data which result in the
greatest benefits.

SUMMARY
A number of possible methods for improving the quality of reporting have
been presented. Theis section summarizes the recommendations viewed as
research priorities. Suggestions for the diary, expenditure interview, and CPS
are provided below.

The Diary

(1) Focus groups and/or in-depth interviews should be conducted with
misleaders and resisters to determine if reporting behavior can be improved for
these groups of respondents. Additional data should be collected on how
respondents complete the diary. These are low cost research alternatives which
should help determine if data collected in its present form can be
substantialiy improved and may suggest some methods for possibly doing so.

(2) The effectiveness of a commitment manipulation should be tested. This
offers some opportunity for improved quality of reporting at a relatively low
cost.

(3) Electronic scanner purchase panels should be used in between subjects
tests of manipulatons so that results can be compared to some (accurate)
criterion. BLS should also begin to consider how electronically scanned
purchases could be used to directly improve expenditure estimates.

CES Interview

(1) Telephone interviews should be conducted with a sample of respondents who
have recently completed the personal interview to guage attitudes toward the

survey and perceptions of the completeness and accuracy of their responses.


APPENDIX
AGREEMENT FORM
That's the last of this set of questions. The rest of the

questions are about health, your daily life, and how you have
been ‘feeling lately. It is important for us at the Ur

of Michigan to get an accurate picture of these ¢

this area, We have selected a small

samp.e of peocle to represent this

these people to give us extra cooperation and try hard to
an accurately, so we Can get accurate :nformation about

health. You are one of the peopie who we hope is willing
to make tnis

fort

Here is an Agreement which explains wnat we are asking
you to do. (HAND OVER AGREEMENT STATEMENT.)
AS you © see, it says, “I understand that the information
fr Interview must be very accurate in order to
be us: This means that I must do my best to give ac-

curate and complete answers. {| agree to do this."’ We are

asking people sign this agreement so that we can be sure
i understand what we are asking them to do. The
Agree: r you to keep for yourself. [t is up to you
to decide ou are willing to agree to do this, we'd like
to sign your name here (POINT OUT LINE). Down
18 a statement about confidentiality, and [
ame here (POINT OUT LINE). (IF R HAS
DY SIGNED): Are you willing to make the
to continue the interview?

wel sign

NOT ALRE

AGREEMENT
! understand that the information from this
interview must be very accurate in order to be
useful. This means that I must do my best to
give accurate and complete answers. | agree to

do this

Signature of Respondent

All information which would permit identifica-
tion of the people being interviewed as a part of
this project will be held in strict confidence.
No information that would allow identification
will be disclosed or released to others for any
purpose.

Signature of Interviewer

From Cannell,C.F., Lois Oksenberg, and Jean Converse, Journal
of Marketing Research, 14 (August, 1977), 306-315.



Q20A.

Q20B

Q20C

7

- "Incapacitate” does not seem appropriate for a survey administered to a
general population, Could you replace "incapacitate" with "keep?" Also,
can respondents accurately report whether or not they will be able to work
in the next 6 months, and if not, is this a problem in this survey?

- "Layoff" is ambiguous. Either ask this in the context of a job to which
they expect to be recalled or ask an additional, specific question about
expectation of recall.

- 20K - This series of questions provides helpful context cues for the
respondent that should aid in the reporting the number of hours worked.
Lengthening the question about how many additional hours (20G,H) would
give the respondent greater time for processing and response formulation.
Other techniques could be used if individual day-by-day processing was
desired. However, if BLS is only interested in whether more or less than
35 hours was worked, such alterations probably would not be cost
efficient, Similarly, BLS needs to construct split sample tests to assess
the benefit of the proposed version to the current survey in terms of
response quality relative to the extra cost incurred.

Lab studies should be used to determine if there are problems associated
with what is perceived as "work," "overtime", etc. Do respondents
perceive picking up a co-worker at the airport or having a company
physical (during working time) as "work?" Do office workers paid a fixed
monthly salary consider an after hours business dinner as "extra work?"

- If this is an important question, it might be interesting to check the
validity of the response by asking in the reinterviews (following the
eight month out-of-sample period) whether or not they had been recalled to

work and if so, when. Respondents out of work for a while can probably


The Consumer Expenditure Survey

The quarterly consumer expenditure survey collects data pertaining to
durable goods and other expenditures that respondents are expected to be able
to recall for a period of three months. Consumer units remain in the sample
for five consecutive quarters with the first quarter interview used as a
bounding interview. Comparisons with the Personal Consumption Expenditures
suggest that there is substantial underreporting in the survey, particularly
for categories such as house furnishings and equipment, apparel, and
entertainment.

Problems and suggestions pertaining to the expenditure survey are
discussed in the following two sections. The first section discusses more
general issues which seem to suggest large-scale changes in the CE survey. The
second section focuses on more minor changes which may improve reporting
without altering the general format and administration of the survey.

Major Problems and Possible Solutions

It appears that the CES in its present form puts a substantial burden on a
large percentage of respondents. There is some evidence that suggests that
this burden leads to a significant amount of error (Gresman 1986). It seems
doubtful that respondent motivation can be raised to a level that will result
in accurate reporting of fairly irregular, non-vivid purchases. Many of the
cognitive tasks faced by respondents would be difficult to perform in
isolation, not to mention performing these tasks rapidly in the midst of a 2-3
hour (or more) interview. For example, a consumption oriented individual has
great difficulty retrieving small, non-vivid purchase events (e.g., socks,
handkerchiefs, small household goods) in conjunction with determining when the

purchase was made (e.g., 2 months ago or 4 months ago). Reducing the time


This effect (called a part-list cuing effect) should lead to greater reporting
for mentioned items and less reporting of non-mentioned items. The comparison
between the 72/73 and 80/81 diaries provides some support for this hypothesized
impact on reporting.

rea The future implications of the diffusion of scanners in grocery stores

should be considered. For example, if respondents can be taught to retain

7 receipts from scanned purchases, perhaps the burden of recording the

, information should be shifted from the respondent to BLS coders.

\ In the longer run (i.e., the next ten to twenty years), BLS should think
about the possibility of setting up an electronic panel. Assuming further
dissemination of scanners, it seems that for a wide variety for expenditures of
interest in the diary, scanned purchase information will be available.
Purchases not electronically scanned could continue to be recorded in the
diary. While scanned expenditures will not be available for all panel members,
having electronically recorded information for part of the sample would be
beneficial in establishing true’ error rates and developing correction
estimators for the conventional pencil and paper panel members. Studies
suggest that even in the best designed diary panels and surveys, non-sampling
errors dwarf sampling errors. Therefore, researchers should at least begin to
think about how we can become less dependent upon the less than perfect
motivation and memory of respondents.

Some experimentation on the best ways for respondents to retrieve
expenditure data may improve reporting when the diary is completed via recall.
It would be useful to conduct lab studies which focus on how much time
respondents spend in attempting to retrieve expenditure information, their

confidence in the accuracy/completeness of their responses, and protocols on


importance of these data be emphasized through feedback and other actions of

the interviewer.

Reduction in respondent burden

Other methods which could possibly improve the quality of the diary data
reduce the burden placed on the respondent. These methods are discussed below.

The provision of individual diaries for each member of the CU over twelve
years of age would decrease the burden on any single respondent and may improve
reporting. The supplemental survey indicated that 14% of other CU members did
not report; while this is obviously a problem, it is unclear how large an
effect this has on data quality. Use of multiple diaries for each CU also
leads to operational problems (e.g., training of individual family members,
constructing different diaries for different CU members, etc.), and these
problems may outweigh any increase in the quality of reporting. Small scale
field experimentation of multiple diaries versus a single diary for each CU
could be conducted to determine the feasibility and effect on reporting of the
use of multiple diaries. If operational difficulties associated with the use
of multiple diaries prove too great, some improvement in getting reports from
all family members may be obtained by (1) setting up an initial appointment
when as many members of the CU as possible will be available, (2) asking each
CU member to enter his own purchases in the diary and then initial and date the
diary daily, and (3) establishing a CU "leader" to check that each CU member
completes his/her daily diary entries.

Further increases in the number of specific examples listed in the
description for major item groupings probably will increase reporting. When
recalling items within a group, the provision of specific items makes the

recall of other items in the group (which are not mentioned) more difficult.


18
provide fairly accurate estimates of the dates in which they are recalled
to work,

- Because the time frame is being changed (one week, six months, four
weeks), clarity may be increased for some respondents by adding a specific
date to the question. ("Even though you eventually expect to be called
back to work, in the past month, that's since » have you been
looking for work?") The interviewer would fill in the blank with the date

one month prior to the current date.

21F,22C - Obviously, many respondents will round off their responses here

21G,22D,G - How do respondents perceive the terms "full-time" and

(e.g., about 6 months, about i year, etc.), or estimate the number of
months and then multiply by number of weeks to supply an answer. There
are also some social desirability problems which may lead to
underreporting; many respondents probably feel guilty and/or incompetent
because of their inability to find a job after a significant amount of
time has passed. Replacing "weeks" with "months" (which is discussed in
the text but not shown in the proposed questionnaire) seems reasonable.
Variations in question wording that attempt to reduce problems of social
desirability may improve reporting.

“part-time?”
Could you simply ask them about jobs in which they work more than or less

than 35 hours per week?

26C - My income was misreported on the CPS because I am paid differently for

teaching in the summer than I am for the fall or spring. When asked about
my normal rate of pay, I thought about the constant pay I receive for the
months of September through May. This potential for misreporting remains

in the proposed survey; however, inconsistency in pay may be too unusual


immediate environment when the diary entries are made, etc. Unobtrusive
observational research would be optimal, but given the operational difficulties
involved, use of telephone surveys immediately following participation in the
diary panel is probably most reasonable. Combining these data with measures of
data quality would increase the understanding of the diary task and hopefully
would lead to the development of specific instructions for respondents that
would aid in the ease and accuracy of reporting.

There are suspicions that there is too little emphasis put on the diary by
the interviewers. Focus groups and/or surveys with interviewers may be
conducted to determine interviewer attitudes and behavior regarding the diary.
It is disturbing to think that the interviewers may be (unconsciously)
communicating to respondents that the accuracy of diary reports are not
important. Perhaps interviewers should be required to spend some minimum
amount of time in going over the instructions and example section during the
first meeting and some minimum amount of time for recalling additional
purthases in subsequent meetings. The interviewer should strongly urge
respondents to call him/her when questions arise. A toll-free number aiso
could be offered to help solve problems since the interviewer may not be
available when needed. A more active (and expensive) option would be to have
the interviewer telephone the CU on the third day of in-home placement to see
if problems had arisen and to reinforce the importance of responding daily.

Lastly, BLS personnel who work with the diary should be required to
complete the diary for their CU every year or two. This focuses attention on
the degree of burden placed on respondents and alerts researchers to potential

problems which may not be evident until researchers occupy the role of the

respondent.


Possible Methods for Improving Response Quality in the Consumer Expenditure

Survey and the Consumer Population Survey

Scot Burton
Louisiana State University

February 20, 1987


tl
non-food and other expenditures not obtained in the current non-durable goods
diary should be tested and results compared to the present survey. Non-
expenditure information could continue to be collected via personal interview.
Alternatives in which all CU members over 12 years of age kept a diary of
expenditures also may be compared to a single diary per CU condition. While
change to a partial-diary format will require major changes in the way in which
estimates are derived, it seems to offer the greatest opportunity for improved
reporting.

The manipulations currently being considered for the CPS reflect the
concern over the negative impact of the difficulty of the cognitive task.
Reduction of the time frame and the length of the interview may improve
reporting to some degree, but the burden and the ability of the respondent to
provide accurate responses may still be questioned. Testing the collection of
the expenditure data in small scale field tests may offer greater potential
benefit. If desired, the diary format could be tested against experimental
conditions in which fewer sections were included and a shorter time frame
employed, as well as the current survey (or control) condition.

Other Recommendations

There are several possibilities which may improve the expenditure
estimates provided by the survey in its present form. As indicated in the
previous section, in the long run, large format~related alterations are viewed
as needed to substantially improve reporting. The ideas listed below are less
dramatic changes involving little or no modification to current procedures

It would be worthwhile to conduct a short supplemental survey for the CES
as has been done for the diary. The survey could examine respondents

attitudes and opinions about the CES and feelings about the completeness and


10

frame of interest to the most recent four weeks will improve reporting, but for
near accurate episode-based retrieval of non-vivid items, I feel that most
respondents probably have difficulty beyond a time frame of about two weeks.

In addition to questions about the accuracy of retrieval and placement in
time of irregular, non-vivid purchases, there are questions concerning the
cognitive strategies employed to estimate the amount of expenditures. I am not
familiar with research in this area, but I feel that respondents would rarely
retrieve specific episodic information on prices paid for non-vivid purchases.
Instead, many respondents probably estimate an average price paid for the
product and report this average. Such a strategy probably leads to large
errors at the individual level, and thus attentuates estimated bivariate and/or
multivariate relationships with other variables (Marqius et. al. 1986.)
However, across respondents errors of underreporting and overreporting may
cancel,

Another substantial problem with the CES is that a single respondent must
report on purchases made for the entire household. Having interviewers call
back other household members on "Don't Know'' responses is commendable, but many
respondents are probably not aware of what they do not know. For instance, my
wife has her own source of income, own checkbook, own credit cards, etc., and
she buys clothes and household items for which I am totally unaware.

Therefore, she may have purchased a number of items that I did not report in
the interview. I did not respond with a "Don't Know” unless I had a reasonably
strong suspicion that she may have made a purchase within a specific category.

In sum, given the needs of the BLS, the difficulty of the cognitive task,

and the burden placed upon the respondent in this survey, use of other

methodologies should be considered. Diaries in which respondents report ail


the strategies used to recall the information. Recalling such information from
two or three days ago is probably a difficult task for many respondents;
interviewers should encourage’ respondents to take their time and search their
memory. Orienting cues such as having the respondent think on a day-by-day
basis about what she/he did in the morning, at lunchtime and early afternoon,
in the late afternoon, and in the evening may increase processing time and aid
in retrieval. A question about retail stores visited on a particular day may
act as an additional cue and provide a "check" on items not recalled. A

special form could be designed to help respondents subdivide and frame the

recall period. Recalling expenditure information is not a simple task and

formalizing and slowing down the process may encourage the "resisters" and
"misleaders" ta exert greater effort in retrieving expenditure events.

Other Ideas and Comments

Focus groups should be conducted with misleaders, resisters, and
accommodators in an attempt to determine why attitudes and behavior were not
favorable. In-depth questioning may reveal possible changes that would lead to
better reporting. It would be particularly interesting to contrast the
misleaders and the resisters in the hope of uncovering how respondents with
negative attitudes can be convinced to provide positive behavior.

The supplemental survey provides important information at little cost and
should definitely be continued, but there is a variety of other useful
information pertaining to how people respond to diaries which should be
collected. For instance, information of interest includes the length of time
respondents spend in filling out the diary, which records are used, the
specific individuals im the CU who are consulted, the order in which the items

are filled out, whether or not all example items are read, the nature of the


21

The results should be useful in determining whether or not a different format
(i.e., diary) should be considered for obtaining the necessary expenditure

data.

(2) If the present interview format is retained, BLS should test the use of an
initial screening interview to determine (a) the importance of having more than
a single respondent available during the interview and (b) the ability to
project which CU member could provide the "best" data. Strong effort should be
made to conduct all interviews with the respondent(s) viewed as capable of
providing the highest quality data.

(3) Lab research would be helpful in determining problems with question
comprehension, response formulation processes, confidence in response accuracy
and/or completeness, etc. for many questions in the CES.

The CPS

(1) Tests should be conducted to determine if specific respondents within a
household can provide superior data. Once a respondent is selected, this
individual should be the target respondent for all subsequent CPS interviews.
(2) Specific question wording concerns for the proposed questionnaire are

given in the text. Laboratory research should be useful in determining the

severity of these problems and in identifying other potential problems.


Lu se ‘ .
y magnitude of errors at the individual-level and serve as a criterion against
which the effectiveness of manipulations may be compared for at least a

significant subset of expenditures.

Potential methods for improving the quality of response to the diary
discussed in the remainder of this section include (1) increasing respondent
motivation, (2) minimizing the respondent burden necessary to provide accurate
data, and (3) a variety of other methods. Specific suggestions for each of
these categories are discussed in turn.

Increasing respondent motivation

A primary problem of the diary appears to be inadequate motivation on the
part of the respondent. Several possible methods for increasing motivation are
: discussed below.
¥ Use of a signed commitment manipulation may increase respondent

» motivation. A commitment agreement reported by Cannell et al. (1977), is shown

wl in the Appendix.
a

A similar commitment could be signed by the interviewer and each available
member of the CU over 12 years of age at the first meeting. In addition, the

a/ mmember of the CU completing the diary should be told to sign and date the diary
A

( a
? Ve Nach day after the purchases for the day have been entered. (A signature and

AE

dating area should be added at the bottom of the page for each day's entry in
\ the diary.) The commitment should contain information about how the data is
used and why it is vital to obtain accurate information. The confidentiality
of the data should also be emphasized. Hopefully, this initial formalized
commitment process and the act of signing and dating each day's entries will
encourage respondents to provide more timely and accurate information.

The downside of the use of a signed commitment manipulation is that it may


accurately identify "the inner core material of the exterior walls?"
Q8b. - Do people understand this question and if so, what do you
realistically expect them to do in formulating a response?

Section 2, Q2c, 2f, etc. - From the respondent's perspective, these questions
asking about the number of payments since the first of the month are very
difficult because of the difficulty of placing the payment in time
(without record checks). For bills that occur regularly (telephone, gas,
electricity, mortgage, etc.), most respondents will report that 3 -
payments have been made in the past 3 months. However, when interviewed in
the middle of a month and asked to report about payments since the first
of the month three months ago, it is difficult to date payments and thus

confidently provide a frequency response. Respondents strongly desire to

use rate-based processing while y

question implicitly promotes episodic

retrieval,

2f. - Do respondents perceive “current month" as the actual calendar month or
the past 30 days?

Section 3 - My wife would have problems answering questions throughout this
section. Also, she would be unable to find the appropriate records needed
to respond. I do not think that everybody understands escrow accounts,
types of mortgages, or even concepts such as principal versus interest.

Section 4, Q3 - Do respondents provide names of companies for both local
telephone service and long distance telephone service?

Section 6-9 - Inclusion of as many sub categories as possible probably
increases overall reporting, but there is a part-list cueing effect that
makes it difficult to recall the “other items not mentioned.

For these sections, it would be interesting to ask respondents how


16

understanding of cognitive and/or conceptual problems and determining how they
may be reduced or eliminated. The BLS - Census Bureau Questionnaire Design
Task Force already has taken a large step in this direction by developing a
proposed questionnaire to alleviate suspected problems and improve reporting.
I think the proposed test instrument is a substantial improvement over the
currently used questionnaire. Because I feel that (1) potential data quality
problems of CPS were not as large as CES to begin with and (2) the survey
design task force has already recognized and taken steps to improve data
quality, the suggestions presented in the following sections are relatively
minor recommendations concerning the wording of specific questions and the
testing of the proposed questionnaire.

Some Recommendations about Specific Questions in the Proposed Questionnaire

Q19 - Specification of the precise days in which you are interested should be
helpful. Respondents do interpret "last week'' ambiguously. For some of
the questions, day-by-day processing is probably desired. To fix the week
of interest in the minds of respcndents in face-to-face interviews,
showing them a calendar with the days of interest circled may be helpful.
In telephone interviews, this introduction should be read slowly to allow
vespondents to focus on these dates. Also, repeating the dates of
interest in a couple of the subsequent “last week'' questions would be
useful in reminding respondents of the dates of interest and would ensure
that the dates had been processed when initially read in this
introduction.

Q20 - Lab research on how individuals with non-routine jobs and those with
occupations performed at home interpret and respond to this question

should indicate if there is a problem.


recall (24%), and some of the disturbing results of the supplemental survey
administered to the diary respondents (and interviewers) at the end of the
second diary week (Giesman 1986; Tucker 1986). Some possible methods for
improving the quality of reporting in the diary are presented in the following
section. ;

Possible Methods for Improving Reporting

General issues

Before offering specific ideas about how reporting may be improved, there
are several issues which should first be addressed. A variety of
recommendations will be provided by the panel, both for the expenditure surveys
and the Current Population survey. Rather than attempt to conduct large field
studies in which only one or two experimental variables can be examined, it
seems that small, laboratory-based studies and small field experiments (where
sample size = 200 or 300) would be a beneficial first step. This would allow
BLS to assess a larger number of experimental manipulations and thus
"prescreen" the possible changes before testing in large, expensive field
studies.

In conducting lab-based or small field studies for the diary and the

interview survey, some valid criteria should be used to assess the effect of

the manipulations, The use of data from National Accounts, Progressive Grocer,

etc., is laudable but some doubt remains about the accuracy of this comparison
data. The possibility of testing survey manipulations with members of
electronic-scanner based panels should be investigated. (Several articles
which identify specific companies with scanner panels are attached to this
report.) While such panels may not produce validating data for ail types of

expenditures of interest, they should provide far greater detail on the


A variety of interesting issues were raised at the Questionnaire Design
Advisory Conference on January 15-16. This report discusses many of the issues
addressed at the conference and offers some recommendations which may lead to
improvements in the quality of the data collected for the CES and CPS.

Comments and suggestions pertaining to the CES (including both the diary and
the quarterly survey) and the CPS are presented separately in the following

sections.

THE CONSUMER EXPENDITURE SURVEYS

The purpose of the expenditure surveys is to provide timely and detailed
information on consumption patterns of households. The information collected
in the surveys also is used as the basis for changes in the Consumer Price
Index. Two surveys are used to gather this information, the Diary Survey and
Interview Survey. Problems and recommendations pertaining to these surveys are
addressed separately.

The

ry Survey

The primary objective of the Diary Survey is to collect accurate data on
inexpensive, frequently purchased non-durable goods. However, the information
obtained in the diary is not Limited to such non-vivid, difficult to recall
purchases; information also is collected on all expenses which the household
incurs during the week. Each household remains in the sample for two
consecutive weeks.

There is evidence which indicates that the estimates of expenditures
obtained in the diary are not as accurate as desired. Some evidence of
problems include (but are not limited to) the following: comparison of total
food at home expenditures to Personal Consumption Expenditures in the National
Income and Product Accounts, "first day bias" or the decreasing trend in

reports by diary day, the percentage of diaries completed by total or partial


Pe luracy of the data provided by "typical" respondents. The supplemental

pe

i
surv'

ey could be conducted by telephone following the fifth CES interview.

we

Interviewers! opinions of the respondent could be combined with the survey

responses. This supplemental survey may provide data on the severity of many
of the perceived problems at a relatively low cost.

Past data should be used to compare the responses of respondents who use
records against those that do not. Interviewers should be instructed to
meticulously record for which specific items records were used. Assuming that
such analyses indicate that use of records lead to higher quality reports, the
requests for record-keeping might be emphasized more strongly. The interviewer
could dwell on the importance of accurate reporting and the difficulty in
recalling items without records. The interviewer and respondent should select
a location in which the record file can be kept. This location should be in
plain sight so that the file will act as a reminder to keep expenditure
records. The interviewer snould "plead with the respondent" to keep these
records and attempt to obtain some sort of commitment from the respondent that
household members will "do their best" in this respect. Finally, a follow-up
telephone call should be made to remind them to keep the records and to answer

any questions several days after placement of the record file in the CU.

i ’ Follow-up reminder post cards should be sent every several weeks during the

J {first quarter in which the CU is part of the sample.

ire

For many households, some fairly large errors probably arise from having a
single respondent report on expenditures of all CU members. The diversity in
today's household composition often makes it difficult for one person to be
aware of all purchases. [It seemed unclear as to exactly how often and for how

many categories interviewers normally made callbacks to non-interviewed CU


15
confident they were about the completeness their reports of purchases for
other family members as well as themselves.

Deciding if a purchase had been made and then placing it in time is an
extremely difficult cognitive task for many of these items. I imagine
that few respondents expend a sufficient amount of cognitive effort
necessary to formulate optimal responses.

Section 12 - These questions are extremely difficult to answer in multi-car
households for cars for which the respondent is not the primary driver or
car maintenance person. How often are callbacks made for this section?

Section 13-15 - Parts of these sections may be very difficult for some
respondents to answer even when good records are available. If records
are not available, heaven help the respondent and the user of the data.

While this is not a complete list of potential problems associated with the

questions, it is sufficient to convey the idea that the number and scope of

problems associated with understanding the question, retrieving relevant
information, and formulating and reporting a response are not trivial.

Requiring that the conference participants respond to the questionnaire was

valuable; I also believe that CES administrators would find participating as a

respondent under actual survey conditions a worthwhile learning experience.

THE CURRENT POPULATION SURVEY

The CPS provides information on Labor force utilization and in a more
general sense, serves as a barometer for the U.S. economy (Bregger et. al.
1986). Information is collected on the number of individuals who are employed,
unemployed, or not in the labor force and the national unemployment rate is
constructed from these data.

BLS is interested in improving data quality through gaining an