Current Population Survey 19

CURRENT POPULATION SURVEY

The Report of the BLS-Census Bureau Questionnaire Design Task Force clearly out-
lines important conceptual issues and questions of understanding and cognitive processes

. The conceptual issues. however. are

that affect the Current Population Survey (CPS
of primary importance. The information desired must be clearly defined before one can

estions succeeds in obtaining it. If respondents

evaluate whether or not a given set of
are not able to provide the information needed to implement the definition required for
analysis. it may be necessary to revise concepts to reflect the way in which categories are
socially and cognitively organized. In the rneantime. however. if the analytic definition of a

ch on the way in which respondents provide

“job.” for example. remains ambiguous. resea

if criteria for evaluation.

answers and interviewers code therm wil! lack c

Many of the suggestions made in the discussion of the Consumer Expenditure In-

though with minor difications. I will not repeat those

jerview apply here as well. a
discussions. The paragrephs below explain how the earlier suggestions can be extended
for evaluating the CPs. {Words and phrases in boldface refer to sections or subsections in

discussion of the CE Interview.}

¢ Motivating and training respondents and deve:oping a rapport that contributes

porting are as desirabie for CPS as for CE respondents. anc similer

to accurate ?

techniques are applicable. Increasing response accuracy is likely to require estabils!
ing a rapport in which the respondent feeis comfortable taking the time needed to

answer: this is particular’, true for reports of actual hours worked

¢ Record use is tess applicable for most CPS questions about hours and income than

for CE questions about purchases. However. increased accuracy and completeness of

reports might: be obtained if. for interviews after the initial interview. respondents

were mailed a self-administered worksheet in advance of the interview. whether

the interview was to be conducted in person or by telepnone. The worksheet would


Consumer Expenditure Interview

who must rely more heavily on recal!.

« In-depth interviews with all adult household members in a smail random sample

of households about their accounting and record-keeping procedures could suggest

interviewers

alternative ways of coliecting accurate information from respondents
could be trained to select the method that promised the most accurate information

with the least burden to respondents.

Self-administered form (worksheet)

schedules. may find participat-

Some respondents. especially those with very b

“they can prepare for the interview in advance. The

ing in the survey easier i

interviews suggested above can provide guidance for developing a workshee: that respon-

bie and record sore or all of the detailed information needed for

dents can use to ass

sheet would resernbie a self-administered questionnaire. If the

view. The wor

the |

worksheet were complete at the time scheduled for the interview. the interviewer would

review the worksheet for compieteness and administer a brief supplermentary recal! section

as weil as sections of the interview rot covered by the worksheet. Responuents who had

not compleied the worksheet would be interviewed using a standard schedule. such a

ef would not be suitable for all respondents. but some would probab!s prefer it to

worksh

hy inters : for some. this Gata collection strategy wii increase accuracy. If sent

to respondents one week vefore the interview. the workshee: would provide the respondent
Ine opportunity to assemble records in a was convenient for trem and to review purchases

with other household members wane might not be present during the interview

e Evaiuation would focus on identifving which sections could be pest adapted to a

might benefit

worksheet format, deve oping 4 procedure to identify which responder
from using such a worksheet. and comparing tne accuracy of results obiained using

e worksheet to that ontained in a conventional interview



Current Population Survey

be evaluated to some extent using information gathered in previous interviews. In most
cases the proposed question (Q23) about whether or not the person has looked for work
in the past twelve months could not be validated directly in the current CPS design. An
example of an exception is a person who says “no” in interview 5 but who reported searches
in any of interviews 2 through 4. Speciai studies, however. can evaluate the accuracy of

recall for shorter time periods.

e The recall of past job-search activities can be examined for up to a 4 month recall

under the current CPS design. Longer recall periods can be used in interviews 5

through ® if reports are restricted to the period covered by interviews 1 through 4.

Whether or not the previous respondent or another was selected would be controlled
Variation in Work Patterns and Earnings

Intensive interviewing can be used to determine how respondents decide whether or
not they have regular hours ;that is. how much variation can a work week have and still
be considered “regular” } and how respondents determine typical or usual work hours (and

earnings! and extra or overtime hours !and earnings)

* Concurrent think-aloud interviews can be used to identify the strategies that respon-

nis use to maxe decisions about whether or not their work schedules are regular

and about warat their typical schedules are. The resuits of these interviews can be
usec to deveiop more structured interviews with a siall heterogeneous sampie of

respondents to determine now much variation there js in these strategies. Validation

studies using diaries wouic be an important supplement to these studies of reporting
strategies.

Miscellaneous Coinments on Proposed Test Questionnaire

hea” this tesi questionnaire is or what its future will be. The

It is not clear now “he

yestionnaire. however. useful as a framework for commenting on some issues in the

CPs. and | have tried to keep the foiiowing comments fairly genera!


Consumer Expenditure [nterview 4

Criteria for Evaluating Changes

While the success of any proposed procedure should be evaluated in an experiment. the

choice of an outcome measure is problematic. This section suggests several criteria: they

vary in how expensive they would be to use and in how strong are the assumptions they

require. Those that are less clearly indicators of improved accuracy should themselves

be evaluated. Although time consuming. such an evaluation would make it possible to

identify criteria that can be used in regular monitoring of reporting quality or added to

ongoing relatively easily for speci

Possible indicators of improved reporting are numbered and described below.

g

}. frequen

of reporting or dollar value of purchases

Reiving on increased reporting assumes that most errors are omissions or underesti-
mates. Increased reporting appears to be the criterion currently used most often in

evaluating accuracy.

2. recall accuracy

The accura’e¢ ~ecaii of items listed for tne third recall month at tne previous interview

referred 10 as tne “item fist” below! can be used to evaniate noth telescoping and

omission err nth recali period under thea

amption that the original

report is accurate. Of the six criteria mentioned here. this is tne only one that offer-

a direct indication of the accuracy of report

e Using the transcrived tem isis to evaluate relative accuracy. whether of an

experimental treatment or of some eternative criterion. requires systematically

coding a.. items reported in the curren: month and their

gories (@.9.. ape

parel. nome furnishing. ete The subsequent interview would record whether
or not the item was correctly mentioned es being purchased in the third re-

call month. The outcomes would incsude whether or pot the item Was omitted


tw

Current Population Survey

enable respondents who could use the form to obtain from all household members
reports of (1) hours worked for ail jobs for the reference week on a day-by-day basis,

(2) earnings for the week from all jobs. and (3) job-seeking activities. before the

interview took p.ace

« Recall aids such as calendars may improve the accuracy of reports about the hours

worked in the previous week, The recall strategies respondents currently use and the

accuracy of these strategies can be evaluated in comparison to the results the results
obtained when the respondent fills in a calendar with number of hours worked each

day (or starting and stopping time each day).

Criteria for Evaluating Changes

The criteria available for evaluating the effects of changes in the CPS interview are

view. This is true because presumptions

more limited than those availabie for the CE ir

Love

er most errors result fro reporting or underreporting are even jess clear

about whetk
for the CPS than for the CE and because the CPS requests jess recall information than

CE interview. still. a few criteria are available to use in assessing response accuracy:

¢ consulting others.

sstionnaire.

e response time or lime to complete sections of the qu

ondent is housenold

OF Spouse.

The comments made avout these criteria in the discussion of the CE

interview apply

ea to be evaluated

the criteria for the CPS would

also. and the usefumess

«© The possibilities for validating reports of hours and earnings are limited. and the few

pay check stub) would not be available

records respondents typically acquire fe

al the time oi interview. Nevertheless. the feasibility of -onducting validity studies

) each worker

nlal treatments in w

should be investuigaiec. For exampie. es


26
REFERENCES

DeMaio. Theresa. 1980. “Refusais: Who. Where, and Why?” Public Opinion Quarterly
4$4:223-233.
Cannell. Charles F., Peter V. Miller. and Lois Oksenberg. 1981. “Research on Interviewing

Technique.” Pp. 389-437 in Samuel Leinhardt. ed.. Soctological Methodology 1981. San

Francisco: Jossey-Bass.

“On the Mode of Administering a Questionaire and Responses to

Groves. Robert. 19

Open-Ended Iten Social science Research 7:257-71.

Lawrence A. Jordan. Alfred C. Marcus. and Leo G. Reeder. 1980. “Response Styles in
Telephone and Household Interviewing: A Field Experiment.” Pubte Opinion Quarteriy
44:2 10-222.

Loftus. Elizabeth. 1984. “Protocol Anaivsis of Response to Survey Recall Questions.” Pp.

61-G4 in Thomas B. Jabine. Miron L. Straf. Judith M. Tanur. and Roger Tourangeau. eds..
Cognitive Aspects of Surrey Methodology. Washington. D. C.: National Academy Press.

Stinchcombe. Arthur. Calvin Jones. and Paul Sheatsley. 1981. “Nonresponse Bias for Attitude
Questions.” Public Opirien Quarterly 45:359-375

Schaeffer, Nora Cate. Irwin Garfinkel. and Thomas Corbett. 1936. “Respondent's Judgments

about Components of the Child Support Reform.” Unpublished manuscript. Universin

of Wisconsin, Macison


Current Population Survey

ures may increase mentions, particularly of rela-

activities are that social desirability pr
tively ambiguous categories, and the time involved in administering the question. Closed
questions. on the other hand, leave less room for interpretation by the interviewer (which ts

an important source of potential bias in coding job-search activities) and present categories

that serve as memory aids to the respondent. Furthermore. the categories complete the
definition of the question by denotation and clarify for the respondent which activities are
meant to be included in answers io the question. When categories are presented in check-
list form (in which the respondent must reply “yes” or “no” to each item in the checklist}

higher reports of most checklist items can be expected as compared to reports obtained

when the interviewer shows a card {see. for example. Jordan. Marcus. and Reeder 1980}

g would affect checklists presented in both personal and

The factors that increase report
telephone interviews.
The checklist form of a closed question. however. has two additional advantages over

other forms of the same question: the question can be almost identical for personal and

telephone interviews and the checklist form may counteract the fast pace of telephone
interviews and the tendency to give fewer rnentions in response to open-ended questions in

tel

phone as compared to personal interviews. This tendency. reported in Groves {1978}.

ma\ nelp account for the decrease in the number of unemploved in the telephone interviews

as compared to personal interviews.

e Whetter or not these advantages are outweighed by the possibilits icreased res

ial desirab pressures or to the increasee cost of

porting of activities due to sc

administering checkiists can be esse-sed by comparing results obtained with different

question forms in botn ‘nrerviewing modes
Discouraged Workers

Tre accuracy of the label “discouraged worker” depends in part on tne reliability of

reports of past job-search activities. The recall of past reports of job-search activities can


Consumer Expenditure Diary Survey

DITURE DIARY SURVEY

CONSUMER EXPE?

The Diary Operational Test incorporates several] promising instrument format changes:

decreasing the size of the diary, decreasing the information requested for each item. and

specifying the items on entry lines. Comments on most @

pects of the diary format are
best Jeft until this experiment has been analyzed. and so my suggestions about the CE
Diary are brief.

e Analysis of the diary data itself can be supplemented by in-depth focused interviews

the

\ with a small heterogeneous sample of CE respondents after their participation

a survey about their problems xeeping the diaries. Such discussions shouid inciude

nould attempt to

tion. and

« ae more and less cooperative respondents. The discussions

distinguish problems of motivation. problems of diary format and preser

problems due to using proxy respondents. because the solutions to these problems

> y differ. In addition, observational studies of a small number of respondents completing

diary in different formats may suggest improvements.

Bg8

Further analysis along the lines begun in Reporting Quality in the Diary Survey may
indicate the need for new feid procedures. It appears that the Diary survey obtains three

of data: Gata from records. data recorded by recall the same day, and data from

jonger recall pertouds. These three kinds of data differ in social and cognitive sources

of inaccuracy. Inaccuracy in data obtained irom records jsuch as receipts} will be due

largeis to transcription errors and incompleteness in the records: procedures to increase

the accuracy of such reports might focus on identifving bias in the records (for example.
are receipts missing for particular household members:. Inaccuracy in Gata recorded at the
tire of purchase or at the end of the Gay should come from similar sources. Inaccuracy
in data for longer recal, periods. however. wil) oe a result of more compen processes:

procecures to increase the accuracy of such reports would focus on determining knowledge

anc aiding recall.


Consumer Expenditure Interview

made to compare the motivations and personal characteristics of those who do and

who do not provide accurate information.

Respondent training and rapport

ng. It should be foilowed by a

The opening presentation begins respondent trai
description of the task. a statement of the importance of accuracy. suggestions about

acy. for example. the suggestion that the task

the strategies most likeiv to increase acc
will be easier if the respondent collects records before the interview begins.) isee Self-

administered form below.}

The interviewer training and the interview schedule should incorporate interviewer

ance of accurate reporting and aid the respor

behaviors that reinforce the impor
meeting this objective.” Reinforcing accurate reporting involves giving positive feedback

d encou

behaviors. requesting that other answers be reconsidered, a

se tne time necessary to recall the information requested. Reinfo

respondent to

accurate reporting can be done most easy in an interview that covers iess materia! than

the

current CE interview for two reasons. First. the feedback and recall 1

the interview, Second. the task should be designed so ‘hat respondents can

Sent deat Che conference. current practice is for tie iiterviewer to decide whether

dordization is tikels

isk tae depeneen ta collect records before che inrerview. Thais lack of

Pett

used tor utilities amd

crease quterviewer varlance. The interview schedule specities that records }

telephones, bar the record requests ire tet miehe at fhe same time. In gey owen uncerview. me preliminary

ugcesnon about using ree was piade cud | assume) thar this was deliberate atthe interviewer
\ pieterred that | avr recep ds miless specitecliy re char request for records at the Gesamte
: Pthe uitersiew Wora cave anmiinesated tte goad of the task mete catecth made responding easier
uel beet dese alten that the the rob Winch cesnioe) in ny maxims funn trip:

upstairs teny les
SSottie of Cie snegestians dn this secten draw on the work simiged in Cedi Miller and Oksenvers

yosd
osha


nsumer Expenditure Inzerview 8

tion: the classes of items for which records were consulted. what records were

. and how often. The current control card item

consulted {bills or checkboo
that asks about record use could be replaced by one that asked the rough pro-

portion of items for which the respondent consulted records.

4. consulting others

Similarly. this criterion assumes that accuracy increases when others are consulted.

whether about the occurrence of purchase or about the price of an item

’ e Detailed analysis of this criterion would also require recording additional infor-

mation: the ciasses of icems for which others were consulted. who was consulted.

‘how often. and whether or not they used records

5. response time or time to complete sections of the questionnaire.

sing this criterion assumes that accurate reporting— whether improved accuracy is
due to reflection or remembering. to record checks. or to consultations with others

the consurmer unit-- takes time

e Detailed analysis of this criterion would require recording the tim

é rejatively sma individual sections of tne questionnaire: the length of ti

rof items answered. T

then be standardized by the nur is criterion r

be appropriate. for example. wher evaluating tne effects of techniques designed

1o increased respondent motivation and commitment

6. respondent is householder or spouse

of assumes hat compared to others. responuents in these ro.es will have
more complete Know lecge of the spending of tnore houseno.d members for more types
of purcrases. A variation isto as. respondents to identify the most knowledgeadie re-

<pondert, (Otherwise knowiedgealie respondents. however. may be biased reporters

about certain areas suca as tne clotinng or entertainment purchases of adolescents


umer Expenditure lnterview

or telescoped forward. After all prompting was completed. respondents would
be asked when omitted items had been purchased: this permits distinguishing
items telescoped backward from those forgotten. although there is likeiv to be
considerable error in these reports. Finally. any items reported in the current
interview but not reported in the earlier interview would be recorded: such in-
formation would be useful in examining the perfomance of different kinds of
respondents for different items. (See the subsection on Respondent rules

below.

If incorporated into exper’ments planned for }988-1992. studies that use this criterion

can compare a one-month and @ three-month recall. Studies using this criterion are

2 studies

likely to be expe. sing the item lists can {1} identify problems
in using increases in average expenditure as a criterion and (2) evaluate the other

ria described beiow. If some aspects of the procedures developed to use these

item lists in research can be routinized. using tne lists for future research projects

hecomes simpler and cheaper.

ed in the future even if a one-month

Assuming that ts will continue to be

reall period is avopted after experiments are completed. tne way

ists are trarsetibec anc coded can be standardized so that thes can be a

research use when needed

use of records

Recoras can be consu.ted for iso reasons: lo detertnine whether or nor a purchase

rterion makes the

occurred and to deters: ue how mm.

assimplion that accuracy increases when records are consulted

used for specific ems is recordec only for

Currentis eter or

ntilities anc teleprone bills

e Detatied analysis of this criterion wonid

6 recoreing ¢


misumer Expenditure Diary Survey

For example, if analysis indicates that reliance on recall increases during the two-week

period, data collection in the second week should include a recall section. In addition, re-
spondents who reiy.on-reeall-for the first. week can be identified and given specia-atsention.
Procedures are needed to help interviewers identify the reason the respondent relied on
recall (for exampie. difficulty in writing) and find a solution (for example. calling during
the week and recording the information over the telephone). Such intensive procedures to
improve reporting may be feastbie only if costs are decreased elsewhere. The advantages
and disadvantages of adopting a one-week diary period should be studied

Some suggestions made earlier about the CE Interiew. are applicabie here as well

motivating respondents may present particular problems with the diary survey and the

cial incentives bears reexamination: developing responden: rules deserves close

use of fin

of the diary.

dx here as well. given the large demand

The Household Characteristics Questionnaire Supplement provides information valu-
abie in analyzing performance on the diary. Some small changes may improve it as a tool
for this kind of anaivsis. For example. recording the number of days on which recall was

used and whether recall was used for al: househoid members or only some would make

at would be usefu! in designing the fleld procedures suggested above.

possibie anaivse~


Consumer Expenditure Interview

interview may increase the interviewer's credibility but is unlikely to motivate the respon-

dent. A brief (about five minutes} narrative emphasizing the uses of the data and the need
for accuracy can be written around brochures such as Your Responses are Vital or How the

s Bureau Keeps Your Information Strictly Confidential. Alternatively. more visually

“ens

attractive materials can be specifically designed for such a presentation. In either case.

important and

the presentation can be used to demonstrate that the data are perceived as

to provide an opportunity to elicit the respondent's commitment to accuracy.

e The presentation script and materials should be systematically pretested to discover
whether or not they succeed in {1] communicating the importance of the survey

and its goals. (2) addressing respondents’ concerns. and (3) holding the interest oi

emis. The effectiveness of the presentation and commitinent can be evalue

respon

usiig one of the criteria described earlier

fficult

The demographic and attitudinal correlates of nonresponse are necessarily

to study tsee for example. DeMaio 1980: Stinchcombe. Jones. and Sheatsley 19s}: In

ition, nowever, (tie is known about why respondents wtio cooperate do so or about

ich motives are associated with increased accuracy. Studying respondent motivation ts

compiicated by the fact that while respondents may know why they refuse. many probably

e.

think of a good reason to re?

at thes cann

do not know whs > COOperate—except t

| desiradijity pressures may influeree answers to

Furthermore, conventionalities and soci

questions about cooperation

small random subsample iabout LOO

e For initial studies of respondent motivation.
cases) of CE respondents a: a small number of locations couid be selected for a brie?

in-Gepth interview about tie respopaent ~ participation in the survey after the last

loLives

interview. The goal of suct soci be to describe cooperative

ang Obstacles iO eNpressin@ these Tot Concerns of respondents revealed in such

interviews couid then pe addresseg -ystecnaticaliy in the description of the survey

. an attempt should be

given to new responderts. Even in sucn « smali-scale +


Current Population Survey a4

Q19 The introduction to this section should establish a frame of reference in such a way
that respondents can use it. The effectiveness of the method used here and the
use of aids such as calendars can be explored in interviews with a small number of

ing a frame of reference, the introduction should

respondents. In addition to establi

discuss the importance of accurate responses.

Q20 I do not understand why this question for the one that it replaces) does not specify
“for pay.” which would clarify the intent of the question. Followup questions for

those who answer “no” would then identify those who were pursuing business during

the week but were not paid for their work. In the current version. “...any work at

al? sounds chailenging and is ambiguous.

Are all “no” answers proved to find out if the person is disabled? {An expli

followup question would be preferable. }

ses the ambiguity of the phrase “on layoff” and it remains ambigu-

Q2UB The report diseu
ous here. If laporatory work does not produce a better phrase. all “ves” answers to

question iike Q20B shoutd be probed [preferably in a feliowup question) to make

sure that the person expects to return ta work

Q20C Ifa person answers “no” to Q2i) and “ves” to Q20B. there may be no code for them

n Q20C {because jobs from which the R is on lavoff are exciuded}.

Q2UE Unless laboratory work suggests an alternative. use a flier question to determine
whether or not the person has regular work hours. {How much variation there is in

ng small samples.)

“regular” hours can be studied us

QROE-1 What does “usualis” mea

ito peopie? How do thes compute or construct

average OF ly pical value.

F Does this question intend to obtain multiple reasons or a



University of Wisconsin —Mai
MADISON. WISCONSIN 53706

Diepartment of Sociolsey
| Science Building
Lee Observatory Drive

March 1. 1987

Cathryn 3 Dippe
Senior Mathematical Staustician
Office of Research anc Evajuation
Bureau of Lab vr Sta
141 G Street NW
Washington. D.C

TISTICS

Dear Dr Dippe

| have enciosed my report from the Questionnaire Design Advisory Conference in January

introduction states. my suggestions focus on the Consumer Expenditure Interview

survey and the Current Population Survey. In some cases the comments are necessarily

bbresiated: please let me know ‘f can expand or clarify any of these suggestions or if 1

he of a ce in some other way

ste

opportunits t& participate in the conference | hope that the delay in the

l appreciated t

report does mot cause any inconvenience

Sincerely

ate Senae

i Professor

Enclosures: as stated



Consumer Expenditure Interview ll

of interviewers — perhaps fifty at different levels of experience — would involve

ion to conduct an interview that would be

bringing interviewers to a central loc

A” observed through a two-way mirror. After the interview. respondents would complete

a self-administered form about their perceptions of the interview and the interviewer

Specific hypotheses wouid focus on the relationship between interviewer behavior and

espondent perceptions and on that between respondent perceptions and an indicator

} iv : \
\y of accuracy or quali:y

Developing Flexible Data Collection Strategies

Because of the great variety in the kinds of recor respondents keep and in the

A level of knowledge of respondents. data cojlection strategies that take this variability into

ccount may both inerease accuracy and decrease burden. Research is needed to develop

respondent rules to identify the respondent who both has information needed and is in a
position to cooperate. In addition. an understanding of how people keep and use records

i of which items records are typicaiiy available for may suggest improved methods of

tered form

aS \fasking about purchases. Finaily. for some respondents. filling out a self-admir

or worksheet before the interview may ke it easier to give accurate information.

Respondent rules

Information ehout househoid mempers’ expenditures and having the oO be an
accurate respondent are not uniformiy cistrizuted within households. Two kinds or re-

t would use one of the criteria described above

search on respondents are needed. The

to compare how respondents in different household roles and in households with different

ehoider, spouse of f sr differ in their use of reco

compositions te. hou UESENGIL

3eing @ COOperative are acehirate respond

their accuracy ntmay be primariny @ function

lifestyle factors: an adult whe Keeps the housenold accounts or one who makes most

purenases may be the most knowiedgeatue ang eccurate respoudent. Tals research would


10

sumer Expenditure Interview

complete it within a reasonable length of time: if respondents are sent a doubie message
that both accuracy and speed are desired. they may heed the suggestion of speed. Ways
of aiding the respondent in accurate reporting can be designed once more is known about

: ) (1) how respondents keep records and (2) the way in which lifestyle variables influence

reporting accuracy. (See the subsection on Use of records below .)
Interviewers should be taught to help respondents and how to do it. Time should
be allowed at the end of the initial bounding interview for the interviewer to expiain the

how to prepare for those inter-

content of future interviews. to make suggestions abou

views, and to discuss problems the respondent anticipates. For respondents who anticipate

cis by telepnone between the bounding interview and

owup con

problems, one or more fc
che next interview may he neces-ary. In this call the interviewer asks what problems the
respondent has been encountering ‘n collecting receipts from household members or in oth-

s with the respondent

erwise keeping track of expenditures: the interviewer then discu

phone call

rom

ways of resolving these prob:ems. In addition to its practical foc.

she interviewer may heip maintain respondent motivation and reinforce the respondent's
perception that accuracy is important

In training respondents the interviewer actively contributes 10 a rapport in which

the interviewer is perceived as someone who can nelp tne respondent be an accurate re-

ipted, however, interviewers may inadvertent]y

porter. Even when “his rapport is highly sc

dermine rapport

e Observational studies of interviewers can identify nonverbal interviewer behaviors

that communicate that speed is more important than aceuracy. These nehaviors may

sistency In requesting record checks +o being poised to record an

rige from

neent bas tormulates answer, lt 1+ possible. for example.

answer before the res:

nat effective recall requires temporary disengagement trom ine social demands of

at speed is desi

a
Vv the interview: if the interviewer con hicales bie, respondents

these periogs of disengagemeri. such a study

may quicken the interview bx onmitcir


|

nsumer Expenditure [nterview

Each of these criteria may be an appropriate indicator of reporting accuracy in differ-

two, the criteria merely specify interview

ent contexts. With the exception of the fir
conditions that may themselves tend to increase accuracy. The item lists can be used to
- of the procedural changes suggested below. In addition. these lists can be

used to evaluate the other criteria. Forexample. while using records and consulting other

evaluate <

household members probably can be presumed to improve accuracy. the presumption is

for response time. or for using

iess strong for increased frequency or dollar vaiue of repor

A the househoider or their spouse as the respondent. (See the subsection on Respondent

rules below.) If these other. less expensive. indicators are indeed associated with improved

performance. they can be cohected on an ongoing basis or periodically for specific studies

Interviewer-Respondent Interaction

6

focus on the interaction between the interviewer and

The suggestions in this seetion

the respondent. Because development and testing these procedures requires extensive
feedback trom: respondents, tne first stages of work can be done with reiatively smai!

tir

sampies. Exait 2 the effectiveness of any resulting procedures. of course. requires a

arger experiment

Respondent motivation

since respondents assume their roe without much experience. it is usually necessary
lo explain what is wanted from therm: Tis description should De simple and direct. At

‘ data. the importance of accuracy. and

imum. iniortiation about

conidentiaiily needs ta he uresented s\ster

cally to all respondents. In addition to

“oriented

mot vating the respundent. saci a prescutation car be designe to build a

vas the general goai of respondent training

rapport and to establish aceurat

read a their own tine before or after the

DIV giving the respondent @ Groci are



Current Population Survey 24

tivities are probably not unifor

in a household kept a diary of hours and earnings for one week could be compared
with the interview method, Alternatively, the interviewer could ask the respondent to
save pay records when they become available. The interviewer would return to collect
validating information about earnings and hours worked. It might be necessary to
use self-administered forms to coilect such information. Also. respondents in some

occupations would ot have records for all earnings (for example. waiters) or for al!

hou or exampie, most salaried workers}. Despite its limitations. such validations

could suggest the limitations of accuracy in other reports

Respondent Rules

Accurate information about employment, hours worked. earnings. and jod search ac-

muy distributed within households. The objective of this

research would be to develop respondent rules to identify the “best” respondent.

e Interviews with a relatively small heterogeneous sample can investigate both vari-

ation acr households in the distribution of this information and whether or not

ember

housenhoid members agree about who is the most knowledgeable househoic
Ideally. one would want to compare the proxy reports about each household member

and |2) the

(Li the most convenient househoic member to interview

provide

housenold member judged “most knowledgeabie. with (5) members’ reports about

themselves \naissis shoula car y identify information that appears untkely te

be known by any groxy. such as some kinds of job-search activ ities.

Job-Search Activities

Among other recommendations. the task force report sugge-ts research about phrases

to use in refering 10 job-search activities and the kinds of activities included in each phrase

{p

hoe

30). In addition to these investigations. the use of checkiisi~ or closed questions should

uvestigated. The principal reasons to avoid c.osed questions tor a topic like job-search



sumer Expenditure Interview

PENDITURE INTERVIEW

CONSUMER ED

Given the limits imposed by costs and by interviewer and respondent energy. the
design of the CE Interview embodies a decision about whether to obtain information on
a large number of expenditures knowing that the data may include guesses or hurried

estimates or to collect information on a smaller number of expenditures attempting to

obtain accurate information. The decision to sacrifice accuracy can be implicit in the design

of a questionnaire: é amount of information requested can be provided within the time

to

availabie (or within

e iime that the interviewer or respondent expects the interview

last. only by reliving on guesses or rough estimates. the interviewer and responden: may

ianding that precision is needed only where specifically requested (as

come to an

it currently is for phone bills and utiities)

Obtaining accurate information. on the other

ed to the

hand. requires an explicit decision: the goal must be svstematically communica

respondent and the interviewer must provide guidance and training in achieving the goal

hen the interview: they assume that the

some of the suggestions made below would leng

an be reduced in other ways 10 a them. The monthiy

scope of the erview

ded in planned experiments. may provide some

period and spit forms. already inch

of the reduction in scope needed in order implement other improvements
The comments on the CE interview are presented in five sections: Criteria for Eval-
uating Changes discuyses possible criteria for evaluating outcomes of experiments on the

surses. Interviewer-Respondent Interaction consists of two subsections that discuss

the issues of respondent motivation and respondent training and rapport. Developing

Flexible Data Collection Strategies discusses respondent rules. use of records, and @
sell-administerec form. huproving Recall focuses on recall strategies. the specific ques-

» CE and recall aids. The finaisection brietiy discuss

tion forms used 1

s Conditioning

Effects.


Consamer Expenditure Interview M4
Improving Recall

Recall within even a one month time frame can be taxing; a quarterly recall period

is more than three iimes as difficult. It is likely that in the experimental comparisons of

recall periods curently planned. one-month recal! will prove superior since confining recali
requests to one month has the desirable effects of simplifying the recall task and shortening

the interview. In any case. research on improving recall can supplement this experiment.

Recall strategies

The effectiveness of recai. strategies probab!y depends both on the time frame and on
the type of information being recalled. Recall! strategies used for the kinds of informmation
requested in the CE interview can be studied using observation and in-depth interviews

as concurrent think-aloud interviewing. This research would

that use techniqnes stuct
identify the range of recail strategies used. propose recall aids and flexible procedures for
using them, and develop hypotheses about which strategies increased accuracy These

examined whether or not the

hy potheses could then be pursued in larger experiments th

techniques improved accuracy.

frame

e Again. interviews with a smal! heterogeneous sample drawn from the sam

npie is indicated: these respondents would nor have to be

as the CE interview sa

cally for convenience. Separate

E respondents and could be restricted geograr

irchases. credit. and so on

studies wouid focus on different kinds of information —

ed in

— to keep the size o e task manageable. Nome interviews would ne condu

rror:

a central location where respondents could be observed through a two-way

other interviews would ve conducted in respondents’ son The task would be

iaterlais [such as scraten

unstructured for respoudent- in the observed condition:

wtor, Would be availabie in the room. mut respondents

paper. a calend

termine for then:

ves how to go about filling out lists of all purchases in

would de

the months specifiec. In both conditions. after respondents reported purchases. thes


onsumer Expenditure [Interview

and in the final instrument. Frequently such qualifiers are ignored: in making judgments
about vignettes, for example, respondents have been insensitive to whether before-tax or

after-tax income was specified (Schaeffer. Garfinkel, and Corbett 1986}.

Aided recall

In their current form, the lists of items handed to respondents to aid recall are over-

ally fatiguing

crowded. difficult to sxim effectively because they lack visual focus. and v
for both these reasons. The ‘ists do not invite close examination: if the rapport of the
interview suggests that speed is desirable. the temptation to give the lists an unfocused
ts should be more attractive. provide a clearer

glance must be quite stron These

focus. and be easier to read than

they are now. Even lists that are more appealing will
not be read or even skimmed unless interviewers encourage respondents to read each list

at tne

own pace.
Conditioning Effects

aied by mean expenditures. occur somewhat erratical.y

Conditioning effects. as

arces of these conditioning effects are necessarily somewhat ad

speculations about the

hifting for-

hoc. Explanations inc.ude. for example. “reimbursements” for insurance.

or transportation, and “ast effort” for gasoline {see Condinoning and Recall Effee

in the Interviewer Surve

«© The criteria that represent interview conditions can be used to ident possible
sources of conditioning eferts: changes in the respondent within the household during

the panel. changes in the tarne per item. changes in the use of filter items. and so

on. Beginning atter the initia: interview. the list of items can be used to identify the

type of memory errors inseived and the type of iver affected bx each

nnd voarresistibje by the end or

The Uerview,


made. and the sampling frame for small-sampie studies must encompass the kind of varia-
tion on important social variables that is expected in the full survey. Whether or not any
of these smal! research projects would then be extended to a larger sample would depend
on the outcome of the initial study.

Two needs are primary. although identifying priorities among the research issues out-

lined below is difficult since tne projects vary in compiexity and expense as well as in

Firs the succe:

. developing criteria to evalua

their potentia: impact on data qual

and the

improvements in question wording and questionnaire design is clearly import

problems that arise in this respect are different in each of the surv Second. developing

both rules to select the best respondent with the best store of information and procedures

e best recall procedures wil! fail with

to support that respondent's efforts are essential: t

a respondent who never knew the answer or whe does not believe that accuracy ix desired



Current Population Survey

Q20G

Q20K

What is the comparison that R uses to determine overtime or “extra” hours? Does
this require computing an average for comparison or are they hours for which over-

time pay is received?

“Civic or military” which appears in Q21 might be useful here also. Why is “own

illness” treated differentiy in Q20K and Q21?

The report suggests that interviewers may vary in the way they interpret and code
job-search activities (pp. 13-15}. Problems of censoring by interviewers may be

comprehensive and specifying each

reduced by providing a list of activities th

a

tivity as unambiguously as possible. Skip patterns and analysis programs can
distinguish between activities which are considered valid job-searches and those that

are not

The report notes on p. 23 that respondents who were previously employed may

reasonably think that the question refers to the length of time since the previous

job ended. For such respondents. the fact that a distinction is required may be

communicated by first asking when the previous job ended and then asking duri

now rmam\ weeks since then the person has searched for work



Questionnaire Design Advisory Conference

Report —- February 1987

Nora Cate Schaeffer
Department of Sociology

University of Wisconsin — Madison

This report summarizes my comments on each of the three surveys discussed at the

e Design Advisors Conference. The report has three principal sections. one

Question

each for the Consumer Expenditure (CE) Interview survey. the Consumer Expenditure

"E) Diary survey. and the Current Population Survey (CPS), Suggestions that are per-

tinent to more than one of the surveys are generajly not repeated: the extensions across
surveys are straightfurward. Nevertheless. because many of the suggestions can be gen-
eralized to ali three surveys, the sections are not completely independent. Most of the

on concerns the CE Interview and CPS: the experiments on the CE Diary de-

discuss

scribed at the conference incorporate changes | would otherwise have suggested

My report focuses on a set of general i + and suggestions for research on those issues.
In adcition. I provide some specific comments on questions in the CPS Test Questionnaire.

The report does no: atiempt to be comprehensive. nor does it comment on current forms

ividual questions
specific research suggestions are identified by e in the text. Much of the research

ll geographically concentrated

suggested can be accomplisheri. at ‘east initially. using

ples

important. however. that these sa

variples of 50-100 respondent:. It

geneous: accurate reporting depends on social as well as cognitive factors. and social and

cognitive processes mas interact in their effects on accuracy. For example. which recali

whether or

strategy is most effective may depend on tne frequency of a behavior. an

eepend on how purchases are made fe.g.. cash. check. or credit

records are availabie rr

actors influence botn the trequeney of purchases and how they are

card). Clearly. social



‘developing

Consumer Expenditure Interview

also examine the stability of respondents over the life of the panel: are the most accu-
rate respondents likely to remain the respondent? Information about typical distributions
of information and about how much variability there is in these distributions would con-

ibute to (1] developing rules for selecting the best available household respondent and (2)

ggestions for he ping that respondent get information from other householé

members

e In-depth interviews with all adult members of households in a smal! {100) hetero-
geneous sample of two-or-more person households would have the objectives of (1!

deter-

identifving who ‘n the household knows the most about what purchases.

1g whether or no. there is consensus about screening items such as “who knows

the most about all the pur es in this household.” (3) identifying the kinds of
purchases most tikely to be known only to the purchaser. and (4) developing meth-

the

ods for systematicalls getting information from household members other tha

Id also discuss the use o! records as described in

respondent. These interviews ¢

The next suusection

Use of records

\eain, two tyes of research subiects are indicated. First. the extent to which accu-

erent

x record checks for items of pes can be evaluatec, secand

racy is improved

nior ion is needed on the ways in wnich households store informetion about different

ribution of and variation in these methods. For ex-

kings of expenditures and the di

de by credit card

in some households. most purchases are r receipts are nol

Kept systematicaliv. however. records about purchases may not be available until one or

two montns after the purchase. In other housenoids. almmosi a. purchases are made by

accurate inforg

provide higt ion. A simple

check so that

reviesing check registers +

screening question about the use of check registers may decrease burden lor respondents

who use checks regulariv and per

inlersiewers to spend more time with respondents


Consumer Expenditure Incerview

would be asked to describe how they went about remembering. what estimating and

averaging strategies were used for which items, what would have helped them

remembering. and how accurate they judge their reports to be.

By describing the variation among respondents in recall strategies, this research would
provide a basis for developing Hexibie procedures that interviewers could be trained to use

ul they

in helping respondents. For example. respondents probably vary both in now hel

how they use one if it is available: some respondents may find public

find a calendar and
events useful anchors while others prefer personal landmarks. Furthermore. effective use
of a calendar probably requires a rapport that permits the respondent to disengage from
the social aspects of the interview: a respondent who feels pressure from the interviewer

to respond quickiv may actually feel increased performance anxiety if offered a calendar.

Specific question forms

A better understancing is needed of the recall and estimation strategies respondents

use with the three principal types of expenditure questions in the CE interview

e Research with a small. heterogeneous. sample of respondents brought to a central

location or inte wed at home would be appropriate. This research would obtain

descriptions of the strategies respondents use to answer each of the es of Questions

fora varie rlicies using Coneurrent think-aloud interviewing. The resucc- would

provide an indication of the variation among responden:‘s in these strategies. The

a accuracy of answers obtained in different ways could be assessed using appros
\ “~priate criterion, Procedures would be developed and question: would be rewritien
mp yo enecur using effective strategies {such as remembering from recent to distant

> events or vice versa: {Loftus 1984)

addition. the way in whicn respondents interpre: sticn questions as “What was

russ amount of 0s | should be investigatec, both in focused intersiews