Sy

age §

Cueing by Listing Specific Purchases

ies)

ng purchased a pr

‘Od
the checklist of sm

Currently, if respondents report not
for example, small household appliances

sD Ee

appliances is not read out. Tr is likely to lead to underreporting,
cause respondents nave to think of a large variety of products,
e if they fail into the category, and because respondents
Ss

ll reduce the number of question:

recali, so if the
vecuced. This pro

ons asked, but may be

consider PF questio
Lown tO De seriousay uncerrepor

thos

1ases dated more
S$ associated with
mer is a more important

purchase.
S that are approori
ge, major illnesses
which types of events
uid write int
example over page),
ubsequent questions. An
the interviewers to
pecially when new set of
ice period. For exampie, the
3 months, that is, since October
.", pointing to that calendar, and mark
pe tne 3-month period. cedure may be par
the reference period excludes the current month,
me <o be desirable that as much as pos the
yould run to the day prior to the day of

use of the time line wi
che appropriate referen:
estions refer tc a cifferent r
erviewer mignt ask "In the

rome

Cu

wn

gZ oy Person

wnen a single question asks respondents to think of purchases by all
members of the family, it is easy for them to forget that a particular
family member made a relevant purchase, even though they knew of the
purchase. Such information is more likely to be volunteered if the

respondent is asked about each family member in turn. Of course, person
by person. questioning would add dramatically to the time of the
interview. Despite this, selective use of the approach for some
seriousiy underreportec items could be us 1. A vesearch project might
investigate which items this technique might be relevant for, by
following the traditional family-based questions with questions asking
other household members to report their purchases, and comparing the
effectiveness of the two approaches in picking up these items.



Page 9

The research proposed below could be conducted using small samples,
perhaps 50 to 60 respondents in each condition.

Dating of the Term "%

the term "last week" is

c/or Saturday, and others might not),
really meant, and not a typical week.
effect of using a calendar, using record
supplemented by laboratory interviews to
the term "last week". Only the
ently conducted in this way,
pondents who are sent a calencar with
ephone may do so.

c look at the

uld usef ons

to be introduced and the 2-week

ntroduction of such calendars snould be

udy. It is pos: e that an event that

the reference period might be used to bound
tne reference period is too short for this

© incoming people who are asked ab
ane ‘also cenefit

he current ques

uu
S might respondents to y questio on
g the last 4 weeks.

Changing the Response Period

If a change of reference period to two consecutive 1-week perio
made, there is likely to be interference between the two weeks.
most recent week is lixely to be recalled with greatest accura
accuracy being particularly nigh if € question is not precede
question concerning the other week. The ¢:
reduce error, but in general the week recal led second is Likel y
recalled as being more similar to the first week than in fact it is.

Making question wording more in line with Respondents’ Knowledge

Questions 21F and 220 of tne proposed test questionnnaire are good
examples of now respondents need to have knowledge structured in a
particular form in order to answer questions. Some respondents may
readily know, for example, what date they were on layoff, but they may
not be sure now many weeks have elapsed since If such oe ce eiae are
revealed in Laboratory researcn, questions should be devised to give
respondents an alternative way of answering. This is particularly easy
in the CATI system, but could be used in face-to-face interviews as
well,



information on two or more time x cot
investigated, too. For example, in Se
nealth insurance, responaents a £
payment in the last 3 months ar

to first taink

payments in the

¢
Ar ed 2. forward order.
oermit prediction as to what is
a] e S, Dut co prov ns
reaers better then others.

mnaires + £ " ¥ fast pace of
ry lewer i 2 keen that

wo
She
et O

©

Bore
@
®
bed

3
io)
5
@
3
5
«

no
fom

rviewers can be
ch which responden
ger before answering,
nend that ISR's traini
iew are investiga
udy as interview con

aw

sO:

Sa GQaw dH

y

BLS does not xnow who makes the best et rc the que:
and ¢ S nee, the current proc 3 ing any
ho answer the questions is reas e. However,
house! S$ are likely to be better res
ne best 5 is depending on the nature of
of product. ng asked about. Research could investigate these
ti rence vesting each adult member of a small number
households, pernaps assuming that higher reported purchases
more accurate recall. If consistent differences are found ryey
itself, particular nousencid members could be sougnt out to answer ail
or part of the questi aire. For example, husbands might be asked some
questicns on cars anc nousenold repairs, and the wives other questions,
jowever knowl ecge, sts, and responsibilities differ

ie a Setter procedure might be to first ask who wo be the
est te answer different ty of questions, or who makes the
purcha: and then ma effort interview that person on these

Some items may de bacly report: ty proxies, in which case small
additional questionnaires or 4 y member
might de considered. Research might also investigate whether the same
or different respondent should ideally be administered the subsequent

questionnaires. if particular household members were found to be

®



Page 5
6. Confidence Ratings

ort how confident they feel about %

che!
ory rese

Many respondents can re
v
i

of some of their an: arcn using Ip
sually assoc with accuracy, $
, the greater his/her likelinood o
Co is by no mean. Ir
answers %

lock at

questions.

considera.
th accuracy on %
asked wh

that their uti

y is the
rom records.
Little risk that
for example, uti y 8,
is is important information
erviewer and respondent
ould be recorded by the
complexity of the task

at

Precra on:

Order of Questions

2 ring, Nancy Mathiowetz pointed out that respondents
hey purecnased an item before being asked what it
this type of information is orga ed in memory
ease with which different types of information can be
wnich item of information better cues the other. The issue of which
question order, if any, leads to Detter accuracy, would be well worth
investigating, ideally in a record checking study. It may be that some
respondents better retrieve price ormation following cate information
whilst otners do the reverse, in which case a flexidle interviewing
procedure would be helpful. And perhaps certain types of products would
be better reported when a particular question order is used.
Alternatively, one order may be consistently superior, both in terms of
time to retrieve the formation and accuracy of the answers.

the

, and

imilar
(Section

S ssues arise on the ms on educational expense

( ), witn eight dif nt questions regarding each educational
expense being asxed in a particular order, Section 17, with cost by the
period being followed by cost per month, and Sections 3D and 3 on

mortgage and ownership costs. The best order of questions when

g

ques
For



Page 8

David J. Mingay,

which

Sure
useful procedures to investiga

proclems with the questionnal ask Force appears”
t tnat the laboratory app Ollow other ac
cause of fund! consideration
A sn take olace
ent and retrospect
not require many resources.
ro home, by 3LS
houl d be conducted
pategies .
y oularly
ioulty cy with some of the
nese who work great ly varying
t fied varough com

OO ww

Tre lab should provide useful information on how respondents are
interpreti ambiguous questions, for example, whether nours worked is
interpreted as including a lunch break, and whether alterna

questions appear to work Detter. Lab work can also indicate how
respondents whe work irregular hours define the term "usual", and how

those not directly employed by a company define the term "business”.
Some ambiguous terms might be interpreted in sufficiently s roa way,
and in a way close enough to the survey designer's intended meaning that
new wording is umnecessary. Unfortunately, this is likely to be the
exception rather than the rule.

likely to be a conditioning effect for the questions on the
number of nours worked last week and usual hours worked. Lab techniques
equid Look at the change in strategies likely when answering whichever
of these questions comes second, as a result of answering the first

one. A study in which respondents answers are validated using company
records might be conducted. Three conditions snould be run: the two
orders and a separation of the two questions by several other
guestions. Self and proxies may behave differently, so both types of
respondents shouic be interviewed.



DEPARTMENT OF HEALTH & HUMAN SERVICES Public Health Service

National Center for Health Statistics
3700 East-West Highway
Hyattsville MD 20782

February 11, 1987

Ms.
Senior Mathemat
Office of Research

my report on possible researcn
NCHS activities, I will be

d be of use to

proposals that you are

cognitive issues in any other

zonduct snort



any of the 3

ppy to talk at

spendents' > tions of questions and
answ: ol techniques would



2 ers could be asked to attempt to
interview these people, dut be ted to interview others if they

should de unWilling or unavaila

11, Redesign of Questionnaire

3 the meeting, the $ should map onto We
de Ddout the topics. nple, Section 20 might
non Currently it der t{ respondents work out
usua. mon This may be 2 for some respondents,

St, Dut not respondents Ss
so that respondents can answer using the
xe (for examp monthly, or 3-mon

Ss being

ear on tne

ata analysis stage).
of "usual" is

if

amount, |

even

ple reso
istant to reporting
ssary arithmetic.

ather radical redesign cf

in any cetail, would

and one which I nave not ceougne
moving to an
ne cecline
happened vecentl y interfe
a phenomenon psyeholog sts oat
t accurate data should therefore
cent event. Respondents might
information, however short or long the

ricds across respondents might
Suggestion with trepidation.

would lose a lot of data on frequentl
be considered as a supplement

pernap , if that change enc
is to “investigated. In order to get sufficient data on inf
doughs nS, such as cars, respondents might be asked to repo
last purenase tney made, however long nas elapsed. Much of thi

to be quite accurate.

Conclusions

i would be happy to discuss any of these suggested projec in more
tail. Most could be examined in a small scale study. I feel that th

nignest pricrity projects are the time line, pace, proxies, and perhaps

certain question ordering anc cueing procedures. Changing time 1
question crdering procedures have the important features that they
should not add to the time tne interview takes, and by being somewhat
easier for tl respondent may even reduce total time.

and



2. The interviewer might
several occasi
for that day.

né tne respondent in the eve
re whether he/she nas complete

\) 4. Ask respondent to maxe a commitment to trying to give
x responses wnen given the sim r to those used
" Res versity of Michigan.

has suggested that this improves
worth considering asking respon
vied to give the
for each day.

accuracy occur to me.
the time that the
n the context of a
hey are relatively
es for getting more of
chases would be a

Savoivel

o
od
5
ct
o
Q
is
o

3

a

3 cognitive psychologists know that recognition is
better than. recall. Reporting of apparel might be improved if
respondents are to lock through their wardrobes and cupboards,
while thinking whether any of the clothing was purchased within the
reference period. Respondents might also be asked to glance °
several rooms looking for recently bought items, pernaps © the

interview.

Cueing by inking of Events Associated with Purchases

Mental cueing might be engaged in. Respondents could de asked
major snopping expeditions, to think about them carefully and
the purchases associated with these visits. Similarly, respond:
ght be cued by being reminded of ¢ ts which often lead to nun
purchases, for example, a wedding, a noliday, or a son/daughter going
college for the first time. Using probes in our iaboratory, I have
found some eviderce that a respondent may forget a whole class of
events. Hence, somedccy whe forgets to recall a wedding when recalling
purchases may forget the clotnes purchased for it, the gifts, the plane
Journey, hotel bills, and other expenses associated with the wedding. A
checklist of infrequent events that are associated with numerous
purenases might be read to respondents at the beginning of the survey,
and subsequently respondents might be occasionally reminded of the

ents they reported as they go through the survey (see also comments on
using calendar).



che most appropriate ways of elicit
who have not filled in the diary.

time, i.e., from day 1 to day 7, or
prove the mos t

ctive,

within 4 day, preceding the checklists with some questions
designed to remine respondents of those activities he or sh

nave done during the day tnat are ely to be assoc

purenases. S include going shopping, eating out, going on
veins with ldren, and paying bills.

s e the diary at the end of
2. Tay ies associated with
pe Xans snack in the work canteen, or a
q is Some fairly general instructions to
Sa a during the day, and the suggestion
tha r cheir nousehold too, might reduce
and h etions might range from the genera
ins during the day, fr

ct @

fa
is]
“RoOdaba

te m list of activities that are T
ass urchases, with the suggestion that these de Kept
/ mind wh: noleting the form. An alternative approach mignt be to have
a ilar set of instructions at the end of each day’s section, with

space for respondents to write in ourct

ases previously forgot

(incidentally, a space for this purpose might be useful anyway, as
respondents might recall products after passin appropriate category
in tt , perhaps because memory for an item cued by recall
item, and be unwilling or unable to locate co
oategory}. They could be encouraged to write nad
previcusly forgotten, The general instruction that respondents snouid
either correct a previous answer or write these newly remembered items

ar

the end of the qu onnaire

ght reduce underreporting.

Other Approaches to Improving Data Quality

Several & ques occur to me that might increase respondents!
when completing the diary, or increase the proportion who complete i
fter eacn day, although the techniques have the disadvan
increasing respondent burden.

1, A specially designed notepad and pen might be attached to
respondents’ wallets or purses, so that they can jot down she cost
and nature of single purchases as they make them.

2. Short diaries, given to other members of household.


oy Davie J.
and Methocology, Nat

ive Labortory tech

tioned

iques mel
tne Consumer Expenditure Survey
ome of the d u
psence of a special

ensus Bureau
powerful and fast
respondents nave with

q
an office ca vape 2 recorder bs
nec
ase
ures,
am

TP part

edn ao

Dire

i sift foulties
means of fi
chat

wo

ne

suggest below do not
respondents. naps 50 te 60 subjects in
with relatively little attention needin
oF respondents. Those
might be replicated, st
represent.

rge numbe

oup would be acequate,
ies

to the demograph

li<

cive of the population.

The new diaries seem very well laid out. The large number of
should increase reporting. [ft might de worth checking in the
whether "etc." is understood sy respondents (for example, 13

probes, ie,, that they are meant to write in items tnat are i

eategories. The extent to which respondents do this mignt be

investigated, ard a greater stress placed on the need to do so if such

rep g@ is inadequate. More generally, the research on the Census

form being cenductec by Betsy Martin could very nicely be applied to tre

ai to get a feeling for the ways in which respondents are filling it
the problems 2 ng the forn.