~miom md coml.sionseqressdinMISdocumentae hoseoffieanti.r(,)

md donotnecssx~yrqresmlm official position oftieBurea”of~r Sla~istim

orhe U.S.Dqmment ofLabor.
●

●

NLS

Discussion

Papers

.
A Comparison of Computer-Assisted Personal

Interviews (CAPI) with Paper-and-Pencil (PAPI)

Interviews in the National Longitudinal

Study of Youth

.

.’

Norman M. Bradbum, Martin R. FraAel, and Reginald P. B&er

National Opinion Research Center

University of Chicago

Michael R. Pergamit

Bureau of Labor Statistics

May 1991

~s paper was prepared for presentation at the annual meetings of tbe Arnericm Association for fiblic Opinion

Research held in Phoenix, Arizona, my 16-19, 1991. Opinions stated in this paper do not necessarily represent “‘

rfreofficial position of policy of the U.S. Dep-ent of Labor.
.

.:

NORC

A Comparison of Computer-Assisted Personal

Interviews (CAPI) with Paper-and-Pencil (PAPI)

Interviews in the National Longitudinal

Study of Youth

Norman M. Bradburn, Martin R. Frankel,

ReginaId P. Baker

National Opinion Research Center

Universityof Chicago

Michael R. Pergamit

Bureau of Labor Statistics

AAPOR Conference

Phoenix,Arizona

May 16-19, 1991
In discussions of retie effects, the survey methodology hteratsrre distinguishes three

modes of data collection-,-face-toface, telephone and self-adrrdnistere~ mere is m extensive

hterature on possible effects of collecting data by each of these modes kause they appear to

differ in fursdamerrti ways. mat has been less notice~ however, is that there are variations

witiiss each of these metiods regarding whether or not they are computer-assisti that is,

whether the questionnaire is represented in elecwonic or paper-md-pncil form. mere is a

pauci~ of hterati on within-mode effects of using computers to assist in the data co~ection

process.

mere are seved reasons for this neglect of resemh on tie effects of computer

assistance. Until recendy here has been relatively Etde experience with computer-sssisti

data collmtion outside the telephone mode. mile the possibility of using computers to assist

face-to-face interviewirrg has bmn recogrdti for a long time, its dity has awaitd the

development of light, portable, inexpensive computers that ordinary interviewers codd carry

mound with hem and use with relatively fitie uaining. Only in tie last few y- have such

computers been avtiable and has CAPI kome a practical retilry. Computer-assisti

seE-administered questionnaires have been used in special settings such as in mdl intercept

studies or in medicd offices to rake medicd histories. As widr CAPI, computer-assisted

seM-adrninistration (CASAQ has been hampered by rhe lack of inexpensive portable

computers and widespread home ownership of compatible computers that codd be refid

upon for use with question-s sent to individual on floppy disks. ~us, in effect, studies

of differences between CAPI and PAPI have been hrnited to the telephone mode.

But computer-assistd telephone interviewing (CA~) begins almost simultanausly

with the spread of telephone interviewing, and inded ti some people’s minds is synonymous

with telephone interviewing. Shifting tim face-reface to telephone interviewing was viewd

as such a big difference buse of tie changes in the way respondents and bterviewers

interactd and tie Merences in the avaikbiEry of visual cues in the two modes. Other

possible differences, such m whether the questionnaire was represented in el=trortic form,

were not viewed as sdous sources of re~nse variation. mere was not a long history of

1
doing studies on the mlephone, so that changes horn PAPI to CA~ we= not seriously

investigated.

With CNI, however, the situation is quite different. We have dec~ of experience

with face-to-face P~I and are osdy now beginning to expIore the fasibfity of CAPI. K

CAPI is intiuced in a survey that has been done for a number of years with PAPI, as is the

case that I am reporting on thy, it is extremely important to know whetier ob~rvd

changes are due to mode effects or to red change in the phenomena I hope that the spread

of CMI will be accompanied by a vigorous research pro- desigssd to investigate mode

effects. If there turnout to be effects that are associated tith the irStiUCriOnOf tie

electronic representation of tie questionnaire, then perhaps we shotid look Mer at whether

or not sometiing similar happens titbirr telephone intewiewisrg.

In discussing mode effects, we can distinguish among three ~s of effects--tiose that

change the interviewer’s behavior, those that change the respondent’s behatior, and those that

change in the interaction between the intetiewer and the respondent. me most obvious

effects of CAPI are those that change tie in:emiewer’s behavior because it is tbe interviewer

that is most affected by a change from PAPI to CAPI. kdeed, it is not immtiately obvious

that there should be any effect on respondents’ behavior kause, from tieir point of view,

they are gernng the s-e questionnaire as they would if the interviewer were woddng with a

paWr-md-~rtcil representation. The use of a computer for ~ording answers, however, may

change the way respondents tiew the task and thus have an impact on their behavior.

FmAIy, reading questions off a computer s-en and typing in responses may change the

qudiry of the interaction betw=n interviewer and responden~ for example by reduced eye

contact or an increased formality in which the computer komes a third party to the

interview.

T~ ROU~ 11 E=ERM~

The ~H is a Iongitudind face-wface smey tilstered by the Bureau of Labor

Statistics @U) now in its 13th year. The sample consis~ of 1I,4d4 pple who were aged

2
14-21 as of January 1,1979 and who have been interviewed every P stice then. The

sample was stratifid by sex, race, ethnicity, and poverty status with oversarnp~ng of blacks,

bispanics, white youths in ~verry and equal numbers of men and women in each group. The

questionnaire is primtily oriented toward hbor market participation, educadon and fefity. 

It is ass exrre-ly complex questionnaire with many skip patterns and extensive rostering.

The complexity of the questionnaire and the difficulties that interviewers have in -g their 

way through it correcdy make it m ided candidate for a computer-msisted fom

NORC began work on a CAPI version of NLS in 1989 using software develo~ by

the Center for Hmnm Resources Research at Ohio State, the prime contractor for this study.

k our fmt experiment, hdf of the Ohio sample for Round 11 of NLS, rsndody selected

from dl Ohio cases, was intefiewd using CAPI. k dl, 301 cases were completed by CAPI

(completion rate = 91.8%) and 264 cases were completd by PAPI (completion rat- 95.6%).

Because some of the cases wem done on the telephone, the &ta presenti here is ody fm

those CMS in the experiment that were completed face-~face (260 CAPI and ~7 P~~,

All 20 Ohio interviewe~ wem trained on CAPI. They completed approtiately hdf

of their cases using CAPI (roughly 16 CAPI cases per interviewer) and hti using PAPL

The PAPI cases were supposed to k done f~st so that the imefiewers were ~sdy tied

on the instrument and had ex@ence witi Round 11 cases. Training on CAPI w= conducted

after the interviewers had kady done some PAPI cases.

A second larger nationwide experiment was conducted as part of Round 12. Data from 

that experiment W k reported in subsequent papem.

~SULTS

The Wfi is an extremely complex questionnaire, and there are a vast number of

comparisons dsat can be made beween the PAPI and CAPI modes. h this paper I s~

extie hypotheses drawn from other mode effects studies to * if some of the same types

of effects occur here.

3
,-

Fwst, however, I can report that out of 139 variables we examined we only found 4

tiat reachd conventional levels of significance. Of these 4, two are reIated to rem of

alcohol use, one concemd the unit of pay and one was respondent attitude tow=d the

interview, which is not pm of the ~ questionnaire itseY. I sh~ orgtie the discussion

around 5 types of possible effects--inttiewer error, length, anonymity, respondent

motivation and the venerable “other”.

Interviewer Error

The major advantage of computer Rpresenration of questionnaires is that the

investigator can better control the presentation of questions and sub~uent bmchisrg patterns

drat we contingent on tbe mspossses. k complex questiossn*s We ~H there ~ many

skip patterns and sections of the questionnaire hat refer to responses earlier in dse

questiorm~. Even witi..extensive &aining, interviewers make many mistakes as they

procmd with the interview. Many of the these mistakes are caught by the interviewer during

the interview itself, others are corrected in tbe centi office by editors, but some shp through

and cannot be corrected even tids extensive cleaning: One of the primary attractions of

CNI is the ability to program skips so that branching errors are efirninated, =rs of this’

type exist only insofm as the proxng has not been able to anticipate every possible

combination of answers and Mowed for a proper nck through the questionn~. Reducing

his we of error can improve the quality of the data and reduce costs by eliminating cosdy

editing and rerneval and reducing the extent of cleaning.

~1 does have a big effmt in this domain. Table 1 presents the average error rates

for the ~PI and PNI cases. It is clear that C~I is working tie way it is supposed to md

reducing the number of interviewer skip errors.

TMLE 1 HERE

4
Note that interviews done with CAPI dso had fewer legitimate missing “~n’t

fiows” and item refusals. It is not cle= why this difference shordd occur. Perhaps

mspndents view WI as a mo~ anonymous form of data recording and are more Wing to

respond to some questions. This hypothesis would be consistent with some suggestive 

differences destibed below. We wiU have to follow up in our analyses of Romsd 12 to see

whetier this finding is repfieati and whether we can understand better what might cause i~

-

One of the major differences between face-t~face imervietig and telephone

interviewing is length. hterviews conducti ova the phone tend to go faster and mspon=s

to individud questions tend to be shorter. Indeed this difference is one of the primary mode

effects. Does anything similar happen witi CAPI? me answer is ciear and simple--No.

Table 2 shows the average length of interview for the two modes.

TABLE 2 HERE

CAPI cases averaged 50.2 minuteslcase; PAP1 cases averagd 51.1 mismtesl- wi~

large standard deviations for both groups. We do not have timing for individud items, as one

can from telephone interviewing, but times for some sections in Round 12 were recor~ and

preliminary analysis suggests that they sdso did not show any difference.

Anonmity

One of the trmst interesting mode differences among face--face, se~-administeti

and telephone interviewing found by Sudman and Bradbum (1974) in thek review of =ponse

effects was on sensitive questions where there may be problems of self-presentation. With

more private modes of -Istration, e.g. self-administration respon&nts were mm tily

to report negative behavior. Whale it is not apparent that ~1 wotid be viewd as more

anonymous than PAPI, since both are face-tmface, it is ~ssible that the fact that the

interviewer is entering the responws into a computer rather than writing tiem down on a

form &at has the respondent’s identifying information on i~ @ tie respondents feel more

5
monyrnous md encourage rhem to repofl more negative information. There is someevidence

that this may be the case, as is shown in TABLE 3.

. TABLE 3 HERE

.- Table 3 presents the frequencies for the CAPI and PAPI cases on a number of items

relating to alcohol use, questions that have been shown to be sensitive ad affected by

wording and format chmges. (~ Bradburn, Sudrnan and Associates, 1979.) ~fie not d of

tie items show differences in the &ction of higher frequencies for CAPI and most the

differences w probably not statistically significant (2 of them are), in 23 out of 26

comparisons reports of ~uency of alcohol-related problems in the last year are greater for

CAPI than for PWI cases. The findings suggest a pattern that is worthy of further

investigation. Data from Round 12, which has a larger sample she, will enable us to follow

up on this ~ssible effect.

Motivation of r6Dondenk

The quality of data is affected by respondents’ motivation to provi& the requestd

data, pticularly in a study ~ie WH which trees the respondents’ memories for reports of

tieir past behavior in many diffe~nt areas and asks a number of sensitive questions, not only

about such Mlngs as alcohol use but dso about detaild information on income and assets. E

the mode of adrninismrion of the survey affects respondent motivation, it can thus have a

profound impact of tie quti~ of da~ fnsofar as we can detect any effects of CAPI on

respondent motivation, it wodd appear to be positive or neuti.

TABLE 4 HERE

There was no difference in the response rates betwmn the two modes, a fiding tit is

replicated in the Round 12 exprirnen~ Since these respondents have &n interviewed for so

many years and have condnued to c~rate, response rare is not a very sensitive rneasum of

motivation. mere was, however, a ~erence in tie interviewers’ ratings of respondents’

6
attitude toward the interview witi the CAPI respondents being mted as mom friendy and

interested than the PAPI respondents. h Round 12 we obtained more &m fim respondents

about their reaction to CAPI and may be able to say more about dte moavationsd effects of

CAPI.

Other effects

I shall end with a puzde. One unexpected difference that has emerged from the

expatiation of the data concerns the pro-on of respondents who report &mg paid by the

hour. As can be smn in Table 5, 7670 of the CAPI respondents report being paid by the hour

while ofly 6770 so report when the interviews were done PAPI.

TMLE 5 HERE

This difference in wage rates is not statistidly si~lcan~ but the proportion bekg

paid by the hour is probably significant. Since we have examirtti a Iwge number differences,

and we would expect some significmt differences on the basi$ of chance, it is possible hat

tis finding is just one of ~ose chance differences. But them are indications from some

prelimin~ asrdysis of tie Round 12 experiment that this difference is appearing there as

we~. So it looks as if it is something red.

E it is a red difference, it is not clear what is causing iL me questions asked of

respondents about this topic are complex and have seved subparts to enable respondents to

mswer in a flexible fashion.

h the PAPI version W of the questions appear on one page together so that the

interviewers can see what is going to be asked and where the question is going. CAPI

interviewers, on the other hand get osdy one question at a time on dreir screens, with the next

question betig deterorind by the respnse to thefit question. Since the set of questions

7
●

✎

✎✍

involve income, pay rates and wheticr one is paid by the hour or some other time period,

respondents may answer dse questions before they are askd The PAPI interviewers,

knowing what is coming, might acmdly use the suggested probe rrmre often and get more

hourly reports in the beginning. But if that were the case, there would be a higher proportion

repordng being paid by the hour when the question is fmt askd As we -n Table 5,

however, the difference shows up only in response to the second question in which they were

asked specifica~y about being paid by the hour. It is possible that intcfiewers in the PAPI

condition were not in fact *g the fo~ow-up question, but we= coding the answer as “no”

on the basis of either what they had heard before the question would have ~n asked or tieti

assumption that respondents who re~ed incomes as ptid by the week or month were not

paid on m hourly basis. Further assdysis is needed before we can ussderstid what is

happening with this question. For Round 12 we have tapes of both CAPI and PAPI

interviews, so we wi~ be able to test out some of these process hypotheses.

Conclusion

We examined the differences for 139 variables between CAPI and PAPI cases in an

experiment where assi~ments had been made ~dotiy to mode of administration. Except

for efftits on interviewer errors tiat were programmed into the WI i=E, in W of these

comparisons we found ordy 4 differences that looked as if hey might even approach

statistical significance. This number is within the number that one might expect by chance

when mting muItipIe comparisons.

There m a few differences, however, that deserve further study bef~ rejating hem

me intriguing hint is that respondents may treat WI as a more anonymous mode of data

collection and increase their repom of sensitive or negative behavior. A second possible

finding is that there may be some question formats designed to give flexibti~ to mspon&nts

that when available to the interviewer otdy one question at a time affects the way respondents

answer or how their answers are recorti Considerable further work is n- to understand

what is redly going on in dds case.

8
NORC

Table 1

Missing Data

P=I C-I

(N=260) (N=248)

Invalid skips .1970 070

Don’t know .4370 .1570

Refusals .1970 .00470
,

.

.

Table 2

Length of Inte.fiew

Average

Minutes/

Case S.D. N.

PAPI 51.1 27.4 247

CAPI 50.2 17.5 259

TOTAL 50.7 22.9 506
NORC

Table 3

Ncohol-Related Behatior Wports

Question P-I C~I 

(N=161) m=149) 

21/60 # times had 6 or more 

drinksin last 30 days

4 or more 17 26

~U65 Ever had strong desire

or wge to drink

Yes 37 42

21/66 Ever dri~ much more

than intended

1 or more times

during the year 42. 55

21/67 Ever found it Mcdt to

stop drinking

Yes 12 19

21/6S Ever tiven car fier

drifing too much

Yes 22 31

21/69 Ever been sick the

modg after d . . “ g

1 or more times

during the year 21 30

21~0 Drinking caused R or

someone to be hurt

1 or more times

during the year 14 12

21/71 Ever drink for a long

period of time

1 or more times

during the year 19 33
.

.

NORC

Table 3

Mcohol-=lated Behavior -Ports (Con’t.)

Question PWI CWI

21f12 Need more to get drunk

now than before

1 or more times

during the year 14 20

21/73 Spouse/ p~er threatened

to leave because of drinking

1 or more times

during the year 4 5

21/74 wed to cut do- drinking

but cotiti’t

1 or more times

during the year 7 8

21~5 Ever shake the morning

after drinking

1 or more times

during the”year 8 7

21M6 Given up on activities

interests to drink

1 or more times

during the year 4 5

21/77 Needed drink so ba~y

Coddn’t thiti

1 or more times

during the year 3 3

2%10 Drink more than before

to get same effects

1 or more times

during the year 7 11

22/11 Stayed away from work

because of dribg

1 or more times

during the year 3 9
NORC

Table 3

I ~cohol-filated Behatior -ports (Con’t.) I

I Question P=I CWI

22/12 Lost ties with fdy

because of dri~g

1 or more times

during the year 1 5

2Z13 D~g instead of doing

something you were supposed to

1 or more times

dting the year 11 18

22/14 Kept drifing even though

threat to health

1 or more times

dting the year 6 11

22/15 Chance for raises hurt

by R’s ~ng

1 or more times -

during the’ year o 1

22/16 Spend a lot ot time driting

1 or more times

during the year 6 14

2%17 Hagover interfered with

someting else

1 or more times

during the year 9 15

2~lS Kept ~ng after

emotional problem

1 or more times

dting the year 3 9

22/19 HearWseen things not

red aftir d .

‘.g

1 or more times

during the year 1 5
.

.

NORC

Table 3

Mcohol-=lated Behavior Reports (Con’t.)

Question PWI C~I

22/20 Taken a fink to keep

horn sh=ng

1 or more times

dting the year 1 5

22/21 Kept fifing tier

problems at home

1 or more times

dting the year 4 5
NORC

Table 4

Respondent Reactions

PAPI CWI TOTfi

(N=247) (N=260) (N=507)

Respo”nse rate 95.670 91.870 93.59q0

Respondent attitude

toward interview

Friendly/

interested 80 91 85

Cooperative 18 7 12

hpatient 1 2 3

Hostile 1 .0 -.

...—

100 100 100
-*

.s

.

●

NORC

Table 5

Respondent Pay Unit

PAPI CAPI

(N=137) (N=172)

Identified as hourly in

first question 49v0 48q0

Identified as hourly in

second question 1870 28Vo

Total houriy 67~0 7670

Average ho~ly wage

men identified in

the first question $6.81 $6.82

men identified in

the second question $8.03 $7.92

-

Total $7.12 $7.22

TOT&

(N=309)

4970

$6.82

$7.96

$7.17
NORC

FIGU&” 1

A. Atogether, how much (do/did) you usually

earn at that job? ENTER IN APPROPRMTE

BonS. PROBE IF NECESSARY Was that per

hour, per day, per week, or what?

111/, /l fJ. JJJ

DOL- Cms

Perhour . . . . . . . (GO TO Q.23)...01

Perday . . . . . . ..(GOTOB) . . ...02

Perweek . . . . . . .( GO TOB) . . . . . 03

Bi-Weekly

(Every 2 weeks) . . (GO TO B) . . . . . 04

Permonth . . . . . .(GOTOB) . . . . . 05

Peryear . . . . . . .(GOTOB) . . . . . 06

Other (SPECIFY) . . (GO TO B)

B. (Are~ere) you paid by the hour on this job?

Yes . . . . . . . . . . .(ASKC) . . . ...1

No ”. . .. . . . .. . . ..(GOTOQ.23) . . . 0

c. How much (do/did) you earn per hour?

////.//1

DOL= Cws
A COMPARISON or COMpOTER-ASS ISTED PERSONAL INTERVIEWS (CAPI)

WITH PSRSONAL INTERVIEWS IN THE NATIONAL LONGITUDINAL

SURVEY OF UOR ~T BEWVIOR-YOUTH COHORT

N.M. Bradburn, M. Frankel , E. Hunt, J :Ingels,

A. Scho=-Glusberg, M. Wojcik

National Opinion Research Center

and

Mic.bael R. Pergamit

Bureau of hbor Statistics

The pur?ose of rhis experiment was to assess the effect of conducting

interviews in Round 12 of the NLS-Y by the Cornputer-&si.sted Persoml

Interview (CAPI.)... method as compared with the traditional paper -and-pencil

personal interview method. The experiment was conducted on one-half of the

to=l sample and excluded respondents who kd to be interviewed outside the

Unized States and/ar in Spanish. Interviewers were assigned cases in the same

geographical region and, where possible , were mcched with respondents for

ethnicizy. Ass igment to the proper experimental or control group was @ne

through random assigment of interviewers. Thus the experiment reflects

actual field practices. The paper will report on the operatiowl problems in

conducting the.experiment.

KEYWORDS

CAPI ; Mode Effects

In this paper we present the first report of an experiment conducted in Round

12 of che Nitiond Longitudinal Survey of Labor tirket Behavior-Youth .,

Cohort (NLS) in which a computer-assisted personal intemiey me.Fhod (CAPI) was

compared with the stindard method of face-to-face PaPer-aad-P.encil PersOnal

interviewing (PAPI) . The purpose of this experiment is co assess the effect

of conducting interviews by the CAP I method as compared with the PAPI ❑ethod.

Previous piloe work has demo~trated the logistical feasibility of conducting

NLS interviews with CAP I, but no iystemaric studies have ~:=en.GOne to

investigate possible method effects on interviewer accuracy or respondents”’

responses

BE NLS AND ~ VIOUS CAPI EXP=IE NCES

The NLS is a 1ongi tidinal face-to-face suney atiinistered by the Bureau of

Labor Sratistics(BLS ) now in its 13th year. The Round 12 sapl e consists of

11,465 people who were aged 14-21 as of JanWv 1,1979 and who have been

interviewed every year since then. The smple Was stratified by sex, race,

ethnicity and poverty status with oversmpling of blacb, hispanics, white

youths in poverty and eq=l nwbers of men and women in each group. The

questiomaire is primrily oriented toward labor market participation,

education and fertility. It is an extremely complex ~estiomaire with mny
skip patter= and extensive rostering. The complexity of the questionnaire

and the difficd ties that interviewers, hve in making their way through it

correctly make it an ideal candibte for a computer-assisted form.

NORC began work on a ‘CAPI version of NLS in 1989 using software developed by

the Center for Hmn Resources Research (CHRR) at Ohio State, the prime

contractor for ttis study.

In a pilot _study, half”6f the Ohio smple for

Round 11 of NLS, rand~dy selected from all Ohio cases, was inteniewed using

CAP I In “all, 301 cases were completed by CAPI (completion rate - 91.7%) . 

fil 20 Ohio inteniewers were trained on CAFI They completed approximately

blf of their cases using CAFI (rougMy 16 CAPI ases per intemiewei) and

half using” PAPI.

Wbil e there were some probl ems that developed during this pilot study, the

pilot was cons idered successful and a more ambitiou experiment was conducted

in Round 12. In the pilof.study we used the Toshih 1.200 bptop Computer

(LTC) with a 20MB krd disk drive, a floppy disk drive, back-lit screen and

2400 baud internal modem weighing 14 lbs. which codd be powered by bacteg or

electrical outlet. From the pi~ot study we learned that:

*

*

*

*

At 14 lbs. LTC weight is a problem but is not an obstacle to we of

the. ~mputers.

Use of battery vs electric power presented no problems to either the

intemiewers or respondents.

Hardware problems were reported by about 5% of the intemiewers,

mostly during che first two weeks of the field period.

Screen reabbility presented a problem, particdkrly in bright light.

Inteniewer difficd ty with the keyboard tuned out !o.be due” to lack of

typing skill, rather than any .specid problem with computer keyboards.

Praccice improved performance.

s~

The softtiare used in these experiments was especially developed _fOr this study

by the CHRR staff, und&r the leadership of Randall J. Olsen, rather tihan

adapting a previously used computer-assistance progrming system as has been

comon in previo=ly reported CAPI experiments (Denteneer, et al. ,1987:

Rothschild and Wilson, 1988”j””~ebestiket al , 1988 ). The software incorporated

the &sic CADAC features of concroll ing skip pattens and ranges, rostering, 

all owing movement within the inst-ent, “fill~ngd preentered tits, and other

minor features. It aiio hndled the complex =lendar operations in the

emplo~enc history section--a critical section which @d given interviewers

considerable difficdty in the PAPI mode.

It is, of cours”e, an advantage to be able to develop CAFI sofware for a

specific sttiy, but it still does not make” it an easy task. The development

and testing of softwife requires considerable efforr and extensive cooperation

between computer progr-ers and survey specialists. It camot be viewed as

-2-
simply designing a quesrionmire appropriate for F~I and then Eurning ic o“er

to rhe programmers

Software development began with..NORG survey Staf’fadvising the prOgra~ing...

staff on the requirements of the software”>for actml field situations When

Cm ?rogramers completed interml testing and judged a software version

ready fOr further testing, the NORC stdf began testing .Dy ,,keybanging’, to

determine if the “software Ebdd be induced to break dom. In.qddition the

software was tested for screen forrnati, consistency between PAPI question text

and CAPI question text, acceptability of ranges, enforcement of consistency

between question responses , correctness of skip patter- , conectness of

roscering and awkwardnesss of usage. When fadts were found in the software,

programing staff first attempted to duplicate”’the error to verify its

exis~ence -d ther,correct it.

Most problems were found, duting the testing stage and quickly Corrected.. A

few more were discovered by the intemie.wers during training and were

corracted before the field period began. Wring the first 2 weeks of the field

period several other problems were discovered and corrected by inktmcting the

interviewers to phone in electronically to receive “fixes;, ?he “fix” was

made by the CR programer and sent co NORC fourchecking co verify its

accuracy and that it did not in turn create additional errors, .~en the fix

was successfd , the Cm programer set up a program so that tbe next time an

inte=i.ewer dided in, the fix was automat idly put..tito her computer.

While one would like to have the CAPI program completely correct by the the

the field period begins, ic is likely that some bugs will show up during

actual field inteniewing even with the most rigorous testing beforehand.

Pencil -ari”d-paperquestiomires frequently have errors also. The possibility

of fixing them after the field period begins i“ssmall With CAPI, ho”ever,

updating of “the programing was possible during the field period and was done

elec=oni tally by the central of.fic”ewithout having t6 rely on interviewers,

comect ing the progras in their o- LTC,s.

THE NLS -12 E=ERMNT

The success of the Round 11 pilot led BLS to consider usage of CAPI...forthe

NLS. They felt, however, that it was first necessary to acquire more

information: (1) What steps tierenecess”ag to.carg out CAPI on a national

scale? : (2) mat was the..tipakt on the”costs of carqitig out the ,survey?; and

(3) what was the impact on the &ta of a change in collection method?

Due to cost considerations, ic was necessa~ to carry out further CAPI testing

in a 1iv~-.production frmework. BLS determined that a controlled experiment

was required. Itiormati6n would be collected which wodd allow”us to evaluate

each of the thee quesrions posed above.

The NLS-12 experiment was designed with two constraints in mind. First, it was

to be a true expertient and strict procedures were developed to insure that

there was no contamination between the experimental and control groups; and

-3-
second, the experiment codd not compromise the qwliiy of the &ia for the

study as a whole, including th& overall completion rate target of 92% which

was necessary co maintain the integrity of the panel.

The experimental design called for division of the ~S s~ple into three

subsam?les: 25Z of the respondents were inteniewed using CAP~, 25% Mere

designated as a control group to be inteniewed ming PAP I; the remaining 50%

were regdar ~S cases chat were not part of the experiment. The &ra from the

CAPI and Control cases will be compared in the analyses of mode effects. The

design serves as a substitute for overlapping samples which one might use in a

repeated cross -section survey. Thus , any mode effeccs from CAPI which cannot

be corr=cted can be controlled for in any future use of the &ti~

Because random seleccion of casesinto cbe’two groupi would not iesdc in cosc

efficienr- clustering of cases for intedewer assigmenc, node selecrion was

based on the attributes of the interviewers and their assi~ent

characteristics e...g.geographical location and pretious HLS experience. T-bus,

25% (77) of the Round 12 incemiewers were randody assigned to zhe CAPI

condition, 25% (77) were assigned to the”control condition and the remaining

170 interviewers were assified” to th&”regdar cases which wer& also done .PAPI

The sample selection process was as follows: field mamgers askZ@ed ~S cases

to each interviewer as in any normal routi. Inteniewers were cbracterized

by assigment size and complexity (the presence or absence of supplemen~~

child cases) , urban vs rural case 1oad, assi~ent type (re@ar vs re@ar -

plus-conversion cases) , and etkicity (black, white, Hispafic) . Inteniewers

with Spanish language cases and cases outside the 48 conti~ous stares were

excluded from the experiment for cost reasons, the anticipated difficdty with

intentional phone transmission of cases, and the cost of protiding

interviewer support by intenacioml long distance. These types of cases are

a ve~ small pofiion of the ~S sample and excluding them from the expeximenc

was judged to have little adverse impact on the generalizability of the

experimental findings.

Intemiewers were randody assi~e”d to conditions by “using procedures which

balanced the conditions on the basis of inteniewer type: re@ar inreniewer

vs convercer; etticiiy; geographic region; and metropolitan/non-mezxopolican

areas. This was acco~lished by sorting the inceniewers with respect to

these four variabl es and then foming groups of four “matched” intemiewers.

Within each matched group, incerti ewers were randody distributed among 4

groups: 1 CAP I group, 1 control group, and 2 regdar groups.

Cases belonging .F.oa .C.APIcondition inceniewe.r were done in the GAPI mode.

those belonging to a control. interviewer’”were in the concrol condirion and

done PAP I The remaining cases were “other,’and done PAPI . To =intain

experimental inregrity inreniewers and cases cotid noc cross mode. that is a

CAP I inzefiiewer codd not tizeniew a respondent in any orher condition as

1ong as rhe interviewer was parricipacing in che experiment. NO PAPI

interviewers were allowed to do cases wing CAPI

-4-
The tOtai nuber of cases assigned co the ~1 mode was 2814; rhe control

sample had 2715 cases.

~ o T PLE

Because ititeniewers were noz allowed to work in more than one mode, cer~in

of the usual fidd economies were not possible in the .Roud 12 data mllection

effort. For instance it “as not possib”li to use one incervi”ewer to conduc;

intemiews with two neighboring respondents if che respondents were in

different ““modes. The s~ple selection process” (selecting asii~en= rather

tbn cases) minimized these effects. Further because of tbe eXCIUS ive MCUre

of the experimental modes and the use of a computer supported field mmgement

system, ic was possible to track experimental costs efficieritly arideasily.

The NORC automated Field Management. System (FMS) is designed to track, on a

case by case basis, production, cost and saple itiormtion. Modifications in

the system set up for Round 12 were designed to allow tracking of “the

experiment and to assure tht no concamiwtion occumed between experimental

conditions.

Executing a field @%pSrimenr of this ma~itude” requires &ontind tracking and

the understanding and cooperation of the field management stiff. Initially

there was some resistance on the part of field =tigers to some o“fche

requirements of the experiment since ir meant tkt cases codd not be freely

transferred from inkeniewer to intemiewer to tiximize completiori rate and

minimize cosrs. Special attention &d to be given to explaining to the field

supervisors the 1ogic of the exQerimentd method and the nkcessicy of keeping

experimental and control cases completely separate. A special procedure was

established so ttic each week requests for reassi~ent of””cai”esbeween

inteniewers were reviewed by the principal investigators to make sure -t no

reass igments were made tht wgdd compromise the.experiment.

Near the end of the field period it became clear that, within” the constiaincs

of the budget, -we codd not achieve the targeted completion. rate for the

entire study and still “maintain strict adherence to the experimen~l

conditions. Thus, for cost reasons , we stopped the”experiment after we had

reached a completion rate of at least 80% each for the CAPI and rhe control

condiciofis (2305 ”””cases, or 81.9%, for CAPI and 2268, or 83.5%, for the

controls ) Tbe remaining cases were completed by whqever method and

inteniewer was convetienc. The overall” completion “race for all Round 12 cases

was 91 .6%..

BTA FOR EVATUATING THE EXPSRIMENT

A great variety of data were gathered as prt of the experiment which will

enable us to analyze potential mode effects. Interviewers filled out

Interti.ewer Feedback Forms on each case. Problem Reporr Forms were used when

problems were encountered as well as a lengthy 15th Case @es.tiowire and an

Exit Questiowai”re. In addition, inteniewers reQOrted any problems to a CAPI

Suppofi Supetisor who assisted them in problem solving. In cum, the @I

-5-
Support staff kepr the central office persome”l apprised of all difficdties

on at least a wee~y ksis . Respondents were also requested so fill out a

very brief questiotiire on the CAPI experience.

BecaWe the tim of the experimental eval~cion of CAPI compared “tirh P-API&=

was to determine what-.if any--mode effects occured, the data hd to be

treated in a comparable -mer once they reached the .dara reduction center.

Once cases were received, the experimental desi~ dic-ced somewhaz clifferent

processing procedures to assure comparability of data:” Control cases were

receipced, edited, coded and &ta entered. A case selected for vdi~tion was

sent to che vditition shop; if it requi”red retrieval, it went to a rerrievd

shop before being retmed to the NLS librag.

C}.PIcases were”-electronically transmitted to NORC and immediately checked for

problems; transmitted &ta were checked against data on the mailed-in diskette

for discrepancies and a dupof verbatim itiomacion (occupation, inhr~,

colleges attended, etc. ) was printed out for coding. Dwps of validation

questions were printed for cases selecced for validation. It was anticipated

that retrieval wodd be extremely rare and indeed this proved to be the case--

there was ody one CAPI case requiring retrieval. CAPI cases, then were not

subjected to retrieval , ediring or cleaning. Control cases were processed

like regdar PAFI cases excepc that after editing, buc before cleaning, a

first -s~ge file of control cases was created for the PAPI/CAPI eval=cion.

These data wiil be zhe core of mch of the -dysis of CAPI and PAPI edited

cases.

Because ~.eare interested in knowing whether CAPI affects the way

inteniewers read the questions or ocher aspects of the interactions bemeen

respondent and inteniewer, tape recordings were =de of the first, second and

tenth inteniews conticted by inteniewers in the C&I and Control conditions.

These tapes will be coded for intemiewer reading accuracy and for any

mistakes in &ta entry by WI int,etiewers that wo”dd not produce a =cbine

detectable error.

Data co”~ection for the e~eriment ended in December _:nd +ta processing is

still .in progress, so”we are not able to report here on modd effecrs on zhe

tiata themselves. We can, however, report on =ny operatio=l aspecrs of

conducting a large scale CAPI effort on a mtion-wide basis.

OPER\T IONAL ASPECTI

Hardwar e

NORC devoted a considerable amount of ctie and effort””to the seleccion of zhe 

hardware that most closely met its nee~ as determined by the NLS-11

experimental resd zs. From che inteniewer, s perspective, the most tiporzant

desiderata were lighter weight compucers and more readable screens. Ten

computers were eval=zed. The Compaq LTE was selected because of its light

weight, bac~it screen, internal modem and 20~ hard disk, and long bacce~

1ife. M though the Compaq computers were priced slightly higher than some of

-6-
the others considered, they were selected because of zheii abilfty “tOmeer.dl

of our needs as well as the support off”eredby the vendor. Each CAPI

inreniewer was provided a Compaq laprop at the start of the fieId period.

~Whe e CAPI

CR I inreniewers set appointments with ~ 12 respondents by the same means

as those inteni ewers doing traditional paper and pencil inceniews CAPI

interviews were conducted in the respondent’s home, at the respondenc, s place

of business , via zelephone and, in several cases, in a peml institution.

With one excepzion, CAPI inte~iewers were able to go “wherever necessary zo

Complete “an inrer-tiew using CAPI In some cases”WI interviewers did n-et

have. access co an alternacing currenc power source and so ““madeuse of”the

rech=rgea~~e hr=e~ rhey were issued with each coquter.

There was ody one occurrence of an interview being refused because of UP I:

prison of~icials in one stice refused to grant an NORC CAP I intemiewer access

to an incarcerate respondent because the contents of the machine were

suspect; and codd not be scanned by prison security periomel in advance.

~How CAP Were ComDlecsd

Each CAPI intefiiewer’s laptop was preloaded in the central office with

elec=onic face sheer data specific to the intemiewer, s caseload; che

ifiormation on a ‘@ven respondent,s face sheet largely determined the defadt

path followed during the inteniew. At the suey ’s condwion, a special

data bachp and architing program executed auto=ticdly, thus saving mo

bactip copies of the completed inteniew (one version copied to the Compaq

hard disk, another copied to a 3.5“ floppy diskette ) and preparing the case

&ta for electronic transmission ro the host computer h Chi~go. “Case data

for interviews I’broken off n before completion were noc archived in preparation

for eleccroni c transmission.

CAPI inteniewers sent ibmpleted case data to Chicago using special

comunicaciori” software loaded into their laptop compurers. They were also

required to send in wee~y rhe 3.5“ bckup diskettes containing case data.

The concencs of the bachp diskettes were compared, byre-by-byte, with case

hta. transmitted electronically. Close amlysis of tits ietumed for e“ve~

completed case” revealed no differences between rhe “two versions. De-ils of

the computer support and transmission procedures are presented in Speizer’s

paper (1991) .

Sevenry -seven men and women were trained as CAPI inteniewers. Since these

NORC inteniewers were randody selected to participate in the experiment,

they possessed va~ing 1evels of skill and experience with both inteniewing

-7-
and computer use. The training program was designed with this variety in

mind. The aim was for complete flexibility: that is, to teach individuals at

all experience levels. The rraining had.three parts: self-study, in-person

training, and post -training evd~cion

Early in rhe field sraffing” process six of the NLS-12 Field-tinage?s were

selected to serve as CAPI field support during the field period. CAPI support

training f“brfield managers lasted one day and focussed on support of

interviewers with CAP I diffic~ti”es. At the conclusion of this.training, each

of the CAP I support staff was issued a.Compaq l=?top computer, loaded ~rith the

CAPI sofr~are and data for 10 mock intemiews This enabled them .to

familiarize themselves with borh ‘the hardware and software. Wing this time

Central Office CAP I training staff were available co answer any questions tbt

arose song the support staff. Support s=ff “familiarity with CAPI tid the

added advantage of all owing us to use them .asass iszant trainers during the

CAP1 incervietier training which took place two weeks after tbe support

training. Members of the central office CAPI “training staff were available to

answer any questions that arose during this period.

Each interviewer was mail ed a CAPI Self-Study.Training Guid~_and instmcted to

spend 30 minute”s completing it before arriving in Chicago for the training.

Rather than focusing on the NLS intemiew itself, the guide focused on basic

computer hardware and terminology. The goal was co ensue tbt dl trainees

arrived at the training with the same basic mders-nding of computer

cerminol ogy. The guide proved extremely valtible in two ways. First, a~l

trainees a~ived at ~aining with ac 1east. a bsic level of corn~ter bowl edge

although a few with previous computer experience possessed more tkn che

others) . Second, the descriptions and completed exercises sened throughout

the field period as valmble job aids. Fur,themore, because of their

f=il iarity with basic computer terminology, even the most tiexperienced

interviewers were able to accurately report any problems they encountered in

the field.

Because interviewers were randody selected to participate in the experiment,

they came co the NLS with va~ing 1evels of interviewing experience, both in

general ad specifikti.y on NLS. fil CAPI intemieweri- received 3 &ys of

NLS -CAP I specif cc crtining after compl ecion of wharever generti and study

specific training was necessa~ because they lacked previous inre~iewing or

NLS experience.

CAP I is a new tectiology,. th=, we hd available OflY a liuite~ nwber Of

experienced CAP I trainers. This forced us to hold the CAP I training in two

separate ctiee day sessions. ..Thefirst session contained all of the non-NLS

experienced interviewers, as well as some experienced intetiewers. The

second session con-ined ody NLS experienced inceniewers.

Ee paid special attention co the needs of trainees for a high level of

persoti, tinds -on experience with the CAFI progra while they were still at

training. The training was designed so C&t the lectue and demonstration

moddes were attended by the entire group. For the practice moddes, the

-8-
b

●

grou? was divided into three smaller working groups and given hnds -on

exercises This process enabled us to standardize the lecture and

demons rrazion modules of”the traiting while still giving each trainee perso-1

attention.

Upon arrival ac che trating, each of the trainees received a.set of.CNI

-cerials..d=.igned to mke. their ititial tiseof CAPI easier and less

overwhelming. The most important of these mterials was “the CAPI Tecticd

Manual which contained seep-by-step insrmctio.ns for performing any of the

tasks asked of a CAP I interviewer. Thus , if inceniewers encomtered a

problem with che C~I software at any point during an inte~iew, they codd

refer to this C= I TechticA Manwl for the answer.

In addition C“O“’theCAP I Techtica Manul, each of che inceniewers. received a ....

set of CAP I administrative specifications, which outlined’’’kheprocedures co

follow when conducting a CAPI interview and a series of job aids co mke the

interviewing task easier.,

The focus of training day 1 was helping inteniewers overcome any initial

compucer fears they =Y have hd. A brief session on how to pack and unpack

the com?ut er was held. (me ❑ain cause of d-aged machines is the failure to

pack them correctly for “shipping.) This se~ent of”the training hd. an added

benefiir when it .b.ecme apparent that computer-novices foud this

f=iliarization the ve~ reassuring. They were able to identify the various

parts of the computer for the first ttie and gain -se in kndling the

equipment.

The rest of the first ‘day was spent in a demonstration of the software,

allowing the trtinees their first tinds -on experience with CAPI”. The first

hands -on session was conducted using a generic non-NLS ~estiomaire so tbt

trainees codd concentrate on the C-I procedties without wor~ing about the

specifics of che ~S inst-ent. The second hdf of the tids -on tr+ting

consisted of practice. with a mock NLS inteniew.

Day 2 covered the GAS I Administrateive Spec.ificaciorii,w%i?h differed

considerably from the paper -and-pencfi procedures, an in-depth demonstration

of””the CAP I special f“eatures such as slide mode, fast fo~ard, review of Face

Sheet Infbrti”rZ6E-,etc. , and more hands-on practice for the interviewers,

hig~igh:>ng the use of the special fe”atures.

The vast ma joriq of Day 3 was spent doing more ~nds -on NLs.practice. The

trainees were required to complete an entire mock interview from che Household

Enumeration co the Intemiewer Remarks. The mocks were done in pairs. freeing

up the trainers co move about tbe room and help the people who needed it mOSC.

mile trainees “were completing these mock inteniews, ❑embers of the training

staff””took pairs of inteniewers ad walked them through the tr=smission

process. Inteniewers were shorn how to enter the transmission portion of the

sofrware, enter “the appropriate codes and password, and transtit their

completed cases. tiso, at this t~e, incemiewers had their actml NLS

-9-
I

caseload loaded into theti compueers, so cht “when they arrived home CMY were

able co com?lere their tele?hone mocks and begin intemiewing.

Day 3 concluded with inteniewers repacking their computers and ..preparingthem

for shipment home. 

Upon returning. home from training, each inteniewer was required to compleze a

telephone mock using the CAPI systeti. The telephone mocks were conducted by

non-CAPI NLS Field Mamgers. .This strategy allowed. us “FO ensure tbt che FK

was able zo assist the inceniewers to resolve any _CAPI specific problems.

This enabled us both to establish a test atmosphere and to model accuately an

accwl incemiewing environment. :Each of che seventy-seven inteniewers was

able to com?lece successfdly the telephone mock and..begin ac.t~ field

intamiewing.

Ue believe the CAFI training was a .co.mpletesuccess. During th& field period,

ody one CAPI inceniewer, out of a total of 77, quit”aid that for reasons

bving nothing to.do ..withCAPI . Ody one required training beyond the.initial

training session. In comparison, the paper and pencil .inteniewers in the

COncrOl group. lost 3 titeniewers. It.shodd b: noted t~t dl of the

intemiewers, who were randody .sele.cred co participate in zhe experiment,

attaded the Craining. No one refused to work on NLS because of CAPI.

9D~ rational Problems

.In plaming for the Rowd 12 ~S experiment NORG established procedmes to

address many problems that we expected to occu dining the field period.

Wile many of these problems did occur, others did not. In@eed, some problems

we anticipated to be major turned out to be insimficant.

One of the problems we ~ticipted was that of developing a cost ef~lcient way

of providing supporr to inteniewers tiring “ngn-bus.i.ness”hems. The first

step in developing this support stmctme was to establish a field support

network. Since the compurer progrmers, the is ve~ expensive, we wanted to

ensure they were involved with problems ofiy they codd ..resolve. The field

support network was desi~ed to screen Calls and reso.lvs as manj.prOblems as

possib”le without progr-er ince~encion.

In anticipation of problems solvable ofiy by a progrmer, however, an NORC 

programer was “on-call”. during non-business hours for the first few weeks of

the field period. We .fomd the nmber of calls ?equlring programer

inrenencion rapidly dwin~ed, and, by the kth week, we ceased ro need

prograwer assi.srance. Ar this point, the field support network took on more

respons ib.ility in dia~osing “problems and soon becme quite pzoficienc. If

che fie~ support network codd not resolve the problem, a member of the NLS

staff was concacted at home. This occurred ody thr@e Cimes .dq$ng the entire

field period.

NORC developed a plan for ~uick replacement of “field computers. Replacement

computers were always sent within 24 hours. If che problem occmred over a

-1o-
weekend, the replacement procedures were implemented first chtig Monday

rooming. The nwber of krdware problem experienced during the field period,

however, was far fewer than originally. expecced. In fact, ody” five machines

broke dow and needed to be replaced; three of these problems were diagnosed

during che training.

..

he to the quick receipt of the CAPI cases, NORC was able to dececc, analyze ,*

and resolve intenieti-er errors much more eff”ectiye~y thn those detecred in

PAP I quesriomaires. klen inteniewers encomtered a problem in CAPI, they

were imtrucred to describe whr happened in a cement field available at any

point during che CAPI interview. A.ddicio~lly, in the Int.emiewer Remrks

section of the CAPI qtiestiomaire, intetiiewers””were asked if they had

encountered any problems. If they responded positively, they were asked again

to describe the problem in detail . ~enever a “Yes” was recorded in response

to this question, CAPI automatically flagged the case as a “problem case”

requiring instant editing.

Problem cases were sent co members of the CAPI staff, ustily the &y after

they were received. Staff reviewed the case to detemine what happened and

how to resolve it. In =ny cases, the enors reported were veg simple

things , e.g. , the inten~ewer forgo= the password to gec into the case and bd

to look it up in the mnul. In other cases, they pointed to more serious

errors. The vast majority of problems encotitered involved difficdty in

entering/editing rosters. A combination of changes to the software and

improved training procedures shodd mitimize this type of error in the fu-re.

Transm-ission Problems

Before begiming the discussion of transmission problems, it is tiportanc to

note cht cr=smission was not considered to be a success tiess dl parts of

the transmission were completed successfdly. Thus, if an inte~iewer

attempted co transmit three cases, and was successfd with the tr~smis sion of

Ody two, the transmission was cons idered to be a failure, even though two

cases were swcessfdly transmitted and received.

Success races in the early part of the field period (Jdy 1990) averaged 80%.

At that time NORC instituted some changes. in !hs.transm;ss iOn :.g<~yar?~hat

● increased che success ta .90% on the first transmission t~, where it remained

. for the rest & the fiel”d““-”-–’”” perxod.““””T“he vast majority of the trmsmission

problems that remained were due to bad telephone line connections. If the

*

interviewer terminated the connection and acrempred to tr=smit again, che

problem was solved. It shodd be noted that the cowication software

prohibited the dispatch of incomplete case data. ‘–

-11-
The NORC Field Manam ~ S1

in an effort co facilitate the management of the CHI experiment, NORC both

.

refined many of its etisting project moni coring systems and developed new

ones. The most nocabl e of these systems was the Field bnagemenc System,

(FMS ), which was used for the first- time on ~_S during Romd 12. In additiOn

co allowing Field &nagers the ability to monitor t.h.e.. progress of their

individul regions electronically, on a case-by-case “basis, the system allowed.

the cencral office and the Yield managers to sbre data in new ways ... Fo~_

inszance, the field ~nagers’ computers were u~ced wee~y as CaSeS were

received in the central office. When a WI case “VaS~electrofially received

in the office a code-was re:umed to the field mmger, through the FMS, to

indicate chat the case tid been transmitted and successfdly received.

be of the biggest ad>.antage of using the ~S was its ability to etiorce the

basic sample concrol ties out~ed earl~er. The ~S was desi~ed so chat

illegal assignments, e.g. ass i~ing Control =ses to a CNI intemiewer, were ...

not allowed. ~W, by using the W, we were able to ezure the integrity of

the expertient.

~~ 0 ware

In some cases, early in the field period, enors in the questio-ire software

were identified and the need for a fix was established. These errors

parallel ed rhe type ttir are zraditiotily corrected with errata memos to the

inteniewers. C~I, however, offered an opporttityto correct these e~Ors

centrally, thus ensming ttit each inzeniewer. was folloting the correct

procedures.

kring Rowd 12 five fixes to the CWI Weszio-ire were needed, dl within

the first few “weeks of the field period.”’ men it was determined tkt a ffi

WaS necessa~, inteniewers were notti-ied and ins=nct@d co dial..in !O receive.

the fix electronically.

&$OO~den r OD inion of CNL

Each C=. I respondent was asked to complete a CHI Respondent Feedback

Questionnaire at the conclusion of the Main intefiew. To date, 1,131

quest iomaires have been part idly analyzed. The preliminav. analysis

indicaces that respondent opinion” about CMI on NLS is generally favorable,

often enthusiastically supportive. MOSC of the CWI respondents stressed tbc

this year’s survey, administered via the laptop computer, seemed easier for

the interviewer; was less stressfd chn in the past, i.e. , tkn with paper A

pencil , and seemed more accurate.

Cost of CAPI Cases

The cost elements for ‘C- I ati””PM I cases are differently distributed, with

training costs being higher and data processing costs lower for CQI as

compared with P-1. These are cosc clifferences ttit will persist” by mode even

-12-
*

.

.

.

af cer the method is firdy established and frequently used, although CAFI

training costs ❑ay come”dom sorn+yhat as more inteuie.wers become generally

familiar wi t“ficomputers. we esctiaze that the 3 days devoted to .CAPI training

might be reduced to 2.5 if all interviewers were experienced computer users

and did not have to be f~iliarized with computers +:ge.nerd .befOre_beginning

training on the parcicdar “CAPI system ad” the particda s.mdy.

At prevailing prices for laptops at the time we conducted this experiment, the

total costs of the CAPI and PAPI =ses were almost exactly the same. me

training coscs ran about 69% higher for CAPi .tbn for PAPI , the field costs

for intemiewing labor were about 1% higher for CAPI, but if you add the cost

of the laprops (amortized over 3 years) to the incemiewing COSCS”,then

interview costs for “CAPI were about 38% higher than PAPi. On the other hand,

&ta processing cos=s–’were ilmost 500% ““higherfor .PAFI t~n for GUI cases

which almost exactiy offset the greater C= I costs of the orher elements.

Thus , the total direct costs for the two modes in this e~eriment were dmosc

exactly the s-e. These COS= do.not reflect the costs of developing and

fixing the,software and other C~ costs nor the costs of.atii”fiisteringthe

experiment itself.

Direct inteniewing costs for CAFI -atiistered cases were appreciably more

exPensive d~ing the first two month of the field period, costing in average

of $8.00 more per completed c=’e “than the PAPI control case t~ough Week 12 of

the field period. After Week 12, the cost of co~leting 6ss4s “via WI

decreased while the cost to-complete cases using traditiowl materials rose

slightly; by Week 17, the last week of the MI experiment, the cdacive

cosc per completed case was ody 1% higher tti cbt of the Control smple.

We must he cautious in generdti’ing this cosc experience to other sumeys

since the final c“os”tdifference will deped on the mix of the elements in a

particdar suney. For NLS, &ta processing coscs are vety large relative to

other costs, thus considerable savings in this elemen~ was achieved by using

CAP 1, and those savings ofFsec the extra expenses for training and the cosr of

=chines.

A big cost factor,. of course, is the cost of the laptops, ht the cost of

acceptable hardware is declining rapidly which will help make CAPI more cost

effective” in the future. The depreciation period used and how on? allOcaces

machine costs across studies will also affecc tbe costs for a particdar

study For this expertienr, we assmed a mchine life o:,3 years and, because

of the lengthy fi”eld period, we assigned all Of the machine COS:S to this

stuay Other studies with a different mix of inceniev complexity. assigmenc

size and data process ing complexity might kve a differ+rircos”c comparison

between the two modes.

GON CLUS ION

We conclufe chat CAFI is a feasible method to We

suneys even with present teckology. For large,

-13-

for large scale national

complex smeys it is
probably cost effective at present- brdware prices if the machine costs can be

spread over a nuber of years and the machines can be used at least 50% of the

time. The logistical pr6blems in maintaining the mchines, traiting ati

support ing the interviewers and in :_ransmitting the dara electroficdl.y are

all well within our ~pabilities. The tits concerting the effect of CAFI on >

tits qmlity or.possible mode differences in responses are still to be

analyzed, but so far we are optimistic tbt there will be few problems, and

=ny advantages. 

REFERENCES

DENTENESR , D ., BETHLEHM, J.G., HUNDERPOOL, A.J., MD =LLER, W.J.

Computer Assisted Sumey Processing, “ in ~. f zhe Thtid ~ ~

Research Conference k Washington, DC: U.S. Bureau of the Census, 112-1~7.

ROTHSCHILD, BETH B. AND WILSON, LUCY B. (1988), ‘Nationwide Food Conswption

Survey, 1987: A

Lndmark Personal Incemiew Survey Using kptop” Computers, ” in =

Pr .eed~s of the Fourrh~~ e ce. Washington,DC:

U.~. Bureau of the Census, 347:356.

SEBESTIK , ~TA , ZELON , WmY , DEWITT , DALE, 0,RSILLY , J-S AND McGOWAN ,

~VIN “(1988),

“Initial Experiences with GAPI, ” in ~~ edinss o urt

~ese ~. Washington, DC U.S. Bweau of the CeW-, 357-365.

.—

SPEIZ~, HOWARD AND DOUG=TY, DOUG (1991) , “Auto-zing Data Tra=mission and

Case ~mgement Functions for a Nationwide GAPI Study, ” presented at

the U.S. Bureau of the Census *w1 Research Cofierence, tili~ton, VA,

1991.

-14-