DESIGNING AN INSTRUMENT FOR COMPUTER-ASSISTED DATA COLLECTION IN

THE CURRENT POPULATION SURVEY

Cathryn Dippo, U.S. Bureau of Labor Statistics

2 Massachusetts Avenue, N.E., Rm. 4915, Washington, D.C. 20212

Over the last decade, there have been two new factors that have significantly influenced the

design of survey data collection--the computer and the theories and methods of cognitive

psychology. Computers were first used by U. S. market research firms in the early 1970s to aid in

the collection of data by telephone. Since then, numerous computer-assisted telephone

interviewing (CATI) systems have been developed. With the proliferation of light-weight

portable computers over the last few years, the development of computer-assisted personal

interviewing (CAPI) has proceeded rapidly. These, plus other computer-assisted survey

information-collection methods, are referred to as CASIC. The application of the theories and

methods of cognitive psychology to survey methodology was promoted by a U.S. National

Academy of Sciences panel on the cognitive aspects of survey methodology (CASM) in their

1983 report. Subsequently, many of the major U.S. federal statistical agencies and numerous

private survey organizations established "cognitive laboratories," which are used in questionnaire

development and for researching other aspects of survey methodology.

In 1986, staffs of the Bureaus of Labor Statistics and the Census initiated a series of meetings

to set goals for a redesign of the Current Population Survey, and, through this process, the

introduction of a new questionnaire and computer-assisted interviewing were given the highest

priority. The interagency team created to develop and test a new questionnaire for full

implementation using CAPI and CATI in January 1994 established two sets of objectives, one

related to data content and one related to data quality. The objectives for improving data quality

included:

• improving the measurement of labor force classification,

• improving longitudinal and earnings data,

• reducing the reliance on respondent volunteered information,

• reducing interviewer error, and

• reducing error in the interaction between interviewer and respondent.

It is through the synthesis of the two new survey research methodologies--CASIC and CASMthat the team has met its objectives. As a consequence, a questionnaire, as defined by the

dictionary or current practice, no longer exists for CPS. That is, there does not exist a list of

questions on paper that can be used by an interviewer to query a respondent. The new instrument

has been designed to use the computer to aid both the respondent and interviewer perform the

mental tasks associated with a CPS interview, i.e., question asking, question answering, and the

resulting interaction.

The question-asking task has been characterized as question reading or recall and question

formulation. In fact, the interviewer's task goes beyond question asking, to include categorizing

responses to open-ended questions when the questionnaire only contains a small set of response
categories and converting sometimes lengthy descriptions of kind of work or important activities

or duties into responses that can be entered on a short line. To aid the interviewer in the

question-asking task, many of the probes currently discussed in interviewer training and included

in the interviewer manual have been incorporated into the new instrument, thus reducing the

memory demands on interviewers. For example, currently interviewers are instructed to probe

about unpaid work if the household contains a farm or business, but the current questionnaire

contains no explicit question to determine if the household contains a farm or business nor does it

specify the wording to be used to probe about unpaid work in the family business. In the new

instrument, the first question in the labor force series asks about the presence of a family business.

If the respondent gives a positive response, appropriate follow-up questions are displayed on the

computer screen for reading to the respondent. To aid the interviewer in the categorization of

responses to open-ended questions, some categories have been relabeled and others have been

added.

To aid the respondent in the question-answering task, many questions have been tailored using

information provided in response to earlier questions during the current interview, e.g., the

presence of a family business or being retired or disabled, or during previous interviews, e.g.,

name of company for which the respondent works or length of a continuing spell of

unemployment. An anchor-and-adjustment strategy is explicitly used to aid respondents in

recalling hours worked last week. Respondent's comprehension is aided by defining some of the

labor force concepts within the instrument, either in the actual question wording or through the

application of specific follow-up questions.

The interaction between interviewer and respondent is aided by the improved flow of the

interview through the use of the computer. Interviewers feel more professional, and respondents

feel more involved. For some data items, verification questions have been included in the

instrument to insure that interviewers and respondents have communicated effectively. For some

data items, a nonresponse results in a follow-up question which includes a motivational statement

in an attempt to elicit a response.

Throughout the development and testing processes, laboratory methods developed by

cognitive psychologists and other behavioral scientists have played a crucial role. Verbal ,

protocol analysis and paraphrasing were used to determine problem questions and evaluate

alternatives. Sorting tasks were performed by interviewers and data analysts to detect problems

in category titles. Focus groups with respondents were used to discuss the interpretation of

questions and concepts. Coding of interviewer and respondent behaviors during actual interviews

was used to evaluate the usability of alternative questions. In some cases, field-based versions of

laboratory methods were developed. For example, respondent debriefings were conducted at the

end of selected interviews.

As a result of our seven-year effort to create and test a new instrument for computer-assisted

information collection in CPS, the interagency interdisciplinary team of economists, psychologists,

social scientists, and statisticians have broken new ground, both in terms of how to develop and

test new data collection instruments and in terms of the design of an interviewing instrument that

can only be administered using a computer.
SUMMARY

Using the theories and methods of cognitive psychology and other behavioral sciences, a new

instrument has been developed for the Current Population Survey. Designed solely for computerassisted interviewing, the new instrument reduces measurement error by aiding both the

respondent and interviewer perform the mental tasks associated with the survey interview.