..

537

A General Model of the Survey Interaction Process

James l. Esposito.Bureauof labor Statistics and

Jared B. Jobe. National Center for Health Statistics

ABSTRACT

In this paper, we propose a general model of the survey interaction process.

Our framework consists of three components: the survey context, the survey

participants, and a descriptive model of the interviewer-respondent interaction

process. Because subjective perceptions of the survey context influence the

way individuals interpret, ask, and respond to survey questions, we first

present a taxonomy of contextual variables. We then briefly discuss our views

on the nature of the human organism, as these views will have implications for

the behavior of interviewers and respondents within the survey context. We

conclude the theoretical discussion with a seven-phase model of the survey

interaction process. In the final section of the paper, we offer some

suggestions for systematizing the survey methodological literature.

KEYWORDS

interviewer-respondent interactions, context, taxonomy, survey framework/model

INTRODUCTION

Our understanding of how to design questionnaires and conduct interviews/

surveys has improved dramatically over the past 10 years. Advances have been

made both in the methodological domain (e.g., questionnaire design) and in the

operational domain (e.g., computer-assisted telephone interviewing). For

example, in the methodological domain, significant improvements in the design

of questionnaires have taken place as a result of the recent emphasis on the

cognitive aspects of survey measurement (CASM) (Jabine, Straf, Tanur, and

Tourangeau, 1984; for a recent review of the CASM movement, see Jobe and

Mingay, in press). In fact, in the past decade, there has been a substantial

increase in the number of publications on a wide range of topics related to

su~vey/questionnaire design and data-collection methodology (e.g., Belson,

1981; Cannell, Miller, and Oksenberg, 1981; Converse and Presser, 1986;

Dijkstra and van der Zouwen, 1982; Groves et al., 1988; Groves, 1989; Hippler,

Schwartz, and Sudman, 1987; Schuman and Presser, 1981; Sudman and Bradburn,

1982; Turner and Martin, 1984).1 Missing from these works, however, is a

single, comprehensive model that describes the sequence of interactions that

take place between interviewers and respondents during an actual interview, and

that also identifies the various contextual and person-specific factors which

collectively influence what goes on in the course of such interactions.

In this paper, we propose a general model of the survey interaction process

that we hope will serve as a preliminary framework not only for integrating

research efforts within the survey field, but also for stimulating new

interdisciplinary research. To be more specific, our purpose is to provide a

theoretical framework that researchers from a variety of disciplines (e.g.,

anthropology, psychology, sociology, statistics) can use to structure and

!/ Among the works cited here are some very thorough reviews of the survey

methodological literature (e.g., Groves, 1989) and of basic questionnairedesign principles (e.g., Sudman and Bradburn, 1982). We could not hope to

accomplish here what these authors have accomplished so well in their

considerably longer works; so, generally speaking, our citations are few and

clearly not representative of the body of relevant research. We recognize,

also, that survey research has a history spanning ~any decades (Converse, 1987;

Jobe and Mingay, in press; Fienberg and Tanur, 1983), and that all of us a~e

indebted to those individuals whose enduring contributions have provided t~e

foundations upon which we build (e.g., Bradburn and Sudman, 1979; Cannell,

Marquis, and Laurent, 1977; Converse and Schuman, :974; Kahn and Cannell, :957;

Payne, 1951; Sudman and Bradburn, 1974).
-

538

co~unicate their research ideas and findings. Two points of elaboration:

First, we offer the framework not only as a theoretical model of the survey

interaction process, but also as a practical tool for planning, analyzing, and

conducting methodological research related to surveys. At the present time,

many investigators appear to structure their research ideas in terms of a fourstep cognitive model of the ~esponse-formulation process (i.e., comprehension,

retrieval, judgment, response; see Tourangeau, 1984; Tourangeau and Rasinski,

1988). As influential as this model has become, it is not a model of the

survey interaction process--we suspect it was never intended to be. It does

not address the broader context in which surveys take place, the influence of

other organismic processes (psychophysiological and biological) on the way

questions are asked or answered, or the interactive nature of most interviews--

all of the above affect the behavior of interviewers and respondents and, as a

result, the quality of survey data. The general model being proposed in this

paper does include these elements and, therefore, can be used as a tool for

evaluating survey research. For example, whenever contextual factors (such as,

interview setting, timing of contact, survey sponsorship, response security)

and organismic factors (such as, the psychophysiological and biological state

of interviewers and respondents) are not addressed in the course planning and

conducting survey research, the findings from such research will be difficult

to interpret and should be qualified in terms of generalizability. We suspect

that many failures to replicate can be attributable to situations of this type

(i.e., to the presence of "uncontrolled" contextual and person-specific

influences on the survey interaction process). Second point: To the extent

that any model is adopted by a community of researchers, it provides a common

theoretical perspective that facilitates communication within and between

disciplines; to the extent that the model is comprehensive, it fosters an

integration of related ideas and theories. The four-step cognitive model

alluded to above may stimulate research activity among psychologists, but it

probably seems overly restrictive or foreign to researchers in other

disciplines. By expanding the scope of the model to include contextual

variables and interactive exchanges, one invites researchers from a broad range

of disciplines to collaborate in the research enterprise--collaborations that

potentially could lead to exciting new ways of studying and conceptualizing the

s~rvey interaction process. We could elaborate further here, but it perhaps

best to move on; we hope that the utility of the model will become more

apparent as each ele~ent is presented.

Our framework consists of three components: the survey context, the survey

participants, and a seven-phase model of the survey interaction process--

the three components together constitute a general model of the survey

interaction process.2 In the following three sections of this paper, we

elaborate on each of these components in turn. We begin with a discussion 0:

the survey context and how contextual variables can influence the wayan

individual interprets, asks, or responds to particular survey questions; in

this section, we present a taxonomy of contextual variables. We then turn our

attention to a discussion of the survey participants. In our view, the human

organis~ can be conceptualized in terms of biological systems and associated

psychophysiological processes (e.g., sensation, cognition, motivation,

emotion). To fully understand human behavior in any context, one must

appreciate the continuous, dynamic interaction of these systems and processes.

The theoretical portion of the paper is drawn to a close with a discussion of

our seven-phase model of the survey interaction process. In the final section

of the paper, we offer some suggestions for systematizing the survey

methodological literature.

THE SURVEY CONTEXT

All behavior occurs within context--or to be more precise, within embedded

contexts (see Bronfenbrenner, 1977)--and these contexts play an important role

in regulating behavior. The context within which a survey takes place is no

exception. Because of their variety and complexity, natural contexts are

difficult to describe adequately, though impressive attempts have been made

~/ ~~e ~odel mig~t,also b~ c~nsidered general in the sense that it is meant to

ap?~y both to cp1n12n/att1tuce surveys (e.g., Gallup P2::S) and fact-gather~ng

s~rveys (e.g., C~rrent Population Survey).
539

(e.g., Barker, 1968). Lacking a comprehensive model of the survey context,

social scientists who conduct surveys would appear to have at least two options

available to them in dealing the effects of problematic contextual variables:

the focused-research option and the limited-control option. The first option

requires an active research program that explores potential sources of error

variability in surveys produced by identifiable contextual factors, such as the

effects of interviewer and respondent racial characteristics on the validity of

surveys responses (e.g., Schuman and Converse, 1971). In the limited-control

option, the survey organization tries to control for factors that are known to

be problematic (e.g., providing assurances of confidentiality or anonymity in

sensitive-topic surveys) and assume that less problematic factors (e.g.,

potential gender effects) balance out via the randomization process. We

believe the success of both options depends on how well social scientists and

survey design groups have identified potential problems, and that success at

problem identification depends on how well these individuals have

conceptualized the survey context--the more ,simplistic the approach, the more

likely it is that important contextual factors will be overlooked.

We now present a taxonomy of contextual variables that identifies aspects of

the survey context which we believe influence the way individuals interpret,

ask, or respond to particular survey questions (see Table 1). We believe,

further, that these contextual variables, acting collectively, will have a

significant impact on the way interviewers and respondents interact. Though

this taxonomy was developed independently, much of what follows may be viewed

as an integ~ation and extention of earlier taxonomic works that have had a

substantial impact in the field of survey methodology (e.g., Kahn and Cannell,

1957; SUQ~an and Bradburn, 1974). The major headings of this taxonomy, and a

brief desc~iption of each, follows.

(1) Interview Setting. This variable refers to the place where the interview

is conducted, or more specifically, to the place(s) where the respondent and

interviewer are situated when the interview is conducted. Generally speaking,

~espondents probably feel most comfortable with surveys that allow them to

complete the interview in the privacy of their own homes (e.g., telephone

surveys), as long as the timing is convenient. Interviewers probaoly differ

widely in their preferences, but if for no other reason than personal safety,

~e suspect that most would prefer to interview respondents by phone, or conduct

face-to-face interviews within the confines of a familiar office. ?resumably,

the more at ease one can make respondents and interviewers, the higher the

quality of the data collected during the interview.

(2) Timing of Contact. This variable refers to the gestalt of past, current,

and anticipated events (proximate and remote) that collectively define the

"psychological climate" of t~e interview. Recency is one of many factors that

influence the salience of events (Sudrnan and Bradburn, 1974). To the extent

that the salience of recent events has the poten~ial to unduly influence survey

responses, the timing of a survey is very i~por~ant. For example, automobile

workers who have just been laid off because of slumping car sales would

probably have some very colorful--if not radica:--opinions to share with an

inte~viewer if called to par~icipate in a survey about the long-range economic

benefits of foreign competition in the automobi:e industry. Those same workers

might have a ve~y different set of opinions if they were all about to be hired

by one of the foreign automo~ile manufacturers ~hat had cost them their jobs in

the first place. It should ~e noted here that whereas timing of contact would

appear to be more critical for opinion surveys, fact-gathering surveys are

affected as well (e.g., respondents are more ap~ to provide accurate earnings

information if they have recently completed their income tax returns or are in

the process of being audited by the IRS).

(3) Respondent/Interviewer Characteristics. These organismic (e.g.,

intelligence), demographic (e.g., age, race, gender), and experiential

characteris~ics (e.g., years of experience as an interviewer) can be used to

construct unique profiles for interviewers and respondents at a given point in

time; and behavioral patterns associated with different profiles may affect the

way individuals interact in a survey situation. For example, there are

several studies in the survey methodology litera~ure that seem to suggest that

individuals with a certain cc~ination of c~ara=~eristics tend to cbtain higher

quality survey data, or appear to have greate~ success at rec~uiting

~espcnde~~s fo~ t~e survey (e.g., Fowle~ a~d ~angione, 1986; Hensen, Cannell,

and Lawson, 1976; Singer, ~~ankel, and Glass~a~, 19~3; cf. Haqenaa=s and
--

540

Table 1. A Taxonomy of Contextual Variables to be Considered in the Analysis

of the Survey Interaction Process*

Interview Setting:

a. loca~ion of interview

(1) the resDondent's home

(2) the interviewer's office (e.g., a cognitive laboratory)

(3) a neutral setting (e.g., train station)

b. characteristics of location

(1) security (e.g., good vs. bad neighborhood)

(2) presence of observers (e.g., family members, authorized observers)

Timing of Contact:

a. state/national/global events

b. local/commu~ity events

c. personal events

Respondent/Interviewer Characteristics:

a. demographic characteristics

(1) gender

(2) age

(3) race

b. socio-economic status (SES) characteristics

(1) education level

(2) income level

c. organismic characteristics

(1) biophysiological status (e.g., good/poor health; fatigue level)

(2) psychophysiological status (i.e., system states: sensory, cognitive,

motivational, emotional, e~c.)

(3) other salient characteristics [i.e., physical appearance; personality

traits (e.g., introversion vs. extroversion); intelligence level;

communication skills]

d. experiential characteristics [Note: This category is, in fact, a subset of

the psychophysiological status subcategory.]

(1) relevant past experience (e.g., as an interviewer/respondent; with

interviewer's/respondent's ethnic or racial group; with particular

sponsor)

(2) knowledge of, and interest in, subject matter (i.e., involvement)

Survey Publicity Method:

a. advance correspondence from the sponsor of the survey (e.g., informational

letter)

~b. "cold contact" procedures (i.e., on-the-spot description of the survey, such

as those used in random-digit-dialing surveys)

c. use of the media (e.g., television, radio, newspapers)

d. word of mouth (e.g., via social networks, such as family and friends)

Survey Sponsorship:

a. public/governmental sponsorship (e.g., surveys such as the Current

Population Survey, the Decennial Census, and the National Health Interview

Survey)

b. university/survey-research-center sponsorship

c. private industry sponsorship (e.g., TV network polls, automobile industry

surveys of consumer satisfaction)

Survey/Interview Administration Mode:

a. face-to-face interview

(1) paper-and-pencil format

(2) computer-assisted format (i.e., CAPI)

b. telephone interview

(1) paper-and-pencil format (i.e., questions read from, and answers recorded

on, a paper copy of the questionnaire)

(2) computer-assisted format (i.e., CATI)

c. self-administered questionnaire (e.g., mail-back questionnaire)
..

541

Response Security Level:

a. complete anonymity (i.e., no connection is possible between the providers of

information and the data itself)

b. confidentiality (i.e., respondents are in principle identifiable, but

binding legal assurances are made guaranteeing that only aggregated data

will be divulged)

c. restricted availability--usually with binding confidentiality statements

(i.e., information is available to legitimate research organizations only)

d. unrestricted availability (i.e., information is available to whoever might

want it)

Attributes of the Survey Instrument:

a. general questionnaire characteristics

(1) the nature and content of the questionnaire [e.g., attitude/opinion

survey vs. fact-gathering survey (e.g., economic, health, sex, or drug

survey; see b.)].

(2) length of questionnaire (i.e., respondent burden)

(3) pace of questionnaire (e.g., fast-paced survey; usually a function of

questionnaire length)

(4) homogeneity/heterogeneity of questionnaire (i.e., item mix; a gross

measure of a questionnaire's susceptibility to context effects)

(5) amenability of questions to self vs. proxy reporting

b. item characteristics

(1) the target of the question (i.e., the person/group to, or about whom,

the question is being asked)

(2) the focus of the question (e.g., beliefs, opinions, autobiographical

facts)

(3) the content and salience of the question [i.e., subject matter of

question (e.g., social, political, or moral issues) and the relevance/

irrelevance of the question for the respondent or the interviewer]

(4) the desirability and sensitivity of the question content

(5) the response format of the question (e.g., fixed vs. open-ended,

verbal vs. purely numerical scaling, number of response categories,

availability of "other" response option)

(6) the length and/or complexity of the question

Incentives:

a. monetary incentives (e.g., respondents may be paid a certain amount of money

for their participation in the survey)

b. tangible, non-monetary incentives (e.g., respondents may be given some sort

of certificate or gift for their participation in the survey)

c. verbal incentives (e.g., in a national health survey, respondents may be

told that their participation will help improve the quality of health care

in the United States)

* NOTE: The contextual variables identified in this table have the potent~al to

influence the way in which individuals interpret, ask, or respond to

survey questions. Though specific features of the survey context may

be perceived as more salient than others (e.g., the sensitive nature

of the questions), it is the mix or configuration of variables that

determines what respondents and interviewers will think, say, and do.

Heinen, 1982; see also Dijkstra and van der Zouwen, 1987). Other research has

shown that when t~e profiles of interviewers and respondents differ, survey

data on particular topics can be biased or unreliable (e.g., Ballou, 1990;

Schuman and Converse, 1971).

(4) Survey Publicity Method. There are a variety of ways to inform potential

respondents of the details of a survey (e.g., use of the media, "cold contact"

procedures). In addition to potentially influencing the way respondents

interpret and respond to survey questions, the use of a particular survey

publicity method may have dramatic effects on nonresponse rates. Though

i~pressionistic, our field experience with the Current Population Survey

suggests that individuals are ~ore willing to serve as participants, and seem

less suspicious c= the interviewer, if they receive an informational letter

describ~ng the survey prior to contact by a Census interviewer; in compar~son,
542

the use of "cold contact" procedures seem far less desirable (e.g., in

telephone surveys where households are selected via random digit dialing).

. (5) Survey Sponsorship. Surveys are sponsored by various groups (e.g.,

.governmental agencies, universities, market research firms) for various reasons

(e.g, fact gathering, attitude or opinion assessment, commerical product

viability); and one can reasonably assume that the behavior of respondents and

interviewers will be influenced both by the survey sponsor and that sponsor's

motives--information that is not always readily available to respondents. For

ey.a~ple, most of us willingly participate in governmental surveys that

contribute to the common good; we are less willing to participate in market

research surveys that we suspect to be exploitive. With respect to the latter

point, Brunner and Carroll (1969) found that response rates increased

dramatically when an introductory letter describing the survey came from a

university-affiliated group rather than from a market research company.

(6) Survey/Interview Administration Mode. Interviews can be administered in a

variety of ways (e.g., face-to-face, via telephone) and these different modes!

procedures may influence the way in which survey participants interpret, ask,

or respond to survey questions. The decision on mode of administration is most

often an economic one (Groves, 1989), but there can also be good psychological

reasons for choosing one mode over another or for mixing modes. The federal

government's Current population Survey provides a case in point. The CPS is an

employment survey in which households are interviewed for four consecutive

months, given the next eight months off, and then interviewed again for four

consecutive months. One interesting feature of the CPS is that the first and

fifth interviews are personal visits, the other six are conducted by phone. In

addition to establishing the legitimacy of the survey (via the presentation of

official documentation, credentials, and personal identification), we believe

that personal visits are important for establishing a sense of rapport and

familiarity between household respondents and the government's field

representatives. And we suspect this sense of rapport carries over when

interviewers call in subsequent months, enhancing the quality of data that one

might have otherwise obtained had all eight interviews been conducted by phone.

(7) Response Security Level. This variable refers to the degree of privacy or

security associated with an interviewee's responses, and can range all the way

from complete anonymity to unrestricted access to a respondent's data. The

federal goverment has laws that protect the confidentiality of a respondent's

data for government-sponsored surveys (Nelson and Hendrick, 1983; see also, Cox

et al., 1985: Plewes, 1985), but for many other types of surveys it is probably

not always clear to respondents how their data are being protected. The

quality of certain types of survey data (e.g., questions on sensitive topics)

can be assumed to vary inversely with respondent uncertainty regarding data

security (Esposito et al., 1990; Nathan et al., 1990; Willis et al., 1990).

(8) Attributes of the Survey Instrument. These attributes refer to

characteristics of the survey instrument (e.g., content, leng~h and pace, item

characteristics) that not only influence whether individuals will agree to

participate in a survey, but also how those individuals interpret and respond

to particular survey questions. For example, in addition to affecting the

nonresponse rate, the length of the survey will probably also affect the level

of cognitive effort respondents expend on certain questions (i.e., the

"satisficing" concept; see Krosnick, in press). To follow through with the

example, it would seem reasonable to assume that the longer the survey, the

higher the nonresponse rate and the greater the likelihood that some

satisficing will occur--especially for questions appearing a~ the end of the

questionnaire.

(9) Incentives. This variable refers to the strategies used by the survey

sponsor (or its agent) to enhance the potential respondent's willingness to

participate in the survey (e.g., money, certificates, appeals to civic

responsibility). On this point, Ferber and Sudman (197~) have found that the

effectiveness of a given incentive strategy depends not only on the amount of

co~pensation respondents receive, but also on such considerations as temporal

cc~mitment (i.e., the amount/period of time responden~s are required to

cooperate) and survey sponsorship. Cannell and Henso~ (197~) believe that

i~centive determina~ion should be viewed as a research ?roje:~ in and of

itself, as overpayment may result in unanticipated (ne~ative) effects. It is

importan~ to recognize, also, that whereas incentives may mo~ivate the
-543

respondent to participate in a survey, they are no guarantee that he or she

will provide accurate data.

To conclude this section on the survey context, we wish to point out that the

categories listed above are not to be viewed as exhaustive; nor are the

contents of specific categories to be viewed as mutually exclusive of the

contents of other categories. Nevertheless, we believe the taxonomy described

above represents a fairly comprehensive blueprint of the survey context. Much

in the way of elaboration, however, remains to be done.

THE SURVEY PARTICIPANTS

Arguably the two most important features of the survey context are the

interviewer and the respondent--the survey participants. And it would seem

reasonable to conclude that if the goal was to understand how the survey

interaction process works, one would start by learning as much as one could

about the nature of the human organism. Needless to say, volumes have been

written on this topic; the very best we can hope to accomplish here would be to

provide the reader with a grossly simplified description of this highly complex

organismic system. Before we begin, however, we want to be explicit about the

assumptions that guide our theorizing.3

The assumptions that follow have been derived from a larger set of assumptions

shared by two very similar philosophical/scientific world views: the organismic

world view (Overton, 1984) and systems theory (von Bertalanffy, 1967, 1968;

Miller, 1978; Laszlo, 1972; Capra, 1982, especially Chapter 9). The first

assumption is that the human organism is inherently active. Individuals

constantly select aspects of their environments to attend to and act upon, they

do not simply react to salient environmental stimuli. Of course, there are

occasions when individuals behave in a very passive fashion (e.g., watching TV

for hours on end); but this behavior pattern is not innate, it has to be

learned. Second, we believe that organismic change is fundamental--and

relative. The hallmark of the human organism is process/change. Some

organismic processes (e.g., aging) require more time than others (e.g., sleep

cycles) and, as a result, produce very slow change; this apparent stability

should not be taken as evidence against the premise of fundamental, ongoing

organismic change. The illusion of stability can be very powerful indeed. For

example, none of us will ever live long enough to perceive continental drift,

but scientists inform us that tectonic plates move several inches every year.

If the very ground on which we stand is not stable, what is? And third, we

prefer to view the human organism (body and mind) as a holistic system, not as

a structured collection of independently operating parts. Survey participants,

therefore, are not to be viewed as passive, unchanging, multi-component

machines; they are to be viewed as active, dynamic, organismic systems with the

freedom to do what they want to do, when they want to do it--and that would

include the freedom not to expend the effort necessary to generate accurate

answers to survey questions.

One final premise: The psychophysiological processes (e.g., cognition,

motivation, emotion) to be described below are theoretical constructs. With

few exceptions, reciprocal interrelationships among these processes and

biological systems are accepted as givens (e.g., the influence of mood on

memory, or motivation on perception, or psychological stress on physical wellbeing). It is the lack of connection or relationship among these organismic

processes that needs to be supported with empirical evidence, and not the

converse. Having stated these assumptions, we now present a very simplified

model of the human organism.

3/ The human organism is a system comprising many subsystems (organs, cells)

and is itself a subsystem relative to the larger systems (groups,

organizations, societies/cultures, supranational systems) of which it is an

integral part (Miller, 1978). The assumptions (i.e., inherent activity,

continuous change, holism, reciprocal interrelationships) that appear in this

section, therefore, apply not only to the human organism, but also to all of

the systems/contexts in which that organism finds itself. To the extent that

the survey interaction process takes place within a given societal/cultural

context and involves individuals speaking a c~mrnon language, these assum?tions

are relevant to the discussion in every secticn of this paper.
544

The human organism can be thought to consist of biological systems and

associated/correlated psychophysiological processes (Miller, 1978); it is the

integrity of the former that makes the latter possible (e.g., an intact brain

is essential for normal cognitive functioning, whereas serious brain trauma may

result in short-term or long-term memory loss; see Poon, 1980).4 Each of us

comprise many disparate but nevertheless interrelated biological systems: organ

systems (e.g., lungs, heart, kidneys), vascular systems (e.g., blood and lymph

circulation), musculo-skeletal systems (e.g., muscles, bones), and the nervous

system (e.g., the brain). One's genetic endowment provides a blueprint for the

growth, development, and differentiation of these systems, whereas the

environment provides sustenance and determines to what extent genetic potential

will be realized.

Depending on one's training and theoretical orientation, psychophysiological

processes can be described and organized in a variety of ways. The framework

provided below is but one of many possible, non-comprehensive arrangements

(for others, see Dippo and Herrmann, 1990; Poon, 1980; Royce, 1973). For the

purposes of this paper, the following processes are considered to be most

important: (1) sensation, (2) cognition, (3) motivation, and (4) emotion.

Other important psychological characteristics (e.g., attitudes, personality)

may be viewed as synthetic or emergent properties of the organism (i.e.,

manifestations of relationships between biological systems and psychophysiological processes). We now briefly describe these processes and

illustrate the relevance of each to the survey interaction process.

Sensation (e.g., vision; hearing) refers to those processes that convert

physical stimulus energy into neural impulses; these impulses are later

interpreted by the brain in sense-specific ways (e.g., images; sounds).

Most of us take the functioning of our sensory syste~s for granted. With the

aging of the A~erican population, however, we suspect greater attention will be

paid to the social and pragmatic consequences of sensory deficits. The

communication problems associated with hearing deficits provide a case in

point. In the survey context, if a respondent has not heard all of a

particular question, the interviewer will reread it for him; but a respondent

with a serious hearing problem may have difficulty with most or all of the

survey questions--even though the interviewer is speaking louder than normal.

The quality of the interaction and the data obtained in this type of situation

can deteriorate very rapidly, especially if the survey is being conducted over

the telephone. Oftentimes, the respondent becomes annoyed or self-conscious

about constantly asking to have questions repeated, and the interviewer begins

to paraphrase questions instead of reading them exactly in an effort to

expedite the survey. For some questions, the respondent may simply provide an

answer to the question he thought he heard, and the interviewer will record

that answer as long as it matches one of the acceptable response precodes.

Whenever it is apparent that a respondent has a serious sensory deficit, the

use of a proxy or special collection procedures should be considered (e.g.,

scheduling a face-to-face in~erview, use of a dual-view CAPI instrument for

interviews conducted in the home) .

Cognition refers to those "processes by which informa~ion is acquired,

analyzed, and synthesized, including perception, memory, thought, and the

processes producing speech and behavior" (Stang and Wrightsman, 1981, p. 14).

Within the confines of this all-encompassing definition, we would include such

processes as encoding, comprehension, rehearsal, and retrieval; included also

would be the processes that make learning (e.g., conditioning, inhibition,

discrimination, generalization) and communication possible (e.g., the

acquisition of language, the development of concepts and other schemata).

It would be difficult to overstate the central role of cognitive processes in

directing and regulating human behavior, yet for muc~ of this century

mainstream psychology ignored these processes. The cognitive revolution of the

late 1960s provided psychology with a new perspective (e.g., Gardner, 1985).

Nithin federal statistical agencies, the cognitive revolution inspired the

4/ The senior author (JLE) wishes to thank Douglas Herrmann for sharing his

views on the issue of relationships between biological/psychophysiological

processes and memory, and for bringing to our attenti~n the thought-provoking

article by Leonard Poo~ (1980).
545

Cognitive Aspects of Survey Methodology (CASM) movement (e.g., see Jabine,

Straf, Tanur, and Tourangeau, 1984). The history and achievements of this

movement have been well documented by Jobe and Mingay (in press). We would

like to mention, however, that although the majority of CASM-inspired research

has been conducted on comprehension processes (e.g., Campanelli, Rothgeb, and

Martin, 1989; Lessler, Tourangeau, and Salter, 1989; Jobe and Mingay, 1989;

Palmisano, 1988) and retrieval strategies (e.g., Bradburn, Rips, and Shevell,

1987; Jobe et al., 1990; Means and Loftus, in press; Smith, Jobe, and Mingay,

in press), more survey-relevant research needs to be done on cognitive concepts

and processes that are "content based". For example, more attention needs to

be paid to "knowledge structures" such as schemata and scripts that make

everything from communication to image management possible (Bartlett, 1932;

Fiske and Linville, 1980; Graesser and Nakamura, 1982; Hastie, 1981; Rumelhart

and Norman, 1978; Schank and Abelson, 1977; Taylor and Crocker, 1981; cf. Alba

and Hasher, 1983; Thorndyke and Yekovich, 1980; for discussions specifically

relevant to surveys, see Dippo, 1989; Mathiowetz, Eisenhower and Morganstein,

1990). Other cognitive processes (e.g., logical operations) are important to

be sure, but clearly it would be impossible to interpret and respond to realworld events in a meaningful way were it not for knowledge structures. These

organized memories serve also as the bases for interrelationships between and

among other psychophysiological processes.

Consider an example of how schemata affect the data collection process and

interact with other psychophysiological processes. Suppose a census

representative calls you at home regarding the Current Population Survey. She

mentions that the CPS is the survey the government conducts every month to

generate employment and unemployment statistics, so you agree to participate.

Three minutes later, after all the relevant labor force questions have been

answered, she begins to ask a series of questions on earnings or on family

planning (e.g., "How much does Mr. Smith usually earn per week at this job

before deductions?"; "How large a family do you want?"). These questions

surprise you at first, because your schema (i.e., script) for this particular

type of survey does not include questions on income or ideal family size; soon

after, you become angry. Emotionally upset, you expend little cognitive effort

in answering further questions--that is, you satisfice (Krosnick, in press)--

and you may even decide to terminate the interview. These things actually

happen, but the point of the example is not to provoke a debate on the merits

of asking earnings questions or attaching supplements to the CPS; it is to

underscore the key role that knowledge structures play in regulating human

behavior. These organized memories not only help us to make sense of the

world, they also provide observers with a basis for understanding

interrelationships between and among other psychophysiological processes.

Similarly, self-schemata may affect how respondents answer survey questions.

For example, if a respondent perceives herself to be health conscious, her

answers to a series of survey questions on disease-preventive behaviors (e.g,

exercise, red meat or alcohol consumption) may be distorted in subtle ways.

Motivation refers to those processes that invite or induce goal-directed

behavior. Psychologists (Murray, 1938; Maslow, 1970) often differentiate

between two types of motives, primary and secondary motives. Primary motives

(e.g., hunger, thirst) are geared to the satisfaction of basic biological needs

(e.g., the need for food and water to nourish the body). Secondary motives are

directed to the satisfaction of basic social-psychological needs (e.g., the

need to interact with others). Maslow (1970) has proposed a hierarchy of human

needs which has implications for human behavior. The most powerful or

influential needs lie at the bottom of the hierarchy, the least powerful or

influential, at the top; the ordering from bottom to top is as follows:

physiological needs (e.g., food, water), safety needs (e.g., security,

stability, freedom from fear and anxiety), lovelbelongingness needs (e.g.,

affection, sOlidarity with others), esteem needs (e.g., desire for achievement,

mastery; desire for status, reputation), and self-actualization (i.e., the

desire to be all that one can be). According to Maslow, it is the satisfaction

of lower order needs that dominates our lives and our behavior; very few of us

achieve self-actualization.5

51 It is worth noting that there is no level of needs in Maslow's hierarchy

speaking to'the issue of survey participation; in other words, most people

probably do not sit by their telephones at night waiting to be called by a
546

Sometimes ques~ionnaire designers are so concerned with the cognitive demands

being placed on respondents (e.g., recall over extended reference periods),

they fail to recognize the potentially overriding significance of motivational

variables. For example, when respondents are asked sensitive questions in a

survey situation (e.g., questions on drug use or sexual behavior), motivational

considerations become very important and may affect responding in unexpected

ways. In an exploratory study of response tendencies to sensitive questions,

Esposito et ale (1990) asked volunteers recruited from a drug treatment clinic

how they thought survey participants might respond to a subset of questions

appearing on a draft version of the National Household Seroprevalence Survey

(NHSS); this survey requests information on behaviors that place individuals at

a high risk for transmitting or contracting the human immunodeficiency virus.

The responses that male volunteers provided to a question pertaining to the

frequency of heterosexual contacts in the previous 12 months are instructive.

Most helieved that participants would overreport sexual activity; a few thought

participants would u~derreport. The explanations provided for both

overreporting and underreporting had to do with image management, a

motivational concern. The former group thought that participants would want to

maintain a macho image of themselves, whereas the few individuals in the latter

group thought that participants might prefer not to be viewed as persons who

engaged in such high-risk behavior. These responses suggest that, for some

groups, the misreporting of sexual behavior is attributable not so much to

faulty cognitive processing (e.g., retrieval failure), but rather to

motivational considerations (i.e., image management).

Emotion refers to those psychophysiological processes that are experienced as

either distinctly pleasant (e.g., happiness, joy) or distinctly unpleasant

(e.g., anger, disgust). Mood, a closely associated concept, refers to "a

passing emotional state ... that tends to pervade and influence an individual's

thoughts, behaviors, and experiences during its occurrence (Stang and

Wrightsman, 1981, p. 60). Research by Bower and his associates (see Bower,

1981) has demonstrated that emotional states can affect such cognitive

processes as free associations and social perceptions. The Stanford-based

research program has also provided evidence for mood-state dependent memory

(i.e., events encoded in a particular mood state are more easily retrieved if

the individual is in the same emotional state during the recall attempt; for

reviews of this literature, see Johnson and Magaro, 1987; Singer and Salovey,

1988). Bower's research carries some interesting implications for conducting

surveys. For example, experienced interviewers can usually avoid angry

individuals when selecting a respondent from the household roster (e.g., they

can ask to speak to someone who is not so "busy" or they can offer to call back

at a "better time"); and doing so would appear to be an especially prudent

strategy in the case of sensitive-topic surveys like the NHSS. Besides the

obvious reasons for not wanting to interview such a person, Bower's findings

suggest that an angry respondent would be much more likely to perceive the

interviewer as a potential law-enforcement collaborator than as a neutral data

collector. Another implication of Bower's research is to expect more retrieval

failures for certain types of questions than for others. For example, a cancer

patient in a positive state of mind (perhaps because his/her cancer is in a

state of remission) presumably would have less success recalling each in a

series of painful chemotherapy treatments than a similar number of pleasant

physical treatments, because it is more likely that the former were encoded

when the respondent/patient was in a negative state of mind (e.g., depression)

(for related research, see Eich et al., 1985; Salovey, Jobe, and Willis, in

[Continuation of footnote 5] market research firm or the Census Bureau. In

fact, to the extent that people have other things they want to do, surveys are

essentially intrusive. The argument that surveys benefit society in the long

run (i.e., by determining what consumers/citizens do, think, or want) may prove

persuasive to the model citizen or to the individual who feels strongly about a

particular issue (e.g., gun control), but it simply will not do for those of us

who have come to expect immediate payoffs for our time and effort. Problematic

as this and the following may seem, the sooner we recognize that most surveys

are imbalanced social exchanges (i.e., the respondent gives and the sponsor/

interviewer takes), the sooner we will take steps to rectify the imbalance (see

Table 1, "Incentives" category). For a more generous evaluation of the nature

of this social exchange, the interested reader is directed to Converse and

Schuman (1974, pages 55-59}..,
~ ...

547

press). It is important not to get carried away here: affective state is but

one of many features of an event encoded in memory (for others, see Underwood,

1969). Inducing mood states as a means of improving recall may not be as

practical or efficient a strategy as using less problematic retrieval cues

(e.g., reading a script of the chemotherapy treatment process and including

photographs of the treatment room) (see Means and Loftus, in press) .

Except when these psychophysiological processes are being actively inhibited

(e.g., during sleep), they are presumed to be ongoing. As for organizational

preeminence, there are good reasons for viewing cognition as the dominant set

of processes (e.g., the central role these processes play in the construction

of meaning and in the regulation of adaptive behavior), but motivation and

emotion also exert powerful influences on behavior in certain situations.

Perhaps the only tenable position, then, is to recognize that all of these

processes play important roles in directing and regulating human behavior--

including the behavior that ensues when individuals agree to serve as survey

respondents.

A MODEL OF THE SURVEY INTERACTION PROCESS

To this point, we have presented a taxonomy of the survey context and have

described in very general terms the biological and psychophysiological

characteristics of survey participants. The third component in our framework

is a descriptive model of the survey interaction process (SIP); the seven

phases that comprise the model are identified below and described on the

following pages:

1. Interviewer and Respondent Orient Themselves Within the Survey Context

2. Interviewer Asks Question

3. Respondent Processes Question and Provides Answer

4. Interviewer Processes and Records Respondent's Answer

5. Interviewer and Respondent Reorient Themselves and Proceed to Next Question

[Recycle to Phase 2, or proceed to Phase 6]

6. Interview Is Concluded

7. Interviewer Reviews/Adjusts Protocol [optional phase]

In the case of face-to-face interviews, the interaction process (see Figure 1)6

is best represented as an ongoing verbal and nonverbal exchange between the

interviewer and the respondent (Kahn and Cannell, 1957; Suchman and Jordan,

1990; Sudman and Bradburn, 1974); hence the behavioral time lines are drawn

parallel to one another with bidirectional arrows in between. In the case of

telephone interviews, the exchange is primarily verbal. Several phases (2, 3,

and 4) overlap with one another, while others are best viewed as temporally

distinct.? Also, the question asking/responding/reorientation segment of the

6/ Of course, reality is never quite as neat and simple as the schematic

diagrams we generate in our feeble attempts to model it, and that is certainly

true of Figure 1. The reader unfamiliar with the actual conduct of surveys

might look at this figure and conclude that the question-asking-and-answering

sequence (phases 2-5) proceeds smoothly and effortlessly; nothing could be

further from the truth. Respondents oftentimes have difficulty either

understanding what information the interviewer is requesting or retrieving ttat

information from memory. They request clarification or provide qualified

answers. Interviewers occasionally have difficulty categorizing a respondent's

answer. They are required to probe when respondents give inadequate answers

and to provide feedback when respondents request clarification. It is anything

but smooth and effortless, interpersonal communication rarely is, and it is no

big deal--language is far from a perfect invention. To summarize, the above

are examples of the types of exchanges that often take place in person-toperson interviews; unfortunately, because their representation would complicate

our presentation of the model, these exchanges have not been made as explicit

as we might have liked (see bidirectional arrows).

7/ To be completely consistent with the organismic assumption of ongoing

change, the reoriention box in Figure 1 should really be represented as

overlapping with phases 2, 3, and 4. We thought that might make the model

overly complex. Again, it is very difficult to represent complex human

interactions with boxes and arrows--and only two dimensions.
- 548

Figure 1. A Model of the Survey Interaction Process

Phase Sequence:

1. Interviewer and Respondent Orie~t Themselves Within Survey Context

2. Interviewer Asks Question

3. Respondent Processes Question and Provides Answer

4. Interviewer Processes and Records Respondent's Answer

5. Interviewer and Respondent Reorient Themselves and Proceed to Next Question

[Recycle to Phase 2, or proceed to Phase 6)

6. Interview Is Concluded

7. Interviewer Reviews and Adjusts Questionnaire Protocol

Schematic Diagram:

Respondent

r--

11 j//~112 @\ \4

H 1~ 1~

11 ~//Jl \3 'l"//~I

L____-

- --,

I EJ \6/~1

1~ H

EJ 16//~1

--1

r ,

17 I

L -

I

Interviewer

Diagram Key:

\///v!Phase with Directed Verbal Communication

c==J Information-Processing Phase

r -.,

I- - I Optional Phase

H Bidirectional Verbal and/or Nonverbal Communication

model (phases 2 through 5) recycles until the interview is either completed or

abruptly terminated. Let us now look at each of these phases in greater

detail.

Phase 1: Interviewer and Respondent Orient Themselves Within the Survey

Context. During this first phase of the interaction process, the interviewer

and the respondent orient themselves to one another, to the setting, and to the

task at hand (i.e., completing the interview as accurately and expeditiously as

possible). It is here that information is usually exchanged between

participants regarding such things as the survey's purpose and the

characteristics of household members (i.e., household roster data). As Sudman

and Bradburn point out, the survey participants assume complementary roles: It

is the interviewer's role "to obtain [accurate] information" and the

respondent's role "to provide [accurate) information" (1974, p.17). The

behavior of each participant is circumscribed by these roles and by other

social norms regarding such interactions [however, see Suchman and Jordan

(1990) for a provocative discussion of how interviews differ from typical

verbal interactions]. To the extent that participants deviate sUbstantially

from their prescribed roles, one can expect to find that the data collected

later in the interview will be subject to response errors (Sudman and Bradburn,

1974). All subjectively salient aspects of the interview context--for example,

administration mode, interviewer and respondent characteristics, response

security level, item characteristics, survey sponsorship, et cetera--have the

potential to affect the outcome of the orientation phase (i.e., enlisting the

support and cooperation of a conscientious respondent in the informationgathering process} and the quality/validity of data collected subsequently

(Su~~an and Bradburn, 1974; Cannell, Marquis, and Laurent, 1977; Kahn and

Ca~nell, 1957; Groves, 1989). In ~ery general terms, the decision to
...

549

p2rticipate will depend in part on what interviewers and respondents think and

feel about one another, and in part on what they think and feel about the

survey.

Phase 2: Interviewer Asks Question. After the initial orientation phase is

over, the interviewer reads the first item on the questionnaire. In

considering the possibilities for miscommunication of question meaning, we must

consider not only question wording (e.g., clear vs. ambiguous terminology) and

item characteristics (e.g., focus, content, sensitivity), but also the position

of the item within the questionnaire itself (e.g., context effects) and the

interviewer's reading style (e.g., enunciation, speed). As Suchman and Jordan

(1990) point out, the issue of question meaning is much more complex that the

sequence of words that comprise a particular survey question; it involves the

entire context, everything that the participants say and do and bring with them

into the interaction (e.g., attitudes, emotions, beliefs, expectations). To

the extent that severe contraints are imposed on the interviewer (e.g., strict

rules against rephrasing any question the respondent finds confusing or

ambiguous), the data collector runs the risk of obtaining something less than

valid information.

Phase 3: Respondent Processes Question and Provides Answer. Even as it is

being read by the interviewer, the respondent is processing the content of the

question and considering his/her response--or the request for clarification

he/she intends to make if the question is not clear. There are several good

models of the auesticn answering process (Cannell, Miller, and Oksenberg, 1981;

Strack and Martin, 1987; Strube, 1987; Tourangeau, 1984, 1989; Willis, Royston,

and Bercini, in press). The model we feel most comfortable with was first

developed by Marquis (Cannell, Marquis, and Laurent, 1977) and later refined by

Cannell and his colleagues (Cannell, Miller, and Oksenberg, 1981). It can be

summarized as follows. Depending on the degree to which the survey participant

is willing to be a conscientious respondent, the number of steps in the

question answering process will vary (Cannell, Miller, and Oksenberg, 1981,

p. 393, Figure 1). The ideal respondent will proceed through the following

sequence: (1) comprehension of the question, (2) further cognitive processing

(i.e., assessment of informational requirements, retrieval of appropriate

cognitions, organization and response formulation), (3) evaluation of response

accuracy, (4) evaluation of response in terms of personal goals, and (5)

co~~unication of response. For the less-than-ideal respondent (i.e., one who

is unmotivated to cooperate or who does not fully understand a particular

question), the response generation/selection process can become sidetracked by

situational cues (steps 6 and 7). In step 6, which can follow any of the first

four steps described above, the respondent internally modifies his/her answer

to a question on the basis of situational cues that emanate from the

interviewer (e.g., status, appearance), the questionnaire (e.g., context

effects), or the respondent himself. In step 7, the cues alluded to in the

previous step result in specific response biases (e.g., desirability bias,

acquiescence bias) and the outcome is a response that is either incomplete or

inaccurate (Cannell, Miller, and Oksenberg, 1981, pp. 393-396). Again, we

believe this to be a very useful model; but in view of the theoretical material

presented earlier in the paper, we want to be explicit in pointing out that the

entire process is also subject to influences originating from biological

systems (e.g., poor health, fatigue) and from other psychophysiological

processes (e.g., extreme emotional states, sensory deficits) .

Phase 4: Interviewer Processes and Records Respondent's Answer. A very

important aspect of the interaction process is the way in which the interviewer

interprets and records the answers provided by the respondent. There would

appear to be parallels between the question-answering process (Phase 3) and the

response-categorization process; in the former, the respondent must interpret

the question appropriately in order to provide a valid answer, and in the

latter the interviewer must interpret the respondent's answer appropriately in

order to properly categorize it. Many of the same cognitive processes are

involved (e.g., comprehension of the response, further cognitive processing,

evaluation of response appropriateness, response categorization), so similar

types of errors can be anticipated (e.g., faulty inferences about what the

respondent said or meant to say; Kahn and Cannell, 1957, p. 19). Even more

basic than the interpretation of a respondent's answers is the interviewer's

understanding of the survey question. If interviewers have only a fuzzy

understanding of the inte~t of a particular question--for whatever reason

(e.g., awkwardly worded question, inadequate training)--they are apt to provide
550

~

inacc~rate fee~ack when responde~~s request clarification or they may fail to

probe when respondents provide an ~nadequate response. Both scenarios

,contribute to response errors.

Now that phases 3 and 4 have been ~escribed, we wish to digress for a moment to

make our position on "response error" explicit. Response error, in the sense

intended here, has been defined by one source (Sudman and Bradburn, 1974, p. 2)

as the difference between the true answer to a question and the respondent's

answer to that question. Although we recognize that, in some cases, response

error can be attributed with justi=ication (a) to interviewers (e.g., for not

following standard interviewing procedures, or for following them too closely;

see Suchman and Jordan, 1990; see also Means, Swan, Jobe, and Esposito,

in press), (b) to respondents (e.g., for not making the effort to retrieve

requested information; see Krosnic~, in press), or (c) to specific questions

or, more appropriately, to questio~ designers (e.g., for wording or phrasing

questions ambiguously; see Belson, 1981), we are more inclined to view response

error as a collaborative effort. ~hat is, we view response error as the

outcome of an interactive process that involves the interviewer, the

respondent, and, in many cases, the survey/question designer as well--a process

that takes place within a particular temporal and situational context, and one

that is greatly affected by that context. The three participants will not

always pla¥ equally significant ro:es, but usually they all contribute

something. &~d if we are to chara~terize response error as a collaborative

effort, to be consistent, we would have to characterize "response accuracy" in

the same manner.9

Phase 5: Interviewer and Respondent Reorient Themselves. Every question asked

on a survey changes the respondent and the interviewer to some degree and adds

something to the survey context. ~_,d to the extent that questions differ in

content, they require some form of reorientation on the part of the respondent

especially. The extent or degree 0= reorientation experienced is intimately

connected with the nature of the question and its placement within the

questionnaire itself. For example, sensitive questions (e.g., "How much money

~/ Let us clarify this point with a~ example that draws on our experience with

the industry and occupation (I/O) series of the Current Population Survey. The

I/O series asks respondents to provide information on the jobs at which they

and other household members are employed (e.g., "For whom does Mr. Smith

work?", "What kind of work was Mr. Smith doing?"). As straightforward as these

questions appear, they can produce some very unrealiable data (Collins, 1975).

Who is responsible for this state of affairs? Is it the respondent? Is it the

interviewer? Perhaps, the question~aire designer is responsible? The answer

to the first question, we will argue, is that all the participants are probably

responsible to some extent. Consider this sequence of events. Mrs. Smith is

asked to provide industry and occupation data about her husband, who, like some

husbands, does not like to talk a great deal about his job; but over the

years, she has picked up some odds and ends about what he does. In response to

the "What kind of work was Mr. Smith doing?" question, she tells the

interviewer what she knows, but because the question is very general, she goes

on for several minutes. The interviewer is hard pressed to keep up; he jots

down those things that seem relevant and lets the rest go. The information on

occupation is later coded by an individual in Jeffersonville, Indiana, who only

has the information that the interviewer could jot down from the discussion

with Mrs. Smith. Let us assume that the information provided is somewhat

ambiguous and, as a result, Mr. Smith's occupation is coded improperly. To

whom do you assign responsibility for the response error? Mrs. Smith? The

interviewer? The coder? The person who designed the occupation questions? We

think it should be viewed as a collaborative effort, and we believe that

response errors associated with othe= types of questions may follow a simila=

pattern.

~/ To accentuate the collaborative nature of this concept, let us operationally

define response accuracy as the appropriate, truthful answer to an unambiguous

question that has been read by the interviewer in the manner intended by the

question designer. What should make us somewhat optimistic about the survey

interaction process is that we oftentimes obtain accurate answers to the

questions we ask even when those questions are worded ambiguously (Belson,

1981) and/or when they have not been ~ead as intended.
551

<

did you earn last year?") can be very "expensive" in terms of psychophysiological processing. In addition to cognitive processing (e.g., retrieval

of dollar amounts for salary, interest, dividends, etc.), there will be a

certain amount of motivational processing having to do with the likely

consequences of answering the question truthfully (e.g., suspicion as to the

sponsor's intent, self-presentation strategies). Such questions can have

dramatic effects on the respondent's orientation (e.g., movement from a neutral

to a defensive posture) and, for just this reason, sensitive items generally

appear near or at the end of a questionnaire. Attitudinal questions provide

another case in point. Due in part to the cognitive reorientation that occurs

after such questions are asked and answered (e.g., priming effects of a context

question on the respondent's interpretation of what information is being

requested in a subsequent target question; see Strack and Martin, 1987), the

relative positioning of attitudinal questions within a survey has been shown to

influence how they will be answered (Schuman and Presser, 1981; Tourangeau and

Rasinski, 1988).

:~

.)

Having completed the discussion of the first five phases, let us now address

briefly the issue of questionnaire length. As we mentioned above, every

question asked and answered on a survey changes the respondent and the

interviewer to some degree. And because questions take both time and effort to

answer accurately (i.e., the continuous recycling of phases 2 through 5),

questionnaires containing a relatively large number of items tend to drain the

conscientious respondent both psychologically and physically; the effects on

the interviewer would appear to be similar. Depending on the nature of the

survey/questionnaire, there are presumably limits to how many questions

respondents will answer before losing enthusiasm. Generally speaking,

respondents do not appear willing to devote excessive amounts of time or energy

to a survey; as a result, one can reasonably expect to find that the quality of

response data will decline for the latter items on excessively long

questionnaires. This is something that survey sponsors must not lose sight of,

because as Groves (1987, 1989) has persuasively argued, there are clearly

tradeoffs between survey costs and survey errors.

Phase 6: Interview is Concluded. The way in which an interview is completed

can have carryover effects for subsequent contacts. Interviewers who are

cordial and carefully explain when and why subsequent contacts are necessary

are more apt to enlist the continued cooperation of respondents than

interviewers who are abrupt or taciturn. And even if subsequent contacts are

not necessary, an appreciative interviewer will improve the quality of the

respondent pool for other surveys.

Phase 7: Interviewer Reviews/Adjusts Protocol. Given the time constraints that

most surveys must adhere to, it may not always be possible for interviewers to

record the entire answer to open-ended questions. In such cases, the

interviewer may jot down a few key words and wait until after the interview is

concluded to review and adjust those items for which he/she has recorded only a

parti~l answer. The obvious problem with this practice is that the

reconstruction process can be biased by the answers the respondent gave to

other questions, or by the interviewer's own inferences about what the

respondent intended to report. According to Kahn and Cannell, recording errors

occur "...because of interviewers' tendencies to round out, amplify, or

otherwise modify responses. The interviewer may attempt, for instance, to make

logical or articulate in his recording an illogical or incomplete response, or

even a response which he is convinced the person was 'about to give'" (1957,

p. 19).

Now that the seven phases of the survey interaction process have been

described, we would like to mention briefly some of the model's more

interesting features: (1) it recognizes and highlights the interactive nature

of most survey interviews (Kahn and Cannell, 1957; Suchman and Jordan, 1990;

Sudman and Bradburn, 1974); (2) it takes a holistic view of the interaction

process and, as a consequence, views response accuracy and response error as a

collaborative effort involving the interviewer, the respondent, and the survey/

question designer; (3) it takes account of behaviors (e.g., interpersonal and

survey-relevant orientations) that models of the question-answering process

tend to ignore (e.g., Tourangeau, 1984; Strack and Martin, 1987); (4) it

incorporates psychophysiological processes (e.g., sensation, motivation,

emotion) that are sometimes pverlooked or understated in these models, and
- 552

(5) it dra,;s attention to the temporal dimension of the survey interaction

process (i.e., the question recycling loop) which, as Groves (1987, 1989)

.?oints out, can have serious cost and quality implications.

A PLAN FOR SYSTE~ATIZING SURVEY METHODOLOGICAL RESEARCH

The survey methodological literature, already quite substantial, is growing

larger every year; and our familiarity with that literature is far too modest

to delineate a research agenda for the 1990s. In this section of the paper, we

are more interested in suggesting ways of systematizing that body of research

than we are in presuming to tell social scientists what specific research

efforts should be undertaken in the coming decade. Before presenting some

ideas on systematization however, we want to be explicit about two of our

research biases. First, because of the inherent complexity of survey contexts

and the collaborative nature of the survey interaction process, we are

methodological pluralists who lean more to descriptive and relational research

(e.g., small-scalelab/field simulations, ethnographic studies) than to

narrowly focused experimental research (Rosenthal and Rosnow, 1984). Second,

because standard experimental and quasi-experimental designs manipulate a

limited set of "independent" variables (and tend not to be replicated), we

favor programmatic survey research that involves sets of investigations which

deal systematically with specific research issues (e.g., the quality of proxy

responding). These research preferences are simply extensions of the

theoretical material presented in previous sections.

Having made our biases explicit, we now present some ideas on systematization.

Before a community of researchers can move forward, they must have a clear

understanding of where they have been. The traditional way of documenting a

large bodi of research is to write comprehensive books on the general topic of

interest; 0 and there have been some impressive efforts made in the area of

surveys (e.g., Groves, 1989). And while we recognize that there will always be

a need for comprehensive reviews of specialized knowledge domains, this method

of documenting and co~~unicating technical knowledge is becoming increasingly

problematic. One reason for this is the proliferation and complexity of

scientific research. There are simply too many studies to be reviewed and too

little page space available for describing research designs and results

adequately. This would appear to be especially true in the social sciences,

where there is a growing realization that interdisciplinary research is the

most efficient way of tackling complex problems. A second reason is that once

a book is written, one cannot reorganize its contents to address specific

research questions. A third reason is the emergence of the computer as an

information-management tool. We think centralized computer databases are a

much more efficient method of storing and communicating technical information,

and we advocate their use for systematizing and utilizing the vast lit€rature

on survey research. This is certainly not a novel idea. In 1974, Sudman and

Bradburn developed an information-management plan in their study of response

effects that we find both attractive and practical. The specifics of such a

plan are discussed below.

To make access to the literature database efficient, survey research must be

coded so that users can select studies that are of interest to them. To guide

the selection process, we recommend the use of a maximally flexible coding

scheme that identifies the key features of a research report without being

overly complicated or burdensome. As a preliminary proposal, we would like to

suggest a coding scheme (see Table 2) that is based on our taxonomy of

contextual variables (for a more detailed coding scheme, see Bradburn and

Sudman, 1974, Appendix A). Studies could be selected for review on the basis

of key words, methods, or any of a variety of other research characteristics.

This would be very similar to comput~rized search systems (e.g., ERIC; the

Survey Methodology Information System maintained by the Inter-University

lQl A more focused approach is the specialized monograph or series report

(e.g., Cannell, Marquis, and Laurent, 1977). The federal government, for

example, sponsors the publication of several statistical series reports that

address survey research issues (e.g., Statistical Policy Working Paper Series,

sponsored by the Federal Committee of Statistical Methodology); these reports

also generally include reviews of the relevant literature.
"

553

.' Table 2. Coding Categories for Systematizing Survey Research

"~~

A. Key Words (list up to

AI:

A2:

A3:

A4 :

AS:

five) :

<--,

B. Research Methodology

BI: descriptive research (e.g., enthnographic method)

B2: relational research (e.g., correlational designs)

B3: experimental research (e.g., 2x3 factorial design)

C. Contextual Variables (code all that apply)

CI: Interview Setting

CIa: location of interview

CIa (I): respondent's home

Cla(2): interviewer's office

Cla(3): neutral setting

Clb: characteristics of location

Clb(l): security

CIb(2): presence of observers

Clc: other salient characteristics (please specify)

C2: Respondent Characteristics

C2a~ demographic characteristics

C2a(1): gender

C2a (2): age

C2a (3): race

C2b: socioeconomic status (SES) characteristics

C2b(1): education level

C2b(2): income level

C2c: organismic characteristics

C2c(1): biophysiological status

C2c(2): psychophysiological status

C2c(3): other salient characteristics

C2d: experiential characteristics

C2d(1): relevant past experience

C2d(2): knowledge of, and interest in, subject matter of survey

C3: Interviewer Characteristics

C3a: demographic characteristics

C3a(1): gender

C3a(2): age

C3a(3): race

C3b: socioeconomic status (SES) characteristics

C3b(1): education level

C3b(2): income level

C3c: organismic characteristics

C3c(1): biophysiological status

C3c(2): psychophysiological status

C3c(3): other salient characteristics

C3d: experiential characteristics

C3d(1): relevant past experience

C3d(2): knowledge of, and interest in, subject matter of survey

C4: Survey Publicity Method

C4a: advance correspondence from the sponsor of the survey

C4b: "cold contact" procedures

C4c: use of the media

C4d: word of mouth

C4e: other (please specify)

CS: Survey Sponsorship

CSa: public/governmental sponsorship

C5b: university/survey-research-center sponsorship

CSc: private industry sponsorship

CSd: other (please specify)

C6: Survey/Interview Administration Mode

C6a: face-to-face interview

C6a(1): paper-and-pencil format

C6a(2): computer-assisted format
554

C6b: telephone interview

C6b(1): paper-and-penc~l format

C6b(2): comouter-assis~ed format

C6c: self-administered ques~ionnaire

C6d: other (please specify)

C7: ReSDonse Security Level

C7a: co~plete anonymity

C7b: confidentiality

C7c: restricted availability

C7d: unrestricted availability

C7e: other (please specify)

C8: Attributes of the Survey Instrument

C8a: questionnaire characteristics

C8a(1): nature of questionnaire

C8a(2): length of the questionnaire

C8a(3): pace of questionnaire

C8a(4): homogeneity/heterogeneity of items on the questionnaire

C8a(S): information source

CBb: item characteristics

C8b(1): the target of the question

C8b(2): the focus of the question

C8b(3): the content of the question

C8b(4): the desirability or sensitivity of the question content

C8b(S): the response for~at of the question

C8b(6): the length and complexity of the question

CBb(7): other (please specify)

C9: Incentives

C9a: monetary incentives

C9b: tangible, non-monetary incentives

C9c: verbal incentives

C9d: other (please specify)

D. Brief Summary of Findings:

ID. Reference Identifiers:

1D1: Author Name(s)

1D2: Title of Work

1D3: Type of Reference

ID3(a): book

1D3(b): published article

ID3(c): series report (e.g., government, survey research

center)

1D3(d): conference paper

ID3(e): other (please specify)

consortium of Political and Soci~l Research, at the University of Michigan);

the difference, of course, is the nature of the information encoded in the

system. Once the database is in place--and we realize getting it in place will

be a monumental task--it can be used for a variety of purposes. For example, a

"research locator matrix" can be developed that enables users to select studies

on the basis of key features (see Table 3); the pool of studies retrieved could

then serve as the basis for a literature review, as a tool for identifying gaps

in the literature, or as a foundation for planning programmatic research.

To conclude, the proposed systematization of the survey methodological

literature would make it possible for social scientists to review an ever

expanding body of research in a very efficient manner; and to the extent that

researchers would also be exposed to a large body of interdisciplinary work,

this system could facilitate the discovery of patterns/interrelationships among

contextual variables. It is our hope that a system of this type will serve to

stimulate ambitious programmatic research efforts, which, in our view, hold the

greatest promise for advancing our understanding of the survey interaction

process.
Category Codes

A B C1 C2 C3 C4 C5 C6 C7 C8 C9 D ID

A

B

C1

C2

C3

C4

C5 a c9 d id

C6

C7

C8

C9

D

ID

555

Table 3. Simplified Research Locator Matrix .

~

Example of Use: An individual interested in the effects of survey sponsorship

(C5) on nonresponse in drug-use surveys would access the relevant literature in

cells "a" (nonresponse, drug use), "c9" (drug use surveys/questions), "d"

(findings), and Hid" (authors, reference type, title of work).

Category Codes:

A. Key Words

B. Research Methodology

C1. Interview Setting

C2. Respondent Characteristics

C3. Interviewer Characteristics

C4. Survey Publicity Method

C5. Survey Sponsorship

C6. Survey/Interviewer Administration Mode

C7. Response Security Level

C9. Attributes of the Survey Instrument

D. Brief Summary of Findings (Not a search

ID. Reference Identifiers

~~

,

y

'., ,

!,

j

variable. )

ACKNOWLEDGEMENTS

The authors wish to thank Pamela Campanelli and Jennifer Rothgeb, both of U.S.

Bureau of Census, for important contributions made to earlier drafts of this

paper. We also wish to thank Cathryn Dippo, Robert Groves, Douglas Herrmann,

Elizabeth Martin, and Michael Welch for their thoughtful reviews and for their

many helpful comments and suggestions.

REFERENCES

ALBA, J.W. and RASHER, L.

BUlletin, 93, 203-231.

(1983), "Is Memory Schematic?" Psychological

BALLOU, J. (1990), "Respondent/Interviewer Gender Interaction Effects in

Telephone Surveys," paper presented at the International Conference on

Measurement Errors in Surveys, Tucson, Arizona.

BARKER, R. (1968), Ecological Psychology. Stanford: Stanford University Press.

BARTLETT, F.C. (1932), Remembering. Cambridge: Cambridge University Press.

BELSON, W.

Gower.

(1981), The Design and Understanding of Survey Questions. London:
~ 556

"

van BERTALANFFY, L. (1967), Robots, Men, and Minds. New York: Braz~ller.

--

."

von BERTALANFFY, L. (1968), General System Theory. New York: Braziller.

"

BOWER, G. (1981), "Mood and Memory," American Psychologist, 36, 129-148.

:c

BRADBURN, N. and SUDMAN, S. (1979), Improving Interview Method and

Questionnaire Design. London: Jossey-Bass.

BRADBURN, N.M., RIPS, L.J., and SHEVELL, S.K. (1987), "Answering

Autobiographical Questions: The Impact of Memory and Inference on Surveys,"

Science, 239 (April 10), 157-161.

BRONFENBRENNER, U. (1977), "Toward an Experimental Ecology of Human

Development," American Psychologist, 32, 513-531.

BRUNNER, .G.A. and 'CARROLL, S.J., Jr. (1969), "The Effect of Prior Notification

on the Refusal Rate in Fixed Effect Surveys," Journal of .Advertising, 9,

42-44.

CM1PANELLI, P.C., ROTHGEB, J.M., and MARTIN, E.A. (1989), "The Role of

Respondent Comprehension and Interviewer Knowledge in CPS Labor Force

Classification," in Proceedings of the Section on Survey Methods Research,

American Statistical Association, 425-430.

CANNELL, C.R. and HENSON, R. (1974), "Incentives, Motives, and Response Bias,"

Annals of Economic and Social Measurement, 3, 307-317.

CANNELL, C.F., MARQUIS, K.H., and LAURENT, A. (1977), "A Summary of Studies of

Interviewing Methodology, 1959-1970," Vital and Health Statistics, Series 2,

Number 69, 1-78.

CANNELL, C.F., MILLER, P.V., and OKSENBERG, L. (1981), "Research on

Interviewing Techniques," in Sociological Methodology, ed., S. Leinhardt,

San Francisco: Jossey-Bass, 389-437.

CAPRA, F. (1982), The Turning Point. New York: Simon and Schuster.

COLLINS, C.L. (1975), "Comparison of Month-to-Month Changes in Industry and

Occupation Codes with Respondent's Report of Change: CPS Job Mobility Study,"

Response Research Staff Report (May 15, 1975), Number 75-5, U.S. Bureau of

Census.

CONVERSE, J.M. (1987), Survey Research in the United States.

University of California Press.

Berkeley:

CONVERSE, J.M. and PRESSER, S. (1986), Survey Questions: Handcrafting the

Standardized Questionnaire. Sage University Paper Series on Quantitative

Applications in the Social Sciences, 07-063. Beverly Hills, CA: Sage

CONVERSE, J.M. and SCHUMAN, H. (1974),

Wiley.

Conversations at Random. New York:

COX, L., JOHNSON, B., McDONALD, S-K., NELSON, D., and VAZQUEZ, V. (1985),

"Confidentiality Issues at the Census Bureau," in Proceedings of the First

Annual Research Conference, Bureau of Census, 199-218.

DIPPO, C. (1989), "The Use of Laboratory Techniques for Investigating Memory

Retrieval Errors in Retrospective Surveys," in Bulletin of the International

Statistical Institute, LIII, Book 2, 323-342.

DIPPO, C. and HERRMANN, D. (1990), "The Bureau of Labor Statistics' Collections

Procedures Research Laboratory: Accomplishments and Future Directions,"

Seminar on Quality of Federal Data, Council of Professional Associations on

Federal Statistics, Washington, DC.

DIJKSTRA, W. and VAN DER ZOUWEN, J. (1987), "Styles of Interviewing and the

Social Context of the Survey Interview," in Social Information Processing and

Survey Methodology, eds., H.J. Hippler, N. Schwartz, and S. Sudman.

New York: Springer-Verlag, 200-211. .,
--

t.

.

fL

,.

.~...

~,.

~

if;

~

P

.'\f;

~,...

~!

i'-=- ..

,.

.,

"

.;

-,

~;~.:!

:~,.

i\~

'!'"i:

;!

.,"

557

DIJKSTRA, W. and VAN DER ZOUWEN, J. (1982), Response Behavior in the SurveyInterview. New York: Academic Press.

EICH, E., REEVES, J.L., JAEGER, B., and GRAFF-RADFORD, S.B. (1985), "Memory

for Pain: Relation between Past and Present Pain Intensity," Pain, 23,

375-379.

ESPOSITO, J.L., KELLEY, C., WILLIS, G., SIRKEN, M., and NATHAN, G. (1990),

"SQR3: An Exploratory Study of Respondent Decision Making," unpublished

research report, National Center for Health Statistics, Office of Research

and Methodology.

FERBER, R. and SUDMAN, S. (1974), "Effects of Compensation in Consumer

Expenditure Surveys," Annals of Economic and Social Measurement, 3, 319-331.

FIENBERG, S.E. and TANUR, J.M. (1983), "Large-Scale Social Surveys:

Perspectives, Problems, and Prospects," Behavioral Science, 28, 135-153.

FISKE, S.T. and LINVILLE, P.W. (1980), "What Does a Schema Concept Buy Us?",

Personality and Social Psychology Bulletin, 6, 543-557.

FOWLER, F.J. and MANGIONE, T.W. (1986), "Interviewer Characteristics and

Nonresponse," in Proceedings of the Section on Survey Methods Research,

American Statistical Association, 299-303.

GARDNER, H. (1985), The Mind's New Science: A History of the Cognitive

Revolution. New York: Basic Books.

GRAESSER, A.C. and NAKAMURA, G.V. (1982), "The Impact of a Schema on

Comprehension and Memory," in The Psychology of Learning and Motivation,

Volume 16, ed. G. Bower, New York: Academic Press.

GROVES, R.M., BIEMER, P.P., LYBERG, L.E., MASSEY, J.T., NICHOLLS, W.L., II, and

WAKSBERG, J. (1988), Telephone Survey Methodology. New York: Wiley.

GROVES, R.M. (1987), "Research on Survey Da'Ca Quality," Public Opinion

Quarterly, 51, S156-S172.

GROVES, R.M. (1989), Survey Errors and Survey Costs. New York: Wiley.

HAGENAARS, J.A. and HEINEN, T.G. (1982), "Effects of Role-Independent

Interviewer Characteristics on Responses," in Response Behavior in the

Survey-Interview, eds. W. Dijkstra and J. van der Zouwen, London: Academic

Press, 91-130

HASTIE, R. (1981), "Schematic Principles in Human Memory," in Social Cognition,

The Ontario Symposium (Volume I), eds. E.T. Higgins, C.P. Herman, and M.P.

Zanna, Hillsdale, N.J.: Erlbaum, 39-88.

HENSON, R.M., CANNELL, C.F., and LAWSON, S. (1976), "Effects of Interviewer

Style on Quality of Reporting in a Survey Inter,:iew," Journal of Psychology,

93, 221-227.

HIPPLER, H.J., SCHWARTZ, N., and SUDMAN, S. (1987), Social Information

Processing and Survey Methodology. New York: Springer-Verlag.

JABINE, T.B., STRAF M.L., TANUR, J.M., and TOURANGEAU, R. (1984), Cognitive

Aspects of Survey Methodology: Building a Bridge Between Disciplines.

Washington, DC: National Academy Press.

JOBE, J.B. and MINGAY, D.J. (1989), "Cognitive Research Improves

Questionnaires," American Journal of Public Health, 79, 1053-1055.

JOBE, J.B. and MINGAY, D.J. (in press), "Cognition and Survey Measurement:

History and Overview," Applied Cognitive Psychology.

JOBE, J.B., WHITE, A.A., KELLEY, C.L., MINGAY, D.J., SANCHEZ, M.J., and LOFTUS,

E.F. (1990), "Recall Strategies and Memory for Health Care Visits," The

Milbank Quarterly, 68, 171-189.
558

JOHNSON, M.H. and MAGMO, P.A. (1987), "Effects of Mood and Severity on Memory

Processes in Depression and Mania," Psychological Bulletin, 101, 28-40.

KAHN, R.L., and CANNELL, C.F.

Wiley.

(1957), The Dynamics of Interviewing. New York:

KROSNICK, J.A. (in press), "Response Strategies for Coping with the Cognitive

Demands of Attitude Measures in Surveys," Applied Cognitive Psychology.

LASZLO, E. (1972), The Systems View of Life. New York: Brazi11er.

LESSLER, J., TOURANGEAU, R., and SALTER, W. (1989), "Questionnaire Design in

the Cognitive Research Laboratory," Vital and Health Statistics, Series 6,

No.1, (DHHS Publication No. PHS 89-1076), Washington, DC: U.S. Government

Printing Office.

MASLOW, A.H. (1970), Motivation and Personality. New York: Harper & Row.

MATHIOWETZ, l~.A., EISENHOWER, D., and MORGANSTEIN, D. (1990), "Recall Error:

Sources and Bias Reduction Techniques," paper presented at the International

Conference on Measurement Errors in Surveys, Tucson, Arizona.

MEANS, B. and LOFTUS, E.F. (in press), "When Personal History Repeats Itself:

Decomposing Memories for Recurring Events," Applied Cognitive Psychology.

MEANS, B., SWAN, G.E., JOBE, J.B., and ESPOSITO, J.L. (in press), "An

Alternative Approach to Obtaining Personal History Data," in Measurement

Errors in Surveys, eds. P.P. Biemer, G. Forsman, R.M. Groves, L.E. Lyberg,

N.A. Mathiowetz, and S. Sudman. New York: Wiley.

MILLER, J.G. (1978), Living Systems. New York: McGraw-Hill.

MURRAY, H.A.

Press.

(1938), Explorations in Personality. New York: Oxford University

NATHAN, G., SIRKEN, M.G., WILLIS, G.B., and ESPOSITO, J.L. (1990), "Laboratory

Experiments in the Cognitive Aspects of Sensitive Questions," paper.presented

at the International Conference on Measurement Errors in Surveys, Tucson,

Arizona.

NELSON, R.L. and HENDRICK, T.E. (1983), "The Statutory Protection of

Confidential Research Data," in Solutions to Ethical and Legal Problems in

Social Research, eds. R.F. Boruch and J.S. Cecil. New York: Academic Press.

OVERTON, W.F. (1984), "World Views and Their Influence on Psychological Theory

and Research: Kuhn--Lakatos--Laudan," in Advances in Child Development and

Behavior, Volume 18, ed. H.W. Reese. New York: Academic Press, 191-226.

PALMISANO, M. (1988), "The Application of Cognitive Survey Methodology to an

Establishment Survey Field Test," in Proceedings of the Section on Survey

Methods Research, American Statistical Association, 179-184.

PAYNE, S.L. (1951), The Art of Asking Questions.

University Press.

Princeton: Princeton

PLEWES, T. (1985), "Confidentiality: Principles and Practice," in Proceedings

of the First Annual Research Conference, Bureau of Census, 219-226.

POON, L.W. (1980), "A Systems Approach for the Assessment and Treatment of

Memory Problems, II in The Comprehensive Handbook of Behavioral Medicine,

Volume 1, Spectrum Publications.

ROSENTHAL, R. and ROSNOW, R.L.

New York: McGraw-Hill.

(1984), Essentials of Behavioral Research.

ROYCE, J.R. (1973), "The Present Situation in Theoretical Psychology," in

General Psychology, ed. B.B. Wolman, Englewood Cliffs; NJ: Prentice-Hall.
...

:;,

~;'"

559

~i

...

",,'

;I'

i

~

RUMELHART, D.E. and ORTONY, A. (1977), "The Representation of Knowledge in

Memory," in Schooling and the Acquisition of Knowledge, eds. R.C. Anderson,

R.J. Spiro, and W.E. Montague, Hillsdale, NJ: Erlbaum, 99-135.

G

~..

..

~

~

'"

~

~

SALOVEY, P., JOBE, J.B., and WILLIS, G.B. (in press), "Assessing the validity

of Retrospective Reports of Physical Pain on Health Surveys," in

Autobiographical Memory and the Validity of Retrospective Reports, eds.,

N. Schwarz and S. Sudman, New York: Springer-Verlag.

}I

f,

<"

SCHANK, R.C. and ABELSON, R.P. (1977), Scripts, Plans, Goals, and

Understanding. Hillsdale, NJ: Erlbaum.

SCHU~~, H. and PRESSER, S. (1981), Questions and Answers in Attitude Surveys.

New York: Academic Press.

~

(:

SCHUMAN, H. and CONVERSE, J.M. (1971), "The Effects of Black and White

Interviewers on Black Responses in 1968," Public Opinion Quarterly, 35,

44-68.

"~:

.'

'"f'..

,.

SINGER, E., FRANKEL, M.R., and GLASSMAN, M.B. (1983), "The Effects of

Interviewer Characteristics and Expectations on Response," Public Opinion

Quarterly, 47, 68-83.

~

_0':

l

"

SINGER, J.A. and SALOVEY, P. (1988), "Mood and Memory: Evaluating the Network

Theory of Effect," Clinical Psychology Review, 8, 211-251.

....

SMITH, A.F., JOBE, J.B., and MINGAY, D.J. (in press), "Retrieval from Memory of

Dietary Information," Applied Cognitive Psychology.

STANG, D.J. and WRIGHTSMAN, L.S. (1981), Dictionary of Social Behavior and

Social Research Methods. Monterey, CA: Brooks/Cole.

STRACK F. and MARTIN, L.L. (1987), "Thinking, Judging, and Communicating: A

Process Account of Context Effects in Attitude Surveys," in Social

Information Processing and Survey Methodology, eds., H.J. Hippler, N.

Schwartz, and S. Sudman. New York: Springer-Verlag, 123-148.

STRUBE, G. (1987), "Answering Survey Questions: The Role of Memory," in Social

Information Processing and Survey Methodology, eds., H.J. Hippler,

N. Schwartz, and S. Sudman. New York: Springer-Verlag, 86-101.

SUCHMAN, L. and JORDAN, B. (1990), "Interactional Troubles in Face-to-Face

Survey Interviews," Journal of the American Statistical Association, 85,

232-241, 252-253.

sum1AN, S. and BRADBURN, N.

Aldine.

(1974), Response Effects in Surveys. Chicago:

SUDMAN, S. and BRADBURN, N.

Jossey-Bass.

(1982), Asking Questions. San Francisco:

TAYLOR, S.E. and CROCKER, J. (1981) "Schematic Bases of Social Information

Processing," in Social Cognition, The Ontario Symposium (Volume 1)," eds.

E.T. Higgins, C.P. Herman, and M.P. Zanna, Hillsdale, N.J.: Erlbaum, 89-134.

THORNDYKE, P.W. and YEKOVICH, F.R. (1980) "A Critique of Schema-Based Theories

of Human Story Memory," Poetics, 9, 23-49.

TOURANGEAU, R. (1984), "Cognitive Science and Survey Methods," in T.B. Jabine,

M.L. Straf, J.M. Tanur, and R. Tourangeau (1984), Cognitive Aspects of Survey

Methodology: Building a Bridge Between Disciplines. Washington, DC: National

Academy Press, 73-100.

TOURANGEAU, R. (1987), "Attitude Measurement: A Cognitive Perspective," in

Social Information Processing and Survey Methodology, eds., H.J. Hippler,

N. Schwartz, and S. Sudman. New York: Springer-Verlag, 149-162.

TOURANGEAU, R. and RASINSKI, K.A. (1988), "Cognitive Processes Underlying

Context Effects in Attitude Measurement," Psychological Bulletin, 103,

299-314. ':
560

TURNER, C. and ~~TIN, E.A.

New York: Russell Sage.

(1984), Surveying Subjective Phenomena, Volume 1,

UNDERWOOD, B.

559-573.

(1969), "Attributes of 11emory," Psychological Review, 76,

WILLIS, G.B., SIRKEN, M.G., NATHAN, G., KELLEY, C.L., KEER, D.W. and ESPOSITO,

J.L. (1990), "The Cognitive Basis of Responses to Sensitive Survey

Questions," paper presented at the meeting of the American Public Health

Association, New York City, NY.

WILLIS, G.B., ROYSTON, P.N., and BERCINI, D. (in press), "The Use of Verbal

Report Methods in the 1>.pplied Cognitive Laboratoty," Applied Cognitive

Psychology.
Bureau of the Census

1991

Annual

Research

Conference

March 17-20, 1991

Holiday Inn Crowne Plaza

300 Army Navy Drive

Arlington, Virginia 22202

Proceedings

-

/...~~

(_'!l 1:-

\~)~.~

U.S. Department of Commerce

Economics and Statistics Administration

BUREAU OF THE CENSUS
I~

~~.

<

561

I

i

~

Discussion

Eleanor Singer

Columbia University

~

'.

t

.,

f

r,

It may be useful to locate the two papers we heard today within a broader framework.

One such framework is what survey methodologists refer to as "nonsampling errors in

surveys" (e.g. Groves 1987, 1989) and cognitive psychologists refer to as cognition

and survey research (e.g. Jabine, Straf, Tanur, and Tourangeau 1984; Hippler, Schwartz,

and sudman 1987). The paper by Shepherd and Vincent is closer to the "nonsampling

error" pole of the typology, whereas the approach of Esposito and his colleagues is

perhaps closer to that of the cognitive psychologists. That is, Shepherd and Vincent

conceptualize a variety of interviewer behaviors as being either correct or not, with

incorrect behaviors hypothesized as leading to "error" in the survey responses

obtained. Esposito and Jobe, on the other hand, take a more neutral stance,

conceptualizing a large range of variables as potentially influencing survey responses

by changing the respondent's definition of the situation. They view responses as

shaped by the context of the interview, by the characteristics of survey participants,

and by the interactions between interviewers and respondents, but they do not

necessarily think of responses as being objectively "true."

An alternative framework in which the two papers can be located is that of Groves'

"measurers" and "reducers" (Groves 1987), with Shepherd and Vincent seeking primarily

to learn about interviewer behavior in order to reduce response errors associated with

that behavior, whereas Esposito and Jobe try to model and measure the behavior of both

interviewers and respondents in order to understand more about the processes involved.

In a sense, the Shepherd and Vincent paper can be located within section 3 of the

Esposito and Jobe paper. Both papers promise significant advances for research. I'd

like to begin by discussing the Shepherd and Vincent paper and then move on to that by

Esposito and Jobe.

Interviewer Behavior

The research by Shepherd derives from, and extends, a long tradition of research

specifically on interviewer behavior and interviewer effects (e.g. Cannell, Oksenberg,

and Converse 1979; Cannell, Miller, and Oksenberg 1981; Groves and Mathiowetz 1984;

Groves and Magilavy 1986; Hyman et al. 1954; Singer and Kohnke-Aguirre 1979; Singer,

Frankel, and Glassman 1983; Sudman--I979; Tucker 1983). Within that tradition, we can

distinguish at least two approaches. One aims to measure the fraction of total

response variance associated with interviewers (e.g. Groves and Magilavy 1986; Tucker

1983). The second is interested primarily in identifying and controlling specific

behaviors or qualities of interviewers that are associated with better or worse

responses (e.g. Cannell, Oksenberg, and Converse 1979; Hyman et al. 1954; Singer,

Frankel, and Glassman 1983; Sudman 1979). The study by Shepherd and Vincent clearly

falls in the latter category.

Past studies on interviewer effects present us with something of a paradox. Those

studies that have concentrated on measuring the contribution of interviewers to total

response variability have generally concluded that this contribution is modest,

especially in telephone interviews. Groves and Magilavy (1986), in a thorough

investigation of this topic, conclude that the intraclass correlation coefficient for

interviewers clusters below .01 on SRC telephone surveys, whereas those from personal

interviews average closer to .02. (Of course, the total effect of interviewers will

increase dramatically when a small number of interviewers take a large number of

interviews, as is generally the case in telephone surveys.)

Furthermore, even though past studies of interviewer behavior, such as those by Blair

(1979), have found that nonprogrammed interviewer speech behaviors occur on over half

the questions in face-to-face interviews, neither Blair nor Grover and Magilavy (1986)

have found any evidence that these deviations affected responses. As Blair puts it,

"The behaviors occur, but they do not affect the data." (1979, p. 50) He goes on to

1. Another interesting, and somewhat disconcerting, finding by Blair was that

nonprogrammed speech behaviors increased (though not significantly) with interviewer

experience. More experienced interviewers "made more reading errors, engaged in more

variations, probed more often, and gave more feedback." (1979, p. 49) Presser and

Zhao (1990), however, found no relationship between accuracy in asking questions and

interviewer experience.
562

say that perhaps different methods are needed to capture the effects of these

behaviors on responses.

So, total variance attributable to interviewers is small, and nonprograrnrned

interviewer speech behavior, though quite frequent, apparently has no discernible

impact on responses. At the same time, studies that have hypothesized specific

interviewer effects -- e.g., the effect of interviewer expectations on response

(Hyman et al. 1954; Sudrnan 1979; Singer and Kohnke-Aguirre 1979; Singer, Frankel, and

Glassman 1983) or race-of-interviewer effects (Schuman and Converse 1971; Schuman and

Hatchett 1974) -- have tended to find support for such effects, though generally they

have been unable to specify the means by which they come about.2 Thus, under certain

conditions, interviewers can have substantively important effects on responses. The

task for research is to specify the conditions under which, and the processes by

which, this occurs.

The study of interviewer behavior by Shepherd and Vincent really consists of at least

two studies and two separate coding systems. One coding system is designed to measure

what they, following Cannell, call "interviewer compliance" -- in the first place,

compliance in asking questions, which means both asking those questions that should

have been asked, and asking them as written; and second, compliance in probing --

which again refers both to probing when it is appropriate and also to probing in an

appropriate fashion.

~pplying this coding system to 6,810 questions on 48 interviews, Shepherd and Vincent

=ind that interviewers follow correct skip patterns 96% of the time, with only 4% of

the questions skipped when they should have been asked. But what is amazing is not

the small percentage of incorrect omissions, but rather how, given the constraints of

a CATI system, such omissions could have occurred at all! Groves and Mathiowetz

(1984), for example, estimate that such errors occurred 1.8% of the time with a CATI

system and 8.8% of the time in telephone interviews using paper and pencil

~uestionnaires. Using a CAT I system, it ought to be possible virtually to eliminate

~his type of error, and such an outcome is precisely the kind of benefit envisaged by

Shepherd and Vincent's research.

S~epherd and Vincent also report that 94% of the questions were asked as written, and

a~other 4% with minor variations. (No figures are presented, in this preliminary

=eport, on interviewer compliance with probing instructions.) Presser and Zhao (1990)

=eport that 92% of questions in their study were asked as written, and note that these

:evels are similar to rates reported in other centralized telephone studies but much

~igher than those in face-to-face studies. If we compare the Shepherd and Vincent

=i;ure with Blair's (in his study, only two thirds of the questions were asked as

~~itten), then it is clear that the combination of computer-assisted and telephone

i~terviewing brings about a tremendous increase in the accuracy of reading the

~~estion. It would be useful to know just how much the computer contributes to the

c=~pliance observed with telephone interviewing alone, and whether the use of CAPI

Aill bring about the same error reduction.

~~e second coding system, designed by Shepherd and Vincent to capture the interaction

==

interviewers and respondents much as Bales' system of categories (1976) was

~esigned to capture the interaction among members of task-oriented groups, is much

~==e elaborate, and the authors have barely begun to exploit its possibilities.

:~ce satisfactory levels of inter coder reliability have been achieved with this

s~-stem, many kinds of analyses will become possible. For example, characteristics of

~~e interviewers, as well as characteristics of the questions, can be linked to

~~~erviewer behavior. Number and type of interactions can be related to question

~~~e, on the one hand, and to data quality, on the other. And other classificatory

schemes -- for example, sensitive and nonsensitive questions -- can be introduced

~~~o the analysis, as well.

~se of this analytic tool should make it possible to achieve the goals of survey

~e~hodologists as well as cognitive psychologists. On the one hand, it should make

~cssible improvement of interviewer compliance with the intent of the questionnaire

In his work on response quality, Cannell has demonstrated that certain programmed

~e~aviors on the part of the interviewer lead to more detailed reporting, or to the

=e?orting of more episodes, by the respondent. But this is different from being able

~c account for hypothesized connections between, say, interviewer expectations or

~~~erviewer race and a particular pattern of responses by specifying the interviewer

=e~aviors involved.
r

I

I

I

I

i

~

i

i

I

~

~

r

t

~;

~,.

r-:

~';

..

>

:::;

"

I

563

developer, and investigation of whether or not compliance, in turn, is associated with

reduced response error and/or reduced survey cost, which are after all the two payoff

variables in research on interviewing. On the other hand, and equally important, it

should also make it possible to study the processes -- the behaviors -- by which

interviewers influence survey responses.

I I

: I

There are two preconditions for these outcomes. One -- the relatively simpler -- is

coding reliability, which the authors are working on. The second, more complex, is

development of a scheme for categorizing interactions which is capable of explaining

the dependent variables of interest. The authors are well on the way to developing

such a scheme, but perhaps that scheme is worth a little more thought. Perhaps the

reason we have failed to find effects of interviewer behaviors in the past is because

we have not adequately conceptualized what this behavior means for the respondent's

motivation or understanding. For example, how should an "Un-hun" by the interviewer

be coded? Is it a reinforcement of the previous response? Is it an expression of

solidarity with the respondent? Is it a random noise the interviewer makes to fill

the pause while she enters a response? Is it an irritant to the respondent? Does

"Un-hun" function in all of these ways, or sometimes in one and sometimes in another

(or does it, perhaps, have a different meaning still)?

I have no better categorization scheme in mind than the one being developed by

Shepherd and Vincent. I would just like to caution that there are many potential

schemes, that they do not inevitably emerge from the data, and that the choice among

them should probably be theory-driven. This promises to be an enormous task, but

potentially a very rewarding one.

A General Model of the Survey Interaction Process

Le~ me turn, now, to the second paper, whose purpose is to provide a theoretical

framework that researchers from a variety of disciplines can use to structure and

communicate their research ideas and findings. It represents an ambitious effort to

construct a comprehensive accounting scheme for all the variables in the interviewing

situation that may affect survey response.

Like the paper by Shepherd and Vincent, it may be useful to locate the present one in

a broader context. And to my way of thinking, the context for this paper is the area

of meta-analysis. Within sociology and survey research, the impetus for meta-analysis

probably carne from Bradburn and Sudman's book on Response Effects in Surveys (1974),

as well as a series of papers by Heberlein and Baumgartner (1978), Eichner and

Habermehl (1978), and Goyder (1982). These papers tried, quite successfully, to

identify the factors affecting response rates to mail surveys by drawing on a data

base of published studies that provided information about the dependent variable,

namely response rate, as well as about a large number of independent variables. Since

then, the techniques and requirements for meta-analysis have been codified, and metaanalysis has become an accepted method for synthesizing a large amount of information

about a given area. (Hunter, Schmidt, and Jackson 1982; Glass, McGraw, and Smith 1981;

Ivolf 1986).

Coding a large number of studies (it doesn't really matter whether they are published

studies, as in the case of Bradburn and Sudman or Heberlein and Baumgartner, or

prospective ones) with respect to the independent variables listed by Esposito and

Jobe might make it possible to identify the major variables having an impact on survey

response rates and response quality. And such an undertaking may well represent a

useful corrective for the tendency of survey analysts (and, for that matter, cognitive

psychologists as well) to focus on one or at most a very few such variables, and thus

potentially to misspecify their equations.

But the difficulty with the "model" proposed by Esposito and Jobe is that it is a

taxonomy rather than a theoretical framework. And therein, unfortunately, lie its

drawbacks as well as its virtues.

The virtue of the taxonomy is its attempt at exhaustiveness -- at specifying all

the variables that potentially affect response rates and, more especially, response

quality. But the very attempt to be exhaustive makes the model extremely difficult

to work with. The list of what is there to be considered is already very long; and

in fact the authors intend it to be longer still. For example, under "organismic

characteristics" in Table 1, the authors list "other salient characteristics (i.e.

physical appearance; personality traits; intelligence level; communiciltion skills),"

What aspects of physical appearance we are to take into account, which personality

traits, and so on, all remain to be specified.
54

m.

....

a~

ir

i~

re

cc

SJ

cj

Sj

g1

er

w:

DE

pJ

pJ

(:

pI

ir

01

m.

m,

p1

i.

s

p:

i:

M

a

p

c

P

?

w

s

T

c

t

a

t

e

a

t

W

u

s

i

::

a

:J

N

:J

il

:i

3.

564

Aside from sheer length, there is another problem with the taxonomy which is

reminiscent of the problem I raised in connection with the paper by Shepherd and

Vincent. Absent a theoretical framework, how are we to know what dimensions of each

of the variables should be attended to? Esposito and Jobe raise the same issue

when they say, on page 14, "Depending on one's training and theoretical orientation,

psychophysiological processes can be described and organized in a variety of ways."

That statement is true not only of psychophysiological processes, but of all the

factors considered in this paper. For example, .does it matter whether the-Interview

is done in the respondent's home or in the interviewer's office, or is it the privacy

of the research interview, or some other aspect of the setting, which matters? Granted

that we may want to describe an interview on all of the dimensions listed, which of

them are we going to select for analysis, and on what grounds?

It is when we come to the analytic phase of research that the need for a theoretical

framework becomes compelling. Esposito and Jobe state, on page 33, that they are

"methodological pluralists who lean more to descriptive and relational research than

to narrowly focused experimental research." But although one may want to use the

taxonomy for descriptive purposes, ultimately, it seems to me, one wants to see which

of the variables make a difference; and in order to do that, one must decide which

variables, and what dimensions, make theoretical sense. For example, even if one were

to decide to focus on a subset of the variables -- say, on motivational variables

only -- one needs a theory to say which of the contextual variables might be expected

to interact with the motivational variables, and which could safely be ignored.

Nor is it clear to me why standard experimental and quasi-experimental designs could

not fruitfully be used to investigate specific research issues related to the

interview. There are at least two ways to achieve what Egon Brunswik has referred to

as a "representative research design." One is to code a large number of existing (or

prospective) studies along a set of theoretically relevant dimensions, and then use

these studies to test hypotheses about the interrelationship of variables under varying

conditions. For example, suppose we have a large number of studies in which

confidentiality assurances have been used. Some have been done by government agencies,

some by universities; some have asked sensitive questions, others have not; some have

given absolute assurances of confidentiality, some have not mentioned confidentiality,

and some have given a qualified assurance. If we are lucky -- and we rarely are,

except over a long period of time and with a deliberate attempt to fill in empty

cells -- we will be able to look at the effect of confidentiality assurances under all

possible conditions described by these factors. But even if we are, note that we have

no surveys done by commercial firms, and our ability to generalize will therefore be

limited by that omission.

The other way of attaining representative

relevant variables and test their effects

It seems to me tnat one could make a good

the first, at least at a certain stage of

presented today provide a good example of

useful.

research designs is to conceptualize the

piece by piece, experiment by experiment.

case for preferring the second approach to

the research. And in fact, the two papers

an area where such an approach would be

The underlying assumption made by Shepherd and Vincent is that the greater the

constraints on the interviewer (in terms of skip patterns, the wording of explanations

to respondents, when and how to probe, and probably other variables as well), the

better the quality of responses is likely to be. Esposito and Jobe articulate what

appears to be the contrary hypothesis: "To the extent that severe constraints are

imposed on the interviewer (e.g. strict rules against rephrasing any question the

respondent finds confusing or ambiguous), the data collector runs the risk of obtaining

less than valid information."

I

It seems to me that this juxtaposition of hypotheses cries out for experimental

testing, and that in order to be most useful, such a test should systematically vary

other factors as well -- for example, the topic of the survey, the educational level

of the respondents, and any others that we believe, on theoretical grounds, might

affect the relationship of central concern.

:i

!1

1

~

.~

There is, as I have said, enormous promise for survey research in both of these papers.

I think that promise is more likely to be fulfilled if we go beyond description,

however elaborate the descriptive framework may be, to articulating a theory, however

primitive, about the processes involved.
...

I

~

---~~----

565

REFERENCES

BALES, ROBERT F.

Chicago Press.

(1950, 1976), Interaction Process Analysis, Chicago: University of

BLAIR, EDWARD A. (1979), "Interviewer Variations in Asking Questions," in Norman M.

Bradburn, Seymour Sudman, and Associates, Improving Interview Methods and

Questionnaire Design, San Francisco: Jossey-Bass, pp. 26-50.

BRADBURN, NORMAN M. and SUDMAN, SEYMOUR (1974), Response Effects in Surveys,

Chicago: Aldine.

BRADBURN, NORMAN M., SUDMAN, SEYMOUR, and Associates (1979), Improving Interview

Methods and Questionnaire Design, San Francisco: Jossey-Bass.

BRENNER, M. (1982), "Response Effects of 'Role-Restricted' Characteristics of the

Interviewer," in W. Dijkstra and H. van der Zouwen, eds. Response Behavior in the

Survey Interview, London: Academic Press.

CANNELL, CHARLES F., MILLER, PETER V., and OKSENBERG, LOIS (1981), "Research on

Interviewing Techniques," in Sociological Methodology, S. Leinhardt, ed.,

San Francisco: Jossey-Bass, pp. 289-337.

CANNELL, CHARLES F., OKSENBERG, LOIS, and CONVERSE, JEAN M. (1979), Experiments in

Interviewing Techniques, Ann Arbor, MI: Survey Research Center.

EICHNER, KLAUS and HABERMEHL, WERNER (1978), "Predicting Response Rates to Mailed

Questionnaires," American Sociological Review, 46: 361-363.

GLASS, GENE V., MCGRAW, BARRY, and SMITH, MARY LEE (1981), Meta-Analysis in Social

Research, Beverly Hills, CA: Sage.

GOYDER, JOHN C. (1982), "Further Evidence on Factors Affecting Response Rates to

Mailed Questionnaires," American Sociological Review, 47: 550-553.

GROVES, ROBERT M.

51:S156-S172.

(1987), "Research on Survey Data Quality," Public Opinion Quarterly,

GROVES, ROBERT M. (1989), Survey Errors and Survey Costs, New York: Wiley.

GROVES, ROBERT M. and MAGILAVY, LOU J. (1986), "Measuring and Explaining Interviewer

Effects in Centralized Telephone Surveys," Public Opinion Quarterly, 50:251-266.

GROVES, ROBERT M. and MATHIOWETZ, NANCY A. (1984), "Computer-Assisted Telephone

Interviews: Effects on Interviewers and Respondents," Public Opinion Quarterly,

48: 356-389.

HEBERLEIN, THOMAS A. and BAUMGARTNER, ROBERT (1978), "Factors Affecting Response Rates

to Mailed Questionnaires," American Sociological Review, 43:447-462.

HIPPLER, HANS-J., SCHWARZ, NORBERT, and SUDMAN, SEYMOuR (1987), Social Information

Processing and Survey Methodology, New York: Springer-Verlag.

HUNTER, JOHN E., SCHMIDT, FRANK L., and JACKSON, GREGG B.

Beverly Hills, CA: Sage.

(1982), Meta-Analysis,

HYMAN, HERBERT H. et al.

of Chicago Press.

(1954), Interviewing in Survey Research, Chicago: University

JABINE, TOM B., STRAF,MYRON L., TANUR, JUDITH M., and TOURANGEAU, ROGER (1984),

Cognitive Aspects of Survey Methodology: Building a Bridge Between Disciplines,

Washington, D.C.: National Academy Press.

PRESSER, STANLEY and ZHAO, SHANYANG (1990), "Attributes of Questions and Interviewers

as Determinants of Interviewing Performance," Unpublished paper, University of

r-laryland.

SINGER, ELEANOR and KOHNKE-AGUIRRE, LUANE (1979), "In=erviewer Expectation Effects:

A Replication and Extension," Public Opinion Quarterly, 43:245-60.

,

I

t

: I

i 1

.

1

i

i
5'

....

~.

a

i

r.

C

S

C

S

g

e

w

)

p

p

(

P

i

J

:n

:n

p

i

s

:)

i

!1

a

::J

:::

::J

::J

N

s

";'

:::

-

3.

3.

3.

v

)

1

)

1

i

566

SINGER, ELEANOR, FRANKEL, MARTIN R., and GLASSMAN, MARC B. (1983), "The Effect of

Interviewer Characteristics and Expectations on Response," Public Opinion Quarterly,

47:68-83.

SCHUMAN, HOWARD and CONVERSE, JEAN M. (1971), "The Effect of Black and White

Interviewers on Black Responses in 1968," Public Opinion Quarterly, 35:44-68.

SCHUMAN, HOWARD and HATCHETT, SHIRLEY (1974), Black Racial Attitudes: Trends and

Complexities, Ann Arbor, MI: Institute for Social Research.

SUDMAN, SEYMOUR (1979), "The Role of Interviewer Expectations," in Bradburn and

Sudrnan, op. cit., pp. 51-63.

TUCKER, CLYDE (1983), "Interviewer Effects in Telephone Surveys," Public Opinion

Quarterly, 47:84-95.

WOLF, FREDERIC M. (1986), Meta-Analysis: Quantitative Methods for Research Synthesis,

Beverly Hills, CA: Sage.
--

~

~

i

,

I

I

I

Ii

~,

~

~

[

t~

~

t"

~

V- I-

~:

~

i"i

't

567

Floor Discussion Jane Polzer

Bureau of the Census

TWo papers were presented during this session which addressed interviewerrespondent (I-R) interaction. The first paper analyzed interaction in a

computer-assisted telephone interviewing (CATI) environment. The second paper

proposed a model for evaluating factors that affect the interviewer-respondent

interaction that would be applicable to all interviewing situations.

Jane Shepherd and Carmen vincent both of Westat, Inc. taped interviews of

college students responding to a 360 item CATI questionnaire. From the replay

of these tapes they developed a series of codes to match to various respondent

and interviewer interactions. They then paired these interviewer-respondent

interactions using these coding systems. Although not complete, they are

working toward developing a I-R model that could be applied when developing

computer assisted interviewing systems, designing CATI questionnaires and used

when developing interviewer training.

James Esposito (BLS) and Jared Jobe (NCHS) then presented a theoretical

framework for evaluating the I-R interaction. This framework takes into account

features of the survey process, the survey context and the survey participants.

For each of these areas the researchers developed a series of contextual

variables that may interact with one another in the interviewing environment.

special emphasis was given to the features of survey participants and the

factors that are assumed to affect their behavior, including psychophysiological

functions. From these data the researchers hope to present a framework that is

more comprehensive than other I-R models currently in use.

The discussant, Dr. Eleanor Singer (Columbia University) spoke of the goal of

research in this area as being to define conditions under which interviewers can

affect respondent responses. She was very interested in the final results of

the Shepherd and Vincent paper and its application to other CAI situations. She

felt that the Esposito and Jobe I-R framework was really more of a taxonomy and

that the actual framework for evaluating I-R interaction still needed to be

defined. She also asserted that the framework presented, because of its size,

makes it difficult to implement.

Gemma Furno of the Census Bureau asked Shepherd and Vincent whether the quality

of the data entry by interviewers on the CATI survey was analyzed. They have

not looked at this yet, however they did build in audit trails on the items and

do plan to assess quality.

Dr. Milton Goldsamt from the National Agricultural statistics service, in

response to the Esposito and Jobe paper, mentioned that adding the

"meaningfulness of the question" as a measure in their framework may benefit

cost analysis. He also mentioned that he would like to see more salience theory

included in the framework.
Bureau of the Census

1991

Annual

Research

Conference

March 17-20, 1991

Holiday Inn Crowne Plaza

300 Army Navy Drive

Arlington, Virginia 22202

Proceedings

/...~~

(-~ \

\~)

~."

u.s. Department of Commerce

Economics and Statistics Administration

BUREAU OF THE CENSUS