Vital and

Health Statistics

Questionnaire Design

in the Cognitive

Research Laboratory

Series 6:

Cognition and Survey

No, 1

Measurement

This report describes a three-phase research project exploring the role of

laboratory studies in designing and testing questionnaires, determining how

methods and theories of cognitive sciences can contribute to designing

and testing questionnaires, and comparing current questionnaire

development and pretesting procedures with alternative procedures,

U,S, DEPARTMENT OF HEALTH AND HUMAN SERVICES

Publlc Health Service

Centers for Disease Control

National Center for Health Statistics

Hyattsville, Maryland

May 1989 (Reprinted April 1993)

DHHS Publication No, [PHS) 89-1076
National Center for Health Statistics

Manning Feinleib, M.D., Dr.P.H., Director

Robert A. Israel, Deputy Director

Jacob J. Feldman, Ph.D., Associate Director forAnaZysi.s

and Epidemiolo~

Gail F. Fisher, Ph.D., Associate Director for Planning and

Extramural Programs

Peter L. Hurley, Associate Director for VW and Health

Statitks Systems

Stephen E. Nieberding, Associate Director for Management

Charles J. Rothwell, Associate Director for Data Processing

and Sewices

Monroe G. Sirken, Ph.D., Associate Director for Reseaxh

and Methodology

Office of Research and Methodology

Monroe G. Sirken, Ph.D., Associate Director

Kenneth W. Harris, Special Assistant for Program

Coordination and Statistical Standards

Lester R. Curtin, Ph.D., Chie$ Statistical Methods St@

James T. Massey, Ph.D., ChieJ Survey Design Staff

Andrew A. White, Ph.D., Chie$ Statistical Technology

Staff

‘T& research was part of a larger project entitle~ ‘T.aboratoryb~d Research on the Co@itive Aspects of Survey Methodology:’

being conducted by the NatIonaI Center for Health Statistics under

~nmG.Sirk.n,Ph.D., wasthepfincip.1investigatorforthisgrant.

nt SES 84-03415 from the National Science Foundation (NSF).

Murray Abom, Ph.D., was the NSF program director.

This research was performed under DHHS contract 282-84-2123

from NCHS to National Opinion Research Center (NORC). Roger

Tourangeau, Ph.D. was the principal investigator. NORC subcontracted

art of the work to Bolt, Berane~ and Newman, Inc. William

~aker,Ph.D., was coinvestigator. JudithT. Leader, Ph.D., was the

NCHS reject officer. The final vexsion of this report was written by

Judith?. Lessler, Ph.D., under~fessional services contract

87A04641720. Jared B, Jobe, P .D., was the project officer for that

contract and is the technical editor and reviewer for Series 6.
Foreword

In 1986, the National Center for Health Statistics

(NCHS) established the National Laborato~ for CoUaborative Research in Cognition and Survey Measurement with

research grant support from the Natiomd Science Foundation (NSF). The National Laboratory’s mission is to promote and advance interdisciplinary research on the

cognitive aspects of designing questionnaires. This research

is conducted in collaboration with the Nation’s universities

and other Federal agencies. By improving questionnaire

designs and the methods of designing questionnaires, the

National Laboratory hopes to improve the quality of survey

statistics produced by NCHS and other Federal agencies.

The National Laboratory’s mission is carried out by its

Questionnaire Design Research Laboratory (QDRL) and

Collaborative Research Program (CRP). QDRL serves

primarily as a workplace in which NCHS and other Federal

agencies conduct cognitive interviews in developing, designing, and testing questionnaires prior to their use in national

surveys. CRP, on the other hand, conducts a contract

research program with scientists who investigate the cognitive aspects of designing questionnaires on specified health

topics, Although these investigations are conducted in the

contractors’ own laboratories, they are designed to yield

results applicable to NCHS surveys.

This report, “Questionnaire Design in the Cognitive

Research Laboratory, “ is the first to appear in Vital amd

Health StatMcs, Series 6. This series is dedicated to reports

on cognition and survey measurement that emanate from

the National Laboratory. The final technical reports of the

contractors participating in the National Laboratory’s contract research program as well as reports of intramural

projects will be published in Series 6.

This report is of a project that investigated the cognitive issues in asking questions about dental health in the

National Health Interview Survey. As important as its

substantive findings are, it is the study’s innovative methods

of conducting cognitive interviews that are most distinctive.

It was the first study to demonstrate the utility of conducting cognitive interviews in a laborato~ setting as a method

for developing and testing designs of survey questionnaires.

That successful demonstration was an important step on

the road that ultimately led to the establishment of the

National Laboratory at NCHS and the questionnaire design

laboratories at the Bureau of Labor Statistics and the

Bureau of the Census.

I was the principal investigator for the larger project

Laboratory Based Research on the Cognitive Aspects of

Survey Methodology, which was supported by a NSF grant

out of which this study was funded. During the period that

she was a Service Fellow at NCHS, Judith T. Lessler

provided technical oversight to the National Opinion Research Center (NORC), the contractor, and Bolt, Beranek,

and Newman (BBN), the subcontractor, for this study.

Roger Tourangeau was the principal investigator at NORC

and William Salter at BBN. Jared B. Jobe was the NCHS

project officer for a professional services contract with

Judith T. Lessler to prepare the final version of this report.

To all these talented people, my personal thanks for making

this a most successful project.

Monroe G. Sirken

Associate Director for

Research and Methodology

.,.

Ill
Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Part A: Exploratory studies.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .? . . . . . . . . . . .

Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Part B: Questionnaire development through laborato~ experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Part C: Field andlaborato~ testing ofquestionnaires . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Methodological results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Summary conclusions andsu~ested protocol for laboratory and field testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Exploratory studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Developmental methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Testing methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Appendixes

I. Pretest of the dental health supplements to the 1986 National Health Intemiew Survey . . . . . . . . . . . . . . . . . . . . .

II. Field pretest ofthe dental health supplements to the 1986 National Health Intemiew Survey . . . . . . . . . . . . . . . .

List oftext figures

1. Model of the question-answering process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2, Answering questions on number of events in a reference period . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3. Strategies for answering questions with unfamiliar terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

4. Experimental design for questioning strategies designed to influence response to questions on mouth rinse and

dietary fluoride . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5. Experimental design for questioning strategies designed to influence response to questionson purpose ofwater

fluoridation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

6. Experimental design for questioning strategies designed to influence response to questionson dental sealants and

own water fluoridation.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

7. Experimental design for questioning strategies designedto influence recall of number ofdental visits . . . . . . . . .

List of text tables

A. Summary oflaborato~ and field-testing activities for the design and testingofthc 1986 dental supplement tothe

National Health Interview Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

B. Summary of recall and estimation strategies used by respondents and methodsto facilitate their use duringan

interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

,..

111

1

4

5

6

10

12

12

14

16

17

17

21

25

28

29

29

29

30

30

31

33

50

4

7

8

12

13

13

13

3

6

v
c. Problems with the initial draft of the dental health supplement detected in the laboratory and the Greensboro

field test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

D. Results of the Part B experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

E. Wording ofexperimental and standard versions of the dental supplement and summary ofdifferences . . . . . . . . 19

F. Summary of Portland fieldtest and Chicago laboratory test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

G. Summary ofresults from the Portland field test for experimental and standard versions of the dental

supplement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

H. Summary of results from the Chicago laboratory test for experimental and standard versions of the dental

supplement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...4. . . ..OO- . 23

J. Summary of major results from Portland field test and Chicago laboratory test . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

vi
I

Questionnaire Design in

the Cognitive Research

Laboratory

by Judith Lessler, Ph. D.,

Research Triangle Institute;

Roger Tourangeau, Ph.D., National Opinion

Research Center and William Salter, Ph.D.,

Bolt, Beranek, and Newman, Inc.

Introduction

For years, survey researchers have developed questionnaires and conducted questionnaire design research. However, the improvements in questionnaire design have been

less impressive than those in other phases of the survey

measurement process, such as sampling and data processing. The reason may be that other survey activities are more

scientifically based than questionnaire design, which remains essentially an art. Is it possible that the cognitive

sciences could provide the underpinnings for questionnaire

design research, comparable to the support provided by the

statistical and computing sciences in the development of

modern sampling and data processing methods?

Both survey researchers and cognitive scientists are

concerned with the manner in which individuals handle

information. However, they have used different approaches

and methods. Survey researchers wish to both measure and

control the errors associated with the survey response task

(Bradburn, 1983). They conduct pretests and quality-check

studies to evaluate the effects on response error of question

wording, response categories, ordering of questions, and

other aspects of the questionnaire (Schuman & Presser,

1981). Survey researchers, especially those in Federal statistical agencies, make relatively little use of controlled

laboratory experiments to investigate questionnaires.

Cognitive scientists, on the other hand, are concerned

with the mental systems people use in processing information. They study thinking, memory, understanding, judgment, and so on. They typically conduct controlled

laboratory experiments, striving to eliminate all factors

except those relating to the cognitive issues being investigated (Neisser, 1982). The possibility of using cognitive

science for designing questionnaires is being explored by

the National Center for Health Statistics (NCHS) through a

comprehensive program for investigating the cognitive aspects of survey methodology (CASM).

NCHS initiated its CASM program following participation by NCHS staff in the Advanced Seminar on the

Cognitive Aspects of Survey Methodology, which was organized by the Committee on National Statistics. The seminar

brought together a group of cognitive scientists and survey

methodologists to discuss potential linkages between the

disciplines of survey research and cognitive psychology

(Jabine, Straf, Tanur, & Tourangeau, 1984). A number of

innovative ideas for collaboration between survey researchers and cognitive scientists were developed at that conference. The project described in this report is one of a

number of such collaborative projects that have been conducted by NCHS under funding from the National Science

Foundation (Interagency Agreement No. SES-840 3415).

Staff from the National Opinion Research Center (NORC)

and Bolt, Beranek, and Newman, Inc. (BBN) participated

in this project.

The project constituted one large experiment in which

we explored the use of laboratory methods and the cognitive sciences for the design and testing of questiomaires. In

the past, laboratory studies have played a limited role in the

design and testing of questionnaires. In part, this is due to

an underlying theory of survey error that states that the

measurements in a survey are a random variable induced

by the general conditions of the survey (Hansen et al.,

1961). The general conditions include the interviewers, the

survey instruments and procedures, the respondents, the

interaction betsveen the respondents and interviewers, and

the general economic and social milieu at the time that the

survey is conducted. This assumption that the general

conditions induce the measurements to behave as random

variables implies that one should study the properties of the

questionnaire under conditions that, as far as possible,

simulate general conditions. However, conducting all studies as field studies entails great expense and complexity

(Lessler & Kulka, 1983).

Sudman and Bradburn (1974) identified three general

types of factors that affect the quality of survey measurements—factors associated with the response task, those

associated with interviewer characteristics and behaviors,

and those associated with characteristics of the respondents. After a review of a large number of studies, they

concluded that factors associated with the response task

have the greatest impact on the quality of the survey results.

Thus, it seems reasonable to hypothesize that laboratory

studies focused on the response task can be used to study

and improve survey measurement instruments, that is,

questionnaires.

In this project, we examined the use of an innovative

method for the design and testing of questionnaires. This

method was a departure from the usual field-testing procedures that were used by NCHS for questionnaire development. Instead, we examined the use of the experimental

1
methods of cognitive science for questionnaire development. The survey questions developed using this innovative

method were compared with those developed by the exclusive use of field testing.

The purpose of the project was to test the methodology

of questionnaire development, not to focus on a single

questionnaire. Thus, it had a generative component directed at integrating cognitive science theories and techniques with the traditional methods of questionnaire

development so that future questionnaire development efforts at NCHS could be more effective. The methods that

we developed will continue to evolve, reflecting accumulating knowledge and experience about the ways in which

those methods can be meshed with the needs of NCHS and

its institutional setting.

Using the planned 1986 dental health supplement to

the National Health Interview Survey (NHIS) as the subject, we investigated three questions:

. What role can Iaboratoxy studies play in the design and

testing of questionnaires?

. How might the methods and concepts of the cognitive

sciences contribute to the design and testing of questionnaires?

. How did the then-current development and pretesting

procedures used by NCHS compare-in terms of cost,

timing, and knowledge gained-with potential alternative procedures that might employ a combination of

laboratory testing, application of cognitive science techniques, and field testing?

NCHS selected NHIS as the subject survey from

among some 20 surveys and data systems it maintains.

NHIS was chosen for two reasons: There has been a long

history of research on various aspects of the survey, and it

was the focus of the CASM seminar. Choosing NHIS

allowed us to take advantage of the knowledge gained

through this history.

NHIS is the main source of information on the health

of the civilian noninstitutionalized U.S. population. In the

survey, conducted annually, data are collected from a national sample of approximately 50,000 households. The

U.S. Bureau of the Census serves as the field agent for

NCHS and collects data using household interviews. An

adult household member may report for himself or herself,

for children, and for other related members of the household who are either infirm or absent during the interview.

The aim of the survey is to provide national data on the

incidence of acute illness, the prevalence of chronic conditions and impairments, the extent of disability, the use of

health care services, and other health-related topics,

The questionnaire has two parts: A core set of health,

socioeconomic, and demographic items, and one or more

sets of supplementary health items, The core items are

repeated each year. The supplementary items change yearly

and are designed to respond to changing needs for dat~

thus, they cover a large variety of topics.

Core items include information on:

Demographic factors—age, sex, race, education, and

family income

Disability days during the 2-week period preceding the

interview

Physician visits during the 2-week period

Health condition responsible for disability days andlor

doctor visits

Long-term activity limitations associated with chronic

conditions and impairment health condition responsible for the disability

Number of hospitalizations during the year preceding

the interview, reason for hospitalization

Interval since last doctor visit

The supplements are desimwd to meet the data needs

of researc&s from universi;es, health care and policy ‘

organizations (both private and public), and specialists

within the Department of Health and Human Services,

such as the National Institutes of Health and various

organizations within the Centers for Disease Control. New

questionnaires are designed each year to meet these special

needs. Supplement topics have included immunizations,

home health care, health insurance, alcohol consumption,

dental visits, health maintenance behavior, and so on.

The design and testing of supplement questionnaires

extend over a 2-year period. At the time that the current

study began, the testing process typically involved two field

pretests, one conducted in the late winter or early spring

and one conducted in the summer of the year before the

supplement was to be fielded. The first and second pretests

typically involved 300 and 200 households, respectively, in

two different sites. The U.S. Bureau of the Census conducted the pretest under conditions that simulated the

interviewing conditions in the survey.

The agency sponsoring an NHIS supplement is heavily

involved in the development process, which is complex and

time consuming. In the past, schedules for the development

activities have not always permitted coding and analysis

of the data collected in the pretests. The pretests were

evaluated through observation of the interviews and

through debriefing of the interviewing staff.

The project described here was a three-phase effort.

All three phases were concentrated on the draft dental

health supplement for the 1986 NHIS, which is used to

obtain respondent reports on visits to dentists and other

dental care practices. (See appendix I.) Part A of the study

involved a variety of exploratory techniques (such as protocol analysis, in which respondents thought aloud as they

answered questions and the resulting verbalizations were

transcribed and analyzed) to identify potential problems in

the draft of the dental supplement and to test preliminary

solutions. Part B involved more formal methods to test

approaches for improving comprehension of items and to

stimulate more accurate recall of dental visits. A complex

factorial experiment was conducted to compare different

2
versions of the dental supplement; item wordings and item split-ballot experiments were conducted, one in a field test

introductions were systematically varied in the different held in Portland, Maine, and the other in a laboratory

versions, The Part B results led to the development of an experiment, conducted at NORC’S offices in Chicago. The

experimental version of the supplement, In the final phase, laboratory experiment included the collection of data from

Part C, this experimental version of the dental question- dental records to check the accuracy of interview reports

naire was compared with aversion that had been developed regarding the number of dental visits. Table A summarizes

using the standard NHIS field-testing procedures. llvo both the laboratory and field-testing activities.

Table A. Summary of laboratory and field-testing activities for the design and testing of the 1986 dental supplement to the National

Health Interview Survey

Time period Laboratory activity Flald activity

1984 None. Development of draft questionnaire and OMB

clearance package for first field pretest.

Winter 1985 Part A

Preliminary Investigation of recall and response Formatting and printing of questionnaire drafts.

issues in the current draft of the 1986

supplement, with major focus on dental Development of Intew!ewer instruction manuals.

questionnaire. Selection of first pretest site.

Investigation of respondent estimation strategies

and inferences from lack of knowledge.

Part B:

Design of experimental studies for Part B.

Submission of OMB clearance package for

Patt B.

Fall 1985

Winter 1986

Part B

Laboratory testing of alternative questioning

strategies and refinements of questionnaires.

Part c

Development of a sampling plan and alternative

questionnaires for full-scale laboratory and field

testing in Part C. Development of,OMB clearance

package for laboratory component of second

field pretest.

Comparison of laboratory results with those

obtafned in the first pretest.

Parl C

Portland test. Observation of interviewing

process by CASM staff.

Debriefing of field interviewers.

Coding and keying of questionnaire data

Prelimlnery tabulation of field data.

Pafr c

Recruitment of laboratory respondents.

Laboratory Interviews. Collection of dental vlslf

data

Comparison of laboratory results.with field

results. Assessment of knowledge gained by

testing mode.

Greensboro pretest of questionnaire. Debrisflng

of interviewers and decisions as to quality of

results.

Review of laboratory resuits and comparison wth

first field pretest resuits.

Development of second pretest questionnaire

and OMB clearance package. Development of

interviewer Instructions and selection of second

pretest site.

Potllsnd field test. Debriefing interview with field

staff.

QuelitatFfeanalysis of field results. Revlslon of

questionnaires.

Prhting of final NHiS 1986 questionnaires.

None.

NOTG OMB IsOfficeof Management and Budget. CASM is cognitive aspscts of survey methodology.

3
Part A: Exploratory

studies

The initial phase of the study was exploratory. In the

Part A studies, we sought to identify plausible hypotheses

for further investigation along with useful methods for

investigating these hypotheses. We focused on the questionanswering process. Oksenberg and Cannell (1977) presented a model of the question-answering process, which is

summarized in figure 1. According to their model, the

question-answering process consists ofi

2. A cognitive processing step in which the respondent

makes some decisions as to the information needed in

order to give an accurate response, attempts to recall

information, and then formulates a response based on

the recall of information.

3. An evaluation step in which the respondent judges the

accuracy of the response. At this point the respondent

may undertake additional thought-again assessing the

1. Comprehension of the question. In this step the retype of information needed, attempting to recall more

spondent tries to understand what is being asked.

information, and reformulating the response.

6

* + 7

+ Choice or modification or respons; ksed on

cues from— Giving of inaccurate rasponse characterized by—

*

e. Interviewer (status, appearance, s. Conformity biaa

behavio~ b. Desirability bias

b. Tha quektion and preceding questions c. Acquiescence hiss

c. Respondent’s own beliefs, vslues, d. Incompleteness

attitudes, and goala e. Other types of inadequacies

. .

.-

1’

“2 &nitive processin9 .

f

1

—. 3 4

5

a. Assessments and decisions

Comprehension of, -

concerning the information + Evaluati& of the - e Evaluation o~ the +

Gi~ng of a raaponse

the quastion needad for sn accurate response in terms response in tarrns. judged to be

answer of its accuracy , of other goals sccurste by

b. Ratrieval of cognition respondent

(attitudae, beliefs,

experiences, facts) -

c. Organization of the retrieved 4

cognition and formulation of

the response on this baais

SOURCE Canndl, C, F., Miller, P. V,. & Oksenberg, L Research on interviewing techniques. In S. Lsinhardb (Ed), Socio/ogica/ Afet/redo/ogy 1981. San .

Francisco: Jossay-aaaa, 1981. Copy@hc reproduced by permission.

. . .

Figure 1. Model of the question-answering process
4.

5.

A second evaluation step in which the respondent may

evaluate the response in terms of other goals. For

example, the respondent may hesitate to report certain

events or attitudes because he or she perceives them to

be socially unacceptable.

As a result of the second evaluation step, the respondent may decide to give a response that he or she

judges to be accurate or may modi~ the response

based on other considerations. These other considerations may include cues from the interviewer as to an

acceptable response, the respondent’s values and beliefs, and cues from other questions. The respondent

may or may not be aware of how these other cues

produced a modification in the response.

The Part A research was focused on the cognitive

processing phase of the question-answering process. Initial

drafts of several supplements to the 1986 NHIS had already

been developed when the study began. We carefully examined each of the supplements and the core interview to

identi~ potential topics for study. We decided to limit the

study to the dental health questionnaire and to focus on two

issues:

. Strategies for recalling or estimating the number of

events in a reference period

● Strategies for answering questions containing unfamiliar terms

These issues had three important advantages as topics

for the initial investigation of the usefulness of the

cognitiveflaboratory approach. First, there was a considerable body of extant research in the cognitive sciences on

both topics. This meant that there were precedents that

could serve as models for our work.

Second, the two issues are important to most surveys

that focus on objective events. Many questions require that

respondents report on the number of events in a specific

retrospective reference period. Examples from NHIS include requests for information on the number of visits to

dentists, to doctors, and to hospitals and number of illness

days. A considerable proportion of the questions in NHIS

contain terms that may be unfamiliar to some respondents.

.

For example, respondents are asked whether any member

of the family is covered by CHAMPVA (Civilian Health

and Medical Program of the Veterans’ Administration)

insurance, whether anyone has diverticulitis, thrombophlebitis, and so on. In many cases, we would not expect the

respondent to know what these things are unless the answer

is “yes,” It is assumed that the respondent will reason that,

if he does not know what the thing is, then he does not have

it, and therefore the answer is “no.”

Finally, these two issues were relevant to almost every

question in the dental health supplement, allowing us to

achieve the applied goal of examining a questionnaire while

still dealing with issues of general theoretical and methodological importance.

In summary, the specific objectives of Part A were

● To gain detailed knowledge of the response process

and use this knowledge to develop alternative versions

of the questionnaire that had potential for facilitating

response

● To evaluate the use of the laboratory setting and the

techniques of cognitive science for discovering problems and suggesting potential solutions

. To compare the cognitive laboratory fiidings with findings from a field test of the same questionnaire

Design

During Part A, a series of laboratory investigations and

a field pretest of the dental health questionnaire were

conducted. The laboratory studies were focused on only the

dental health questionnaire; in the field pretest, the dental

health questionnaire and two other supplements—one on

health insurance and one on vitamin and mineral use—

were tested. The dental health questionnaire is shown in

appendix L

Because one of the goals of the study was to develop

the laboratory approach, a variety of laboratory methods

were examined during Part A. The response process and

problems with the questionnaire were examined by using

detailed comprehension probes and by using a standard

technique drawn from the cognitive sciences called protocol

analysis, in which respondents are encouraged to think

aloud as they solve a problem or, in this case, answer a

question. (The terms protocol analysis and think-dud

protocols are used interchangeably in this report.) For the

laboratory testing, we used several different versions of the

questionnaire. These versions were designed to reveal the

components of the response process and were not intended

to be those that would ultimately be used in a national

survey. Parallel lines of research were carried out in

NORC, BBN, and NCHS.

NORC used an intensive laboratory pretesting approach, concentrating on the issue of how people respond

to items containing unfamiliar terms. A total of 32 interviews were conducted; items were tested using different

questionnaires, often with several followup questions. In

the followup questions, respondents were asked about the

strategies they used to answer the question or about their

understanding of a particular question. For example, the

draft dental supplement contained a number of items on

the use of fluoride products. One version of the supplement

used at NORC included a series of probes on whether

respondents had ever heard of fluoridation, which were

followed by a series of questions about specific products

that might contain fluoride. The purpose of these items was

to assess the familiarity of a number of terms (such as

fluoride mouth rinses and fluoride tablets) that appeared in

the draft supplement. Another version of the supplement

did not contain these items but did contain a question on

how respondents knew whether their own water was or was

not fluoridated. This followup probe, it was hoped, would

help to identify the process by which respondents answered

the question about public water fluoridation.

At BBN, the research focused on the items in which

respondents were asked about the number of times they

had visited a dentist during the 2-week and 12-month

5
periods preceding the interview. Protocol analysis was

used. In parallel items, respondents were also asked about

doctor visits and visits to automobile mechanics. The aim in

using several parallel items was both practical and theoretical. The practical point was to increase the number of

observations per respondent. The theoretical point was to

test the generality of the estimation and recall strategies we

observed. Although we did not expect to see major differences between recalling dental visits and recalling other

sorts of similar events, we thought it important to check this

possibility. A total of 12 respondents were interviewed at

BBN.

In the NCHS study, a mixed approach was employed.

The entire dental questionnaire was tested, collecting thinkaloud protocols for selected items and using a range of

followup probes for others. A total of 26 interviews were

conducted, the bulk with NCHS staff in Hyattsville, Md.

The details of the design of the Part A laboratory studies

are in the Part A report (Lessler, Mitzel, Salter, &

Tourangeau, 1985).

The field pretest was conducted in Greensboro, N.C.,

during March 1985. Experienced NHIS interviewers were

trained in the use of the three supplements and conducted

some 380 household interviews. Each interviewer was accompanied by an observer, who took notes on the problems

experienced by the respondents. Some of the staff who had

participated in the laboratory experiments observed the

NHIS pretest. The pretest findings were summarized in a

series of debriefing sessions and debriefing reports. The

pretest was designed to simulate the general conditions of

NHIS. Therefore, specific probes on comprehension were

not used during the interview. However, at the end of the

interview both the observer and the interviewer were encouraged to query the respondents about difficulties that

they had with the questionnaire.

Results

We obtained several types of findings from the Part A

studies. We gained insight into the question-answering

process, identified problems with the questionnaire,

learned much about the operation of laboratory studies,

and began to develop an understanding of the relative

effectiveness of the laboratory and field-testing approach

for exploring problems with the questionnaire. In this

section, we discuss each type of finding.

Substantive results

Recall strategi”es-It was apparent that respondents

used a range of recall and estimation strategies for dealing

with the items concerning the number of dental visits.

Table B, reproduced from Lessler et al. (1985), summarizes four of the major strategies observed in the Part A

interviews. It is worth noting that the anchoringand-adjustment strategy (first identified by Tversky &

Kahneman, 1974, in a different context) may be particularly well suited for dental visits. Because most people know

they are supposed to go to a dentist twice a year for a

checkup, “two visits” provides a natural starting point (or

anchor) for their answers. Another strategy used by some

respondents was to decompose the questions about dental

visits into simpler or more specific questions. For example,

respondents might first recall visits to the orthodontist

and then recall visits to other dentists. A third

Table B. Summary of recall and estimation strategies used by respondents and methods to facilitate their use during an Interview

Sfrafegy How tO facllflafe

Anchoring and adjustment

Initial recall or responee followed by reasonableness assessment, further recall, Ask res~ndents to evatuate their confidence In their answers. Suggest streteand adjustment. gkre of parformlng reasonableness assessment. Invoke a deccrmpoelfbn strefegy to stimulate further recall.

Decomposltbn

Additfvsx Recall of visits for speclflc reasons, problems, parsons, lime of year, Use checklist of treatments, types of Welts, and fypae of provkfers to stimulate

etc., followed by summalbn. Well of additional categories, Suggest dacomposlng the year Into smaller

urrltsto ease reoatl.

Multiplicative Recall of rate of Weltsand multlplicatiorr by duration of reference Probe to sea if SPSCMCrecall of v!slts can verHy that the calculated number k

period. Recall of Interval end divisbn Into duratbn of reference parbd. correct.This Is a way to Invoke an anchortng-and-adjustment strZdegy. Asslat In

divlsbn or mulllplicatbn.

Temporal skeleton

Building in the mind a conception of the reference parbd with markers at the Suggest personal landmarks to the respondent that m~ht assist In anchorlrrg

ends. Recalling events and landmarks associated with them in order to assess elfher the reference perbd or the faoalkd events. Use a temporal skeleton In a

overlap with reference parbd. decomposiiiorr strategy (Le., ParMbn recall perbd Into smaller unlfs).

Context

Cues surrounding the even!s used by the respondent to stimulate recall. Cues Suggest spactilc cues (e.g., ask who usually takes the children to the dentist).

Include getting there, bekrg there, problems wtth teeth, treatments recefved, Read a checklist of conceptual Items. Albw respondents to provkfe a narrathfe

paying the dentist, thoughts and feelings at.mul the visits, es well as eplsode-fo- description of their own or the family’s dental Weltsto facilitate person-to-pmon

episode and person-to-person cuing. end epkoda-to-apiscde cuing. Encourage the respondent to use personal recall aids such as checkbooks and calendam during reporting.

SCIURCELessleret al., 1sS5

6
strate~ involved building a temporal skeleton for the 1-year

reference period involving landmark events, such as the

beginning of the school year or holidays. Sometimes use of

a temporal skeleton triggered specific memories, and sometimes it served as a basis for dating events recalled using

other strategies or cues. A final stratepy was to organize the

search for events by retrieving contexts for the events. Some

respondents, for example, remembered specific visits by

thinking about where they go to the dentist and how they

get there.

Before respondents engaged in a detailed recall of

events, they made an assessment of whether there was

anything to recall. In essence, they first asked themselves a

question similar to, “Did I ever go to a dentist in the

approximate timeframe?” If the answer to this quick assessment was “noJ’ the recall process ended. This is illustrated

in figure 2.

There appeared to be two general strategies for making

the initial assessment—respondents could recall a fact or a

specific visit. They then judged whether this recall information implied that there were visits in the reference period.

For example, one respondent commented that he followed

the “twice yearly rule:’ meaning that he attempted to

adhere to the recommended two dental checkups per year.

Others appeared to recall a specific visit to decide that they

had no events in the reference period. This is typified by a

respondent who, when answering the first question on the

number of visits in the previous 2 weeks, commented that

the last time he went was just before Thanksgiving. This

was several months prior to the 2-week reference period, so

he quickly concluded that he had no visits in that period.

In subsequent phases of the research, we attempted to

use this information to influence the response to questions

on the number of dental visits. Because the respondent

stops trying to recall events if the answer to the quick

assessment is “nofl a false negative to this first stage of the

response process is a serious problem. Thus, we speculated

that we could improve the cognition phase of the questionanswer process by preventing false negatives in the initial

step and by providing cues to assist in the recall. One

consistent finding of previous research in cognitive science

and survey methods (Tourangeau, 1984) is that a recall is

improved it

. Multiple cues for recall are given

. The respondent is allowed more time to recall events

● The respondent puts more effort into the recall process

In subsequent sections of this report, we explain how

we tried to improve recall and our success in doing so.

Unfamiliar terms-Perhaps our clearest conclusion

from Part A concerning items with unfamilim terms is that

respondents answer these questions even when they do not

understand the terms, and they do so without asking for

clarification of the terms. Other studies (Schuman &

Presser, 1981) have shown that respondents are reluctant to

give “don’t know” responses to attitude questions about

very obscure issues. The Part A studies revealed a similar

reluctance for items about behavior. Figure 3 summarizes

u

Ever went in

approximate

timeframe?

r-iThink

mm

I

“visit II fact I

--l-------T--

1 1

Yes

Recall and Negative

dating of response

visits continues formulated

I

I I

I

bThink

Figure 2. Answering cnrastions on number of rsvents in a refarance

period

the strategies that respondents used when trying to understand and interpret unfamiliar terms.

In three items on the initial questionnaire (appendix I),

respondents were asked about use of dental care products

with which they might be unfamiliar-fluoride mouth rinse,

dietary fluoride, and dental sealants. Some respondents did

employ the strategy assumed in construction of the questionnaire: They reasoned that because they did not know

what the item was, no one had used it. This is illustrated by

the far right branch in figure 3.

Some respondents incorrectly interpreted the terms.

For example, some interpreted dental sealants to mean

fillings; others confused fluoride mouth rinses with ordinary

mouthwashes. These incorrect interpretations generally resulted in incorrect answers.

7
Comprehension

and

interpretation

4

go :“ - 4

Correct Incorrect Partial Not known

m

.->

%

Similar

:

Superordinate

category

~

Yes category s

-1 “’”

I

~

l“i1

Yes

Irtformative

Yas Recall

Recall No Judgment

O

No

‘t: f

:

.-

No

z

E

8

1“o

Yea gc

2

informative

No Judgmant

&.D

Don’t know

ar

.->

,% Donut know

E

z

.3

s

+

El Base rate

Figure 3. Strategloa for answering questions with unfamiliar terms

In some cases the respondent gave a partial interpretation of the term, most often considering some superordinate category. Often this was sufficient for the respondent

to answer the question, Respondents who did not use any

mouth rinses could safely answer the question even if they

did not distinguish fluoride mouth rinses from mouthwashes. Respondents who used no dietary supplements did

not need to know what dietary fluoride was. If this partial

interpretation strategy was not sufficient basis for an answer, the respondent had to further consider the response.

This is illustrated by the respondent who commented that

he used mouthwash but did not know whether it contained

fluoridq those who took vitamins but were uncertain

whether they contained fluorid~ and those who had had

something placed on their teeth but were uncertain as to

whether they were dental sealants.

When respondents were forced to rely on a judgment

process rather than recall of specific events or facts their

judgments took the form of inferences that were apparently

based on the relative frequeney, or base rate, of the behavior in question as well as the diagnostic value of the absence

of specific information. Such base-rate inferences were

made by respondents who reported that their water was

fluoridated and gave as the rationale for their answer the

belief that “most places” have fluoridated water. Other

respondents appeared to reason that they would know if

8
fluoride had been added to the water. Because they did not

specifically know, they assumed that their water was not

fluoridated. Such inferences based on lack of knowledge

have been observed in other domains by Genter and

Collins (1981).

The results suggested several strategies for improving

the response to questions with unfamiliar terms. We speculated that we would achieve improved response if the

terms were defined in the question or if base-rate information were supplied. In addition, we noted that several of the

questions that contained unfamiliar terms concerned multiple behaviors. For example, in the question on fluoride

mouth rinse, the respondent was asked to simultaneously

consider home, school, and work use of fluoride mouth

rinses. This complicated the formulation of the answer, and

in subsequent phases of the study we decomposed the

complex question into several simpler questions.

Problems with the questionnaire

We were interested in whether the problems that were

identified in the field test would be identified in the laboratory studies. Table C summarizes the problems detected by

the two sources of testing.

The laboratory studies and the field studies were successful in identifying problems with the initial draft of the

dental health supplement. In both sites of testing, comprehension problems were identified; however, the laboratory

study was more efficient in detecting the problems (assuming, of course, that all problems detected in the laboratory would also be detected in the field). Fewer respondents

were used in the laboratory (62 versus 380). In fact, the

problems with the question on reasons for visit, which was

the question that had the most severe problems, were

identified after the first six laboratory interviews. Travel

costs were less, and staff working in the laboratory had the

opportunity to quickly modify the questiomaires in order

to explore in more detail the ideas that came up in the

initial interviews. In contrast, because of the long planning

time, the need to train a fairly large interviewing staff, and

the necessary process of obtaining clearance from the

Office of Management and Budget, staff working on the

field test did not have the flexibility to use a variety of

techniques.

The greater efficiency of the laboratory is directly

attributable to the techniques that were used. Many of the

comprehension problems that were detected would probably have remained undetected with the same number of

respondents in afield test. Respondents in the pretest often

answered the questions confidently without noticeable delay and did not reveal their underlying confusions. The

confusions became apparent only when the respondents

were probed about their understanding of items or asked

specfic followup questions designed to assess their comprehension of items. In the field, the observers and interviewers did not directly attempt to assess understanding

during the interview and had to depend either on the

respondent’s spontaneously indicating some problem or on

inconsistencies in answers. Because respondents may not

themselves be aware that they have misinterpreted a question and because they are apparently reluctant to volunteer

lack of knowledge (which was true in the laboratory as well

as in the field), the frequency with which problems are

detected will be much lower in the field setting.

Another technique that contributed to the greater efficiency of the laboratory was the manner in which the

observations were recorded. In the field, observers took

notes on apparent problems; the interviewer continued with

the questioning. Thus, the observer, who was writing an

observation of the response to a previous question, could

miss observing problems with succeeding questions. In the

laboratory, video and audio recordings were used to make

the observation of the response process. Thus, an on-site

observer was not needed, and deliberate detailed observation of the interview could be made by reviewing the

recorded session.

The use of a greater variety of techniques in the

laboratory also provided somewhat deeper insight into the

causes and extent of problems. All of the research techniques that were used in the exploratory studies-response

protocols, comprehension probes, and the provision of

alternative items designed to vary in speciilc ways (e.g.,

questions that varied the amount of information given to

the respondents) -provided converging evidence for the

conclusions. In addition, the response protocols allowed us

to identi& not only problems but also response strategies

that were not revealed in the field testing.

The field test did reveal several problems that were not

evident in the laboratory. Interviewer reactions to the

questions were not noted in the laboratory, and the disruptive use of flash cards was not detected. Field interviewers

also had excellent insights into the language problems

experienced by respondents and generated hypotheses for

improved wording. An NHIS interviewer was the source of

the suggestion that the term “fluoridation” be replaced by

the phrase “adding fluoride to the water; a change that

was subsequently proven effective.

Based on this comparison, the laboratory should be the

main setting for discovering problems with questionnaires

and field tests should be reserved for testing more refined

instruments.

Laboratory methods

The findings horn Part A suggested a number of broad

conclusions about the methods for pretesting instruments

in the laborato~ setting. The intensive observational techniques employed in the laboratory allowed us to identilj

problems using only a few respondents. However, it was

apparent that it was important to test a range of respondents. For example, it was difficult to study items on the use

of fluoride products in Chicago because its water supply is

fluoridated and few people actually used fluoride vitamins

or fluoride mouth rinses. The few interviews that were

conducted in a nonfluoridated area produced interesting

differences from the Chicago results. In addition, it was

9
Table C. Problems with the initial draft of fhe dental health supplement deteotad In the latmratory and the Greensfxxo field test

Dental vklfa kr prbr 2 weeks

LeLWetory

Diffkulty in Iocafkrgthe boundaryof the mferenee pertodand decldlngIf a

rwent visitwash or out of the pedod.

Greensborofteldtest

No problemsnokxt.

DentalVMISin prbr 12 months

Laboreto~

Tendenoy to give approximate answers based upon wlf-pemeivad usual

behavior.

Difficultyin Ioczdkrgthe boundaryof the referencepertod.

Problemsraczdlingvisitsfor otherfamily members.

Difflcuttydaterrntningthe numberof vtslfewhen there ware Largenumbersof

visits.

Greensborofieldtest:

Approximateanswersencouragedby use of the word “about.”

Reasonsfor dental vtsifs

Laboratory

Tendency to reportthe type of tmafmentreceivedes the reasonfor visiting

the denfkt, whereas the sponsorwas Interactedin assssslngthe irnpasfof

recallsystemson the reasonsfor vkils to the denfist.

Inability to choose a single category because the cfdegorles ware not

mutuallyexclusive.

GreensborofWd test

Reportingof treatmentratherthan motivetkm.Respondentsware intendedto

reporton a serW of Weafmenfsbut dkl not.

Reportingof reasonfor firstvisitin previous12 months,which dkf not meet

the sponsor’sgoal of detarmlningwhy peoplewho had had a seriesof vistfs

for a particular~blem initiatedthfsserbs.

Use of toothpaste

Laboratory:

Tendency to thinkthat the initialpartof thtsquesrbn was siltybecause most

people use toothpaste.

DifferantInterpretationsof the time pedod for the quasfbn. These ranged

from reportingon the brand that wee used on the day of the Interviewto

brandsused mostoftenin the pastyear.

GreensborofieldWC

Toolhpasfewas used by mosteveryone.

Feelingthat the use of a responseoardwas unneoesswy.

apparent that the response task differed for self-versusproxy response. Parents who reported for their children had

a more difficult recall and reporting task than those who

reported only for themselves.

The think-aloud protocols were useful for ident@ing

response strategies as well as potential problems. Although

protocol analysis has its limits (Ericsson & Simon, 1980), if

used carefully, it can provide strong indications regarding

the process underlying response to survey questions. The

difficulties inherent in accurately recalling and placing dental visits in a reference period were manifest during the

collection of the response protocols. In some cases the

respondents spent 20-30 minutes trying to accurately recall

visits for their entire family. Only short recall periods were

observed in the field tests, indicating that there is great

potential for inaccurate reporting in the actual survey.

Numerous issues can be studied when designing and

testing a questionnaire, and often the observations have an

impressionistic quality that makes it difficult to communicate the results to others. This contributes to the conclusion

that questionnaire design is an art rather than a science.

10

Use of dbtary fluoride

tatnmrfory

Mfsundmtendhg M the term “dktery fluoride.”

Respondentsdld notask for an explanatbn of the term.

&MdUSfOfl of dlafaryflouddewithdiet drcys.

Some reqmndenfs dkf not report separafefythe home and sohoolcomponents of the quesfbn.

Greensborofieldtest

Confusbn of dietaryfluoddewithfluorkfemouthrinse,ordineryvltemlns,and

weightraduofbn products.

Use of fluorkfemouthrinse

bllmratory:

Confusbn of fluorktemouthrinsewiih ordinarymouthwash.

Greensboro field teat

Corrfusbnof ffuottdemouthrinsewithordinarymouthwash.

Dental sealants

Laboratory

ConfWsbn of dentet seetents with other products, including flllktgs and

fluoridetreatmentsthat are applbd at the dentist.

Grwnsboro fieldtest

Confusionof dental sealantswIfhother produsfs.

Lossof all lldllfd teeth

Laboratory

Corrfusbn because of awkward placementof quastbn. Some mspondenfs

thought that the quasfbrr referred to a member of their faml!y who Was

elsewhere.This was besause they had spent some time talking about the

sara of teeth, whichkn@ii that everyone in the householdhad teeth.

Greensborofbld test

No problemsnoted.

Publk waterffuorkfatbn

Laboratory:

Misunderstandingof the term “fruorkfefbn.”people had no spacifk knowtedge of whether their wafer wee fluorkfeted end based their arrswemon

Greensborofieldtest

Misunderstandlrrgof the term “fluorldetbn.”The phrase “addkrgfluorkteto

the water” wouldbe betterund~ti.

Dlsmptiveuse ofthe flesh card, whkh was not nemsswy.

The decision to organize the research around the two

general issues of recall of the number of events in a

reference period and response to unfamiliar terms proved

to be effective. It provided useful rubrics for thinking about

and organizing the findings.

Conclusions

The use of the laboratory for the design and testing of

questionnaires is an evolving process. New methods will

continue to be developed, and NCHS will continue to seek

the best balance between laboratory studies and field testing. However, based on our experience, we conclude that

the laboratory is an effective site for exploratory studies

directed at ident@ing the response process and response

problems. We suggest the following protocol.

At least three organizations participate in the development and testing of NHIS supplement questionnaires—

NCHS, the U.S. Bureau of the Census, and the sponsoring

agency or agencies that need the survey information.
Typically, the sponsoring group specifies its information

requirements and may provide initial drafts of the questionnaires. Staff of NCHS and the U.S. Bureau of the Census

then develop and test the survey instruments. Many surveys

have staff of these three general types. Once the information needs are identified the three types of staff jointly

should develop a plan for both the laboratory and field

testing. If previous questionnaires are available or drafts

have been provided by the sponsoring groups, the stall

working on the instruments can conduct an appraisal of the

forms to identify response issues and potential problems.

These issues and problems can then be classified into

broader categories that can be used to guide the research.

We found that this process contributed to a fuller understanding of the needs of the sponsoring group and also

pointed up uncertainties in the sponsor’s goals. The first

steps in many sciences are observation and classification.

An explicit focus on these steps before beginning experimentation and testing may contribute to moving questionnaire development into the realm of science.

The laboratory can be used for the initial stages of

testing. The range of laboratory respondents should reflect

the characteristics that are known to affect response. Given

such attention to variety and the use of the intensive

observational and probing techniques, the number of laboratory respondents need not be large (20-50). The laboratory testing will be most effective if it is guided by explicit

hypotheses as to the response task implied by the items,

problems that arise in the response process, and methods

for improving the response. Think-aloud interviews that are

recorded and reviewed can be used to generate such hypotheses. Unanticipated processes and problems will appear even with careful prior analysis.

Because of the wide experience of the field staff, we

conclude that the early laboratory results should be supplemented by ~eld tests on nine or fewer respondents. Our

laboratory studies did not incorporate this feature, and we

think that some of the excellent insights and experience of

the field interviewers will be useful for guiding the laboratory studies. Staff from the laboratory can observe the field

test so that the laboratory efforts can adequately reflect the

household situation.

Our studies were improved by the need to write reports

in which we described our findings to others. We found that

this caused us to reflect on the results and generated

discussions that improved subsequent efforts. The parties

involved in the development of the survey instruments

should meet frequently to summarize and assess the current results. Severe time pressures face staff when developing the survey instruments. The laboratory seems to be very

well suited to an iterative testing process with short cycles

of testing and evaluation. This type of approach is almost

impossible to implement in a full-scale field test. Thus, the

laboratory method is perhaps most efficient if it is designed

to take advantage of this cyclic potential, and the periodic

reporting and assessment can contribute to assuring the

accumulation of knowledge about the question-answer

process.

11
Part B: Questionnaire

development through

laboratory experiments

The goal of Part B was to conduct a series of laboratoq

experiments directed at identi&ing questioning strategies

that influence the question-answering process. In Part A we

observed respondents as they answered different types of

questions and noted differences among respondents and

questions. Although we had a notion as to what types of

differences we might observe, our work was not guided by

formal hypotheses as to the nature of these differences.

Based on the Part A observations, in Part B we developed

specific hypotheses concerning the effect of different types

of questions on the recall and comprehension of items in

the dental supplement. Again we had a dual focus. We were

trying to find improved ways of asking the dental health

questions, and we were examining laboratory methods that

might be used to identiij the improved questions. Whereas

in Part A we examined observational methods, in Part B we

examined experimental methods.

Part B was carried out over a 3-month period. We were

exploring our ability to quickly build on the Part A exploratory studies by using the laboratory method. We were

concerned with a number of questions. Can we design and

test different questionnaires in a short time? Will we be

able to recruit respondents, administer the experiments,

and analyze the results within the 3-month period? Will the

small sample sizes be sufficient to detect differences? Given

that sample sizes are small, are there ways in which we can

magnifj the differences we are trying to detect?

In summary, in Part B we focused on answering questions concerning methods. As our test vehicle, we included

experiments directed at (1) improving recall and estimation

of the number of dental visits, (2) increasing the comprehension of items that contain potentially unfamiliar terms,

and (3) fostering specific strategies for dealing with unfamiliar terms.

Design

Experimental design

Comprehenswn issues-The results fi-om Part A concerning items with unfamiliar terms and other comprehension issues suggested four hypotheses about techniques that

might improve answers to the dental supplement:

●

●

12

Encourage respondents who do not understand a question to give “don’t know” responses

Use simpler language or define technical terms

●

●

In

Decompose complicated items into a series of simpler

items

Provide accurate base-rate information

each case, the rationale for these approaches was

straightforward. We reasoned that people who understood

the question were more likely to answer it correctly.

Therefore, respondents might be prevented from giving

erroneous substantive answers to questions that they did

not understand if they were encouraged to give “don’t

know” responses instead. Alternatively, respondents’

comprehension might be improved by defining unfamiliar

terms or eliminating them entirely in favor of simpler

terms. Some of the experimental items contained a

number of qualifying phrases intended to clarify the exact

scope of the questions. The Part A results suggested that

these qualifications sometimes created more confusion

than they cleared up. For this reason, decomposing items

with qualifying phrases into a series of simple questions

seemed a promising approach. Finally, respondents often

seemed to rely on base-rate information in answering

questions with unfamiliar terms. These answers could

perhaps be made more accurate if respondents were given

more accurate information regarding the relevant base

rate.

Several items lent themselves to the investigation of

these hypotheses. Figure 4 summarizes the experimental

design for the fluoride mouth rinse and dietary fluoride

questions. Versions of these items varied in whether a

definition was given for these terms, decomposition was

used to simplifj the items, or the then-current draft was

followed. For the question on the purpose of water fluoridation, we investigated both the effect of lowering the

threshold for giving a “don’t know” response and the use of

Comptehensbn

Betterdefintbns DscompcsHbn Con!lvl

Number of res~nden!s

32 32 32

Hypctkeses

1. Eetterdefinitlomerxl decompositionIrrprow undemtandlngof Items,therebyreducing

~!Joflh3 of u= of fluoridemcuthrinse.

2. Bstterdefinltiocsanddecon-pcdlionImprovepmstlriervl.swcamp+wmlon teatonthaw

Items.

Figure 4. Experimental design for questioning strategies

designed to influence response to questions on mouth rinse and

dietary fluoride
defined wording. Figure 5 summarizes this experiment. We

also examined the impact of better definitions and the

provision of base-rate information in the questions on

dental sealants and whether respondents’ own water is

fluoridated. This design is summarized in figure 6.

Recall issues-In the dental supplement (appendix I),

respondents are asked to report the number of dental visits

made by each member of the family in a 2-week reference

period that immediately precedes the week of interview.

The 2-week items are followed by similar questions about

dental visits during the previous year.

The Part A results suggested a number of possible

strategies for increasing the accuracy of recall. Context cues

were often used by respondents in recalling specific dental

visits. In Part B, we sought to encourage the use of context

cues by asking a series of items regarding the context of the

visits (Is there a particular dentist’s office, dental clinic, or

some other place that usually goes for dental care?

Does anyone in the family go to an orthodontist? When

needs to go to the dentist, how does he/she usually

get there?). We thought that these context items would

encourage retrieval before respondents could make a

quick, possibly erroneous judgment that they had no visits

to report. In addition, the context items might facilitate the

process of retrieving individual visits. We varied whether

respondents received these context items in the versions of

the supplement that included them, the context items appeared immediately before the 2-week dental visit items.

The Part A results also suggested that people often

recalled dental visits by thinking of the reasons for specific

visits, To encourage the use of this cue for retrieval, we

included in some questionnaires a list of the reasons that

people might go to the dentist. The reasons list was included as part of the introduction to the second dental visit

item, which concerned the total number of visits to the

dentist during the past year.

Finally, based on Part A results, we surmised that

respondents might recall additional visits if they tried to

answer the question a second time; in addition, landmark

events were often used by respondents to date visits. This

strategy can result in a reduction in the number of visits

reported as well as an increase because the respondent gets

a better idea of when in time the visit occurred. To

encourage the use of this strategy, we added to some

questionnaires an item in which respondents were asked to

recall an important event near the beginning of the l-year

reference date and to try to answer the 12-month visit item

again using the landmark event. The landmark probe is

obviously complex, involving a second try as well as the

generation of a landmark event. In order to separate the

two effects, we also developed a second-guess probe, in

which respondents were simply asked to answer the

12-month visit question a second time without generating a

landmark event. The landmark or second-guess probe appeared immediately after the 12-month item in the versions

of the supplement in which they were used.

Altogether, then, there were three treatment variablex

context items versus no context items; reasons list versus no

Lower thresholdfor

a “Don’f know” No threshold

Comprehension response man@ulatbn

Number of respondents

Defkredwordlng . . . . . . . . . . . 24 24

Orlglrralwordlng . . . . . . . . . . . 24 24

Hy@kses

1. Mined wordirgwill prcduce more people Irxorrecfly stating the WPOSS of ~bllc water

fluoridation.

2. Lowering tbe thres.tmld for a “Dcm’t knew” PSSPOW will pmduca more people who say

they do not know the purpose of public w8!er fluoridation.

3. An Interectlon effect with threslwld rnsnlpulation will hava a graater impact on those thst

rscelve the original wording.

Figure 5. Experimental design for queatfoning strategies

designed to influence response to questions on purpose of

water fluoridation

Bass-rate manipulatbn

COIr7DBh6VfShf Hioh Low None

Number of respondents

Defined wording. . . . . . . 16 16 16

Original wordlrrg. . . . . . . 16 16 16

Hypotheses

1. Reptlmg of the IRS of dental see.lsnts will be greater for ths Hgh be..ss-rste group than for

the low ba.sa-rats group.

2. @oflln9 ~ Wn water fluOrldatiOn will ~ gre~er fOrths high bsss-rste and no bsse-rste

grurps.

3. Thrwgh srr Infersction effect, better definltiora will raduca the Impmt of threshold

rnanipdatlon.

Figure 6. Experimental design for questioning strategies

designed to influence response for questions on dental sealants

and own water fluoridation

Contextcues No contexicues

Reason No mason Reason No reason

ProLra cues cues cues cues

Number of responderrls

Landmark probe . . . . . . . . . . . 8 8 8 6

Second-guess probe . . . . . . . . 8 8 8 8

Noprobe . . . . . . . . . . . . . . . . 8 8 8 8

Hypotheses

1. Context cues and reason cuss will prevent falsa negatlvss to tha Mid promss, tfmraby

lmreaslng ths rumberof reeprderrte who repmtat /eust0n9 visit.

2. Ccatext and -n cues will Inpove recall of derrfsf visits, thereby incresslq the

number of visits reported.

3. Follcwup probes will enhsnce ttm reporting of tha number of visits. Lsrdrrark probe will

decreaseh nurrberofreportedv191!sby assisting In the test phese of the recall prwees.

%.xmdquess probe will Immase the number of repmfed visits by dirr!ulatlng additional

recall. Effect of both probes [s likely to ba greater arneng three wtm did not receive

Introductory wee.

Figure 7. Experimental design for questioning strategies

designed to influence recall of number of dental visits

reasons list; landmark probe versus second-guess probe

versus no followup probe. These variables were crossed,

giving a total of 12 versions of the dental visit items, each of

which was administered to eight respondents. Appendix I

shows the exact wording that was used. Figure 7 summarizes the experimental design and our hypotheses.

Procedure

Because the experiments concerned different items in

the dental supplement, it was possible to carry out all

13
experiments with a single group of 96 respondents. A

unique questionnaire for each respondent was printed using a microcomputer. Treatments for one experiment were

nested within treatments for other experiments and ordered

from highest to lowest and then lowest to highest. This

assured that a particular experiment was balanced across

all levels of the other experiments. Respondents were

randomly assigned to one questionnaire version. The ability

to use a microcomputer to print quickly alternative questionnaires prevented collating errors when assembling the

treatments for each respondent and assured correct random assignment of the treatments. An additional eight

respondents were administered the original version of the

dental supplement.

Personal interviews were conducted with the 104 respondents. The respondents were recruited through advertisements and handbills in which the study was described in

very general terms, including the facts that it concerned

health issues and would last about an hour. Respondents

were promised $10.00 for participation and were given a

telephone number to call. Potential respondents were

screened over the telephone to eliminate persons under 18

and students from the University of Chicago. The interviews were scheduled in a second telephone contact.

The 104 respondents were residents of Hyde Park and

neighboring areas. Although we had hoped to select

roughly equal numbers of persons with and without children and with varying levels of education, our recruitment

efforts did not yield a large enough pool of potential

respondents to balance the sample on both variables. We

did succeed in recruiting equal numbers of respondents

living with children and with no children. However, we were

unable to recruit equal numbers at each level of education.

In addition, the high-education group included fewer respondents with children than the other two groups, Most of

the respondents were black (81 respondents) and female

(65 respondents).

The respondents were interviewed at NORC’S offices

in Chicago. The interviews were conducted in person by

three experienced NORC interviewers. Before starting the

interview, the interviewer gave each respondent a consent

form that explained that the purpose of the survey was “to

develop better questions” for the National Health Interview

Survey and gave assurances regarding the confidentiality of

their answers. All of the respondents agreed to participate

and were then interviewed, The interview began with a

series of items on household composition and demographic

characteristics; this portion of the interview included eight

standard items drawn from the core NHIS questionnaire.

The dental health supplement followed these initial

questions.

After the dental supplement was completed, respondents filled out a short questionnaire designed to assess

comprehension of some of the terms in the supplement and

were asked to sign a permission form allowing us to

recontact them for a second interview. All the respondents

agreed to the reintemiew. Respondents were then paid

$10.00 for their participation. The initial interview generally

took about one-half hour to complete.

After a week to 10 days, respondents were contacted

and interviewed over the telephone. We were able to

complete the reinterview with 73 of the respondents. The

telephone reinterview included only the questions from the

dental health supplement. Each respondent received the

same version of the supplement and the same reference

dates as in the initial interview but was questioned by a

different interviewer. During the reinterview, respondents

were asked to check their answers to the dental visit items

by consulting dental bills, checkbooks, and other records; in

addition, they were asked to check the brand names of

toothpastes and mouthwashes actually present in their

homes.

Any government activity that requires collection of the

same data from nine or more people requires clearance by

the Office of Management and Budget (OMB). OMB

provides an oversight function directed at assuring that data

collection efforts are scientifically sound, that the information is actually needed, that the requests for information do

not place undue burden on the public, and that citizens

have an opportunity to comment on planned data collection

activities. The entire OMB clearance process takes some

3-5 months, including the within-agency clearance. Anticipating that we would want to administer the same questionnaire to more than eight people during Part B, we began

the OMB clearance process well before Part A began. This

created some operational difficulties. We had not anticipated that the Part A studies would be as successful as they

were in suggesting different questioning strategies and had

anticipated that in Part B we would still be searching for

insight into the question-answer process. Instead, when we

reached Part B we were ready to posit specitic hypotheses

about the impact of different questions. It would have been

helpful at that point to collect validation information from

an external source to confwm our laboratory results. We

were not able to do this because we lacked the necessary

OMB clearance.

Results

Experimental results

An alpha level of .05 was used throughout Parts B and

C. Marginal results are reported for an alpha of .15 or less.

Statistical values are presented only for significant and

marginal results. Results from the Part B experiments are

summarized in table D, which shows those experiments in

which we found differences between the experimental

groups.

Results on comprehenswn—In the first experiment on

comprehension, two items using the term “public water

fluoridation” (standard version) were contrasted with two

using the phrase “fluoride added to the water” (simpler

version). This apparently minor variation in the questions

had considerable impact on the answers. The simpler

14
Table D. Results of the Part B experiments

Dental vlstiexperiments

1. Percentof respondentsw~hat least 1 visitIn prior12 months

Self reports

Reasoncues No reasoncues

Conlextcues . .,, ,., . . . . 67 42

No contextcues . . . . . . . . . 75 66

Reportsof otherfarnllymembers

Reasoncues No reasoncues

Contextcues . . . . . . . . . . . 77 58

Nocontext cues . . . . . . . . . 70 56

2, Mean numberof dental visitsin prior 12 months

Self reports

Reasoncues No reasoncues

Contextcues . . . . . . . . . . .

Nocontexf cues .,..,....

Contextcues . . . . . . . . . . .

Noconlext cues . . . . . . . . .

2.34 o.a4

2.13 1.47

Reportsof otherfamily members

Reasoncues No reasoncues

1.95 0.75

0.93 0.81

Purposeof waterfluorkfatbn

Percentof respondentscorrecflyansweringby type of wording

Deflnad Orlglnal

word!$ig wording

Preventtoothdecay. . . . . . . 60 43

Other, . . . . . . . . . . . . . . . 23 28

Don’tknow . . . . . . . . . . . . 17 2e

Use of flourktemouthrinse

Percentof respondentswho gave correctanswers

Original Defined Decomposed

wording wording wording

Correct, . . . . . . . . . . . . . 56 69 75

Incorrect. . . . . . . . . . . . . . 44 31 25

wording yielded more correct answers regarding the purpose of fluoridation (60 percent) than the standard wording

(43 percent). The difference approached significance.

(Logit analysis was used; z = 1.79 andp < .08.) Compared

with the standard wording, the simpler wording also increased the number answering “don’t know” in response to

the question about whether their own water was fluoridated

(52 versus 21 percent) and decreased the number answering “yes” (23 versus 66 percent). Both of these related

effects are significant (“don’t know” rate: z = 2.72, p <

.03; “yes” rate: z = - 4.3, p < .001). In Chicago, the water

is in fact fluoridated, so the simpler wording reduced the

rate of correct responses. These results suggest that when

people understand the question better, they are more

willing to admit that they do not know the answer. This

increased willingness to give a “don’t know” response may

or may not affect the rate of apparently correct responses,

depending on respondents’ strategies for guessing the answer and the actual situation in their communities.

The other two comprehension studies produced one

more noteworthy finding. Results for the fluoride mouth

rinse item su~ested that either defining the term or

breaking the question into several simpler ones (e.g., Does

anyone use a mouthwash? Does this mouthwash contain

fluoride?) reduced apparent overreporting. Overreporting

could be assessed using responses to a later item in which

respondents were asked the brand name of the mouth

rinse. In both experimental versions of the question, the

distinction between fluoride mouth rinses and ordinary

mouthwashes was emphasized; most overreporting involved

confusion between the two.

In two experiments, we tried and failed to affect answers by providing base-rate information. Thus, we are not

entirely certain what processes respondents use to answer

questions when they have little specific information on

which to base their answer. TWo-thirds of the respondents

in the standard wording group correctly answered that their

drinking water was fluoridated; however, the bulk of them

did not know the purpose of fluoridation and, based on the

Part A results, we doubt that their answers reflected definite information about their own water supply either. However, their answers were unaffected by the base-rate

information we provided, and what exactly served as the

basis for these answers remains unclear to us.

Results on recall—The variable with the most consistent impact on the reported number of dental visits was the

reasons list. Respondents who were read the reasons list

prior to the 12-month visit item reported making an average of 2.1 visits during the past year. Respondents who

received a version without the reasons list reported an

average of 1.2 dental visits. Despite the small sample size,

this difference approached significance (F(l, 91) = 3.72,

p c .06). The results were similar when the proportion of

respondents reporting at least one visit was examined.

The picture was somewhat less clear for visits by other

members of the respondent’s family. Again, the reasons

variable had a marginally significant effect (F(l, 71) =

2.29, p < .14) on the average number of visits per other

family member. This main effect appeared to be qualified

by an interaction with the other two experimental variables.

In particular, the reasons list produced the highest level of

reporting when combined with the context items and either

one of the followup probes. For the interaction of both

context and reasons, F (1, 71) = 2.42,p < .13. Because, on

the average, fewer visits were reported for other family

members than for self, this result suggests that recall for

events involving other people is more difficult and a number of retrieval cues may be necessary to prompt further

recall. More generally, this result underscores the importance of examining both self and proxy respondents in

assessing the effects of an experimental variable.

Use of laboratory methods

The Part B studies clearly demonstrated that the laboratory approach lends itself to testing a large number of

variations in a short time. Although it is not impossible to

carry out large-scale field experiments, it would be extremely difficult to do so at comparable cost or with

comparable speed. In addition, the logistical dit%culties of

15
carrying out a field test with so many versions of the

questionnaire would be considerable, if not

insurmountable.

Conclusions

The Part B studies clearly demonstrated that the laboratory approach lends itself to testing a large number of

variations in a short time. A number of the posited effects

of different questioning strategies were demonstrated. The

multiple versions of the questionnaire embodied specific

hypotheses about the source of reporting error rather than

a scattershot effort to identifi the best version. Although

the laboratory method does not absolutely require the

development of hypotheses, the control that the laboratory

affords and the strong tradition of hypothesis testing within

the laboratory setting foster the development of general

hypotheses.

The laboratory approach was not without its limitations. We encountered two major problems in Part B. First,

the samples that we used were too small and too

homogeneous. Second, we observed little variation across

respondents on several key items.

The samples for laboratory studies can be larger and

more varied. Our study was the first of its kind. We did not

have the opportunity to develop a pool of potential respondents with known characteristics. A continuing laboratory

program could develop such a respondent pool by tapping

alternative sources of subjects, offering larger incentives,

and developing an ongoing recruitment effort.

&other limitation of the laboratory studies is that the

setting is very different tlom the setting of most surveys, In

the laboratory, the respondents are not distracted by children, neighbors, or the television; they have made some

commitment to the response task by showing up at the

laboratory and they can give full attention to the task. This

is an advantage because it permits the use of more detailed

observational techniques. It is also a disadvantage because

there is no guarantee that the processes observed in the

laboratory are the same as those used by respondents in the

field. In Part C, we examined the transferability of results to

the field setting.

16
Part C: Field and

laboratory testing of

questionnaires

Part C was the final phase of the project. Again we had

several goals. We compared two versions of the dental

supplement, a version that was based on the laboratory

development activities (the experimental version) and one

that was based on the field pretesting activities (the standard version). The goal of the comparison was to determine

if the questioning methods that appeared to be successful in

the laboratory would be successful in the field. The general

conditions of field surveys are quite different from those of

the laboratory. The laboratory will not be a good setting for

the design and testing of questionnaires unless it can be

used to examine processes that operate in the field. We

need to know what types of effects are observed in both

settings and, when differences across settings are observed,

the reasons for these differences. Thus, we were concerned

with whether questions that seemed to be better in the

laboratory would also seem to be better in the field.

Again, we undertook several activities directed at examining testing methods. The tsvo questionnaires that were

compared in the field were also compared in the laborato~

in order to obtain more information about factors that

might have different effects with a change in setting. Because of tight time schedules and the operational problems

associated with testing alternative versions in afield pretest,

previous NHIS development efforts had been focused

largely on a qualitative assessment of a single version of the

supplement. This assessment consisted of oral debriefings

of the interviewers immediately after the pretest and both

written and oral debriefings of the observers. The comparison of two questionnaires in a field test offered an opportunity to examine whether a more quantitative assessment

would yield different results from a qualitative assessment.

Thus, we keyed and tabulated the results from the pretest

and compared qualitative and quantitative conclusions.

The gathering of validation data in a laboratory setting

was also examined. Validation studies for health surveys

usually consist of record-check studies in which the information in medical records is compared with respondent

reports. These studies are difficult to conduct. Locating the

health care providers is difficult, as is the matching of

events reported in the survey and in the records, obtaining

permission to gain access to records, and gathering the

information in a timely manner. We examined the gathering of validation data in the laboratory setting by attempting

to gain access to the dental records of the laborato~

respondents,

Finally, we made an initial attempt to bring some of the

unique features of the field setting into the laboratory for

examination. During observation of the field tests, we noted

that there was a big difference in the pace at which the

interview was conducted in the laboratory and in the field,

Several of our treatments were designed to induce the

respondent to devote more time to the question-answer

process. We speculated that these treatments would be less

effective in a fast-paced interview. The rapid pace of the

field interviews is probably due to the social context in

which the interviews are conducted. Interviewers visit respondents’ households without an appointment and most

often encounter respondents who are in a hurry to move on

to their everyday activities. NHIS interviews are often very

long, averaging 1% hours, and respondents are often

reluctant to spend this time, putting interviewers under

pressure to complete the interview quickly. In contrast,

laboratory respondents come in prepared to devote an hour

or so to the response task. Thus, we attempted to systematically vary the pace of the laboratory interviews to begin

to get an idea whether we could simulate field conditions

and observe their impact.

In summary, we had several objectives in Part C

● To determine if the questioning strategies that appeared successful in the laboratory were also successful

in the field

● To continue to examine the impact of site of testing on

the results

● To assess how conclusions based on observation alone

would differ from those that also included tabulations

of the pretest data

● To begin to gauge whether some of the general conditions of the field setting could be studied in the

laborato~

Design

Questionnaire versions

In an ideal comparison, the two versions of the dental

supplement would have been developed in complete independence, but, in fact, there was some interaction between

the two teams developing the dental supplement. Staff of

this project served as observers during the Greensboro

pretest and so had access to the results of the pretest. The

report on the first phase of this project was disseminated

17
widely, so the team developing the standard version of the

dental supplement had access to early laboratory results.

The two versions, thus, do not differ as dramatically as they

might have. Both the standard version and the experimental

version of the dental supplement questionnaire are shown

in appendix II. The major differences between the two

versions involve items covering four areas: (1) public water

fluoridation; (2) 2-week and annual dental visits; (3) fluoride mouth rinses; and (4) dietary fluoride. These are

summarized in table E.

In addition to the differences detailed in table E, there

were several other differences between the two versions

that we mention only briefly here. Both versions of the

dental supplement contained items to determine the reason

for the last dental visit by each family member. The standard version contained a single item for this purpose; the

experimental version contained a preliminary item concerning the treatment received and a followup item to determine how the person ended up receiving the treatment. The

two versions differ so sharply that it was impossible to make

quantitative comparisons between them. The interviewers

at the Portland pretest expressed a preference for the

experimental version. Both versions also included items

about dental sealants and provided definitions of this term;

the exact wording of the items differed somewhat, but,

perhaps because of the relative rarity of this treatment, no

marked quantitative or qualitative differences between the

two were observed. Finally, both versions contained items

to elicit the brand of toothpaste that family members used.

There was only a slight difference in the phrasing of these

items. The Portland interviewers seemed to prefer the

experimental version, but no quantitative differences between the two versions were apparent.

Portland pretest

Interviewers-Twenty-four experienced interviewers

fkom the U.S. Bureau of the Census were flown to Portland, Maine, to carry out the pretest interviews. They

received 1 day of training to familiarize themselves with the

instruments to be used in the pretest. Aside from the NHIS

core questionnaire, the instruments included the two versions of the dental supplement and supplements on health

insurance coverage, vitamin and mineral intake, and occupational health issues.

Respondents– Each interviewer was assigned 16 dwelling units on a block (or set of adjoining blocks) that had

been preselected and prelisted. The blocks were selected

purposively to represent different income levels. Additional

addresses were listed on each assignment sheet to allow for

vacant units and nonresponse. The respondent for a family

was selected using the standard NHIS respondent rule: Any

adult family member could serve as the respondent for the

family other adult members @esent were also encouraged

to participate.

Although the preselected dwelling units were to have

been sent an advance letter explaining the purpose of the

interview, most of the respondents did not appear to have

received any letter and, thus, were approached “cold” by

the interviewer. A total of 385 interviews included at least

some data from the dental supplement and are analyzed

here.

Experimental procedure–In the Portland pretest, two

factors were systematically varied-the version of the dental supplement questionnaire and whether the full NHIS

core questionnaire or a shortened version (the partial core)

was administered. The shortened core included the household enumeration items, various demographic items, and

items covering limitation of activity. The assignment of a

household to an experimental group was not, strictly speaking, random. Rather, the first eight households interviewed

by each interviewer received a full core; the remainder

received the partial core. Experimental and standard versions of the dental supplement were simply alternated.

The interviews were generally conducted with an observer present. The observers included staff from this

project, from NCHS, from the U.S. Bureau of the Census,

and from other interested agencies. The qualitative results

described later, however, reflect the comments of the interviewers only.

Chicago laboratory test

The NORC experiment differed from the Portland

pretest in several respects. The most dramatic differences

were that the respondents were interviewed at NORC’S

offices in Chicago rather than in their own homes and that

they were asked if they would permit us to contact their

dentists in order to verify the accuracy of the answers.

These and other differences in the two studies are summarized in table F.

Interviewers-Two experienced U.S. Bureau of the

Census interviewers from the Chicago area agreed to join

NORC’S staff temporarily to carry out the Chicago experiment interviewing. One of the interviewers had also participated in the Portland pretest. The two interviewers were

trained in a half-day session in which the dental supplement

was emphasized.

Respondents—One hundred forty-six persons completed the interview. Roughly one-sixth of them were selected from the files of two cooperating dental practices in

the area. After persons were selected from the dental

practice files, they were telephoned by a member of the

project staff and asked to participate. The participation rate

was low. Of the 176 persons initially selected, only 22

actually completed an interview. The purpose of recruiting

some of the respondents through a dental practice was to

ensure that records would be available for at least a portion

of the sample and to enable us to “salt” the sample with a

number of respondents who had visited dentists during the

2-week reference period.

The remainder of the sample was recruited via posters

placed at stores in the neighborhood around NORC’S

offices, advertising in a local newspaper, and a recruiting

table set up at a nearby hospital. In the poster and advertisements, volunteers were requested for a study concerning health. Respondents were promised $10.00 for their

participation, which, they were told, would last about an

18
Table E. Wording of experimental and standard versiona of the dental supplement and summary of differences

Evpwlmenfal version Sfandard version

Public water fluoridation

Wording:

la As you understand If, what is the purpose of adding fluorkfe to the Now t’m going to ask you some questions about WATER FLUORIDATION.

publlc drinking watar’?

b, Does the water that you drink at home come from a public water As you understand n, what ts the PURPOSE of public WATER FLUORIDATION?

system or is k from another source, such as a well?

c. Has this public water supply had fluoride added to tt? a Is your home drinking water supply parf of a PUBUC water system, or

is tf from a well, spring, or cistern?

b. Is YOUR home drinking water supply FLUORIDATED?

Difference Experimental Vareion used simpler language In order to facilitate comprehension of the ttems.

Dental visits

Wording:

The next questions are about receiving dental care. HAND CALENDAR

2a.

b,

3a.

b.

c.

4a.

b.

5a,

b.

c.

d,

Is there a particular dentist’s office, dental clinic, or some other place These next questions are about receiving dental care.

that you usually go for dental care?

Altogether, how many DIFFERENT PLACES do family members go for 3a.

dental care?

Does anyone In the family go to an orthodontist?

Who iSthis?

Anyone else? b.

When needs to go to the denfist, who usually makes the c.

appointment for ?

d.

When needs to go to the dentist, how does 4s.

usually get there?

During the 2 weeks (outllned In red on that calendar), beglnnlng

Monday (date) and ending this past Sunday (date), dkt anyone In the

family go to a dentist? Include all types of dentists, such as

b.

orthodontlsls, oral surgeons, and all other dental speckdlsts, es well as

dental hygienists.

Who WS3 this?

During those 2 weeks, did anyone else in the family go to a denfist?

Ask for each person with “Dental visit” In 5b.

During those 2 weeks, how many times dkf go to a dentist?

The next questions concern vlsifs to the dentist that anyone in the family may

have made In the pest year. To help you remember possible vistfs I wIII read a

list of reasons some people have for going to the dentist. Do not answer as I

read the list. It is just to jog your memory.

Some people go to the dentist for a check-up and to have their teeth cleaned, or

to have a tooth filled or capped–

Some go because they are in pain or because a tooth broke or a filling fell out–

Some people go se part of a series of treatments for gum disease, a root oanal,

or to have false teeth fllledAnd some go as part of a series of orthodontic treatments–to have their teeth

straightened.

6a. During the past 12 months, (Ihat is, since (12-monfh date) a year ago),

how many visits dld make to a dentist? (Include the

(Number In 50) visit(s) you already told me about.)

b. When did IAST go to a dentist?

If respondent mentions month and year and NOT an Interval, fill 6C and mark 6b.

c. What was the month and year of this visit?

For the questions that I have just asked you, if is very important that we get the

most accurate answers possible. We know that n is sometimes hard to

remember whether a visit was before or aflar (LWronth dafe). We have found

that R helps some people to think of important events that happened about a

year ago. For example, this might be a birthday, a new job, a holiday, or a

vacation.

During the 2 weeks (outlined in red on that calendar), beglnnlng

Monday (date) end ending this pest Sunday (dale), dkt anyone in the

family go to a dentist? Include all types of dentists, such as

orfhcdontiste, orat surgeons, and etl other dental spaclaliste, as well as

dental hygienists.

whO WSSthis?

Durfng those 2 weeks, did anyone else In the family go to a dentist?

During those 2 weeks, how many Umea did go to a dentisf7

During the past 12 months (that Is, since (12-nronUr data) a year ago),

how many vtstfsdid make to a dentist? (Include the

(Numtmr /n 30) visit(s) you atready told me about.)

How long has if been since LAST went to a dentist?

19
Table E. Wording of experimental and standard versions of the dental supplement and summary of differences-Con.

Evperimenletversbrr Stendatrl versbrr

7.

S.%

b.

c.

d.

e.

f.

what Imporiantevent happened about a year ago?

(Specify)

You report tha~ made no VIMSsince (12-mor7Urdate).

When you think about (Ew?rd/n 7) do you remember any visiis that

made since (12-morrttrdate).

How many?

You reported that made (NumberIn 6a ) visitsslnos

(12- monthdate).

Thinking about pent /n 7) dkt any of the (NumberIn 6a) visitsthat

made occur before (12-morrftrdate)?

How many?

When you think about (Eventin 7) do you remember any additional

visits that made since (f2-monttr date).

How many?

Diftarancew Experimental version used three approaches to stimulate recall. Items 2-4 concerning the context of dental visit~ the list of reasons why people might visit

the dentiit that introduces question& and a reasklng of the question on number of visKain the prbr 12 months after asking the respondents to generate a Iandmartr

event.

Fiuorlde mouth rinse

Wording:

Sometimes people use fluorkfe to protect thalr teeth. For example, some mouth Se, Recently, some MOUTH RINSES have been developed that conlairr

rinses contain fluorkfe, others do not. FLUORIDEto reduce tooth decay. Does anyone in the farnliy now use

12a Schools and work places may have fluoride mouth rinse programs.

a mouth rkree that oontarns FLUORIDE, such as AOT, Fluorigard, LIsterDoes anyone In fhe family take part In such a program?

mint with Fluoride, StanCare, or a similar produot?

b. whO IS this?

b. Who is this?

c. Anyone else?

c. 1sthis at school or at work? d. Does use thk ftuorkfe mouth rinse at home, at school, or at

ri. Is anyone else in a mouth rinse program? work?

e. Does anyone in the family now use a mouth rinse or moulhwash that e. Whet is fhe name of the mouth rinse?

has fluoride in it?

f. whO is this?

9. Anyone else?

h. What ISthe name of fhe fluorkie mouth rinse that uses?

Dlfterenc~ Experimental vetsbn defines fluoride mouth time and decomposes the questbn so es to etlow the respondent to separately sonskfer *hool and horna

use of tluorkfe mouth rinses. Standard versbn defines tluorkte mouth rinse by expliottly Itsling brands of fluoride mouth rinse that were Oommerdelly available at the

time of the pretest.

Dietary fluodde

Wording:

Sometimes dostore or dentlats prescribe pills or drops with fluorlde In them. 9. Does now use FLUORIDEtablets, drops, or FLUORIDEvNa13a. Doss anyone in the family now take vltarnlns with fluorkte In them? mln supplements whioh are intended fo be SWAUOWED?

b. Who is this?

c. Anyone else?

d. Does anyone In the family now take any other kind of fluoride drops,

pills, or fablets?

e. Who is this?

f. Anyone else?

Difference!x The axparimantal versbn atfampts to improve understanding by decomposing the quastion so that the different types of products are asked separately,

20
Table E. Wording of experimental and standard versions of the dental supplement and summary of differences-Con.

E@arimental version Standardvarsion

Dental seatants

14a. Dental sealants are special plastic coa!ings that are painted on the tops 1Oa The chewing surfaces of teeth maybe coated with plastic dental eeafof the back teeth to keep them from decaying. They are put on by a ants to prevent tooth decay. These sealants are NOT fillings, caps, or

dentist or a dental hygienist. They are different from fillings, caps, crowns. Hae anyone In the family had plastic dental SEAMNTS applied

crowns, and fluoride treatments. Hes anyone In the family had dental to the teeth?

sealants placed on their teeth? b. Who is this?

b. Who iSthis?

c. Anyone else?

c. Anyone else?

Differences: The experimental version describes the procedures used to apply the sealants in order to promote understanding. Both versions specflcally exclude the

types of treatments that were confused with dental sealants in previous rounds.

hour, They were instructed to call NORC to set up an

appointment. Volunteers were asked a few screening

questions over the telephone to eliminate persons under

18 years of age, full-time students, and persons who had

not seen a dentist within the past 3 years. Those recruited

at the hospital were screened and given an appointment in

person.

Experimental procedure —A total of eight versions of

the interview were developed. A 2 X 2 X 2 factorial design

was used—questionnaire version (standard or experimental) and core (full or partial) were the levels of the first two

factors. These were the same factors employed in Portland.

Finally, one-half of the interviews were administered at a

fast pace; the other half were administered at a slow pace.

The rationale for including this last variable was simple: It

represented an effort to increase the realism of the laboratory interviews. & respondents showed up for their appointment, they were randomly assigned to an experimental

condition. The interviews were typically conducted in an

unoccupied office. A few were observed by project staff,

and an additional half dozen were tape recorded, but the

vast majority were not observed.

Dental information-After the interview was completed, respondents were asked to list all dentists they and

other members of the family had seen within the last 2

years. The respondents were then asked to sign a form

giving us permission to contact their dentists and to obtain

information to verify their answers to several of the items.

Parents living with minor children were also asked to sign

permission slips on behalf of their children. When other

adult family members were covered by the interview, the

Table F. Summary of Portland field test and Chicago laboratory test

respondent was given permission forms to take home for

the other adults to sign and return. Followup calls were

made to encourage the return of these additional permission slips and to resolve questions about the addresses of

the dentists.

At the conclusion of the interviews, each dentist who

had been named by a respondent was sent a data abstraction form with the person’s name and a copy of his or her

permission slip. The data abstraction form contained questions about the number of visits the person had made

during the 2-week and l-year reference periods. If the

dentist had not replied within 2 weeks, a followup call was

made.

Results

Portland pretest

By means of the Portland field test, we were able to

examine how well the questioning strategies that were

developed in the laboratory could be transferred to the field

setting. Table G summarizes the results by questionnaire

version for the Portland field test. In performing the statistical tests for differences, we took into account the clustering within interviewers and segments. Thus, the error

variances were based on only 23 degrees of freedom and

included the impact of variance among interviewers. Because of this approach, few of the differences were statistically significant.

Dental visits—Relatively few people reported any visits

to the dentist during the 2-week period preceding the

Portland pretest Chicago laboratory test

24 experienced Intewiewere 2 experienced interviewers

Purpostve sample of blocks Respondents recruited from dental practices and through advertisements and

Interviews conducted In respondent’s home

recruiting tables

Interviews generally observed Interviews conducted in laboratory

2-way ditslgn varying version of dental supplement and completeness of core

Interviews generally not observed

No validation data

3-way design varying version of dental supplement, comptetenees of core, and

pace of interview

Records data from dentists

21
Table G. Summary of results from the Portland field test for

experimental and standard versions of the dental supplement

Evperimentat Sfandahi

COP3 supplement supplement

Total . . . . . . . . . . . . . . . .

Shortenedcore . . . . . . . . .

Complete core . . . . . . . . . .

Total . . . . . . . . . . . . . . . .

Shorfenadcore . . . . . . . . .

Complete core. .,...,...

Total . . . . . . . . . . . . . . . .

Shortenedcore . . . . . . . . .

Complete core . . . . . . . . . .

Total. . . . . . . . . . . . . . . .

Shortenedcore . . . . . . . . .

Complete core . . . . . . . . . .

Tokit.. . . . . . . . . . . . . . .

Shorfenedcore . . . . . . . . .

Complefecore . . . . . . . . . .

Toile t . . . . . . . . . . . . . . .

Shortenedcam . . . . . . . . .

Complete core . . . . . . . . . .

Average numberof dental

VMS In prbr 12 months

1.8 1.6

2.1 1.6

1.6 1.6

Percentof personsusing

ftuoridemouthrinseat home

11 10

15 8

6 11

Percentof parsonsusing

fluoridemouthrinseat school

6 1.5

6 1

6 2

Percentof correctanswerson

householduse of fluoridemouthrinsel

42 57

49 57

28 56

Percentof correctanswerson

purposeof publicwaterfluoridation

76 67

60 88

76 66

Percentof respondentsglvlng

correctanswerson

own waler fluorkfetbn

46 37

46 39

44 49

‘Judged cormcl if bramdmporred mntalned fluoride.

interview. Overall, the proportion of people with one or

more 2-week visits was less than 10 percent, The analysis of

variance indicates that the proportion did not vary significantly as a function of the experimental variables.

Table G shows the average number of annual visits

reported in the four experimental groups. It appears that

the combination of the shortened core and the experimental version of the supplement produced the highest levels of

reporting, but the analysis of variance revealed that neither

the main effect nor the interaction was significant either for

the initial answers to the experimental version or for the

final answers that reflect responses to the landmark followup probe. The kmdmark probe did tend to increase the

level of reporting, netting an additional 49 visits.

Fluotide mouth rinse-Both versions of the dental supplement included items on the use of fluoride mouth rinses,

and both contained a definition of the term. The items

differed in that the experimental version contained separate

questions about school programs, and the major brands of

fluoride mouth rinse were listed in the standard version.

Both of these features appear to be effective. Table G

I

shows that more participants in school programs are picked

up through the experimental version, but use of the standard version reduces reports mentioning nonfluoride

brands. The analysis of variance revealed a marginally

significant main effect for the version used (F(l, 22) = 4.1,

p < .10),with the experimental version producing higher

overall rates of reported use.

Public waterfluofidah.on-Use of the experimental version of the dental supplement produced a higher rate of

correct responses concerning the purpose of public waterfluoridation than the standard version; 78.0 percent of

respondents given the experimental version answered “to

prevent tooth decay” or gave a related answer, compared

with 67.0 percent of respondents given the standard version. This effect was marginally significant (F(l, 22) = 2.9,

p < .11).

The item regarding whether the respondent’s own water was fluoridated yielded small and inconsistent differences between the two versions of the supplement.

Dietay Ji’uoride-The two versions of the supplement

produced no significant differences in the reported use of

dietary fluoride supplements. The general feeling among

the staff was that overreporting would be the biggest problem with this question, and the experimental version did

have marginally fewer reports.

Chicago laboratory test

The Chicago laboratory test was carried out by two

interviewers. Because so few interviewers were used and

because they were not selected randomly, it made little

sense to attempt to generalize across interviewers in analyzing the data from the NORC study. Instead, we treated the

interviewer variable as a fixed factor in the analysis, along

with the more substantive variables in the experimental

design-the version of the dental supplement, the use of

the full or shortened core, and the pace of the interview.

The basic analyses thus involve four-way analyses of variance, or logit analyses. We will focus on the effeets of the

substantive variables and discuss interviewer effects only

when the main results are qualified by interactions with the

interviewer variable.

Note that, for some of the items, we cannot expect to

replicate the findings from Portland. The use of supplementary sources of fluoride is considerably lower in Chicago, where the water is fluoridated, than in Portland,

where it is not. This reduced sharply our ability to look at

comparisons involving these items,

Time of aziministration-On the average, it took respondents nearly 9 minutes to complete the dental supplement portion of the interview. The pace variable, as

expected, had a marked effeet on the times; supplements

conducted under the fast pace took 7.9 minutes to complete

versus 9.6 minutes for those conducted under the slow

22
pace. The difference was significant (F(l, 130) = 16.8,P c

,001). The two versions of the supplement also took different amounts of time, with the experimental version requiring an average of 10.6 minutes, compared with 6.9 minutes

for the standard version. This large difference was also

significant (F(l, 130) = 75.4, p < .001). None of the other

experimental variables, either singly or in combination, had

a significant effect on the time for the supplement, although

one of the interviewers was significantly faster than the

other. The pace variable thus seems to have been successfidly implemented, and the experimental version, with its

added items and more elaborate introductions and definitions, required more time to administer than the standard

version.

The pace variable also significantly affected the time it

took to complete the core interview, the slow pace adding

an average of more than 3 minutes (F(l, 130) = 5.99,p <

.05). The full core required almost 13 more minutes, on the

average, than the shortened version, a highly significant

difference (F(l, 130) = 80.87,p < .001). Table H summarizes some of the results tlom the Chicago laboratory tests.

Dental vi+sits-As expected, none of the experimental

variables had significant effects on responses to the 2-week

dental visit items. We examined responses to the l-year

visit items separately for the respondents and for other

family members.

The number of self-reported 12-month dental visits

varied considerably, with responses ranging from zero to 50

visits. The distribution is quite skewed, with most respondents reporting two or fewer visits. In order to reduce the

impact of the few cases with a large number of visits, we

deleted data for persons reporting 12 or more visits, and we

analyzed the square root of the number of visits reported as

well as the raw number. The results from all analyses were

the sam% therefore, we present the findings based on the

raw figures here.

Table H shows the average number of self-reported

visits for the l-year reference period. There was an interaction between the pace and version variables. As we

expected, with the slow pace, the experimental version

produced higher levels of reporting than the standard

version (2.7 visits for the experimental version versus 2.0

for the standard). This pattern was reversed for the respondents interviewed at the fast pace (1.7 for the experimental

version versus 3.4 for the standard), a reversal we did not

expect. The overall interaction was significant (F(l, 129) =

6.7, p e .05). The pace variable also interacted with the

shortened core, and slow pace with the complete core led to

relatively high levels of reporting (3.1 visits for the fast

shortened group and 2.9 visits for the slow complete

group). The interaction was significant (F’ (1, 129) = 4.8,

p < .05).

Table H also shows the mean number of annual dental

visits reported for other family members. (For households

with more than one additional family member, a mean was

first taken for the other family members.) For other family

members, the completeness variable interacted with the

version of the supplement. The experimental version produced higher levels of reporting with the shortened core (an

average of 2.3 visits versus 1.4 visits for the standard core);

with the complete core, the hvo versions did not differ as

much (1.2 for the experimental version versus 1.9 for the

standard version). This is the same pattern that was observed in the Portland pretest. In Chicago, the interaction

was significant (F(l, 83) = 6.4, p < .05). However, the

pattern is further qualified by an interaction with the pace

variable: It is apparent only when the interview was conducted at the fast pace. The interaction involving three

variables was also significant (F(l, 83) = 4.6, p < .05).

Fluoride mouth rinse and diekny j7uoride-The use of

dietary fluoride supplements and fluoride mouth rinses was

relatively rare in our sample This comes as little surprise,

as Chicago’s water supply is fluoridated. Neither item

revealed any differences by version or by any of the other

experimental variables.

Public waterj7uonifation-As in Portland, the experimental version of the item concerning the purpose of public

water fluoridation produced a higher percentage of correct

Table H. Summary of results from the Chicago laboratory test for experimental and standard veraions of the dental supplement

Fast pace Slow pace

Total, Shortened Complete Shortened Complete

Supplement and respondent both par%e Total core core Total core core

Average number of dental visits In prtor 12 months

Experimental supplement

Self . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 1.7 1.7 1.8 2.7 2.3 3.2

Others . . . . . . . . . . . . . . . . . . . . . . . 1.7 2.0 3.0 1.1 1.5 1.8 1.2

Standard supplement:

Self . . . . . . . . . . . . . . . . . . . . . . . . . 2.7 3.4 4.5 2.4 2.0 1.3 2.7

Others . . . . . . . . . . . . . . . . . . . . . . . 1.7 I .e 1.2 2.5 1.6 1.7 1.4

Parcent of respondents giving correct answer to question on the purpose of publtc water fluorkfation

Experimental supplement. . . . . . . . . . . . . 76.7 77.1 68.2 66.7 76.3 75.0 77.8

Standard supplement . . . . . . . . . . . . . . . 52.9 60.3 77.2 44.4 45.9 50.0 42.0

Percent of respondents giving correct answer to Ihe question on own water fluoridation

Experlmantal supplement. . . . . . . . . . . . . 53.5 56.8 50.1 52.6 40.0 m.7

Standard supplement . . . . . . . . . . . . . . . 67.3 :: 55.6 61.1 75.7 66.7 84.2

23
responses than the standard version. Apparently, at least

some people know what fluoride is for but may not recognize the term “public water fluoridation.” A Iogit analysis

revealed a significant (p < .05) main effect for the version

variablq 76.7 percent of the respondents who were given

the experimental supplement correctly named the purpose

of fluoridation, compared with versus 52.9 percent of those

who received the standard version. The logit analysis also

revealed a marginally significant (j e .15) advantage for

the shortened core (71.2 percent correct versus 57.5 percent for the complete core) and an uninterpretable marginal interaction involving the version, completeness, and

interviewer variables. The item in which respondents were

asked whether their own water was fluoridated revealed

only a marginal main effect for version, with the experimental version producing a lower rate of “yes” responses and a

high rate of “don’t know” responses. This replicated the

prior Part B results.

Overall results: Summary

The experimental version of the dental supplement

differed from the standard version in four main respects. It

had simpler language than the standard version for the

items concerning public water fluoridation. In addition to

the main items regarding annual dental visits, it included a

series of warmup items regarding the context of dental

visits, a list of reasons for possible visits, and a followup

item asking respondents to try again. It gave a general

definition of fluoride mouth rinses (as opposed to the

standard version’s definition, in which particular brand

names were mentioned), and contained a separate question

regarding participation in school mouth rinse programs.

Finally, an item about the use of dietary fluoride was

decomposed into a series of items. The last difference

produced no marked effects, perhaps because so few people use dieta~ fluoride supplements that it was difficult to

observe much variation across versions.

Portland results—Overall, the Portland pretest, despite

its larger sample size, yielded few significant effects involving the version of the supplement and no effects at all

involving the length of the core interview, A number of

factors reduced the power of the design. The sampling

error estimates include interviewer variance and are more

substantial than might be inferred from the nominal sample

size of 971 persons. The data were clustered by household

and by segment and interviewer. (These last two sources of

variation are completely confounded.) Because of the clustering, the variance estimates for most analyses are based

on 23 degrees of freedom. This conservative approach

makes it unlikely that results will reach conventional levels

of significance. We therefore relaxed our criterion for

reporting results in this summary, discussing findings “significant” at the .10 level, especially when the findings are

replicated in other studies reported here.

Despite these sources of imprecision, the Portland

pretest produced a few noteworthy results involving the

version of the supplement. First, respondents interviewed

with the experimental version were likelier to correctly

identify the purpose of fluoridation (78.0 percent versus

67.0). Although only marginally significant, the result is

consistent with the earlier Part B findings and, as it turned

out, with the results from the Part C Chicago experiment,

Second, the Portland results suggested that the experimental version produced higher levels of reporting of dental

visits only when administered in conjunction with the shortened core. We suspect that the shortened core probably

produced a more leisurely interviewing pace and that this

slower pace was a prerequisite for the effectiveness of the

warmup items in the experimental version. This hypothesis

was put to the test in the Chicago experiments, where it

received some support. The final two findings involve the

fluoride mouth rinse items. The experimental version produced significantly higher levels of reported usage overall

and, most dramatically, higher levels of reported participation in school programs. This difference apparently reflected the decomposition of the mouth rinse item in the

experimental version, which contained a separate question

on school programs. The standard version produced lower

rates of erroneous reports involving nonfluoride brands.

The standard version explicitly mentioned the brand names

of the relevant mouth rinses.

Chicago results–The Chicago experiment confirmed

the Portland results in several respects. Once again, the

experimental version with its simplified kmguage produced

higher rates of correct answers to the item about the

purpose of fluoridation. The expected pace-by-version interaction did, in fact, appear in answers to the annual dental

visit items. There were a number of surprises as well.

The pattern for annual dental visits was more complicated than we had predicted. We had expected the two

versions to differ only when the pace was slow, with a slow

pace, the experimental version would, we thought, produce

higher levels of reporting. The results for the slow-pace

respondents confirmed this prediction. With a fast pace, for

which we expected little difference between versions, the

standard version in fact produced the higher level of reporting. In addition to the pace-by-version interaction, pace and

the completeness of the core interview had an unexpected

interactive effect on self-reported visits. Finally, for visits by

other family members, we observed the same interaction

originally noted in Portland-the experimental version produced higher reporting only with the shortened core—

rather than the pace-by-version interaction we had predicted. This interaction, however, was present only in the

interviews conducted at a fast pace,

The Chicago results departed from the Portland results

regarding the two versions of the mouth rinse items. In

Portland, the experimental version produced a higher level

of reporting, especially reporting of participation in school

programs; in Chicago, the two versions did not differ

significantly. The absence of school mouth rinse programs

in Chicago probably accounts for the difference.

Compankon-Table J provides a summary of the results of the two experiments. Considering the number of

methodological differences between the studies, we find the

convergence in the results quite impressive, Although there

24
Table J. Summary of major results from Portland field test and Chicago laboratory test

Item Porlland results Chkago rewlls

Public water fluoridation

Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . Experimental version produced marginally Experimental version produced sign”~antly

higher Identification of purpose. higher Identfka!ion of purposq marginal effect

for shortened core.

Ownwater . . . . . . . . . . . . . . . . . . . . . . . . . No effect. Experimental version produced marginally

higher rate of “don’t know” responses and

mapjinally lower rata-sof “yes” responses.

Annual dental visits

Levelofrepoftlng . . . . . . . . . . . . . . . . . . . . No effecc cell means Indicate that experimental S!@ficant pace X version interaction: Wdh fast

verelon produced higher average reporting only

with shortened core.

pace, standard version produced higher

reporllng; wHhslow pace, experimental version

produced h@her reporting; for others,

experimental verskm produced higher reporting

only wHhshortened core.

Fluoride mouth rinse

Level of reporting . . . . . . . . . . . . . . . . . . . . Experimental version produced significantly Experimental version generally produced

higher rate of reporting overall and an especially somewhat higher rate of reported use, but

high rate for participation in school programs. differences not significant.

Reporilng of brands . . . . . . . . . . . . . . . . . . . Standard version produced fewer reports of No effect.

nonfluorlde brands.

are differences between the results of the two studies, they

are generally differences of degree (e.g., a finding that is

significant in one study is apparent but not significant in the

other) or differences that reflect the fact that Chicago has

fluoridated water, whereas Portland does not.

The pace and completeness variables produced some

interesting findings. The Chicago results for annual dental

visits by other family members resembled the Portland

results more closely when the interviews were conducted

under the fast pace. Further, the Chicago dental visit results

resembled those of our earlier Part B study when the core

was shortened and the pace was slow. Under these conditions, the experimental version produced higher levels of

reporting than the standard version. In the Part B study,

only a few demographic items were used in place of the

NHIS core, and the interviews were conducted at a slower

pace than those we observed in the Portland pretest. These

findings su~est that important characteristics of the field

setting can be duplicated in the laboratory, and that it is

important to do so if the results are to be generalizable.

Methodological results

In this component of the study, we also examined the

advantages and disadvantages of various methods of testing

a questionnaire. As stated earlier, we were particularly

interested in (1) the impact that site of testing has on the

results, (2) whether qualitative assessments yield different

conclusions from more quantitative assessments, and (3)

whether it is possible to replicate some of the general

conditions of the field setting in the laboratoq. We think

that an understanding of these matters is very important

because it will provide guidance as to how laboratory

testing can be integrated with field testing. Therefore, in

this section, we review some of the difficulties we had in the

operation of both the field and laboratory testing and

present some potential ways to avoid these problems. In

addition, we present our results on the differences between

a qualitative assessment and a quantitative assessment of

the two instruments, and we comment on the need for and

possibility of taking account of the general survey conditions during laboratory testing.

Field test

When this study began, the NHIS questionnaire process included two large field pretests. They were evaluated

qualitatively by means of a debriefing of the interviewers

and the observers. Because of the time pressures, data were

not always analyzed quantitatively and thus were not always

used in the evaluation of the results, Because generally only

one version of the questionnaire was used in each pretest,

there would have been no basis for comparison even if

tabulations had been done. In this study, however, the

results of the Portland pretest were mailed to NORC from

the field, coded, keyed, and tabulated in time for the

debriefing of the observers. Thus, the quantitative results

can be compared with those of the qualitative assessment.

Operationalfeatures of the pretest–Although offering

advantages for analysis, the introduction of two versions of

the questionnaire created a number of practical problems.

The laboratory version of the questionnaire arrived late at

the U.S. Bureau of the Census; consequently it was not

formatted and printed as carefully as the standard version.

Moreover, training for the experimental version was relegated to the last position in the training session. Finally,

procedures for administering and coding the experimental

instruments departed in various ways from what had been

standard procedure for NHIS interviewers in the past.

Interviewers strongly objected to such items as the

25
landmark followup probes and the reasons list, which differed markedly from the standard version.

These problems resulted from the isolation of this

project from the previous testing environment and our

resulting insensitivity to the NHIS institutional environment. The interviewers face a complex task that requires

following a standard set of procedures and maintaining

consistent behaviors throughout the interview. Because the

training sessions were brief, it was especially difficult for

them to adapt to the changes required for the experimental

questionnaire. The problems with the printed version of the

questionnaire aggravated this difficulty. In retrospect, we

see that we should have worked more closely with the U.S.

Bureau of the Census to overcome these problems.

In fiture design studies, efforts should be directly

aimed at overcoming the transition from laboratory to field

testing. To alleviate the operational problems outlined

earlier, we concluded that field staff should be included in

the exploratory and developmental phases. The success of

the testing phase is dependent on having procedures that

can be implemented in the traditional NHIS environment.

This means that the field staff should be brought into the

design process sooner, should be considered a partner in

the development of the questionnaire, and should be kept

informed of the findings and reasoning that motivate decisions. Field staff have excellent insights into interviewers’

reactions to certain procedures and can anticipate respondents’ difficulties in answering certain questions. We did

little to take advantage of this expertise in the current study.

Qualitative and quantitative assessment of differences–

The qualitative assessment of the results consisted of (1)

oral debriefings with the interviewers immediately after the

pretest and (2) both written comments and an oral debriefing of the observers. The qualitative assessment of the

results on the dental visit items indicated that the interviewers had severe problems administering the landmark followup probe. They were very critical of these items and were

not able to administer them correctly. Interviewers also

objected to reading the reasons list that preceded the

12-month question, and they reported that some respondents objected to hearing the list. Interviewers did not have

problems in administering the 12-month question; they

merely found its treatment jarringly different from the

treatment used for other questions. These difficulties could

have been avoided through consultations with interviewers

earlier in the design process.

Analysis of the quantitative results revealed no significant differences between the experimental version and the

standard version, although prior laboratory tests had registered differences between them and the pattern of differences was later replicated in the Part C laboratory study.

Because no significant differences were found between

the standard and experimental versions of the dental visit

items, we can conclude that, in this case, qualitative assessments are not necessarily reliable guides to quantitative

results. Interviewers’ strong objections to the experimental

version would have led one to assume that it was unworkable. However, the tabulated results indicated few

differences, and those that were present suggested that the

experimental version produced more complete recall.

Quantitative analysis of field pretest results on items

with unfamiliar terms substantiated findings from the previous two stages of development. The most dramatic instance involved the question on the use of fluoride mouth

rinses in school programs. Portland has an extensive fluoride mouth rinse program in the public schools. In the

experimental version of the question, home and school use

of fluoride mouth rinses was decomposed. Nearly four

times as many children who used fluoride mouth rinse were

identified under this strategy, clearly indicating the superiority of the approach.

Tabulation of the results on the home use of mouth

rinse revealed that the standard version was more successful in reducing overreporting. In the experimental version

of the question, we tried to stress the distinction between

fluoride and nonfluoride mouth rinses. In the standard

version, we made the distinction by mentioning the four

commercial brands of mouth rinse that contain fluoride.

The success of the questions was evaluated by asking the

respondent to report the brand used. The superiority of the

standard version in reducing overreporting of non fluoride

mouth rinses substantiates a conclusion reached in earlier

laboratory work The clearer the distinction between the

specific item in question and similar items, the more accurate the response.

However, the qualitative assessments of the items with

unfamiliar terms did not always give the same results. For

example, at the end of the debriefing sessions about the

question on the use of fluoride mouth rinses in school, we

concluded that the better question was the briefer version,

which did not contain a separate question about school use

of fluoride mouth rinses. Tabulation of the data M to the

opposite conclusion. Similarly, for the question on the use

of dietary fluoride, observers at the debriefing concluded

that the shorter standard version would be preferable in

that it would reduce the likelihood of overreporting of these

products. However, the tabulated data showed less reporting on the experimental version.

After observing the interviewing and debriefing and

tabulating the data, we conclude that both the qualitative

and quantitative assessment are needed. The interviewer

reactions and problems—a valuable source of information

about problems in the questionnaire and possible sohltions—are noted in the qualitative assessments. However,

the qualitative assessments are not sufficient for noting

things that appear only in the aggregate results. Each

interviewer and observer sees about 10 completed interviews. Thus, only those differences that can be noted with

sample sizes of 10 or less are apparent in the qualitative

results. The quantitative assessment is needed to detect

differences that are apparent only with larger sample sizes.

There is a basic inconsistency in conducting a large-scale

pretest and not tabulating the results. The purpose of large

sample sizes is to detect small differences or rare problems.

These will be apparent only from the tabulation of the data,

To facilitate the quantitative assessment, questionnaire

26
data should receive the same editing in the pretest as in

the main survey.

We also think that experimental designs with more

than one version of the questionnaire should be employed

in pretests. This takes advantage of the large sample sizes

that are used and increases the information yield from the

field test. Having the results from more than one version of

the questionnaire provides some basis for determining why

certain questions result in problems and what potential

solutions exist. Exploratory and developmental studies in

the laboratory can provide the basis for deciding which

items to test further in the final testing phase. However, our

experience tells us that special efforts will be needed to

smooth the transition between the laboratory and the field.

Laboratory tests

,., The final laboratory study in Chicago yielded relatively

little new information. The two versions of the questionnaire were not radically different, and most of the differences between them were already apparent in the field

experiment, The site of the final Iaboratoxy study—a fluoridated area-may also have reduced its power to detect

differences in the items involving use of fluoride products.

Although laboratory testing could be substituted to

minimize the deleterious effects of not having a field test, it

is not recommended. The main differences between the

results of the two testing sites can probably be attributed to

the fact that the laboratory experiment was not conducted

in a fluoridated area, whereas the field test was. To make

maximum use of the laboratory method in the last round of

testing, the sample sizes would need to be increased. Also,

special procedures, such as postinterview comprehension

probes, might be used to increase the value of the laboratory testing. However, our recommendation is that field

testing be done, if at all possible.

Our results indicate that, although there is consistency

between the laboratory and the field, the general conditions

of the field survey have an important effect on the outcomes. Currently, a field testis the best way of making sure

that the questionnaires and procedures operate well under

these conditions.

Both the laboratory testing and the field pretesting 4

conducted in the current study were subject to large selection biases. The field pretest was conducted using a nonranb dom sample of blocks; in addition, no attempt was made to

persuade nonrespondents to participate. The nonresponse

rate in the pretest was thus many times greater than that in

the national survey. Relative to the actual survey, the

pretest contained a higher proportion of people who were

easy to interview. This may give an overly optimistic picture

of the adequacy of the questionnaire. The laboratory respondents were also a nonrepresentative group. The small

monetary incentive disproportionately attracted low-income respondents and persons located near the laboratory.

The selection biases inherent in both testing sites must

be kept in mind when interpreting the results. Those associated with the field pretest could be reduced by requiring

the interviewers to obtain a higher response rate, which

they are capable of doing. Screening can be used to reduce

the selection bias among the laboratory respondents. However, the bias cannot be eliminated in this way, since certain

groups will never volunteer for screening. Recruitment

among special subpopulations may be needed to overcome

some of the selection bias associated with the laboratory

approach. Contacting groups of retired people, mothers

with children in day care, and so on may serve to increase

the range of people in the laboratory studies. However, we

did not try these procedures, and their effectiveness remains to be demonstrated.

V&iation study-The validation study conducted in

conjunction with the Chicago laboratory study was something of a disappointment. We encountered a number of

difficulties in obtaining validation data regarding dental

visits. We had hoped to reduce the data collection costs by

recruiting a large portion of the sample from a small

number of cooperating dental practices. Although two

practices agreed to participate in the study, the patients

themselves were very difficult to recruit. Less than onesixth of the sample ultimately came from the ~o practices.

Thus, the validation study was no more efficient than one

that could have been conducted in connection with a field

experiment.

We encountered all of the usual problems with recordcheck validation studies. The respondents made errors in

the names and addresses of the dental practices. The

dentists took a long time to return the validation questionnaires, and many did not do so at all. In one case we had to

visit the dental ot%ce to get the information. Approximately

40 percent of the respondents lacked complete validation

information. The lack of agreement between respondent

reports and reports of the dental office was sometimes

difficult to interpret (as when dentists denied ever having

seen a patient), perhaps indicating a problem in the records

or an error in identi~ing the dentist. We made no attempt

to resolve differences between the respondent’s report and

that of the dentist, so we have no information as to what

produced the discrepancy. Resolving such differences

would have been an expensive and time-consuming effort.

A more successful validation study would have required

greater effort than the resources of this project would

allow.

If a sample of persons is to be selected from a health

care provider, a means for directly contacting the patients

must be negotiated. The return rate on a mail solicitation

(which was used with patients at one practice) is much too

low to obtain an adequate number of respondents. Alternative procedures that could be tried are (1) obtain permission to contact the patients by telephone; (2) obtain the

cooperation of the health care provider early and use a

prospective recruitment effort (e.g., place a recruitment

poster in the office); or (3) hire the staff of the provider to

make the direct contact with the patient,

Another way in which the validation study might be

improved would be to use the respondent to help resolve

differences between the survey data and the record data.

These discrepancies could be discussed with the

27
respondent, and matching errors (such as contacting the

wrong provider) could be corrected. Debriefings, reinterviews, and interviews with other family members could also

be used to assess the quality of the responses.

Conclusions

On reviewing the results of the final phases of our

work, we arrive at a number of general conclusions. We

conclude that field testing with two or more versions of the

survey instruments should be employed in the final phases

of testing. The evaluation of the field test should include

tabulations directed at answering specific questions as to

differences among the alternative methods as well as a

directed qualitative assessment of the operational characteristics of the various methods. A formal debriefing questionnaire for each of the interviewers and observers could

be used to direct the qualitative assessment toward the

most important issues. This could be combined with group

meetings that allow discussion and elaboration of the

observations.

The instruments that are included in the field test could

be developed through a series of laboratory studies similar

to those that we conducted in Parts A and B of our study.

The laboratory studies should be augmented with smallscale field tests and consultations with field staff in order to

ease the transition from the laboratory to the field. Our

experience and our analytic results tell us that the general

conditions are important when one is considering the methods to be employed in a survey and that some results can be

transferred from the laboratory to the field setting. Our

attempt to manipulate the pace of the interview is a preliminary indication that it will be possible to study field conditions in the Iaboratoxy. Much, however, remains to be

learned about the limits and the scope of our ability to

study field conditions in the more controlled laboratory

setting, This should be a topic in a continuing assessment of

laboratory and field methods.

28
Summary conclusions

and suggested

protocol for laboratory

and field testing

We conclude that laborato~ testing and the cognitive

\

sciences can play an important role in the design and testing

of questionnaires. Using laborato~ testing and such cognitive techniques as protocol analysis, comprehension probes,

and experimental manipulation of questions, we were able

to identifj some of the processes respondents used in

formulating answers, to analyze the problems they experienced, and to develop hypotheses about how questions

might be improved. Many problems that were identified in

the first field pretest were pinpointed in the laboratory in

less time, with fewer respondents, with less professional

effort, and at lower cost. The laboratory setting can also be

used to gain greater insight into the source of respondent

difficulties. Concepts from the cognitive sciences were used

to wide the development of alternative questions, and

subsequent laborato~ testing revealed that these changes

were often successful in improving responses. Many of the

laboratory findings were confirmed during subsequent field

testing,

Field testing remains a vital component of the questionnaire development process. Problems with handling

and administering materials became apparent only in field

tests. Moreover, interviewers are in a position to offer

excellent su~estions for clarifying the wording of some

questionnaire items, We conclude that laboratory testing

can increase the efficienq of field testing. The procedures

used by NCHS for the design of the NHIS supplements can

be improved by using a combination of both approaches.

Based on our evaluation of both the Iaboratoxy and fieldtesting activities, we recommend the following prototype

for the design process. Our prototype takes the form of a

series of individual recommendations for each stage of the

process.

At NCHS and in many other settings the survey research staff often consists of three types of people

(1) statisticians and survey methodologists concerned with

the development of the measurement and analysis methods,

(2) survey methodologists and field staff concerned with

refinement and implementation of the measurement methods, and (3) substantive experts who are sponsoring the

survey in order to answer specific substantive questions. We

see the laboratory testing as a way to bring these three

parties into closer, more effective collaboration.

Our suggested protocol includes the following four

phases: (1) planning, (2) exploratory studies, (3) laboratory

development and testing of instruments, and (4) field testing. This phasing represents a progression in which each

successive phase becomes more like the setting (general

measurement conditions) of the actual survey.

Planning

The research effort devoted to the development of the

questionnaires needs to be carefully planned. This is, of

course, a precept for all research efforts which is easier to

state than execute. Careful and early planning is particularly important in the government setting, in which time is

tight and government clearances are required for some

testing activities. The transfer of methods from the laboratory to the field will be eased if all three components of the

research staff are involved in the early planning.

We found that an early attempt to categorize the

research issues was helpfid in guiding our research. We

therefore suggest that response issues be identified for each

survey item and that these issues then be classified into

more general categories.

Exploratory studies

The theories and methods of cognitive science are an

excellent tool for exploring the measurement issues associated with the survey items. Initial stages of testing using a

variety of respondents can be done in the laboratory. The

range of laboratory respondents should reflect characteristics that have been known to affect response. Given such

attention to variety, the number of laboratory respondents

need not be large.

We suggest that the laborato~ testing be guided by

explicit hypotheses as to the response task implied by the

items, problems that arise in the response process, and

methods for improving the response process. Protocol analysis (think-aloud interviews) can be used to generate such

hypotheses.

Early laboratory results can be supplemented by field

pretests on nine or fewer respondents. Using experienced

field staff to conduct these tests and having laboratory staff

observe the tests may facilitate the development of measurement methods that fully take into account the conditions of the field setting.

Experienced field staff can also serve as consultants in

the laboratory, assisting in planning and commenting on

results. The three types of staff that are concerned with the

survey should be involved in all stages of testing. Time

29
should be devoted to interpreting the resultsat each stage

and planning for the next stage.

Developmental methods

The development of questionnaires should involve iterative laboratory testing and experimental designs. It maybe

desirable to include techniques that increase the differences

among alternative methods. This type of experimentation

may reveal the underlying response process and allow

subsequent development of questions that facilitate this

process.

Techniques that will be useful in the development

activities include debriefing the respondents and recording

the interviews. The researchers sponsoring the survey can

review the recordings, thus gaining direct experience with

the measurement methods and problems.

The important differences to be measured in the survey

should be reflected in the area where the laboratory study is

conducted and the group of people interviewed. To fulfill

this requirement, it may be necessary to move the laboratory to different areas.

The tabulated laboratory results should be the basis for

choosing the preferred questioning strategies, The laboratory setting allows for detailed analysis of the results and a

flexible iterative testing strateW, so discoveries in each

round of testing can be used in succeeding rounds.

Again, all parties should be involved in the questionnaire development program so as to ease the transition

horn the laboratory to the field.

Testing methods

Results of the laborato~ testing will suggest alternative

approaches to asking the questions. These approaches can

be tested in a large-scale field test that will be used as the

basis for the final questionnaire construction. The tests

must be designed so that the information needed to evaluate the design is included, and the data from the field test

should be tabulated as a guide to decisions about the

questionnaires.

In choosing the versions to be tested, the site of testing,

and the sample sizes to be used in the field test, planners

should consider which choices will provide maximum statistical power. The field staff and the sponsor should participate in the design and interpretation of the field test.

In addition, based on our experience, we have the

following practical suggestions concerning operation of a

laboratory testing program. I

Respondents can be recruited in a variety of ways.

Sufficient incentives and a convenient testing site are

needed. Respondents should be screened by telephone and

scheduled for appointments. Mailed reminders of appointments are likely to increase the participation rate.

The place dedicated to testing should include access to

audiovisual equipment and, if possible, a one-way glass that

permits observation.

If necessary, laboratory testing should be moved to

different sites to ensure representation of specific subgroups.

A variety of staff will be needed. We recommend a mix

of survey methodologists and statisticians, cognitive psychologists, clerical assistants, and field interviewers.

The testing process will be most efficient if quick

reaction to discoveries at different rounds of testing is

possible. Thus, OMB clearances must be broad enough to

allow such modifications.

30
References

,

Bradbum, N. M. Response effects. In P. H. Rossi, J. D. Wright, &

A. B. Andemon (Eds.), Handbook ofwtvey m=search.New York

Academic Press, 1983.

Ericsson, K. A., & Simon, H. Verbal reports as data. P~chologz”cal

Review, 1980,8,215-251.

Genter, D., & Collins, A. Studies of inference from lack of

knowledge. Menwry and Cognition, 1981,9, 434-443.

Hansen, M. H., Hurwitz, W. N., & Bershad, M. A. Measurement

errors in censuses and surveys. Bulletin of the International Statk

tical Institute, 1961,38, 358-374.

Jabine, T. B., Straf, M. L., Tanur, J. M., & Tourangeau, R.

Cognitive aspects of survey methodology: Building a bridge between

disciplines. Washington, D.C.: National Academy Press, 1984.

Lessler, J. T., & Kulka, R. A. Reducing the cost of studying

measurement erro~ Is a Iaboratoxy approach the answer? In T.

Wright (Ed.), Statistical methods and the improvement of data

quality. New York Academic Press, 1983.

Lessler, J. T., Mitzel, H., Salter, W., & Tourangeau, R. Cognitive

aspects of questionnaire design: PartA teport. Contract No. 282-84-

2123. Hyattsville, Md.: National Center for Health Statistics.

1985.

Neisser, U. Memoty observed. San Francisco: W. H. Freeman and

Company. 1982.

Oksenberg, L., & Cannell, C. F. Some factors underlying the

validity of self report. Bulletin of the International Statistical

Institute, 1977,48, 325-346.

Schuman, H., & Presser, S. Questions and answers in am”tude

surveys. New York Academic Press, 1981.

Sudman, S., & Bradbum, N. Response effects in surveys: A review

and synthesis. Chicago: Aldine, 1974.

Tourangeau, R. Cognitive science and survey methods. In T. B.

Jabine, M. L. Straf, J. M. Tanur, & R. Tourangeau (Eds.),

Cognitive aspects of survey methodology Bw”ldinga bridge between

disc@lines. Washington, D.C.: National Academy Press, 1984.

Tversky, A., & Kahneman, D. Judgement under uncertainty

Heuristics and biases. Science, 1974,185, 1124-1131.

31
Appendixes

Contents

I. Pretest of the dental health supplements to the 1986 National Health Interview Survey . . . . . . . . . . . . . . . . . . . . . 33

Part A pretest questionnaire . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Part B questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

IL Field pretest of the dental health supplements to the 1986 National Health Interview Survey . . . . . . . . . . . . . . . . 50

Part C field pretest questionnaire-standard version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

Part C field pretest questionnaire-experimental version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

32
Appendix 1

Pretest of the dental

health supplements to

the 1986 National

Health Interview

Survey

PartA pretest questionnaire

Section O. DENTAL CARE PAGE PERSON 1

HAND CALENDAR.

Th=s. rmxt quumtlon. -r= -bout mc.i.lng d.ntal car..

1¤ DurIn~ th. 2 w..fw Ioutllnod In rod on that aml.nd=r]. boglnnlng Monday

Liafd snd ●ndhts th:= Past ttund=v fdatd. dld =.YO.O I. tho fmmtly so to a

dantlst? !nclud= ●11types of dmntiatw. ●mh .s orthodcmtl.t.. wml

SMWBOIIS, ●td ●ll Otfmr dantd spacldists, ●C w.11 m=d.ntd hyglanlct-. Cl Yes O No (2J

----- --------------------------------------------------------- . —--- -— — _________________

b. Who w-. ah?.?

Matk “Dental vlsh~r box In parson% column. 1b. I ❑ Dental visit

--. ----------- . ---------, ______________________________________ - .—-.

o. During thmm 2 w=ak8. dld ●nyon. ●lm fn the family go to ● dwtist?

-— — __________________

o Y.. (Reaak lb ●.dc) a No

-------------------------------------------------------------- - .—— - .—— —_________________

d. Ask for ..ch p.raan with ‘Wental VISWVIn 7b:

Dtwkna tha.a Z w-.US. h-w many urn.. did -- a- X- ● dmrttlst? . ‘- U..mb.

Mark box If undsr two ynar; old. SS8 O Und.r 2 INP)

2m. Dwknu the past 12 months (thmt Is, ●lnco 72-mc.nrh dare)= yaar ●go . ●bout

how mmny wlslts dld -- rnsk~ tr#. d~tk.t ~~

2a.

wIsN(s) you .Jmady told ma about.)

visit8

----- ---------- =-- ——----— -—--- .—----— ———————————-—————- —————-——.----- ————————.___________

00LIO None

Mark ..2-wuk dantal VISIY. box ~ p.r.an% cofunm 77vhit[s) reported in 1d.

b. ADOUT h- IO”U h-- kt h-n ~n~a -- LAST W=ilt *O m dmntlst?

~ a P88t 2 waoks not rcponed

b. (Mark 76, -sk 7dJ

Z n 2-week dental vlslt

J a Over 2 weeks. loss

than 6 months

a U 6 months, less than 1 year

S D t YC8r. !~SS than 2 yaafs

0 ❑ 2 wars. ta==than 5 years

7 ❑ 5 yams or mom

o a Nevw

01 Rater to number of vlslrs in 2.. o~ 1 n At least 0“. “hit In 2S t3)

● n other (fVP)

HAND CARD 07.

1. Whlsh on= of th=mm mDoons BE8T dows,lba. why -- vt,lwd tfm d.nttst

during tha p~at 12 months?

a. 1234S

8Z

{/ moro thmn orm vhlt kn2s:

?I*MG slwm ftM tkN mason fortha FIRST Vhft.

1. Cmllmd for -ok-up by d~tlm.~ oNkce

z. W-tit fOF chmck Up 081 OWn

J. W*nt bmmtm= of pmln or dlsoomf~

Spaclry

4. Adwls.d b~ f ●milv mmnb=r or frfond

6. Adwlsad by phyglclmt or m’mthor dsntlst

S. 8om8 othnr muon Cire& OQ* raa.on only.

L=. D~a ●nyon. In tho Camlly now usm todthpmta?

u Y*S U No (6) n OK (S)

------- —- —— —-------- ---------------- —— -------- ----------------- .---- --------------—---—-

b. Who h this?

Mark ‘TeothNm8”, bpx in P8rson% aokmm. 4b. s n T00thp8mt9

-=---”----------------------------- ------- —----- -----—————————— --------—---—--———————-

0.Anvomm ●I**?

❑ Y.. fReask 4b ●id.) ❑ No

-- — ---------------- --- —---- —- ———--— —---------------------- —- —-— — .- ——. - ——— ——— ——_. ——— -—-—---

A8k for ●ach pW80~ with .’Toothpaste “ in 4b:

d. Doom -- tlOW us- ●ny ONE of thma brand=?

HAND CARD 02. d. 12345678

1. cOl(fmt2. Crmmt

a. fkleam

4. Aqumfrw8h

6. Mmelowrs

6. Alm

7. Atky othar toOthIMO~ Whh “fluo,ld.

S. Any oth-r 8oothpmstm.

Nmuffiph brand=: Wfdoh bruid dou -- Moo m-t oft.n? Circle one &and cmfy.

Im. D-. .Iwo?t. In th. t.mlly now us. dl.tmrv fluorld. drops, tabl.ts, 08

vlt=mln Nuorld- =uppl*m@nu wbhh am Int.ndti to b ●8vallcmwDA,

●mm? ●eftorrm or ●* .Chook? 13 Yes ~ No [6) a DK (sJ

----- ---------------------- ------ ———-—-— ——----- ——-----— —— -—-—- . . ---- —- — —------—----—-—--

b. Who ID this?

Mark .“Fluorkdaauppfomonts”” %x id porwn.w twkumm s b. t a Pluerido supplements

------- ------------------------------------------------------- . - ——. - ——— —--—— —---—--—-—--

E. Anycma tin?

❑ Yes (Reask 6b ●nd c) = No

lml M1-,,sm,.x,,,M, ,S+, sl P.m. 10

33
Section O. DENTAL CARE PAGE, Continued

6-. Does ●nyone in tho family now use ● fluorld~ mouth

rinss which is Intondod NOT to be swdlowsd?

•l Yes ❑ No {7) ❑ 13K (7)

. ---------—-- - —-------- -- —————-- —--- -.

b. Who Isthis?

---——-

Mark “Fluoride mouth rinse” box in person’s cofumn.

------- --- —- ————- ———---— ——---------------- —- ——----- .----- ——-—.

c. Anyonn sise?

❑ Yes (R?ask 6b and.) ❑ No

--------------------- ~------ L---- ---------- ---____—______—_——.

Ask for each person with “Fluoride mouth rinse” In fib:

d. Does -- uscthis ffuorldg mouth rinsoat home, ● school,

or ●t work?

7a. Dental sealants●a plmtfc cosffngs usad to &avont tooth

drmayon tht chewing surfacesof tooth. Has ●nyormIn

tfw famlfy had dcntai sealantsplarmdon hiWirortooth?

❑ Yes ❑ No (8) ❑ DK {8)

—- ———--— ——----------------------- ----- ———----- —-------------—.

b. Who Is tltls?

Mark“Dantal aaak+ms” box in person’s WfUMn.

--------------------- ————- ——-— ———__ -— ------- ----—————————--——-

C. Anyorw ●lso?

“• Yes fReask 7b andc) ❑ No

SD. 1sthsm anyorwIntho family who has loatALLof his or hw

natural taeth?

•l Yes u No (9J —---------------------------- - —--—_____ -------- ----—————- -----

b. Who is this?

Mark “Lost sII teeth” box in person’s column.

-------------------- —------------------ -— -— ------------ -------

c. Anyone ●Isc?

Q Yes (Rassk 8b wrdc) ❑ No

Now I’m go”mgto zsk ●bout fluoridation.

HAND CARD 03.

9. As You undorstmtdft,which ono of the rossorwon this csrd

bsstdtsorfbm tftopurposo of public watw fluoridation?

1. To rsduos pollution

2. To Improw thotasteof water

3. TO reduca toothdSCSy

4. ?0 purIfy wmw

9. Don’t know Circleonemason only

Ask only if 3 circled irr 9.

10. 1sYOUR horn. drlrrkingwatorsupply ffuorfdatad?

.

--

6b.

---

--

d.

—

--

7b.

—-

Bb.

--

—

-

—

—

PERSON 1

----- _____ __________

I n Fluoride mouth rinse

-------- ____________

---------- ________ -.

1 ❑ Home

2 ❑ School

3 ❑ Work

------- ___________ -_

I ❑ Dents! sealmjts

-—--------- _________

1❑ Lost all teeth

12349

t •1Yes

20No

sDDK

w Hls-lwrxl Ilmel 11+s51 Pegs12
Part B questions

Thenextquestims are abcut receiving dental care.

C.1 Is there a particular dentist’s Officei dental clinic, w aw Jes Jez --yts JM Jzs

othsr pkp that uaual!y goes fcs dmtal care? _59 _3Q _riff _!M _9 @

C,2 FIN tcq~therf hcu Rallydiffafent plates [doyin/dofaaily MdlK5)

go for dmtal care?

places

W, lb ycddoasanycmein the fully) go to M crthodmtist?

_yc$ _rlo (C.4)

b, NIO is this?

Hark ‘orrhodcstia’ box in person’sco!tm. . _orhtomtia _tvthodontia _wthodont~t _orthodmtia _orfhodontia

c, PilycmWe?

-_Yes Lfeask CJ muc) __no

:sr::=::::::::::::::::::::a-:s:::s:s:::::-%2:::::::::s::s::::::::::G-::::::------ .-

--------x::==~==::~=:::=.: :=~=,:,:=:::=s:: ===:: =:-n:::::.-=::::: ::::

Skip ii resprndmt MS alone. Othmisc:

Askfor each fady mber W

Ilk Mlm nafxts to go to the dentist, whousually sates

thg appointment fw his/her? —— . ——

- Specify $ucify Speciiy Speciiy Specify

---------- ........................................... ..------------ ------------------------,=.xsx:x,=x---..-.-s:sz:s:=,=====.=,=:::=,,,:~::: 2----------------------------------------------------------------------------------- -------.

CQ5Ml needsto go the the dentist, hon dces hek$e

usu~ll y qet there? —. ——

Sptciiy Specify Specify SJccify Specify

Thr next questim are ahcut receiving dental cam

W&m 1 -F-3tw z F?Rx’43 im!m44 i&-Wi 5

35
.--------- —--—---------------------- ----------------------------------------------------------------------------------- -----— --—— --—-—--.---—- ----- ------—. ——--— —----- ——--——----- ———------- —---- ———-—--—--—

!LM WHIM

la, Curingthe 2 wedscutlinedin redm that calendar,beginning

f!onday(dat~jandendingthis pastSunday(date)did

anyrmein the fasily goto a dmtist? Includeall typesof dentists

suchas KthfXlCdi5t5, oralsurgem, andall othsrdentalspKialists~

as Al as dmtal hygimistst

_yes ~~(j]

---

- ~---.-———---— -----—.-.—-..— ———... —- —..

b. Mowasthis?

dark ‘derdai wit” box in person’s co!uan. _dentai _deda~ _ien t ai _dentd _o’entai

visit Yisit wit Pislt Pisit

-------—--- ——-— —-—-— ————- — ——.--..——

c, iluringthose2 m?eks,did anyoneelsein thefaailygoto a dentist?

__yes (fieask lb aad c) _ 20

--—----—— ---- ----—.-.---————.

,4Afor eachpersonxith ‘dentaiwit’ in fb:

— ——- -

d. Wringthose2 weeks,honmy tises did goto

the dentist? —-

nuzoer nusber nuaber nusber number

Thenextquestionsccocernvisits to thedentistthat you/andyour

faailyaayhaveaadein thepastyear, Tohelpyouremberpoesible

visitsI willreada list ofreascm%mepeqle havefw goingto

the dentist. h notansweras 1readthe list. It is just to jog

ycurmescry.

We paxqlegoto thedentistfof a check-upandto havetheir

teethcleaned,...

w to havea toothfilled w capped,..

!he gobecausetheyare in painK bKau5ea toothbrokeor

a filling fell wt...

We people go as part of a SwissK treatwts for gumdiwase,

a rmt canal,K to havefalse teethfitted...

hd soaegoas part of a seriesof wthodciitictreatmnts-to

havetheir twth straightened.

2a.Duringthe past 12maths,that is since(!?-sonthdate)a

yearago,honmy visits did saketo a dentist? (Include

the i’nusoerin M visit(s) youalreadytold ae about,) ---— ---- --— ----- .----

wsier rmsb~r nwber nuxber nusber

36
_Aw’# _?-iVet

_!heeh,

(0-alx!tfis

_6-#orith5,

(1-year

_6-soorn5,

(f-year

_6-sonths,

(1-year

_6-Iollths,

(1-year

_i-ye3r,

(Z-years

_l-year,

{J-years

_f-Year,

(Z-years

_?-years,

(5-year5

_2-years,

(S-years

_2-years,

{$-years

—-~-years,

(5-year5 I

‘1 _5+ years _5+ years _5+ years _5+ years

I _never _never _never _never

::=::::=:::::: ::::::::::::: :=:::::::::::: ::::::::::::::::::::::::::: :::::::::=: ::::::::::::::::::::::: :=:::::::: :::::::::::::::::::::::::::::::::::::::--

bWMRKPROBE

2c. For the questims that I have just asked ycu, it is very isportant

that M get the coat accurate ansuers possible. Smetioes it is hard

to reseW’ tiether a dental visit w bSfK@ or after (!2-sonrh daRJ

It can help to think of iefmtant events that happened about a year ago.

Fcr emple,this Eight be a birthday, a nea job, a holiday, or a

vacatim Mat i~ortant event happened almt a year ago?

!igecify

Thinking about(hferd :n 2c) I mld like you to anew the

questim again.

Ming the past 12 mmths, that is since (12-mi oaw)a

year ago, how eany visits did sake to a dmtist? —- ---- -—-

~ug~er rmber ;;~;r mber cwber

---.----—-—--——- ——-— —--—--— ——-—.—. -— -——— —-

::::=:::: :::::::::::::::::::::: ::: :::::::::::::::::::::::: :=:::::::::: ::::::::::::: ::::=::::= :::::::: :::::::::::::2 ::::::: :::::::::=:::::::=== : s=:::::==

tECfWOGUESSRWE

2c, Frr the queeticm that J just asked ym it is very iqmrtant to get -

the best amwrs w can. It can be hard to red.mr EMCTLYhat

happened since(fl-sonth date), Please think abcut the past year

again,

Non please anew the question again.

During the past 12 mths, that is since (H-sonth o’aie)a

year ago, hcu sany visits did sake to a dentist? -— .—

rwber mier }~~ber mher nu~ber

::::::: =:::::2: :::::::::: ::::::::::::::::::::: :::::::::::::::::::::::::: :::::::: =::::=:::::::::::::::: ::::::::: =::::::==::=::::::::=::::::::===:=:=:::=:

37
tie find that winy people do not kncu the reasons for sose of the

things that are done to the public drinking water,

&. Do you know the purpme of public teeter fluoridation?

— yes

rlo .—

— Jtbgr “

----- ——----------

Sfecify

~

M! W TMES?ii?13)! I)5NE0 WIRCIW

!3

* find that manypecqle do not know the reasm fcr sm of the

things that are addd to the public drinking nater,

3a. lb you km the purpose of adding fluoride to the mater?

yes -—

no ----

3!I. As you understand it, uhat is the purpose o+ adding {Iuwide to

the public drinking water?

W W lE.4 L47[GOHH

!Vevert: tooti decay, protect testh, -—

or r2hted re:ponse

Other -— --..-----.—~--—--

tpecify

h. As you understand it, Mat is the purpose of public water

38
Other ---- .-----—------—--

$peczfi

---- lx

3a.I%youunderstandit, that is thepurposeof addingfluorideto

thppublicdrinkingnatw?

MM’ IIEAf) C.4TEGfWES

__ Preventtoothdecay,twtectteetil,

orre!atedresponse

-_ Other-.—--—-----.-——

Speclfif

---

ix

@

0k?4H(UMFLIN!MTICN

HIGHBASEWE XCIRfENTWRDM

!!

‘h! eajorityof peoplelive in placesnhwetheuaterisflumidated.

k, Is WI?hoeedrinkinguatwsupplyflwidated?

—- w

---

—:

II

H16HPAX RATEI 0EFM5 HWDIN3

!4

Timujority of peoplelive in placeswhereflumidehasbeenadded

to the publicdrinkinginter.

3c,HasYOURhrmedrinkingtutwhadfluorideaddwltoit?

---- yes

--- fio

ok’ ---

g

LOMWE RPITEI MRENT NIRDM

39
1~

.—-------—-—------. -—.-- ———--—-------- —---—-—.

-—---------— —-. ------—-——--—----——-——----—

Nearly half of Inencans live in placesHITHMpublicWr

flwidation.

k. Is WRhmedrinkingnatersupplyfluoridated?

--

yes

--— w

-—-

ix

I&arlyhalf of all Aaericanslive in plaseethat haveNOTadded

fluorideto thepublicdrinkingwater.

3c,KasYllRhomdrinkingWr hadfluorideadddto it?

--- yes

no ---

— 0{ -.

&

NOWEIWEMNIFIJLATION XCURRENT NORM&

!3

——— ——-—

X. Is YEWhmedrinkingwatersupplyflwidatd?

--- yes

—

--- :“

fj

NOW RATEM!tiIFWATIC$i I DEN&IiUU31W

!6

3c. HasVIMhoeedrinkingwaterhadfluorideaddedto it?

---- yes

-- no

-— M

EmN 1 FRscti2 PERm3 FERSON4 PER!iw5

Ncuw aregoingto ask about somethings that people maybe doingto

takecareof their teeth.

4a, Doesanyonein the familymmusetoothpaste?
y~~ ad h ..—. ---- -----

.- ---------------------------------------------------------------- -----—-- —.--------- —-------—---—----—— ——— —---

b. MOis this?

Ilari ‘tooritpaste~Dox2n person’s cdum. _toothpasce _toothpaste _toothpasie .-tooth?asts _ruothpaste

c. Anyuwme?

--- yesMedsk4band:!

---- M

-------- .-.- .. ...-.. -. —-- —-.. —— -———-——-.---—-----——---——------——————

Ask for each person #iti “toothpaste’ in O:

d, Rxs _ nouw anyU&of thosebrands?

MM olfw 02

1. colgatP

2, mat

3. ma

40Fquafreah

5. Haclem

6, Aia

7, ihyothwtoothpasteadthflumide

E,klyothartoothpaats.

lf adtipie brands,:Michbranddora m mat Oftan?

1 1

2 2

3 3

4 4

5 5

b b

7 7

a 8

1

2

3

4

5

b

7

8

Circh oniy one brand.

41
Sowetiaespeopleuwfluwideto protecttheir testh, Fw exampleme

south rimes ccntainfluoride,(lthwsdonot.

5a. DIManymein the faaily ncu use a south rime or amth WA that

has fluoride in it? ~is cmld be either at hose w at echml or

at wk.

yes no w“““ --_/K (6) -— —

-—---- ---—. —-—-— -—------ —-. -—--— —-—— - ---—

b. Mo is this?

M “fiucride Iouth rinsd’ box In person’s cdum, _{lg~ide _flucride _fiuKide _fluoride _flwids

muih rinse acuth rinse muth rinse emthrinw muth rinw

--- ..—

-. —---—---— -—----. -.-- —- ——- —-. ..-—--—- —-

AW for each person tiith ‘fiwiie Iofith rjnsea jn 5b:

d. Iiiwe dma I& this fluoride wth rinse: at hme,

at schml ,K at wxk? _hoae Jose --hose _hote _hoae

_s~hooj

._school _school _schooJ _schooJ

_wark _wk _wk _hwk _iwrl

Soaetiees peqle take pills with fluoride in thea,

6a, Dces anyone in the family take either vitwins nith fluoride in

themw otherkindsof tablets, pills, w dropsthat havefluoride

in thee?

yes no (ii W ~i) -— -— ----

-—-—-—-—.—— —--..——---———-—---- -—

b. Iho is this?

-.

(Hark “fluoria’e sup@ewntsU box in person’s CG!WI. _fAforide _fiuoride _f!u~rjde _ffuorzde _fluoride

sup@eIerits sup@Berit5 supplements suppiesents supphwents

----— - ——--.-.---— .. —-—-------—-- — -—---— -—-—-

c. ilnyme eke?

_yes (MIsk ib and cl no -—

SKTK!N2.2 (ECUIMIS43QI.ESTIONS)

Fm)Nl m2 PERWN3 EF#ON4 F’ERSiMs

—— —-——— —-—. -——w— _______ .—— ——. — - -——— —— ———— —-

4a. Does anyone in the faaily non use tmthpaste?

_yes __no i)h -—

42
----------- ---. ----—-—-——----——-—-— .

c. Anyoneelse?

_yes O?eask4bandc) __t!o

.-.. -..--. ---—.-—- ———--—--

M foreachpersonwith“toothpaste’ !64:

d,Qoee nonwe anyCK ofthosebrands?

HMOCARD02

L C!llqat@

2. bat

3sMa

4. fquafreah

5, Hacleans

b. M

7. hy othertoothpae+te tith flwide

8. ltiyothertmthpaste,

If Wipie brands:Michbranddw _ w mat often?

1 1 1

2 2 2

3 3 3

4 4 4

5 5 5

b 6 b

7 7 7

E 8 B

1

2

3

4

5

b

7

ti

Circleonlyme iwaad.

43
ffi:;* J

:!%!!534? W5E’i 5 ~*~ ~ ;%!f!$i[*5

=-=s.—.—.a=.s~ee= —-—--. —e—.— —-—————————— .—-——

5a, Does any one in the faeily non use a math rinse?

.

—.

ves _ ho (5) _M (6)

--—— .—---———-—- -———-—

c. Ihyoneelse?

Jes Ueask5bwd ci _-no

Askfor eachpersonNitb%outhrinser in 5iI:

d, Ibesthe emthrinsethat uses cmtain fluwide?

Hark‘coRiaiasfluoride” boxin person’scohn. _co9tains _co8tains _corftains _corttains _contairfs

fluoride fiuoride fjuorjde fluoride fluorzde _K _iK _lX _iK _M .-””

e. Mere dw usethis fluoridesouthrinse at hw+,

at echool,orat mrk? _hoIe _hoIE _hose _hoae _ho]e

_school _sciloof _school _school _school

_wrk _wk _wri _wrk _wk

ml PEiwN2 F’RW3 PERW4 m5

6a.Ooesanymein the faeilynwtakevitasinswithfluoridein thee

either at hoesw at schml?

Jes _no {6dl _oi w

—--———-—— —— ——

b. ldIois this?

Hark‘fhiorzde Ares” boxin person’scoium. _fluoride _fluoriie _fluoride _fJuoride _fluoride

;Nalins vitaains rita8ins Fitazins vitasins

c. hyrmeelse?

_yes (Reask ib and c) _no

d. k &nymein the fasiiy rw take any other kind of flwide drqs,

pills K tablets?

yes noii) -— — -_N (?j

e. MOis this?

i(ark ‘other fluoride su;piesent’ boxiri person’scoiuan. _otiier _2tiw _cthK _other _oth2r

fhmie fM!fid2 ilu@Lce {Mice fluor!de

44
-:-:::::-_-::g !=m;~q!-----::::-_::---_:-:a?%?t ~J@~,~

- ---------------------------------------------------------- —.—

f, hnyme else?

UIWNT WR51tkS

Pwm 1 FEMN 2 F!ZWi3 FIERWi4 F.&%i~

=-&-AG-n—a--Gw—- —.—————===--—----——————— —--

4a,Meaanymein the faaiily non use toothpaste?

--w __n~ _-_~~

..--. ---.——--— ----------------- —--------------- —— --—--

b. MOis this?

ffarkVocr@asteJboxin persori’scciwn, _tmthpas:e _toothpaste _toothpaAe _toothpaste _tooihpaste

-.-..-- —--— ---. --—_____ —____________________ ——

c, hnyoneelse?

_-_ye: Wea3kMaiidc) --_no

.---- —----------------------- .-_---- —--——-

Askfor each person#irh “toothpaste”ZB4b:

d, O(W ncwuseanyIX ofthosebrands?

HMOL411DF2

L Colgate

2, CM

3, 61ee9

4. fquafresh

5. Hacleans

b, his

7, lhyothertoothpasteNith{Wide

Il.(wyOthK tmthpaste.

If dti@e brands: Michbrand dma U* mat often?

1

2

3

4

5

b

7

E

1

2

3

4

5

b

1

s

1

2

3

4

5

b

7

e

1

2

3

4

5

b

7

i!

Circie oniy one brand.

45
5a, Doesanymein the faeilynonusea fluoridemuth rinse

tiich is intmdedMITto bewallwed?

——--—----- —-—--—---—-——-—----— —-----—-—-—— -——---—---. ---— ------ —----

b. Mois this?

HariYiwride IIWII rim’ OGXin person : COW, . _flucrllie _f1uwid2 _fluorlde _flucride _flucr:(ie

mouthrinsa muthrinse moutiirnse sdh rme wtk rme

c. hyme else?

_ye5 Reask5iJandc) _-_no

M for eachperso?M ‘fkorlde south rin:e” iri 5b:

d, Ooee usethis fluorideecuthrinss at how, at school,

or at lmrk? _fiose _ho#e _hoae _hote _hoae

_schmi _Xhool _5ciooi _school _schoo!

_uork _wrk --xwk _x’ork _#ork

ha, Doeeanytmein the fmily mmw dietary fluoride drQ5, tablets,

or vitamin fluoride wppleamtstiich are intendedto beW1OWA,

eitherat hcmeor at school?

—yes — no(71_Jwi (7J

—-—— - --—— —— -—--— ---—-—

c. ilnyone eke?

_yes Lfeaskib ard c) —- 20

CMTALSEPLI!ATS

HIGHEWRATEXCK”KNTI!EFI}!ITIDN,

:! mfm~ ,b,w-,I E..’9?32 PmifN 3 PEWS4 ~~;$~~

—--__--____----—— --------------------------------------------------- --—. —---— —-—- —-------——-—

NWI ae goingto ask’youabut a procedure that has becme

46
sorepcqularrecently.

7a,Dentalsealantsareplastic coatingsusedto preventoothdecay

onthe chwingsurfacesof the teeth. Hasanymein the familyhad

dentalsealantsplacedonhis/herteeth?

yes -----noi:) j~ {&

---- .----

------------ .---. ——--------— -—-------------- —------ ——-—— —— -—----—---——

b, Hho is this?

~ar~‘dentaise;]ants~hut in person“Sco!wn. _kmi _kntai _05mtal _i)mtal _Deniai

5Edant SMiailt sealant sealant SMi Uli

c. t%yoaeelse?

_yes Neasi 7b and c) _no

*

HENBME MITE 1 PZTTERJEFINITWL

!4

F’ER?lM! ImilN? PERW3 PER33N4 PEREUN5 —--..-..—.———-———— ------———-—————-—-————

la.

Non I ah going to ask ym about a praedure that has bsa!ee

we popular rwtly.

Dental sealants can be used to prevmt tooth dKW. TheY are put m by

a dmtist w a dmtal hygimist. Theyare diffKMt frcm fillings, caps,

crowns, and flmride treatwnts. Dental sealants are special plastic

coatings that are put m the tops of the back teeth to keep thee free

decaying, NM anyme in the faeily had dmtal sealants placed

m his/her teeth?

Yes __no (8} ---, _M W

----------- ---------—.---———--——

b, MO is this?

-——————-————-----—-

tiark “dental seaimtd box in person’sccifm. _&ntal J2ntai _:#~ _iiental _!kniai

*~iUlt ;ed ark ;MM sealant s.ealam

-------- ----——-— —-.. —--------—- —-—— —--—- ———

c. (syone else?

----- yes [i’east 7b and c1 no ---

@

KMMURATE1CLWNTDEFINITION

I&n I aa going to ask ycu abcut a new procedure that is rarelyused.

7a. Dentalsealantsareplastic coatingsusedto prevmttooth dKay

m the chwing surfaces of the teeth. Has anyme in the faeily had

dmtal sealants placed m his/her teeth?

_yes _no [8) _M (w

—-------——-- —— ————.

b, Mo is this?

.—.—

Hark “dentaisealants”boxin person’smium. _kntal _Eental _k.i2i _iMal _iMai

sealant seakn: sealant ssala7t 5ealant

---------------------------------------------------------------- -—

c. Mycme else?

__ye5 :t?east Tb 3na c) --— no

47
----. -----—-—-------c ---———-. -——-.

-—----- ——_-—-----————-----— -——-— — ------------

- —-

——— .———

Ncu I u g&ng to ask ycu abmt a new prKedure that is rarely med.

7a, Dental walants can be used tD prevent trmth dKay, They are put m by

a dmtist w a dmtal hygimist.Theyarediffermtfrm fillings,caps,

crcms,andfkuride treatmts. Dentalsealantsarespecialplastic

coatingsthat are put cm the tops of the back teeth to keep thee from

dKaying. Has anyone in the faeily had dmtal sealants placed

on his/hK teeth?

_yes _ no w“-’” _---N N)

——-—-— ———-— ——.-__-_—_— ___

b, ho is this?

——— ——-— ---

ihri %’ental seaiants’ box in person’s colum, _Dentai _Emtal _Mal _l)mtal J)ental

eealant sealant sealant sealant e.ealant

c. Anyone eke?

__yes O?easkib and c) -_no .,

7a. kIItal sealants are pl@iC COiitingS used to prevmt tooth dKay

m the chewing mrfacea of the teeth. Has anyme in the faaily had

dmtal sealants placed m his/har teeth?

--_-y~s _rlo (w _uK (:)

—----—-.—————— —--. —-— ----: —--—-—_________

b, Mo is this?

——-—--—.

ifark ‘dentai ssa)ants” box in pe?son’s cohn. _lkntd _Mntal _Dental _Dental _Oental

Slant seaiant sealant 5ealant 5ealant

-—------————--— --- —-—-——e—--— __________ —.————-—-

c, &lyme eke?

—- yes {tkasi ib and c) -_no

d

W EM WI PMUW4ATIUNX KTTERDEFINITION

7a, Oental sealants can be wed to prevent troth dKay. They are put m by

a dentist K a dmtal hygienist, They are differmt froa fillings, caps,

crams, and flouride treatemts. Oental sealants are spKial plastic

coatings that are put m the tqs of the back teeth to keep th~ free

dKaying, Has anyme in the faeily had dmtal sealants placed

m his/her teeth?

yes 50 (~,i -_--~~ {~)

-—— —,,

——--------- —---------— --—___ ——— ——.. —___________

b, Mo is this?
c, ihyme else?

-.._-Yes W?ask7b andd - -— M

49
Appendix II

Field pretest of the

dental health

supplements to the

1986 National Health

Interview Survey

Part C field pretest questionnaire–

standard version

...”-..”.. . . --...--..=- h.- m“=

Now I.rn SIOIIVJ to -.k vou .-mm qu=.t;o”. about WATER FLUORIDATION.

1. A. you und.,.l.nd It, what 1. th8 PURPOSE of pub:lc WATER

FLUORIDATION?

Do not read ●nswer categories. circle the ONE that best fits respondent% ●ns wer.

1. 1 ’37 s

1.Prmmnt tooth decay. proxact tenth, or related rasponse

8. Some other reaatm fSpccifyJ

S. Oon-t know

2-. Is your horn. dtlnkitig water SUPPIV p.ra of - PUBLIC

wmt.r systm tn. ar 18 It from. bv.11, .PAw. or d.t.m? 2a.

~

1 0 Public wmer system

D ❑ Othar

sDOK

------------- ———--—-—-—-— —— y—-—-—--- ———-—-------- ——— —---- ——-— — ——- —_______________

b. 1= YOUR horn. drinking wstor supply FLUORIOAYEO? la. 1 n Y..

>

K

2UNC.

o=OK

HAND CALENDAR.

Thmsm tmxt qummtlonm 8.. ●bout MOSIUJWI dmtd au=.

3=. Dwhtg tho 2 w-ok, (outlhmd III r-don thst calond+,

b-ehtnfns MOndaY wmnd ~dhtg Chls past Sundaw

fc!ZISL dld ● nwn. In tho family 00 to ● dat118t? Itieludo all

typmm of d-.tist., .ueh ●s orthodont1s18. Oral sur@80n8.

-“d ●II eth., d-”~.l .p-d.tkt,... w*I1 m. d.”t.l

hygl-nal.ti. ❑ Yes ❑ No (4)

----- — --------------- --- —----- --------------------------------

--- -— — _____________

b. Who w.= thi.? ., K

Mark .-D.mal visit,. box In paman.s column.

ab. 1n Oontal VIBIC

------ --- ______________________________________________________ ___ _____

=. DurlwI the.- 2 w-.ks. dkJ .t,y~tlm ●t.- 1“ th.

im’nlly *- no . &dm? = Ycw iffensk 3b ●rid.) a No

--- —- —----- —_____ —— ______ ______________________________ ___ ________________ _ -

Ask for each parmm with ‘-Dent81 vNkeo[n 3b:

d. Durlns the,. 2 wok.. how rnmty wnw. dtd -- se to.d0rtwst7 d. = N“.buo,t,.e~

Mm;k box if tmd,r IWO ym,rs aid. . SS8D Under 2 JNP) ]11-13

4m Ouriosg lh. p.st 12 tnottthm {thin 1’, zlno. {12.month dat

U“dId.&*fNumb

@ ● year ●go).

hew tn. ” wilts dld _ - rnmk. to . d.”tl.,

48.

win 3dJ i?ks:t[s}you _lrmmdy total rnm ●bout.)

visits

oo~ No”.

______________________________________________________________ ___ __________

Mark .%2-w..k dw)td “[s[t,. box in Porson”a column If “hltiaJ ,eported in Sd. w

b. HL9W Im”g ha. W b..” .1”=* -- LASrw.”t ,- ● d.”ti,,? I ❑ Past 2 winks not rapoflad

b.

(Mark 3b, ●sk 3d)

2 ❑ 2-wink dmmmlvblt

s a Dvmev:t~=eeb, lass th,n

● o e months. l*sa tlmn 1 y.,r

s n 3 v..,. lass than 2 v..,*

6 n 2 yaw=. 19.8 rh.n 6 y..rs

70 S vmms or mm.

o D Never

PI

tClOnly 1 vlalt (5a) &

Refer to 4a.

P1 2 ❑ Mom than 1 Vhlt ft$b)

6.

[

s ❑ Other (NP)

a. What wm= th. REAsON -- last want ec! thm d&kti.t7 1

~

b. What Was tfm REASON -- boswI th swiss of dontml vIsI1*? 8. 12S4

BYL

Do no; read answer Cat0gOr&6. c[rch all that apply.

1. To mm my teeth cleaned

2. Went In fors check-up or ●amin-fro” o. my owm

S. Was call=d In for 8 .heck-”p or ●xmminmthmbvth. demist

(speww

4. Sm’nethl”g was wrong, bothering or hurting me

S. Other

‘OOTNOTES

.-.. .... .,.. .”. ...-. .- . . .

Ig* 4 — “.- .!---- ,.. -, ,,.. -,
I

Section ---- .—. P. f2ENTAL HE AI-TU PAGE _ Cn.litlugtj I PERSON 1 - - ——------- .. . .. . . . . ---- . . .....

3A Is there nnyons In tha fnmlly who h=, lost ALL of hi. or h.r

ttaturaftc@th7 Q Yes ❑ No (7J

b. Who ISthis? '------------------------------------------------------------- ‘- “--;;-.:;;::E-----E 6b.

Mark“’Lostall teeth”’ box in person’s column.

--------------------------------------------------------------

---- . ____________________

c. Anyonm dse?

❑ Yes (Reask 6b and C) ❑ No

Refer to 6b.

~

P2

1•l “lostall teeth”

P2

markedin6b (NPJ

s D Other [W’)

,

Do nocaskforjwrsons with “’LostM teeth”’marked h P2

~

7. (Now Iam#olngto askabo@whatpamon8 lnthofamllyus*wkntiy bm;btidr Wsti.)

What doos -- usowhen brushing -- tooth?

7. 123456839

Ifmorethen orrccategory, ssk: Whiohhm -- us-d mostoften“duringthepbsl 2 weeks?

{

Colgato 2,

{

Closo.up

{

o Donquol 6. TOPOI

1. Crest Ultra-ffrito s.

Sonsodyno

Alm 3: Popsodani Protmct 8. Otberproducw @pscify)

Aquafrcsh Promise 0. Don”t know

4. Cheek-up

Ea.ftmxntly, somo NtOUTHRlNSE8 hmsabeM davclopodthat

contain FLUORIDE to r.duco toothdecay.Dom anyormIn

tho family now usa ● mouthrtnsmthm eontaincFLUORIOE,

suah m ACT, Fluorigard,Lktermint with Flurrrldo,

StanCare.or ● .Imilar product? •1 Yes ❑ N. (P3) ❑ .lDKfP3J

----- -------------------- ---------------------------- -—----——- .-

---- ______________

b, Who k this?

hfsrk *‘Phnwfdemeurhrfnse.’boxh parson’s coi”nm.

8b.

,0 FfUcrrid,nrouthd”selw

-------- ----- —- ——-------- -------- —- —- —-- —- —--- ——-------- ,———--— . —-. —--- ______ —_________

e. Anyono ●kB?

❑ Yes (Reesk S6 #nd c) ❑ No

------- -------- —----------- ———---— — —--------- ---------------- -

---- _______________

Ask 8dandg fo+each &rwrconw/th ‘°Fhroridemouthrhtd’ kr 8b: t ❑ Home P

d. Do08 -- uss thk ffuorldomouthrfns- at homa, ● school.or ●t work? d. 2 ❑ School

3 n Work

--- ——_____ ------ ______ __________ ______ _____________________ ----

s. What k thonamoof themouthrktm? ‘-.● ----------------E

Nsme

S9❑ DK

P3 I

26

Rofw tO age. P3 t 0 Under17 (91

20 17ertdovar (NP)

0. Doss -- now us. FLUORIDE tabkfs, drops,or FLUORfOEvitamin s. 27

●rppkmenm whkh sre Intamkd to bs SWALLOWED? 1•1 Yes

2DN0

sDDK

On,Ths chowlng surfmxs of toothmay becoatedwithplastic

dental●oalanfe to pravwtttooth dermy.T’haswsgzkrrta●e

NOT fllIhrg8,caps,or crowns. Has●nyenoin ths fsmily

had phetfc dentefSEALANTS ●pplledto ths taoth? •l Yes D No

❑ DK )

(Section0)

------ ______ ________________________ ___________ - ___ _

b. Who iSthfs?

Met&“Oe;tblsealants” box in person’s c~umn.

1Ob. m I ❑ Oemaisealants

-- —____________________ __ _____ ___ __, - ___

o.Anyone ●fm?

❑ Yes (Reask 10b●ndC) D No

20TNOTES

.- .- ....- .,-. ”.....- .. ----

51
Part C field pretest questionnaire–experimental version

I m 69

Section P. DENTAL HEALTH PAGE PERSON 1 I

3-4

18. As you under’ctand it, whet k thm pu&m”of mfdlrig fhmrlde to tha publk drlnkkg wstsr? 1 ●.

~

Do not read answer categories. circle the ONE that best fits respondent’s ●nswar.

1

87 9

1. ,Prevant tooth decay, protect tenth, m related raxponsa

8. Sores other reaeon

9. Don’t know

(Specify)

— - — - — -------------- .---— ------- —---- ——— ———--------- ———-— ——— ——. --- -—----—-----—-—- ---

w

b. Dots ths wmtar tfmtyou drkrk athomo come from ● puhfic water svm,m or 1sIt from ●notfwr b. I •l Public water cystem

,Oureo, suck ●s ● Wsll? 8 ❑ Othsr 90UWS (2J

----------------------- L -------------------------------------- --

- ----------------m

o. Hs8 this publlc wmor+upply h-d flourhk ~ddod to It? o. 1 •1 Yss

ZONO

sODK

~

Ths nmtf quostlons ●m about rscskring dsntsl c-m.

Mark box /f under two years old. o ❑ Under 2 (NP)

2=. k therms patikukr dentist% offlos, dsntsl ollnlc, or sores othst pfsos that -- 2C. 1 c1 Yes

usually goos for dwitcl car.?

}

(NP)

2DN0

—— _____ ------------------------------ ------------------------- --

- ----------------m

b. Aftogotfwr, how many DIFFERENT PLACES do family mombsrs go fos dsntal cars? b.

Pkces

3a. Do*a ●nyc+m In tfm family go 10 in orthodontist? o Ycs ❑ No (4)

_ --------------------- .----—--- ------- — ---------- ------------ ---- ---------------—

b. Who isthk?

Kz

3b.

Mark “Orthodontia’” box in person-a oofumn.

I ❑ Orthodontk

——--— —- ——— ———-—---- ——-— —— —--------- ----- ----——--————————————— --— - —-------------- ----- ,

c. Anyone dse? . ❑ Yos ff?eaak3b sndo) ❑ No

-=

If mzpondent Iiva8 done go to 6. 0 n Undw 2 INP}

Mark box if under t wo yaara old.

4s. Whmr -- needs to go t6 Um dmtkt, who ucuslly makas tho sppolnbrwnt fnr --? 4C.

. . (sp#cIfyJ

—— —___________________________ ______ ----- _______ _____________

‘- -----’-””--------=

b. Whm -- nssdc to go ttw tho dsntist, how dws -- usually sst lha,r=? b.

fSPecWJ

HAND CALSNLJAR

58. Durfrtg the”2 wasks (outllnsd In rod on thst cslandm),

bsghming Monday date and ●riding this past Sunday date),

did ●tyorm in ths fsmw y go to ● dsntkt? Inoluds ●ll types-% o

dettllata, such ss orthodontkfs, oral surgeons, ●nd ●ll otfwr

dmtal SYddkt& ● t Wen ss dsntsl hysknkta. Cl Yes D No f6J

—---- ______________________ _______ _____________________ ___ __L. -— —— -- ---

b. Who W=B tflk? -L-K

Mark “Dental vlaft” box in paraon”s mluj-m. 6b . 1❑ Dental visit

— -- —-—______ ________ ----- ____________________ _____ ----- ______ ___ ___ > _________ --

c. During those 2 woaks, did onyons slaa [n tho family go to

● dentist?

❑ Yes(Reosk 5b wrdc) ❑ No

----- _- —— ____ ________ ______ ________ _______________________ ___ ___ -

---------------H

Ask for each psrson with “Dsntal vlsft”” In 5b. ‘

d. Durfng thoss 2 weeks, how msny tim as did -- go tmthe dmtIat? d . ~Numbarofthrs

‘OOTNOTES

. .

1
,. Section P. DENTAL HEALTH PAGE - Continued I ~pERsoH1

TIM nextquistiorrsa~orn vfsltato tin dmttlstthot●nyosiafrrtfmfwrrffymeyhsvs mmfskitfrapastyomr.

To helpyou remembarpoesibfavisfts1wiltmad ● listof masonssoma Psoplohavafor gofngto thod.ntist.

~

‘Do not ●nswer●s I red * ilst.It lsJssO,ttoJoffyorsr msmory

. .

Somo parrplogo w“thadmtlat for n chaokwp●nd to hsvothefrtaadrclesned,

or to hSW@ s tooth fiiklf or capped -

Somo go baofiso they●eln pefrsor becauaa● tooth brokoor o hillrrgfallout -

8omo peopfe~ ospertofa eerioaoftmetmmt~ for gurri.dfeeeso,● root CMd, OStO h~~fa~ ts~ fi~ -

~d some 00 m P* of ● sad., of ssrthrrdontic.tra&nents. to ~ve tfrdr,taathetrelghtenuf.

Miri box if under two yam old. 9980 Under 2 (fVP)

O& Dhring the pebt12 months, (that IsArcs 112-month A@ s yOW ●go), how lllwsy vIsftS dfd -- meka to &. 000 ❑ NQne

6 dentist?Onoltrchthd~Mmrber’hr5dJvfdt(s) you dr~dytqld M* s~@.1 .

wits

. .

.——___— ——_______ L __________________________ ___ _________________ _ _,

&f#r~q w&k dentai VrsrtrO’box 12?- /nprrrson;srilumn if vuitfs~repartedrn i%i.’.

b. When dld --

1❑ f%8t 2 weeks IIOt 10p0rU3d

LAST 90 to’m darttfst? fMark 5b; ask 5d)

If f@pOt%f#ntmtM& pwnth endyaar snd NOTan lntervaf, fflr6Cand mark f3b.

2 ❑ 2-week dantal visit

. . . 3 •l Over 2 weeks, iess

. .

than 6 months “.

b. 4 ~ 6 months, (ass than

}

(6C)

h year.

6 •l 1 year, iess than

2 years

a ❑ 2 years, less than

6 years

7 D 6 years or mom

}

(NPI

O ❑ Nwer

----——— . —____________________________________________ ___ __ _________ ___,

L

c. What vies th~month ●nd yoerof fhievisit?

. . .

.“C.

— -—

Month , Yaar

. . .

.-

PI

~

.Refs( to 6b for all parsons.

I ❑ 2+ yaars or NEVER in 6b

PI for all parsons (100)

s ❑ Other (7)

.,. . . . .

.,

.’. .

. .

ga8 .

. ~ HIS-1(S5).%3(1SW 17-31401

53
Section P. DENTAL HEALTH PAGE - Continued

For the questions that ! haw Justaskedyou, it IBvary important thatW* gatthamostaccurate

●nswerspossible.We know thatNis ●omollmashard to rsmamborwhmthw● vlsltwasbaforo or ●ftw

(l Z-month d8@. We hrwofound th,t it hnlpa.mma P*OPI*to lhink of Importantwent- thath~ppanad

●bout a year●go. For ●xamplo. this might ba = bfrthday,8 n@wJob,● holldsy,or ● vacation.

‘. What Importanteventhapponod ●bout ● ycm ago?

fSpec;fyl

I

P2 Referto 6aand b.

Mark first applicable box.

VW mpotio(f that -- mad- “() ~sI@ ,Inca lz.~~nth (f~~a ,

la. Wharryou think about (Event in 7/do you ramomber●nyvlsftathat -- mado sine. (l Z-month dare).

--------- _____________ ________________ ___________________

b. How many?

---------- ----------------------------------------------------

YOU ro~cf that -- mtda (fJumb~~’Jn6~) “Islt. •ln~ fz.~~nfh ~ate)m

o. Thlnklng about (Event In 7) dld mtyof the (Number In 6aJvlsltethat -- mad. @oourb,f~r. (lz.m~nth dater

-- ——_____________________ _____________________________________

d. How mmny?

----—-—- -—___ _____ ______ ______ _______________________________

. . When you think ●bout (Evbt in 7} do you rmmembarWW●dditionalwlsitathat -- madeslnoe(lZ-month date).

—-_______—_ ___________ ______ __ - ____________

f. How many?

HANDCARDPI

9a. Which of tfwtromtnmntaon tfrh card did -- WO.1”. ●t -- l~~td.n~l “i~t?

Mark allthat gpply. Follow first ●pplicable skip.

A. Teeth straightened, that is, orthodontia.

B. Tenthcleaned

C. An x-ray taken

D. An examination or check-up

E. A fluoride treatment

F. Tooth Sfled

G. Treatmant for gums

H. Work done on a crown or cap

L Work done on e rootcanal

J. Work done on a bridge

K. Work done ons partial danture

2., Work done on ● oompleta danture

M. A tooth pulled

N. Other oral surgery

0. Othar treatment fSpac/fyJ

--- —-- —___—________________ -- ________________ ______

b. w- -- “Otffiad by tha d.”t,f.t-, ~ffl=~ that It w.* tlm. f~~ . &.&+p Or .j(.mln.tf~”?

---------- ----- ________ ______ ____________________________ ____

ci. Did -- gojuet to get fTreatment E-E in 9u)or d[d -- go baoaua- sonwthkrg was botharfngor hwtfng -- ?

------------------ ------- ______ ______________________ ________

{Some pxople go to thedwttlstbacmtaathoythfnk tfmttheymfghthave●probfcm;otfwrpaopfogo

to ths darttht for M ●xamlnatfonor a ofraok-up●nd thedontfetdiasovorstbettheyhava● problam.)

HAND CARD P2

d. Which of thtso beetdoaorkboshow -- s~d Up getthtg (TreMrrrent F-O in 981?

Thought there might bee problem before seeing dentist (self)

Dentist discovered the problam (dentist)

lge 8

2

1.

.

L

..

c.

-.

d.

--

a.

.-

f.

m.

b

,-

a

.-

d

PERSON 1

1❑ 2+ yaers in 6b fNPJ ~

L❑ No viaita in 6a (8a)

I a 1 + visits In 6a (8c)

I •l Yes

t❑ No INP)

.--------------E

f9aJ

Number

---------------w I ❑ Yca

2 D NO fga)

---------------E

Number

---------------E

1 ❑ Yes

z ❑ No (9a)

----------------EE

❑ A (NP)

❑ B

❑ C

}

(9b)

1%

❑07

(Qd)

(Specify]

.——____ -— —----- - ---

k

I ❑ Yrra INP)

20No

01U2)K

----------------

m

1•l Routine cara only

2 ❑ Problem oofy

3 ❑ Both

}

[NP)

en DK

‘:~-;e;f----------F

2 ❑ Dentist

a •l Other ~

K3racW

Fo6Mtw.lumx3 {well {7.31
Section P. DENTAL HEALTH PAGE - Continued PERSON 1

10s. Is ;hem anyoneIntfmfamilywho hasfostALL of his

or her naturalteeth? ❑ Yes ❑ No (11) ❑ DK (71)

----- ———————__ —- —————--—-__— ———~----- —- ———--------- ——--— ——-— —- - -- -—-——-—————--——— - -

b. Who lathis? 10b. LE

Mark ‘lost allteeth” box inperson’s column.

I ❑ Lostallteeth

----- —— —-- ---------------------------------------------------- -- ——-———--————————————

o. Anyonmdea? ❑ Yes (Reaaklobandc) ❑ No

~

P3

In “Loetelltaeth’’m arked

Refqr to 10b. P3 M 10b (NPl

ED Other (fVP)

Do not ask for persons with “Lost all teeth” kr P3. ~

11●. {Now I ●m gohig 10askaboutsomathingsthat IWOPICmaybedoing to takecarsof iholr teeth.)

I ❑ Toothpaste

118. 8❑ Other ~

whstd~-s -- use~he” -- brurhoa -– t-ash — toothpaste,toothpowder,or sc.mathlngelsa?

(Speclfv)

--------------------------------------------------------------

-— ——-—-—————— >— ---- ——-

Ask foreechperson with “Toothpsata” in 11s. U&!

b. What b,a~ doom-- US-7 b. IZ34668~g

Mrnore than one category, ask: WIMr has-- usedmoxtoftottdurfngthopasttwo waako?

{

Colgmte

2“ {

C[oeo-up

.{

Dwufuef 6. Topof

1. Cra*t Ultra-Brlto

s.

Smtmcfwm

Alm S. Pop90dattt Prot9ct

8. Ottmr

Aqrmfmsh 4. Chackup Promise 9. Don’t know (Specify)

SommfmaspxoplrrUS-fluorldoto protecttfwlrteeth.For●xampfasomomouthrfnsm eontaln

fluorfdo, otharxdo not.

12=. Schools ●nd work placesmayh=vofluorldo mo&h rlnea .

programs.Does MYOIWInthe famlfySxkopm’sInsuah●

program? ❑ Yea ❑ No (12e) ❑ DK (12eJ

-------- ------------------------------------------------------ —

b. Who Isthla?

—- ——--—__— ———_-—--

12b. -lzz

Mark “Prouram8’ box[nperson’s column,

1•l Program

------ —--- --- —_--- ——----— _--- ———---- ——— ———----—_ —-—----—- —-——- ——- —————-———-—————-

Ask for each personwith“program’”in 12b: c. -E!z

c. ISthlx ●t schoolor ●t work?

1❑ School

ZD Work

--------- ——— —------— -_ ———---— __ —---- —--- —_ —- ——--— ——-— —— —---—-_ -—- -— ——— ——— ——— —— -------

d. InMyorw ●IxnIn ● mouth rfrmoprogram? ❑ Yea (Reesk?2b,c,andd) ❑ No

----- -— -—-— _—— ------.& ____ ————--,— -—-- ——___—- ———--—_ —__ —__ —_-__—

Somotlmeefluorido mouth rlnxasxtouead●t home.

——- ———————————————--——-

. . DOOD●nyone Inthofamifynow us- ● mouth rfnxsor

mouth waah that has fluotfdein lt? ❑ Yes ❑ k (P4J ❑ No (P4 )

------- L ------------------------------------------------------ --- -— ——----— ——— —--—

f. Who h this? f. E

Mark ‘“FluorJdomouthrlnea”boxinperson’s column. 1•l Fluoride mouth rinse

-——------——— ----- -— —_____ -—-___ -- ——--— —--- _———_ ———————-- ———--

g, Anyorm den?

—————————————--—-—--—-

❑ Yes fReaak 72f●ndg) ❑ No

----- ----—— -—-- ——------ ---------- ——--— ——----- ———-— —----- ———--- -—-

Aak for sash person with “Fluoride mouth rinsa” In 72f:

——-—-— ————- ———————-.

h. Wh=t Isthenams of ths fiuorldemouth rfrwathat -- usae? h.

Nsme E

P4

~

Refer to age. P4 I❑ Urrder17 (13)

2017 andover (NP)

Som-tfmas doatoreor dxntistepraacrlbaplllaor dropswfthfluorldeintfwm.

1Ss. Dose srryonsInthofsmlly now takevftxminawith

fluorida in tham? ❑ Yes ❑ No (f3d) ❑ DK (13d)

------ —--- -- ————-------- —---- .---------- .—-—-— ————---- ——---— —----

b. Who h this?

—-. ——_—____———————- - -

13b. @

Mark “Fluoride vltamhrs”box in person-a column.

1(1 Ffuor”kfovitamins

------------------------------------------------------ -------- ---

c, Anyona aha?

————————————-— —_ ——__

❑ Yes (Reask 13band c) ❑ No

-------------------------------------------------------------- ——.

d. Doaa ●nyans Inths famifynow taka●ny otharkindof

-— —————-— ————--—-—--

fluorlds drops, pllla,or tablets? ❑ Yes ❑ No (14) ❑ DK (14)

---------------------------------------------------------- -——- —-. -——————-————————

S. Who h this? w. T_zE

Mark “Othar fluoride supplement” box in person”skdumn. I ❑ Other flourlde supplement

----—————— ---------------------------------------------------- ——

f. Anyone .1ss?

——————---———--—-—-—-

❑ Yes (Reaak13sandfl ❑ No

14a.Dantelseelarrtawaspseialplastlcmatingsftrsteropelntedonfhs

tc.paof tfrebackt-h tokaepthamfromdecaying.Theywaput

onby ● dontlxtormdentalhyglmlst.Theyeradtixr.nt from

fillings,capa,orowns❑ndfluorkfetrsetrnenta.Has●nyoneIrI~he

familyhaddwrtelaealemaplacedorrtheirtacth? ❑ Yaa ❑ N.J (Scot/onQ J ❑ DK LSection Cl)

-------------------------------------------------------------- -—-

b. Who 1sthla?

————- —__ ——————_-

14b. m

Mark “Dantalaealanta” box in person’s column

I ❑ Dental sealant

-------------------------------------------------------------- ——- -—----——-—----—. ——-—.

c. Anyone .18-? ❑ Yea (Re.xk 14b andc) ❑ No

mu Inc,,m,. ,,.,”,--.!., fig 10 . “.... ..-.,-,- , ..-.,,.-.
Reviews of

New Reports

National Center for Health Statistics

Use of Dental Services and Dental Health:

United States, 1986

Series 10, Number 165

(PHS) 88-1593

For information contact

Paula Summerour

Scientific and Technical Information Branch

3700 East-West Highway, Rm. 1-57

Hyattsville,MD 20782

Tel:(301)436-8500

An important measure of

adequate dental care is the proportion

of persons visiting a dentist, In 1986,

more than half of the U.S. population

age 2 years and over (57 percent)

reported having had a dental visit in

the previous year. Although the vast

majority of Americans had visited a

dentist, more than 11 million persons

including 6 million children 2-4 years

of age had never seen a dentist.

Data contained in a new report by

the National Center for Health

Statistics state that this large number

of persons with no dental visits maybe

due to low incomes and lack of dental

insurance. The proportion of persons

in the lowest income group who had

never seen a dentist (7.1 percent) was

more than double the rate of persons

in the highest income group (2.9

percent),

The report, “Use of Dental

Services and Dental Health: United

States, 19867 presents national

estimates of the volume and timing of

dental visits, coverage by private

dental insurance, and use of

preventive dental products.

The average number of dental

visits per person per year was 2.0,

slightly more than the average number

reported in 1983 (1.9). Females had a

higher number of dental visits per

person per year (2.2) than males (1.9).

White women had higher dental visit

rates (2.3) compared with their male

counterparts (2.0), to black females

(1.5), or to black males (1.2).

For children aged 2–16 years, girls

(63.7 percent) were as likely as boys

(61.4 percent) to have visited the

dentist in the past year. The

proportion of white children with a

dental visit in the past year was about

25 percent greater than the proportion

of black children (64.8 percent and

50.8 percent, respectively).

In 1986, 9 out of 10 children used

fluoride toothpaste. About 13 percent

of all children used a fluoride mouth

rinse at home, and about 7 percent of

all children had dental sealants

applied.

Nearly 38 percent of the

population reported having some type

of private dental insurance coverage.

Males were more likely than females

to have insurance coverage. Persons of

minority groups were less likely to

have private dental insurance-29

percent of black persons compared

with 39 percent of white persons, and

31 percent of Hispanic persons

compared with 38 percent of

non-Hispanic persons.

These statistics were gathered

during the 1986 National Health

Interview Survey, a cross-sectional

household interview survey conducted

by the National Center for Health

Statistics. Questions on dental health

care were included for persons 2 years

of age and over in the civilian

noninstitutionalized population of the

United States.

Authors: Susan S. Jack, M.S. and

Barbara Bloom, M.P.A.

Turn over for ordering information

U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES

Public Health Service

Centers for Disease Control
PublicationOrder Form Mail to:

Superintendent of Documents

Government Printing Office

❑ YES, please send me

Washington, D.C, 20402

copies of Use of Dental Services and Dental Health: U.S. 1986

GPO Stock Number 017-02241048-6 Price $4.00

The total cost of my order is $ . Foreign orders please add an additional 25%.

Prices include regular domestic postage and handling and are good through April, 1989. After that date, please call Order and Information Desk at

(202) 783-3238 to verify prices.

Please Type or Print

(Company or personal name)

(Additional address/attention line)

(Street address)

(City, State, ZIP CODE)

(Daytime phone including area code)

Please choose method of payment

•1

•1

❑

Check payment to the Superintendent of Documents

GPO Deposit Account Unnn-n

VISA, MasterCard, or Choice Account

II HIIIIHIHIHHI1

(Signature)

(Credit card expiration date)
U.S. DEPARTMENT OF HEALTH AFJO

HUMAN SERVICES

Pub[[c Health Setvtce

Centers for Disease Control

rJat[cmal Centsr for Health Statistic?,

3700 East-hVest H[ghway

Hyattsville, Maryland 20782

OFFICIAL BUSINESS

PENALTV FOR PRIVATE UEIE, $300

BULK RATE

POSTAGE & FEES PAID

PHS/NCHS

PERMIT NcI. G-281

DHHS Publication No. (PHS) 8$1076, Series 6, No. 1