NATIONAL CENTER FOR EDUCATION STATISTICS

Technical Report November 1997

National Household Education Survey

An Experiment in

Random-Digit-Dial

Screening

U.S. Department of Education

Office of Educational Research and Improvement NCES 98-255

NATIONAL CENTER FOR EDUCATION STATISTICS

Technical Report November 1997

National Household Education Survey

An Experiment in

Random-Digit-Dial

Screening

J. Michael Brick

Mary Collins

Westat, Inc.

Kathryn Chandler

National Center for Education Statistics

U.S. Department of Education

Office of Educational Research and Improvement NCES 98-255
U.S. Department of Education

Richard W. Riley

Secretary

Office of Educational Research and Improvement

Ricky T. Takai

Acting Assistant Secretary

National Center for Education Statistics

Pascal D. Forgione, Jr.

Commissioner

The National Center for Education Statistics (NCES) is the primary federal entity for collecting,

analyzing, and reporting data related to education in the United States and other nations. It fulfills a

congressional mandate to collect, collate, analyze, and report full and complete statistics on the condition

of education in the United States; conduct and publish reports and specialized analyses of the meaning

and significance of such statistics; assist state and local education agencies in improving their statistical

systems; and review and report on education activities in foreign countries.

NCES activities are designed to address high priority education data needs; provide consistent, reliable,

complete, and accurate indicators of education status and trends; and report timely, useful, and high

quality data to the U.S. Department of Education, the Congress, the states, other education

policymakers, practitioners, data users, and the general public.

We strive to make our products available in a variety of formats and in language that is appropriate to a

variety of audiences. You, as our customer, are the best judge of our success in communicating

information effectively. If you have any comments or suggestions about this or any other NCES product

or report, we would like to hear from you. Please direct your comments to:

National Center for Education Statistics

Office of Educational Research and Improvement

U.S. Department of Education

555 New Jersey Avenue, NW

Washington, DC 20208-5574

November 1997

The NCES World Wide Web Home page address is

http://www.ed.gov/NCES/

Suggested Citation

U.S. Department of Education. National Center for Education Statistics. An Experiment in Random-Digit-Dial Screening, NCES 98-255,

by J. Michael Brick, Mary Collins, and Kathryn Chandler. Washington, DC: 1997.

Contact:

For content:

Kathryn Chandler

(202) 219-1767

(e-mail) NHES@ed.gov

http://www.ed.gov/NCES/NHES

For free single copies of this publication, call the National Library of Education at 1-800-424-1616 or

send a FAX request to (202) 219-1696
TABLE OF CONTENTS

Section Page

ABSTRACT ........................................................................................................ 1

Introduction......................................................................................................... 1

Background of the National Household Education Survey ...................................... 1

Rationale for an Experiment in Screening........................................................... 2

Respondent Selection............................................................................................. 3

Mailings to Respondents........................................................................................ 4

Design of the Experiment..................................................................................... 4

Data Collection Procedures.................................................................................... 6

Outcome Measures................................................................................................ 8

Findings................................................................................................................ 11

Cooperation, Conversion, and Response Rates....................................................... 11

Eligibility Rates..................................................................................................... 16

Resources.............................................................................................................. 16

Conclusions.......................................................................................................... 17

References............................................................................................................ 21

Appendix A: NHES:96 Field Test Screener ....................................................... 23

Appendix B: Advance Letter.............................................................................. 41
iv

LIST OF TABLES

1. Number of telephone numbers, by screen-out question and

advance letter........................................................................................... 7

2. Number of telephone numbers dialed, by final Screener response

category and screen-out question .............................................................. 9

3. Number of Screener, Screener refusal conversion, and extended

interview cases, by cooperation rates, screen-out question,

and advance letter..................................................................................... 11

4. Differences in cooperation rates, by screen-out question and

advance letter........................................................................................... 12

5. Predicted Screener response rate, by screen-out question and advance

letter........................................................................................................ 13

6. Number of households and gross Screener response rate, by screen-out

question and advance letter....................................................................... 15

7. Administration time of Screeners, by experimental condition

and household type................................................................................... 18

LIST OF EXHIBITS

1. Definitions of Screener results for calculation of outcome

measures.................................................................................................. 9
1

ABSTRACT

Much of the literature on RDD response rates focuses on the benefits and drawbacks of various screening and

sampling procedures. The common assumption is that enumeration is more invasive, leading to lower response

rates, but there are concerns about population coverage and self-selection with other methods. When the target

population is found in a subset of households, a screen-out question may be used to eliminate ineligible households

prior to employing sampling methods. This experiment examined the impact on screening response rates of (1)

full enumeration of all households (no screen-out) versus a screen-out question and (2) mailing an advance letter. 

An RDD sample was divided into four quarter-samples: screen-out and letter; screen-out and no letter; no screenout and letter; no screen-out and no letter. Response rates were significantly higher in the screen-out condition. 

The advance mailing increased cooperation in the no screen-out condition, but not in the screen-out condition. 

The no screen-out condition consumed substantially more resources than a screen-out sample of the same size, but

also provided more data for each completed case. Implications of this experiment for the design of future surveys

are discussed.

Introduction

One of the challenges of random-digit-dial

(RDD) surveys is to obtain the cooperation of

households at the beginning of a household

contact. Unlike list-sample surveys, in which

sampled persons are asked for by name and are

often connected to a sponsoring or cooperating

institution, little is known about the telephone

numbers selected for an RDD study, and

respondents may perceive the call as a

telemarketing attempt or an unwanted intrusion.

A household screening response rate in the 1995

National Household Education Survey

(NHES:95) that was significantly lower than

previous NHES administrations led to an interest

in identifying the source of the loss in response

and exploring survey designs that maximize

response rates at the screening level.

Because the NHES:95 was the first survey in this

ongoing data collection system to use full

enumeration of all household members in all

sampled households, it was suspected that this

approach was a likely factor in the decline of the

screening response rate. A systematic

experiment was developed and executed to

examine the impact of the full enumeration

approach on survey response. The experiment

also included a test of an advance letter to

households for which addresses could be

obtained, based on the success of a nonresponse

letter utilized in the NHES:95.

This report presents the design and results of this

experiment in RDD screening. It begins with an

overview of the NHES survey system, then

proceeds through the design, data collection and

analysis of the data from the experiment.

Background of the National Household

Education Survey

The National Household Education Survey

(NHES) is a data collection system of the

National Center for Education Statistics (NCES),

which has as its legislative mission the collection

and publication of data on the condition of

education in the Nation. The NHES is

specifically designed to support this mission by

providing information on those educational issues

that are best addressed by contacting households

rather than schools or other educational

institutions. The NHES provides descriptive

data on the educational activities of the U.S.

population and offers policymakers, researchers,

and educators a variety of statistics on the

condition of education in the United States.

The NHES is a telephone survey of the

noninstitutionalized civilian population of the

U.S. Households are selected for the survey

using random digit dialing (RDD) methods, and

data are collected using computer-assisted

telephone interviewing (CATI) procedures. From
2

45,000 to 64,000 households are screened for

each administration, and individuals within

households who meet predetermined criteria for

populations of interest in the given survey

(usually defined by age or grade in school) are

sampled for more detailed or extended interviews.

 The data are weighted to permit estimates of the

entire population. The NHES survey for a given

year typically consists of a set of screening

questions (Screener), which collects household

composition and demographic data, and two

extended interviews on different education-related

topical components. In order to assess data item

reliability and inform future NHES surveys, each

administration also includes a subsample of

respondents for a reinterview.

Throughout its history, the NHES has collected

data in ways that permit estimates to be tracked

across time. This includes repeating full topical

components on a rotating basis in order to

provide comparative data across survey years. In

addition, each administration of the NHES has

benefited from experiences with previous cycles,

resulting in improvements to the survey

procedures and content. Thus, while the survey

affords the opportunity for tracking phenomena

across time, it is also dynamic in addressing new

issues and including conceptual and

methodological refinements.

A new design feature implemented in the

NHES:96 is the collection of demographic and

educational information on members of all

screened households, rather than just those

households potentially eligible for a topical

component. In addition, this expanded screening

feature included a brief set of questions on an

issue of interest to education program

administrators or policymakers. The total

Screener sample size was sufficient to produce

state estimates of household characteristics for

the NHES:96.

The NHES was conducted in 1991, 1993, 1995,

and 1996. Topics addressed by the NHES:91

were early childhood education and adult

education. The NHES:93 collected information

about school readiness and school safety and

discipline. The 1991 components were repeated

for the NHES:95, addressing early childhood

program participation and adult education. Both

components underwent substantial redesign to

incorporate new issues and develop new

measurement approaches. In the NHES:96, the

topical components were parent/family

involvement in education (PFI) and civic

involvement (CI). The NHES:96 expanded

Screener included a set of questions on public

library use.

The experiment discussed in this report was

conducted in preparation for the NHES:96 full

scale survey. Although most of the issues raised

are pertinent to RDD surveys in general, some of

the specific features of the NHES:96 such as the

Parent PFI/CI and Youth CI interviews, and the

public library questions included in the expanded

screener are mentioned. Readers interested in

more detail on the topics covered in the NHES:96

can refer to Collins et al. (forthcoming).

In addition to its topical components, the NHES

system has also included a number of

methodological investigations. These have

resulted in technical reports and working papers

covering diverse topics such as telephone

undercoverage bias, proxy reporting, and

sampling methods. This series of technical

reports and working papers provides valuable

information on ways of improving the NHES.

The present research reflects a continuation of

this concern for methodological issues by

evaluating factors associated with response rates

in RDD surveys and the resources associated

with selected approaches.

Rationale for an Experiment in

Screening

As noted above, the idea of conducting an

experiment in the NHES on factors associated

with Screener response rates was precipitated in

part by a lower than expected screening response

rate in the NHES:95. While the NHES:93

response rate at the screening stage was 82

percent (Brick et al. 1994), the NHES:95

screening response rate was 73 percent (Collins

et al. 1996). As mentioned above, the topics

addressed in the surveys were different, but both
3

survey administrations were related to

educational issues and the topics covered were

not expected to be associated with the screening

response rates.

A key design difference between the 1993 and

1995 surveys was the screening method. In the

1993 survey, only the 30 percent of households

with at least one person age 18 or younger or in

12th grade or below were enumerated in the

Screener. A screen-out question was asked in the

Screener and households without any potentially

eligible members were eliminated without

enumerating all the household members. In the

1995 survey, however, every household was

potentially eligible for an extended interview and

all the members of every household were

enumerated in the Screener. It was suspected

that this full enumeration of all households was a

major factor in the difference in response rates.

When it was clear that the Screener response rate

in the NHES:95 was going to be much lower than

planned, letters were mailed to nonresponding

households, except those designated as "language

problems." The addresses were obtained from a

commercial service and fewer than half of the

telephone numbers could be matched to valid

addresses of households. Since addresses were

not available for all telephone numbers, the letter

was mailed only to those households for which

addresses were obtained. The results of this

NHES:95 nonresponse mailing showed that those

who were mailed letters responded at a much

higher rate than those who were not sent letters.

This raised the question as to the potential

effectiveness of an advance mailing to an RDD

sample.

The variation in response rates for RDD surveys

of households is well documented, but not well

understood. Some RDD surveys obtain response

rates of 50 percent or less while others have

response rates of 80 percent or more (Groves and

Kahn 1979; Thornberry and Poe 1982;

Friedrichs 1987; Alexander et al. 1986; Salmon

and Nicholls 1983; Hagan and Collier 1983). 

The variation in telephone survey response rates

is associated with factors including the content of

the survey, sponsorship, the length of the

interview, and data collection procedures such as

callback protocols and refusal conversion efforts

(Groves and Lyberg 1988). However, it is

difficult to estimate the relative effects of specific
4

aspects of surveys because these aspects vary

concurrently and there

are no definitive experiments on the impact of

these factors on response rates. As a result of

this uncertainty, an experiment involving some

factors that were feasible in the NHES was

thought to have value. This research provides the

opportunity to test the effects of certain

experimental conditions while holding constant

the survey content, timing, interviewer training,

and calling protocol.

Respondent Selection

Much of the literature on response rates and

screening methods in RDD surveys focuses

primarily on procedures used to sample adult

household members and the relative benefits and

drawbacks of various procedures. This literature

examines alternatives to a full enumeration

approach like the Kish method of respondent

selection, which is similar to the method used in

the NHES:95. The Kish method involves the

enumeration of all adult household members by

age and sex at the start of the screener or

interview (Kish 1949). Because of concerns

about the intrusiveness of collecting such detailed

household information early in an interview and

the amount of time required to conduct the

enumeration, there has been concern that the

Kish method has a depressive effect on response

rates (Lavrakas et al. 1993; Forsman 1993;

Oldendick et al. 1988). A number of alternative

procedures for selecting adult respondents within

households have been developed and compared to

the Kish method.

The Troldahl-Carter method (Troldahl and

Carter 1964) involves ascertaining the number of

adult household members and using an

alternating procedure to select the oldest male,

oldest female, youngest male, or youngest female.

 Another method that has been used increasingly

in recent years is the "last birthday" (or

sometimes "next birthday") method (Lavrakas et

al. 1993). Under this procedure, the screener

respondent is asked to identify the adult age 18 or

older in the household who has most recently had

a birthday (or will be the next to have a

birthday). It is necessary to ascertain the total

number of adults in the household for appropriate

weighting.

Much of the research literature on RDD

screening focuses on differences in rates of

refusal in studies that manipulate the method of

respondent selection (Oldendick et al. 1988;

Salmon and Nicholls 1983; Hagan and Collier

1983; Czaja et al. 1982) or the placement of the

enumeration within an instrument (Bercini and

Massey 1979; Alexander et al. 1986). The

results of these studies are somewhat mixed, and

it is not clearly demonstrated that the alternatives

to enumeration methods provide uniformly higher

response rates (Czaja et al. 1982; Oldendick et

al. 1988; O'Rourke and Blair 1983). In addition,

these methods of respondent selection may also

have an impact on other features of the sample

such as population coverage and inappropriate

respondent self-selection (Bryant 1975; Carr and

Hertvik 1993; Hagan and Collier 1983; Lavrakas

et al. 1993; Romuald and Haggard 1994).

In general, this literature focuses almost

exclusively on methods of sampling one random

adult within households. The sampling of

children as a unit of analysis is seldom, if ever,

addressed. When children are sampled or there is

the potential to sample more than one person in a

household for a survey, as in the NHES,

enumeration becomes an essential design element.

 In each of the four full-scale NHES collections

conducted to date, households with children could

contain one or more eligible members, and in two

of the collections, adults were sampled based on

data collected in the Screener. The enumeration

of households members in these situations may

be needed so that information required for

sampling household members can be collected in

an unambiguous way. Similarly, when it is likely

that all, or nearly all, households include at least

one person eligible for a study, the full

enumeration of all households may be important.

 However, when the population of interest is

found in a relatively small fraction of households,

a preliminary screen-out question may be used

prior to enumeration to eliminate those

households without any eligible members. Such

eligibility criteria can be based on age or on other

characteristics that can be reliably reported by an
5

adult household member responding to a

Screener.

Mailings to Respondents

The literature on advance mailings to respondents

in RDD surveys is quite sparse. It is likely that

little has been done in this area because of the

limited proportion of an RDD sample for which

addresses can be obtained and the costs

associated with the mailings. Camburn et al.

(1995) report the results of an advance mailing

for a national RDD survey about children's

immunization. They found that mailing an

advance letter in an RDD survey can lower the

rate of refusal and improve overall response

rates. No other pertinent work on advance letters

for RDD surveys was located in the literature.

Design of the Experiment

Based upon information available in the extant

literature and on experiences with previous

NHES data collections, particularly the

NHES:95, an experiment was developed and

implemented as a part of a field test for the

NHES:96. This research provided the

opportunity to test the effects of the experimental

conditions while holding constant the survey

content, timing, interviewer training, and calling

protocol. The experiment tested the effects of 1)

screening out ineligible households before

enumerating all household members and 2)

sending respondents a letter about the survey

prior to interviewer telephone contact. The field

test sample was partitioned into four equal

samples to examine the effects of four conditions:

n No advance letter, no screen-out question;

n No advance letter, screen-out question;

n Advance letter, no screen-out question;

and

n Advance letter, screen-out question.

The screen-out question used in this experiment

asked the Screener respondent whether any of the

people who normally lived in the household were

age 20 or younger. If no one in the household

was age 20 or younger, the Screener ended at this

point. In households with members in the target

age range, household enumeration was

conducted. A copy of the questions in the

Screener is given in Appendix A. The box at

the

bottom of the first page of the instrument shows

that the screen-out interview ended if no members

were in the eligible age range. If there were

eligible household members or the household was

part of the no screen-out sample, the interview

continued in exactly the same way for both

samples. The last part of the Screener contains

the public library use questions.

The advance letter used in the experiment

identified the U.S. Department of Education as

the sponsor of the survey, provided an overview

of the purpose and content of the survey, stated

the average amount of time required to complete

interviews, and stressed the importance of

participation while emphasizing that participation

was voluntary. A page of commonly asked

questions about the survey was also included. A

copy of the advance letter and the other

attachment are given in appendix B. These were

mailed first class in a standard envelope with

permit postage paid by the U.S. Department of

Education. The letters were mailed

approximately one week prior to beginning data

collection.

One hypothesis addressed in the experiment was

that the lower than expected response rate

observed in the NHES:95 was related to the full

enumeration of all households even if no person

in the household was selected for an extended

interview. In the NHES:93, only households

with children age 18 or younger or in 12th grade

or below were fully enumerated. In the

NHES:91, only households with children or those

sampled for Adult Education (AE) component

screening were enumerated, and only persons in

the eligible age ranges (2 to 9; 16 or older) were

enumerated. In each of these earlier studies, the

interviewers could usually complete the interview

quickly for households with no children, and this

might have been used to encourage respondents

to continue the interview, since it only involved

answering a few items. This speculation led to
6

the introduction of the screen-out part of the

experiment for the field test.

A second hypothesis was that the advance

mailing would improve response rates. This was

suggested primarily by experience with 

a

nonrespondent mailing in the NHES:95,

discussed above. Since the NHES:95 experience

was not an experiment (obtaining an address and

response propensity were confounded), the

results were not conclusive. However, the

NHES:95 experience strongly suggested that

mailing a letter to respondents might be effective.

In addition to these major hypotheses, several

other results of the experiment were investigated

to determine the consequences of surveying under

the different conditions. One outcome measure

studied was the percentage of households with

children eligible for the Parent PFI/CI interview.

 A household was eligible if it had any member

from age 3 through 12th grade and up to 20

years old (that is, preschoolers age 3 and older

and children enrolled in grades K through 12, up

to a maximum age of 20). The percentages of

households eligible for the Parent PFI/CI

interview under the different conditions can be

compared to estimates from other sources and

serve as a rough indicator of the quality of the

screening.

Another aspect of the experiment examined is the

resources required to conduct the interviews

under the different conditions. The no screen-out

interview is clearly longer for households with no

members eligible for an extended interview

because it contains more questions than the

screen-out interview. However, these additional

questions were added to increase the utility of the

Screener. In the screen-out option, the Screener

only serves to determine eligibility for the

interview and to sample members for the

extended interview components. In the no

screen-out interview, the Screener is also used to

obtain data on households for making national

estimates. For example, in the NHES:96 the

Screener was used to make national estimates of

household public library use.

A list-assisted sample of 10,001 telephone

numbers was selected from the Genesys sampling

frame used in the NHES in 1995 and 1996. The

sample was systematically divided into two halfsamples; one of the two halves was made

available for interviewers at Westat's Rockville,

Maryland Telephone Research Center (TRC) and

the other was made available to interviewers at

Westat's Frederick, Maryland TRC.

Screen-out condition. The screen-out question

was included in the interviews conducted at the

Frederick TRC, while all households were

enumerated at the Rockville TRC. The screenout question was restricted to one facility rather

than assigning interviewers at both facilities to

test this experimental condition in order to

eliminate the contamination that might have

occurred if interviewers at the same facility

talked to one another about the study. The

interviewers at the facilities were not informed

that a different version of the interview was being

tested at the other site.

Although this design eliminated the

contamination between interviewers, it had the

negative consequence of confounding the facility

with the screen-out condition. Before this design

was considered, the NHES:95 initial cooperation

rates (defined in the next section) in the two

facilities were compared and found to be nearly

identical (60 and 61 percent). Westat strives to

ensure there is no "facility effect" in their

telephone centers by using standard training and

supervision methods and monitoring the

outcomes by center.

Advance letter. For the advance letter

component of the experiment, all 10,001

telephone numbers were sent to Telematch, a

commercial firm that matches telephone numbers

to addresses. Rather than mail to all the

addresses found, a random sample of half of the

numbers assigned to each facility was selected

and the advance letter was mailed only to the

addresses obtained for those in the selected halfsamples. Letters were sent via first class mail. 
7

Table 1 summarizes the design of the experiment

and shows the number of telephone numbers

assigned to each experimental condition, along

with the results of sending the numbers to

Telematch. Across all experimental conditions,

advance letters could have been mailed to 33

percent of the telephone numbers (3,348 of

10,001), but under some experimental conditions,

no letters were mailed. Of the 1,684 letters

mailed, 184 (10.9 percent) were returned by the

post office as undeliverable. Note that having an

address associated with a telephone number does

not necessarily mean that the correct household

received the letter. Postmaster returns are

accounted for in the analysis presented in this

report, but it is likely that some additional letters

were neither received by the households with the

sampled telephone numbers nor returned.

Data Collection Procedures

The last step in the design was to select

interviewers to conduct the interviews.

Interviewers with experience working with the

NHES:95 were selected in both facilities.

Westat's telephone center management then

examined the cooperation and refusal conversion

rates of each of the interviewers from the

NHES:95 to ensure that the interviewers in both

facilities were comparable and that the selected

interviews did not have unusually high rates.

Twenty to 25 interviewers were trained at each

telephone center. Following the training, the

interviewing began.

The sample of telephone numbers was sorted into

random order and then released for work

according to a standard call-scheduling protocol.

This protocol required that at least seven

attempts be made to contact a sampled telephone

number and complete a Screener, with additional

calls allocated to any extended interviews in the

household. The calls were staggered at different

times of the day and week, with two daytime

weekday calls, two weekday evening calls, and

two weekend calls on separate weekends. Initial

refusal cases were held for a period of 13 days,

at which time specially trained refusal conversion

interviewers attempts to convert these cases.

Unlike full-scale NHES surveys, no interviewing

was done in Spanish for this experiment. The

data collection procedures were identical in the

two facilities. For example, if an interviewer

reached an answering machine when a number

was dialed, the call outcome was recorded and
8

Table 1.—Number of telephone numbers, by screen-out question and advance letter

Experimental condition Number Percent

Total............................................................... 10,001 100.0

Screen-out question

Total ......................................................... 5,001 50.0

Sampled for advance mail.................... 2,501 25.0

No mailable address ........................ 1,661 16.6

Mailable address.............................. 840 8.4

Mailed, returned ........................ 90 0.9

Mailed, not returned................... 750 7.5

Not sampled for advance mail.............. 2,500 25.0

No mailable address ........................ 1,640 16.4

Mailable address.............................. 860 8.6

No screen-out question

Total ......................................................... 5,000 50.0

Sampled for advance mail.................... 2,500 25.0

No mailable address ........................ 1,656 16.6

Mailable address.............................. 844 8.4

Mailed, returned ........................ 94 0.9

Mailed, not returned................... 750 7.5

Not sampled for advance mail.............. 2,500 25.0

No mailable address ........................ 1,696 17.0

Mailable address.............................. 804 8.0

NOTE: This table reflects the design of the field test experiment. As discussed in the text, 350 telephone numbers were

removed from each half sample after the experiment began. The final sample numbers appear in table 2.

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey

(NHES), field test of the 1996 survey.

no messages were left on the machine by the

interviewers in either facility. The data collection

period for the experiment lasted 5 weeks

As the data collection continued, the number of

completed interviews was monitored. After it

was determined that most of the field test goals

for the number of completed extended interviews

of various types were being accomplished, it was

decided to eliminate 350 of the telephone

numbers in each facility that had not yet been

worked at all. Since the sample was sorted

randomly, this reduction in the sample was done

very quickly without risk of bias.

Table 2 gives the results of calling the telephone

numbers by interviewing facility. This table

provides some detailed information on the results

of the call attempts. The counts for numbers that

were not residential show the telephone numbers

in the two facilities were very similar. The

percentage of telephone numbers that were not

residential in the two facilities were
9

approximately equal (51 percent in the Frederick

faculty and 52 percent in Rockville).

Furthermore, the distributions of not residential

numbers by type (not working, business, and not

ascertained) were also very similar. The

completion and nonresponse outcomes are

discussed in more detail in the Findings section.

Outcome Measures

Before discussing the results from the

experiment, some terms are introduced that are

needed in the evaluation of the experiment. The

three types of measures used to evaluate the field

test experiment are 1) cooperation, conversion,

and response rates; 2) eligibility rates; and 3)

resources expended to complete the interviews.

Generally speaking, higher cooperation rates are

the single most important evaluation criterion.

Eligibility rates are an important consideration

because they are one indicator of the extent of

possible nonresponse bias in a survey. Resource

measures reflect the cost associated with

obtaining completed interviews; it is incumbent

upon study designers to evaluate the effort

required to achieve desired response rates and the

resources required to support those efforts.

Three cooperation rates and two response rates

are examined in this experiment. The most

important cooperation rate in the NHES is the

Screener initial cooperation rate. When a

household is contacted initially, the result can be

either a completed screening interview, a refusal,

or another result (such as a language problem or

a temporary outcome like an appointment to call

later). The initial contact with the household

may be preceded by several call attempts that do

not result in speaking to a person, such as a ring

with no answer or an answering machine result. 

The initial cooperation rate is the ratio of the

number of initially completed interviews (C) to

the number of interviews that are initially either

refused (R) or completed (C); thus, the rate is

calculated as [C/(R+C)]. In some cases, the

respondent neither refuses nor completes the

interview, and a temporary outcome, such as an

appointment to call later, is generated. These

results are only included in the cooperation rate

once they become either completes (C) or initial

refusals (R).

The Screener refusal conversion rate is a second

cooperation rate used in this study. If a Screener

interview is initially refused, an attempt to

convert the refusal into a completed interview is

tried some time later. The refusal conversion rate

is the ratio of the number of these cases that are

completed during refusal conversion (C2) to the

total number of interviews that were either

refused a second time (R2) or completed (C2);

thus the ratio is calculated as [C2/(R2 + C2)]. As

noted below, only one refusal conversion was

attempted in this experiment. However, this

attempt might have required several telephone

calls to contact the household. Other outcomes

such as “maximum calls,” business, language

problems, and nonworking numbers are not used

in the computation. Hostile refusals, for which

no conversion efforts were made, were also

excluded from calculation of the conversion rate.

The third type of cooperation rate discussed in

this analysis is the extended interview

cooperation rate; the extended interviews are

those conducted with sampled persons or parent

respondents with sampled children within the

screened households. Thus, it includes

interviews sampled within households with

completed Screeners. This rate is the ratio of the

number of completed extended interviews (CE) to

the number of interviews that are refused (RE) or

completed (CE); thus, the rate is calculated as

[CE/(RE + CE)]. If a sampled child was not at

least 3 years old or under 21 and enrolled in 12th

grade or less, the child was ineligible for the

interview; these children are excluded from the

computation of this rate. Outcomes such as

“maximum calls,” language problems, and

nonworking numbers are also excluded from this

rate. No attempts were made to convert the

extended interviews that were initially refused

during this experiment.1

 

1

 NHES field tests typically do not include refusal

conversion attempts. Such attempts were made at the

Screener level for this experiment, but not at the extended

interview level.
10

Table 2.—Number of telephone numbers dialed, by final Screener response category and screen-out 

question

Screener response category

Screen-out question

(Frederick)

No screen-out question

(Rockville)

Number Percent Number Percent

Total............................................................... 4,651 100.0 4,650 100.0

Complete ........................................................ 1,721 37.0 1,381 29.7

Complete, no extended interview............... 1,213 26.1 877 18.9

Complete, Parent PFI/CI interview only..... 226 4.9 238 5.1

Complete, Parent PFI/CI and Youth CI

interviews..................................................

282 6.1 266 5.7

Not residential ................................................ 2,357 50.7 2,417 52.0

Not working .............................................. 1,405 30.2 1,459 31.4

Business.................................................... 553 11.9 540 11.6

Not ascertained.......................................... 399 8.6 418 9.0

Nonresponses, finalized .................................. 430 9.2 559 12.0

Language problem..................................... 29 0.6 37 0.8

Maximum call ........................................... 115 2.5 130 2.8

Refusal ...................................................... 284 6.1 377 8.1

Other nonresponse..................................... 2 0.0 15 0.3

Nonresponse, not finalized.............................. 143 3.1 293 6.3

Language problem..................................... 10 0.2 25 0.5

Maximum call ........................................... 47 1.0 197 4.2

Refusal ...................................................... 86 1.8 71 1.5

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey

(NHES), field test of the 1996 survey.

Exhibit 1.—Definitions of Screener results for calculation of outcome measures

Result Definition

C Completed Screener with no initial refusal

R Initial Screener refusal

C2 Screener completed in refusal conversion

R2 Screener refused again in refusal conversion

CE Completed extended interview

RE Refused extended interview
11

An overall or gross Screener response rate is

also used to evaluate some of the results from the

experiment. The gross response rate is the ratio

of the number of completed interviews to the

number of known residential numbers. While the

gross response rate is used in most NHES

studies, the field test gross response rate is not

very indicative of the expected Screener response

rate for a full-scale NHES for several reasons. 

In a full-scale NHES, data are collected over a

longer period of time and this generally increases

the percentage of households that are contacted

and the response rate. A full scale study would

also include interviews conducted in Spanish to

help reduce the percentage of language problem

cases. Furthermore, the call scheduling protocol

used in the field test would be replaced by more

extensive dialings to telephone numbers that

could not be interviewed and refusal conversion

would be attempted for some cases a second

time. The use of these more intensive methods in

a full scale study, a standard for the NHES,

reduces the number of unresolved numbers and

increases the gross Screener response rate.

Despite the shortcomings of this measure, the

gross Screener response rate is used in a limited

way to compare the results across experimental

conditions.

Because the gross Screener response rate for the

experiment is not very useful for predicting the

actual response rate in a full scale study, an

alternative, called the predicted Screener

response rate, is introduced. The predicted

Screener response rate is the initial cooperation

rate for the Screener multiplied by one minus the

Screener refusal conversion rate. Let i be the

initial cooperation rate for the Screener and let r

be the refusal conversion rate for the Screener,

then the predicted response rate is 100{i+(1-i)*r}

percent.

For example, in the NHES:93 the initial

cooperation rate was 69.7 percent and the refusal

conversion rate was 45 percent, so the predicted

Screener response rate was 83.3 percent

(100*{.697+(1-.697)*.45}=83.3%). The actual

Screener response rate in the NHES:93 was 82.1

percent. For the NHES:95, the predicted

Screener response rate was 74.2 

percent

(100*{.61+(1-.61)*.339}= 74.2%) and the actual

Screener response rate was 73.3 percent. In both

cases the predicted Screener response rate is

about 1 percent higher than the actual rate. 

Thus, the predicted response rate provides a

reasonable predicted value using only initial

cooperation and refusal conversion rates. This

rate is particularly useful because the initial

cooperation and refusal conversion rates do not

depend on the calling protocol (number of call

attempts, maximum number of attempts, number

of times refusal conversions are attempted, etc.),

and the calling protocol for the experiment differs

from the standard NHES protocol. The predicted

Screener response rate is robust to these

differences in protocols.

The eligibility rate can be another useful statistic

in assessing the survey outcome. The eligibility

rate is the percentage of the respondents who are

the target of the survey. For example, if the goal

is to conduct interviews with parents about

children and youth age 3 through 12th grade,

then the eligibility rate is the percent of

households with children in this population. 

Examining the eligibility rate is a means of

assessing the extent to which the cooperating

respondents reflect the population on the key

characteristic that determines eligibility for the

study. Significant deviation of this rate from

known population parameters suggests

nonresponse bias at the screening level.

The measures of resources that are used to

evaluate the experiment are the number of

telephone calls, the length of the Screener

interview, and the number of air time interviewer

hours. The length of the interview is the

administration time for the interview. It begins

when the respondent answers the telephone and

ends when the screener is completed. The air

time is the amount of time used to make the calls,

get the respondent on the telephone, conduct the

interview, and code the result of the call; it

includes both those attempts resulting in contact
12

and those not resulting in contact. These

measures are provided at the Telephone Research

Center level so that they can be used to compare

the screen-out question condition of the

experiment but not the advance letter factor.

Table 3.—Number of Screener, Screener refusal conversion, and extended interview cases, by cooperation

rates, screen-out question, and advance letter

Screener* Screener refusal conversion Extended interview

Experimental condition

Complete Refusal

Cooperation

rate Complete Refusal

Conversion

rate Complete Ineligible Refusal

Cooperation

rate

Screen-out/letter......................... 802 272 74.7% 67 144 31.8% 342 6 17 95.3%

Screen-out/no letter.................... 769 269 74.1 83 135 38.1 284 10 23 92.7

No screen-out/letter.................... 671 348 65.8 71 179 28.4 299 9 21 93.6

No screen-out/no letter............... 572 387 59.6 67 196 25.5 255 11 15 94.7

Screen-out................................... 1,571 541 74.4 150 279 35.0 626 16 40 94.1

No screen-out ............................. 1,243 735 62.8 138 375 26.9 554 20 36 94.1

Letter .......................................... 1,473 620 70.4 138 323 29.9 641 15 38 94.5

No letter...................................... 1,341 656 67.2 150 331 31.2 539 21 38 93.6

*At this stage, refusal conversion attempts are not included.

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey

(NHES), field test of the 1996 survey.

The costs of sending an advance letter are

dependent on a number of factors that may vary

from one survey to the next. In this study,

Telematch charged $0.05 for each address found

for a telephone number rather than charging per

telephone number searched. Other types of

arrangements with commercial firms who do

these services might be arranged, especially for

large volume mailings.

Other components of resources are the costs for

printing the advance letter, printing the

envelopes, preparing the mailing, and postage. In

this study, a simple letter was prepared (see

appendix B) and the postage was paid by the

U.S. Department of Education. As with the

methods for obtaining addresses, different

arrangements for conducting the mailing, such as

using a company that specializes in these

activities, or using regular stamps rather than

government permit, and a different number of

mailings could result in overall costs varying

substantially. Because of these variations, the

focus in the next section is on resources used in

the interviewing rather than the mailing. These

are also clearly the predominant expenditures in

this type of survey.

Findings

The findings of the experiment as they relate to

the screening cooperation, conversion, and

response rates, eligibility rates, and survey

resources requirements are presented below.

Cooperation, Conversion, and Response

Rates

Table 3 shows the number of Screener cases

prior to refusal conversion, Screener refusal

conversion cases, and extended interview cases

for each of the four experimental conditions, as

well as aggregates for each of the two conditions.
13

 The cooperation rates are computed using the

methods described above.

As noted above, the initial cooperation rates are

the most important in any RDD survey, and the

differences in these rates between the

experimental conditions are obvious. The initial

Screener cooperation rates for the screen-out

conditions are much higher than for the no

screen-out condition.

Table 4 presents the cooperation rates and their

standard errors. It also gives the t-value for each

combination of experimental conditions, where

the t-value is the ratio of the rate to its standard

error. The t-values in bold are statistically

significant at the 95% confidence level after

controlling for the multiple tests being performed

(Bonferroni adjustment with family size 7).

For the Screener initial cooperation rates, the

results are striking. The difference in rates

between the screen-out and no screen-out

conditions is 11.5 percent (e-f). The screen-out

condition is the dominant factor for determining

the level of the initial cooperation rate for the

Screener in this experiment. The difference in

the initial Screener cooperation rate due to the

advance letter is 3.2 percent (g-h), but this

difference is significant only for those interviews

conducted with the no screen-out question (c-d). 

The screen-out question leads to a substantially

higher initial cooperation rate. If no screen-out

question is included, then the advance letter

increases the initial cooperation rate (c-d), but

not to the same level attained by using the screenout question (a-c).

Table 4.—Differences in cooperation rates, by screen-out question and advance letter

Experimental condition Screener cooperation* Screener refusal conversion Extended interview

Rate s.e. t-value Rate s.e. t-value Rate s.e. t-value

(a) Screen-out/letter................... 74.7% 1.3 31.8% 3.2 95.3% 1.1

(b) Screen-out/no letter.............. 74.1 1.4 38.1 3.3 92.7 1.5

(c) No screen-out/letter.............. 65.8 1.5 28.4 2.9 93.6 1.3

(d) No screen-out/no letter......... 59.6 1.6 25.5 2.7 94.7 1.3

 Difference (a)-(b) .................. 0.6 1.9 0.3 -6.3 4.6 -1.4 2.6 1.8 1.4

 Difference (a)-(c)................... 8.8 2.0 4.4 3.4 4.3 0.8 1.7 1.7 1.0

 Difference (a)-(d) .................. 15.0 2.1 7.3 6.3 4.2 1.5 0.7 1.7 0.4

 Difference (b)-(d) .................. 14.4 2.1 6.9 12.6 4.2 3.0 -1.9 2.0 -1.0

 Difference (c)-(d)................... 6.2 2.2 2.9 2.9 3.9 0.7 -1.0 1.9 -0.5

(e) Screen-out............................ 74.4 0.9 35.0 2.3 94.1 0.9

(f) No screen-out........................ 62.8 1.1 26.9 2.0 94.1 1.0

 Difference (e)-(f)................... 11.5 1.4 8.0 8.1 3.0 2.7 0.0 1.3 0.0

(g) Letter................................... 70.4 1.0 29.9 2.1 94.5 0.9
14

Experimental condition Screener cooperation* Screener refusal conversion Extended interview

Rate s.e. t-value Rate s.e. t-value Rate s.e. t-value

(h) No letter............................... 67.2 1.1 31.2 2.1 93.6 1.0

 Difference (g)-(h) .................. 3.2 1.4 2.2 -1.3 3.0 -0.4 0.9 1.3 0.7

*At this stage, refusal conversion attempts are not included.

NOTE: t-values equal to or greater that 2.7 are significant at the 95 percent confidence level. Calculations of differences are

based on unrounded data.

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey

(NHES), field test of the 1996 survey.
15

In addition to the differences in the cooperation

rates, a substantial difference was observed

between the screen-out and no screen-out

conditions in the numbers of nonfinalized

maximum call cases (see table 2). While only 47

cases were in this status in the screen-out

condition, there were 197 such cases in the no

screen-out condition. This may be a function of

the relative ease of completing the Screener with

ineligible households in the screen-out condition

compared to the no screen-out condition. Another

factor may be that some maximum call cases are

hidden refusals. In the NHES:95, 41 percent of

maximum call cases that were refielded for

additional attempts were finalized as refusals

(Collins et al. 1997). The greater incidence of

maximum call cases in the no screen-out

condition, is consistent with higher rate of refusal

for that condition.

For the Screener refusal conversion rates, only

two of the differences are statistically significant.

The screen-out question not only results in a

higher Screener initial cooperation rate, but it

also has the desirable feature of resulting in a

higher refusal conversion rate. The increase of

8.1 percent is large enough to be of substantive

importance. It is worth noting that most of the

difference in refusal conversion rates in the

screen-out question occurs when no advance

letter is mailed.

No statistically significant differences were

observed for the extended interview initial

cooperation rates. All of these rates were

uniformly high.

A goal of the experiment was to examine the

probable Screener response rate under these

different experimental conditions. The predicted

Screener response rate shows this better than any

other single measure, although the rates are about

1 percent higher than the actual response rates.

This rate represents the expected Screener

response rate under the full NHES data collection

protocol; it is discussed further on page 10. 

Table 5 shows the predicted Screener response

rates for each of the experimental conditions. 

The predicted Screener response rates using the

screen-out question are about 82 to 84 percent,

while enumerating all households gives predicted

Screener response rates in the range of 70 to 75

percent. The gross Screener response rate can be

used to measure the difference in response rates

by whether or not an advance letter was actually

Table 5.—Predicted Screener response rate, by screen-out question and advance letter

Experimental condition

Number of

telephone

numbers

Initial

cooperation

rate

Refusal

conversion rate

Predicted

response rate

Screen-out/letter ........................ 2,326 74.7% 31.8% 82.7%

Screen-out/no letter.................... 2,325 74.1 38.1 84.0

No screen-out/letter.................... 2,325 65.8 28.4 75.5

No screen-out/no letter............... 2,325 59.6 25.5 69.9

Screen-out.................................. 4,651 74.4 35.0 83.3

No screen-out............................. 4,650 62.8 26.9 72.8

Letter......................................... 4,651 70.4 29.9 79.2

No letter .................................... 4,650 67.2 31.2 77.4

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey

(NHES), field test of the 1996 survey
16

mailed to a household. The gross Screener

response rate, as noted earlier, is the ratio of the

number of completed interviews to the number of

residential numbers. This rate is the only one

used for examining the detailed results because

the other rates, especially the predicted rate,

become more unstable when based on fewer

cases. For the purpose of examining the impact

that mailing the letter has on the Screener

response rate, the gross Screener response rate is

used. As noted earlier, the fact that a letter was

mailed does not mean that it was delivered to any

household, that it was delivered to the correct

household, or that the Screener respondent ever

saw the letter. Interviewers were aware that the

mailing to some households was conducted, but

had no information about whether specific

households had been sent a letter. The

information available is whether or not the

advance letter was mailed and, if so, whether it

was returned by the postmaster as undeliverable.

 The postmaster returns are only available for

those advance letters that were in the part of the

experiment in which the letters were actually

mailed.

Table 6 presents the number of households and

the gross Screener response rates by the various

experimental conditions. The gross Screener

response rate shows the large difference in

response rates associated with the use of the

screen-out question, as did the cooperation and

conversion rate measures discussed above. Under

the screen-out condition, no significant difference

in Screener response rates is observed between

households who were and were not sampled for

mailing (74.8 and 75.3 respectively). The

comparison of these rates examines whether there

is an improvement in response rates when an

advance mailing is part of the data collection

protocol, and not whether households with

mailable addresses respond at different rates

regardless of whether a letter is sent. The latter

question is of less interest, but is discussed

below. Under the no screen-out condition, those

who were sampled for mailing had a 5.9 percent

higher Screener response rate (64.7 percent) than

those who were not sampled for mailing (58.8

percent). These findings suggest that the advance

mailing may help to increase Screener response

rates under a no screen-out design, but not under

a screen-out design.

Although the main issue is the effect of doing an

advance mailing, it is interesting to examine the

difference in Screener response rates for those

with mailable addresses under the two conditions.

Under the screen-out conditions, the difference

between the gross Screener response rates for

those sampled for the advance letter with a

mailable address (79.5 percent) and those not

sampled for the advance letter with a mailable

address (78.0 percent) was only 1.5 percent (t =

0.7). Under the no screen-out condition the

difference between the two groups was

significant at 7.4 percent (t = 2.7). Thus, there

appears to be an interaction between the two

experimental conditions and the effect of the

mailing is different depending on the use of the

screen-out item.

The lower Screener response rates associated

with the no screen-out approach raise the

important question of the source of the loss in

response. As noted previously, the literature

generally regards enumeration as more invasive

than other respondent selection approaches.

However, data collection experience on the

NHES indicates that most refusals occur prior to

the enumeration matrix (Brick et al.,

forthcoming). Some researchers (Bercini and

Massey 1979) have suggested that the lower

response rates may be related to the interviewers’

perception of the task. Specifically, Bercini and

Massey reported that collecting household

members’ names had a “direct negative effect on

interviewer performance.” The findings of this

experiment are consistent with that hypotheses

and are further supported by the fact that most

refusals occur prior to enumeration, and cannot

be attributed to respondent reaction to the

enumeration (Brick at al. 1997). Following this

screening experiment, efforts were made in the

NHES:96 interviewer training to emphasize that

only a brief interview (screening questions,

personal characteristics, and library use

questions) was to be conducted in most

households. However, the NHES:96 Screener

response rate was at the lower end of the range
17

Table 6.—Number of households and gross Screener response rate, by screen-out question and advance

 letter

Experimental condition

Number of

households

Gross response

rate t-value

Total............................................................... 4,527 68.5

No mailable address ....................................... 1,965 64.5

Mailable address............................................. 2,562 71.6

Screen-out question

Total ......................................................... 2,294 75.0

(a) Sampled for advance mail .............. 1,162 74.8

 No mailable address..................... 527 69.1

(b) Mailable address ......................... 635 79.5

 Mailed, returned ................... 52 76.9

 Mailed, not returned ............. 583 79.8

(c) Not sampled for advance mail ........ 1,132 75.3

 No mailable address..................... 463 71.3

(d) Mailable address......................... 669 78.0

Difference (a) - (c)

Difference (b) - (d)

-0.5

1.5

-0.3

0.7

No screen-out question

Total ......................................................... 2,233 61.8

(e) Sampled for advance mail .............. 1,147 64.7

 No mailable address..................... 499 60.7

(f) Mailable address......................... 648 67.7

Mailed, returned................... 52 65.4

Mailed, not returned ............. 596 68.0

(g) Not sampled for advance mail ........ 1,086 58.8

 No mailable address.................... 476 56.9

(h) Mailable address........................ 610 60.3

Difference (e) - (g)

Difference (f) - (h)

5.9

7.4

2.9

2.7

NOTE: t-values equal to or greater that 2.7 are significant at the 95 percent confidence level.

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education

Survey (NHES), field test of the 1996 survey.
18

expected for a no screen-out approach with an

advance mailing (the NHES:96 Screener

response rate was 70 percent). Thus it appears

that direct experience with conducting a given

survey shapes the interviewers’ perception of the

ease or difficulty of the task, which in turn has an

effect on response rates.

Eligibility Rates

As stated earlier, differences in eligibility rates

could indicate potential response problems.

Based on the October 1992 CPS (the most recent

October CPS file available at the time of the

initial NHES:96 sample design), the percentage

of households eligible for the NHES:96 was

estimated to be 30.4 percent. The percentage of

households eligible for the NHES:96 based on

the completed Screener interviews using the

screen-out question was 29.5 percent (508

households out of 1,721 completed Screeners),

within 1 percent of the CPS estimate. The

percentage in the numbers assigned to the

interviews with no screen-out question was 36.5

percent (504 households out of 1,381 completed

Screeners), about 6 percent higher than the CPS

estimate. (t=2.52). This result initially led to

speculation about potential nonresponse biases

due to differential response rates by whether or

not there were any children in the household. 

However, further examination of the estimates

from the CPS reduced the concern, because the

CPS estimates were found to vary over time. 

The estimated percentage of households with

eligible children from the October 1992 CPS was

30 percent, while the October 1993 CPS estimate

was 33 percent and the October 1994 CPS

estimate was 38 percent. Since the eligibility

rates under the two experimental conditions were

within the range of the CPS for the three years, it

is difficult to determine which of the

experimental rates is closer to the population

value. 

The difference in the eligibility rates by

experimental condition may be less problematic

than they first appeared for another reason. 

Brick and Broene (forthcoming) found that the

calling procedures used in the full scale NHES

surveys to increase response rates (e.g., 

second refusal

conversion attempts, additional attempts to

contact households that did not respond in the

first 7 callings) virtually eliminated the eligibility

differences that could be measured. Specifically,

comparisons of the characteristics of households

from the NHES:95 and from the Current

Population Survey revealed relatively few

differences in characteristics associated with

eligibility, and the observed differences were

generally quite small. Exceptions were 1-person

households (for which the NHES:95 estimate was

2.1 percent less than the CPS estimate) and

households with at least one adult aged 65 or

older (2.7 percent less than the CPS). Thus, the

statistically significant eligibility rates found in

the experiment would probably be reduced if the

standard NHES calling procedures were used.

Resources

The other part of this evaluation is the

consideration of three measures of resources

expended to conduct the surveys: 1) the number

of telephone calls made, 2) the length of the

interview, and 3) the air time required. As noted

earlier, these measures are provided at the facility

level so that differences between the screen-out

question and the full enumeration can be

evaluated. The difference due to the advance

letter cannot be examined, but this is only a

minor restriction since the advance letter was not

expected to reduce the length of the interview.

The advance letter could have an impact on

reducing the number of calls needed to finalize a

case and this can be evaluated.

The mean number of telephone calls per case

made in the field test did not differ substantially

by the use of the screen-out question. The mean

number of calls with the screen-out sample was

2.76 and the average number of calls with no

screen-out question was 2.83. The difference is

small and of little practical consequence for a

survey like the NHES. About 6 percent of the no

screen-out interviews were not finalized and only

3 percent of the screen-out interviews were not

finalized. It is likely that the mean number of
19

telephone calls required to finalize the cases so

that the percentages finalized were equal would

have been somewhat greater for the no screen-out

numbers. However, the differences are probably

small, because similar results are obtained if the

mean number is computed separately for only the

completed household screening interviews (2.69

for the screen-out question and 2.73 for no

screen-out question).

The average length or administration time of the

Screener differed by whether or not the screenout question was used. The Screener contained

questions on the educational and demographic

characteristics of household members and items

on public library use2. These questions were

asked in all households in the no screen-out

condition, but were only asked in the screen-out

condition if there were household members age

20 or younger and the Screener was continued.

The mean times for households that were not

sampled for an extended interview were very

different for these two screening conditions. The

mean for households not sampled for an extended

interview with the screen-out question was 2.1

minutes and for those without the screen-out

question the mean was 8.1 minutes (table 7). 

The very brief time for those without extended

interviews in the screen-out condition reflects the

fact that most of these households were screened

out with only a few questions. While these

differences have cost implications, it must be

remembered that much of the additional survey

administration time is associated with collecting

data on the characteristics of households and

their members that would not be available for

analysis otherwise. The difference in cooperation

rates discussed earlier suggests that some of the

additional time is required for refusal conversion

activities, i.e., persuading persons who initially

refused to complete the interview. However,

instrument administration time alone is not an

appropriate comparison of resources associated

 

2

 The questions on public library use were asked in the

Screener if no household members were sampled for an

extended interview; if one or more household members

were sampled for extended interviews, the public library

use questions were asked during the first extended

interview in the household.

with the screening experiment, since more data

were collected under the no screen-out condition.

The survey administration time discussed above

reflects effort expended on completed cases. Air

time reflects interviewing effort for both

responses and nonresponses. The total air time

was 520 hours in the Telephone Research Center

with the screen-out question and 645 hours in the

Center without the screen-out question. This

difference is primarily due to two factors: the

length of the interview for households with no

extended interviews (which was reflected in the

survey administration time discussed above) and

additional time for refusal conversion efforts.

The increase is nearly 25 percent above the air

time devoted to the telephone numbers that had

the screen-out question. Since air time is

probably more highly correlated to the total data

collection cost of the survey than any other single

measure, it is reasonable to speculate that the

cost of doing the no screen-out method and

collecting the household-level data (demographic,

educational, and library items) for all households

is about 25 percent greater than the cost of

conducting data collection with a screen-out

question under these or similar eligibility

conditions.

Conclusions

The results of this experiment were definitive.

The use of a screen-out question to eliminate

ineligible households prior to enumeration results

in considerably higher response rates. The cost

of interviewing is also about 25 percent lower for

this approach if the items included in the

expanded screening interview are similar to those

used in the NHES:96 and about one-third of all

households had members eligible for extended

interviews. Some of the additional resources

required for the no screen-out condition are

associated with the collection of these additional

data. However, the cost differential is also

associated with the need for a larger refusal

conversion effort.
20

In the wake of the lower than planned response

rates from the NHES:95, the screen-out question

approach could be used to obtain high response

rates in future NHES collections and other

surveys in which only a subset of households is

eligible for the extended component interviews.

Since the advance letter does not increase

Table 7.—Administration time of Screeners, by experimental condition and household type

Experimental condition and household type Number Time in minutes

Mean Minimum Maximum

Screen-out question

No extended interviews.............................. 1,213 2.1 0.7 23.4

Parent PFI/CI interview only ...................... 226 5.3 2.9 13.2

Parent PFI/CI and Youth CI interviews....... 282 5.9 2.9 16.0

No screen-out question

No extended interviews.............................. 877 8.1 3.6 35.7

Parent PFI/CI interview only ...................... 238 5.6 2.8 24.7

Parent PFI/CI and Youth CI interviews....... 266 6.5 3.3 15.9

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey

(NHES), field test of the 1996 survey.

response rates when the screen-out approach is

used, its use under a screen-out design should not

significantly increases response rates. However,

a letter might still be useful for refusal or other

nonresponse conversion purposes under these

conditions.

Despite the advantage of a higher response rate,

the screen-out approach is not acceptable if

estimates of the characteristics for all households

are necessary to meet the goals of the study. The

experiment showed that mailing an advance letter

was effective in increasing the Screener response

rate when the full enumeration of all households

was conducted. Thus, the advance mailing

should be used in such designs.

Under a full enumeration design, other

procedural methods besides the advance mailing

should be considered. As noted earlier, some

researchers (Bercini and Massey 1979) have

suggested that the lower response rates may be

related to the interviewers' perceptions of the

task. If this is true, then training methods that

emphasize the simplicity of the job might

improve response rates. For the NHES:96, the no

screen-out method was used and only personal

and household characteristics and a few library

items were collected in the 70 percent of

households that did not have extended interviews.

 The interviewers in the NHES:96 were trained to

expect a short interview for these households.

Despite this training approach, the Screener

response rate for the NHES:96 was 70 percent

which is in the lower end of the range predicted

based on the results of this experiment. It is likely

that interviewer’s actual experience in conducting

the survey overrides messages conveyed in

training about the ease or difficulty of the task.

Thus, training may not be adequate to overcome

the lower response rates.

Other procedural methods used in the NHES

have been shown to improve response under both

screen-out and full enumeration designs. These

include second refusal conversion attempts and

refielding nonresponse cases such as those with

maximum numbers of contact attempts and those
19

with only noncontact (e.g., answering machine)

results. Extending the data collection period to

allow refusals to be held for a longer period prior

to attempting conversions and to provide a longer

period over which to contact other nonresponse

cases may be helpful, but this approach needs to

be considered in terms of the attendant costs

versus benefits and the time constraints under

which the survey must operate.

During the development phase of a survey, it is

incumbent upon the researchers to consider the

balance between the substantive goals of the

study and the response and resource implications

of the designs that could be used to meet those

substantive goals. This experiment indicates that

the approach to enumeration used in an RDD

survey has implications both for the screening

response rate and for the resources required to

carry out the survey.
20
21

References

Alexander, C.H., Sebold, J., and Pfaff, P. (1986). Some results of an experiment with telephone sampling

for the U.S.-National Crime Survey. Proceedings of the American Statistical Association Section on

Survey Research Methods. Arlington, VA: American Statistical Association. p. 351-356.

Bercini, D.H., and Massey, J.T. (1979). Obtaining the household roster in a telephone survey: The impact

of names and placement on response rates. Proceedings of the American Statistical Association

Section on Social Statistics. Washington, DC: American Statistical Association. p. 136-140.

Brick, J.M. and Broene, P. (1997). Unit and Item Response Rates, Weighting, and Imputation

Procedures in the 1995 National Household Education Survey. NCES Working Paper No. 96-06. 

Washington, DC: U.S. Department of Education, National Center for Education Statistics.

Brick, J.M, Collins, M.A., and Chandler, K.A. (forthcoming) An Overview of Response Rates in the

National Household Education Survey. Washington, DC: U.S. Department of Education, National

Center for Education Statistics.

Brick, J.M., Collins, M.A., Nolin, M.J., Ha, P., Levinsohn, M., and Chandler, K. (1994). National

Household Education Survey of 1993: School Readiness Data File User's Manual. NCES

Publication No. 94-193. Washington, DC: U.S. Department of Education, National Center for

Education Statistics.

Bryant, B.E. (1975). Respondent Selection in a Time of Changing Household Composition. Journal of

Marketing Research. 12:129-135.

Carr, K., and Hertvik, J. (1993). Within-household selection: Is anybody listening? Proceedings of the

American Statistical Association Section on Survey Research Methods. Arlington, VA: American

Statistical Association. p. 1119-1123.

Camburn, D.P., Lavrakas, P.L., Battaglia, M.P., Massey, J.T., and Wright, R.A. (1995) Using advance

respondent letters in random-digit-dialing telephone surveys. Proceedings of the American Statistical

Association Section on Survey Methods. Arlington, VA: American Statistical Association. p. 969-

974.

Collins, M.A., Brick, J.M., Kim, K., Gilmore, S., and Stowe, P. (1996). National Household Education

Survey of 1995: Adult Education Data File User's Manual. NCES Publication No. 96-826. 

Washington, DC: U.S. Department of Education, National Center for Education Statistics.

Collins, M.A., Brick, J.M., Loomis, L.S., Nicchitta, P.G. and Fleishman, S. (1997) Design, Data

Collection, Interview Timing, and Data Editing in the 1995 National Household Education Survey

(NHES:95). NCES Working Paper No. 97-08. Washington, DC: U.S. Department of Education,

National Center for Education Statistics.

Collins, M.A., Brick, J.M., Nolin, M.J., Vaden-Kiernan, N., Gilmore, S., Chandler, K., and Chapman, C.

(forthcoming). National Household Education Survey of 1996: Data File Users Manual, Volume I.

NCES Publication No. 97-425. Washington, DC: U.S. Department of Education, Office of

Educational Research and Improvement, National Center for Education Statistics.
22

Czaja, R., Blair, J., Sebestik, J.R. (1982). Respondent selection in a telephone survey: A comparison of

the three techniques. Journal of Marketing Research. 19:381-185.

Forsman, G. (1993) Sampling individuals with households in telephone surveys. Proceedings of the

American Statistical Association Section on Survey Research Methods. Arlington, VA: American

Statistical Association. p. 1113-1118.

Friedrichs, J. (1987). Effects of mailing a questionnaire in a telephone survey. Paper presented at the

International Conference on Telephone Survey Methodology. Charlotte, NC. November 1987.

Groves, R., and Kahn, M. (1979). Surveys by Telephone: A National Comparison with Personal

Interviews. New York: Academic Press.

Groves, R.M., and Lyberg, L.E. (1988). An overview of nonresponse issues in telephone surveys, in

Groves, R.M., Biemer, P.P., Lyberg, L.E., Massey, J.T., Nicholls, W.L., and Waksberg, J., (eds). 

Telephone Survey Methodology. New York: John Wiley & Sons. p. 191-211.

Hagan, D.E., and Collier, C.M. (1983). Must respondent selection procedures for telephone surveys be

invasive? Public Opinion Quarterly. 47:547-556.

Kish, L. (1949). A procedure for objective respondent selection within the household. Journal of the

American Statistical Association. 44:380-387.

Lavrakas, P.J., Bauman, S.L., and Merkle, D.M. (1993). The last-birthday method and within-unit

coverage problems. Proceedings of the American Statistical Association Section on Survey Research

Methods. Arlington, VA: American Statistical Association. p. 1107-1112.

Oldendick, R.W., Bishop, G.F., Sorenson, S.B., Tuchfaber, A.J. (1988). A comparison of the Kish and

last-birthday methods of respondent selection in telephone surveys. Journal of Official Statistics. 

4:307-318.

O'Rourke, D., and Blair, J. (1983). Improving random respondent selection in telephone surveys. Journal

of Marketing Research. 20:428-432.

Romuald, K.S., and Haggard, L.M. (1994). The effect of varying the respondent selection script on

respondent self-selection in RDD telephone surveys. Proceedings of the American Statistical

Association Section on Survey Research Methods. Arlington, VA: American Statistical Association.

p. 1299-1304.

Salmon, C.T., and Nicholls, J.S. (1983). The next-birthday method of respondent selection. Public

Opinion Quarterly. 47:270-276.

Thornberry, O.T., and Poe, G. (1982). NCHS research on the telephone interview: some observations. 

Proceedings of the American Statistical Association Social Statistics Section. Arlington, VA: 

American Statistical Association. p. 296-301.

Troldahl, V.C., and Carter, R.E. (1964). Random selection of respondents within households in phone

surveys. Journal of Marketing Research. 1:71-76.
23

Appendix A:

NHES:96 Field Test Screener
24
25

NHES:96 Field Test Screener

SCRN

S1. Hello, this is (INTERVIEWER) and I'm calling about a research study sponsored by the

United States Department of Education. Are you a member of this household

and at least 18 years old?

HHMEMB18

YES.................................................................................... 1 (GO TO S4)

NO..................................................................................... 2 (GO TO S2)

PROBABLE BUSINESS............................................................ 3 (GO TO S4)

GO TO RESULT ................................................................. GT

S2. May I please speak with a household member who is at least 18 years old?

SPEAKMEM

AVAILABLE .......................................................................... 1 (GO TO S1)

NOT AVAILABLE.................................................................... 2 (GO TO RESULT, CALLBACK

 APPT.)

THERE ARE NONE ................................................................ 3 (GO TO S3A)

GO TO RESULT ................................................................. GT

S3A. May I please speak with the male or female head of this household?

SPEAKHH

PERSON ON PHONE.............................................................. 1 (GO TO S4)

OTHER PERSON, AVAILABLE .................................................. 2 (GO TO S3B)

OTHER PERSON, NOT AVAILABLE ........................................... 3 (GO TO RESULT, CALLBACK

 APPT.)

GO TO RESULT ................................................................. GT

S3B. Hello, this is (INTERVIEWER) and I'm calling about a research study sponsored

by the United States Department of Education. Are you a head of this household?

HHMEMBHH

YES.................................................................................... 1 (GO TO S4)

NO..................................................................................... 2 (GO TO S3A)

GO TO RESULT ................................................................. GT

S4. Is this phone used for...

SFONEUSE

Home use, ........................................................................ 1 (CONTINUE)

Home and business use, or............................................... 2 (CONTINUE)

Business use only? ........................................................... 3 (GO TO THANK1)

GO TO RESULT ................................................................. GT

For Parent/Youth Interview, read SCRN_20.

For Adult CI Interview, read PRE_LIB.
26

SCRN_20. The U.S. Department of Education is conducting a voluntary and confidential study about the

educational experiences of children and how they learn about their communities and

government. We are also interested in how all households use public libraries. Are any of the

people who normally live in your household age 20 or younger?

YES.................................................................................... 1 (READ STMT1)

NO..................................................................................... 2 (READ STMT2)

GO TO RESULT ................................................................. GT

Split-half 1: If SCRN_20 = 2, then terminate, else continue. 

Split-half 2: Continue.

STMT 1. These next questions are about the people in your household and usually take 5 to 7 minutes.

(GO TO S6.)

STMT 2. Then I just have a few questions about libraries and about the people in your household. They

usually take 5 to 7 minutes. (GO TO LINTRO.)

PRE_LIB. The U.S. Department of Education is conducting a voluntary and confidential study about how

people use public libraries and how they learn about their communities and government. These

questions usually take 10 to 15 minutes.

First, I'd like to ask about libraries. (GO TO LINTRO.)

PRE_S6. Now I'd like to ask about the people in your household and their education and background.

ENUM

S6. Starting with yourself, please tell me just the first names and ages of all the people who normally

live in your household. What is your first name, please?

[HOUSEHOLD MEMBERS INCLUDE PEOPLE WHO THINK OF THIS HOUSEHOLD AS THEIR PRIMARY PLACE OF

RESIDENCE. IT INCLUDES PERSONS WHO USUALLY STAY IN THE HOUSEHOLD BUT ARE TEMPORARILY

AWAY ON BUSINESS, VACATION, IN A HOSPITAL, OR LIVING AT SCHOOL IN A DORM, FRATERNITY, OR

SORORITY.]

What is

[your first

name/the first

name of the

next person]?

How old

[are you/

is (he/she)]?

Is this person

male or

female?

SCREENER

RESPONDENT

FNAME AGE SEX screspx

S6VERF1. [VERIFY THE NUMBER OF HOUSEHOLD MEMBERS LISTED ON THE MATRIX.] Have we missed

anyone else who usually lives here who is temporarily away from home or living

in a dorm at school, or any babies or small children?

MATRIX CORRECT ................................................................ 1

RETURN TO MATRIX.............................................................. 2

GO TO RESULT ................................................................. GT

If AGE >= 3, -7, -8 (person age 3 or older), ask SX7.
27

SX7. [Are you/Is (PERSON)] attending (or enrolled in) (school/nursery school,

kindergarten, or school)?]

SENROLL

YES ................................................................................... 1

NO ................................................................................... 2

REFUSED...........................................................................-7

DON'T KNOW......................................................................-8

If AGE >= 18 or -7, -8 (person age 18 or older), autocode SX8

= 2 (not home schooled) and go to box after SX8. Else, if

AGE = 5-17, ask SX8. Else, if SX7 = 1 (person is enrolled in

school), go to SX9. Else, go to first box after SX12.

SX8. (READ FIRST TIME: Some parents decide to educate their children at home rather

than sending them to school.) Is (CHILD) being schooled at home?

SHOMESCH

YES ................................................................................... 1 (GO TO SX8A)

NO ................................................................................... 2 (GO TO BOX AFTER SX8A)

REFUSED...........................................................................-7 (GO TO BOX AFTER SX8A)

DON'T KNOW......................................................................-8 (GO TO BOX AFTER SX8A)

SX8A. So your child is being schooled at home instead of at school?

SHOMECON

YES ................................................................................... 1 (GO TO SX10)

NO ................................................................................... 2 (RECODE SX8=2 AND GO TO

GO TO BOX)

REFUSED...........................................................................-7 (RECODE SX8 = -7 AND

GO TO BOX)

DON'T KNOW......................................................................-8 (RECODE SX8 = -8 AND

 GO TO BOX)

If SX7 = 1 (person is enrolled in school), go to SX9.

Else, go to first box after SX12.

ENUM
28

SX9. What grade or year of school [are you/is (PERSON)] attending?

[PROBE FOR T OR P: Is that before or after kindergarten?]

SGRADE

NURSERY/PRESCHOOL/PREKINDERGARTEN/HEAD START ......... N (GO TO SX11)

TRANSITIONAL KINDERGARTEN (BEFORE K) ............................ T (GO TO SX11)

KINDERGARTEN .................................................................. K (GO TO SX11)

PREFIRST GRADE (AFTER K)................................................. P (GO TO SX11)

FIRST GRADE ..................................................................... 1 (GO TO SX11)

SECOND GRADE.................................................................. 2 (GO TO SX11)

THIRD GRADE..................................................................... 3 (GO TO SX11)

FOURTH GRADE.................................................................. 4 (GO TO SX11)

FIFTH GRADE...................................................................... 5 (GO TO SX11)

SIXTH GRADE ..................................................................... 6 (GO TO SX11)

SEVENTH GRADE ................................................................ 7 (GO TO SX11)

EIGHTH GRADE................................................................... 8 (GO TO SX11)

NINTH GRADE/FRESHMAN IN HIGH SCHOOL ............................. 9 (GO TO SX11)

TENTH GRADE/SOPHOMORE IN HIGH SCHOOL ........................ 10 (GO TO SX11)

ELEVENTH GRADE/JUNIOR IN HIGH SCHOOL ........................... 11 (GO TO SX11)

TWELFTH GRADE/SENIOR IN HIGH SCHOOL ............................ 12 (GO TO SX11)

UNGRADED ELEMENTARY/SECONDARY ...................................U (GO TO SX10)

SPECIAL EDUCATION ............................................................S (GO TO SX10)

VOCATIONAL/TECHNICAL AFTER HIGH SCHOOL....................... 15 (GO TO SX9A)

COLLEGE (UNDERGRADUATE) ............................................. 16 (GO TO SX9B)

GRADUATE, PROFESSIONAL SCHOOL.................................... 17 (GO TO SX9C)

REFUSED...........................................................................-7 (RECODE SX8 = -7 AND

 GO TO BOX)

DON'T KNOW......................................................................-8 (RECODE SX8 = -8 AND

GO TO BOX)

[IF T: In this interview, we will be referring to that as "kindergarten." 

IF P: In this interview, we will be referring to that as "prefirst grade."]

EXPA

SX9A. In terms of credits earned and requirements fulfilled, what year of vocational/

technical school [are you/is (PERSON)] in now?

XVOCYEAR

FIRST ................................................................................. 1 (GO TO SX11)

SECOND OR HIGHER............................................................. 2 (GO TO SX11)

REFUSED...........................................................................-7 (GO TO SX11)

DON'T KNOW......................................................................-8 (GO TO SX11)

SX9B. What is [your/(PERSON's)] class standing? That is, [are you/is (PERSON)] a freshman,

sophomore, junior, or senior?

XCOLYEAR

FRESHMAN.......................................................................... 1 (GO TO SX11)

SOPHOMORE....................................................................... 2 (GO TO SX11)

JUNIOR............................................................................... 3 (GO TO SX11)

SENIOR .............................................................................. 4 (GO TO SX11)

REFUSED...........................................................................-7 (GO TO SX11)

DON'T KNOW......................................................................-8 (GO TO SX11)
29

SX9C. In terms of credits earned and requirements fulfilled, what year of graduate

or professional school [are you/is (PERSON)] in now?

XGRAYEAR

FIRST ................................................................................. 1 (GO TO SX11)

SECOND ............................................................................. 2 (GO TO SX11)

THIRD................................................................................. 3 (GO TO SX11)

FOURTH OR HIGHER ............................................................. 4 (GO TO SX11)

REFUSED...........................................................................-7 (GO TO SX11)

DON'T KNOW......................................................................-8 (GO TO SX11)

ENUM

SX10. What grade would [you/(PERSON)] be in if [you/(he/she)] were (attending a school/

attending a school with regular grades)?

SGRADEQ [PROBE FOR T OR P: Is that before or after kindergarten?]

NURSERY/PRESCHOOL/PREKINDERGARTEN/HEAD START ......... N

TRANSITIONAL KINDERGARTEN (BEFORE K)............................ T

KINDERGARTEN .................................................................. K

PREFIRST GRADE (AFTER K)................................................. P

FIRST GRADE ..................................................................... 1

SECOND GRADE.................................................................. 2

THIRD GRADE..................................................................... 3

FOURTH GRADE.................................................................. 4

FIFTH GRADE...................................................................... 5

SIXTH GRADE ..................................................................... 6

SEVENTH GRADE ................................................................ 7

EIGHTH GRADE................................................................... 8

NINTH GRADE/FRESHMAN IN HIGH SCHOOL ............................. 9

TENTH GRADE/SOPHOMORE IN HIGH SCHOOL ....................... 10

ELEVENTH GRADE/JUNIOR IN HIGH SCHOOL ........................... 11

TWELFTH GRADE/SENIOR IN HIGH SCHOOL ............................ 12

UNGRADED/NO EQUIVALENT..................................................U

VOCATIONAL/TECHNICAL AFTER HIGH SCHOOL....................... 15

COLLEGE (UNDERGRADUATE).............................................. 16

GRADUATE, PROFESSIONAL SCHOOL.................................... 17

REFUSED...........................................................................-7 (RECODE SX8 = -7 AND

GO TO BOX)

DON'T KNOW......................................................................-8 (RECODE SX8 = -8 AND

GO TO BOX)

[IF T: In this interview, we will be referring to that as "kindergarten." 

IF P: In this interview, we will be referring to that as "prefirst grade."]

If SX8 = 1 (in home school), go to first box after SX12.

Else, ask SX11.

EXPA
30

SX11. [Do you/Does (PERSON)] go to a public or a private school?

XPUBL PUBLIC ............................................................................... 1

PRIVATE ............................................................................. 2

REFUSED...........................................................................-7

DON'T KNOW......................................................................-8

If SX9 or SX10 = N, T, or K or SX7 = 1 and AGE >=16 or -7, -

8 (person enrolled in nursery school or kindergarten or age 16

or older and currently enrolled in school), then ask SX12. 

Else, go to first box after SX12.

SX12. [Are you/Is (PERSON)] now enrolled in school full time or part time?

XFULTIM

FULL TIME ........................................................................... 1

PART TIME .......................................................................... 2

REFUSED...........................................................................-7

DON'T KNOW......................................................................-8

EXPA

Ask SX7 to SX12 for next person enrolled in school.

After last person, go to next box.

If (SX7 and SX8 = 2, -7, -8) or (SX9 or SX10 = 15, 16, 17, -7,

-8, and AGE >= 16 or -7, -8) (person age 16 and older who is

not currently enrolled in grade/equivalent 12 or below,

ungraded elementary or secondary, or special education),

then ask SX13 to SX15. Else, go to first box after SX15.
31

SX13. [Now I have a few questions about (you/you and the other adults(s) in your household).] 

What is the highest grade or year of school that [you/(ADULT)] completed?

XGRADE

XGRAD1 UP TO 8TH GRADE................................................................ 1 (ENTER ACTUAL GRADE,

 GO TO SX14)

XGRAD2 9TH TO 11TH GRADE............................................................ 2 (ENTER ACTUAL GRADE,

 GO TO SX14)

12TH GRADE BUT NO DIPLOMA............................................... 3 (GO TO SX14)

HIGH SCHOOL DIPLOMA/EQUIVALENT ...................................... 4 (GO TO SX15)

VOC/TECH PROGRAM AFTER HIGH SCHOOL BUT NO

 VOC/TECH DIPLOMA ........................................................... 5 (GO TO SX14)

VOC/TECH DIPLOMA AFTER HIGH SCHOOL ............................... 6 (GO TO SX14)

SOME COLLEGE BUT NO DEGREE............................................ 7 (GO TO SX14)

ASSOCIATE'S DEGREE .......................................................... 8 (GO TO BOX AFTER SX14)

BACHELOR'S DEGREE........................................................... 9 (GO TO BOX AFTER SX14)

GRADUATE OR PROFESSIONAL SCHOOL BUT NO DEGREE ........ 10 (GO TO BOX AFTER SX14)

MASTER'S DEGREE (MA, MS) ............................................... 11 (GO TO BOX AFTER SX14)

DOCTORATE DEGREE (PHD, EDD) ........................................ 12 (GO TO BOX AFTER SX14)

PROFESSIONAL DEGREE AFTER BACHELOR'S DEGREE

 (MEDICINE/MD; DENTISTRY/DDS; LAW/JD/LLB; ETC.)............. 13 (GO TO BOX AFTER SX14)

REFUSED...........................................................................-7 (GO TO SX14)

DON'T KNOW......................................................................-8 (GO TO SX14)

ENUM

SX14. [Do you/Does (ADULT)] have a high school diploma or its equivalent,

such as a GED?

SDIPLOMA

YES ................................................................................... 1

NO. ................................................................................... 2

REFUSED...........................................................................-7

DON'T KNOW......................................................................-8

EXPA

Ask SX13 to SX14 for next person age 16 and older who is

not currently enrolled in grade 12 or below, ungraded

elementary or secondary, or special education. After last

person, go to next box.

If AGE >16 or -7, -8 (person age 16 or older), then ask SX15.

 After last person, go to SX16.
32

SX15. What is [your/(ADULT'S)] marital status? [VERIFY IF KNOWN.]

XMARSTAT

MARRIED/REMARRIED........................................................... 1

SEPARATED ........................................................................ 2

DIVORCED .......................................................................... 3

WIDOWED........................................................................... 4

NEVER MARRIED .................................................................. 5

REFUSED...........................................................................-7

DON'T KNOW......................................................................-8

XSCR

SX16. Not counting the Reserves or National Guard, (are you/is any member of

your household) currently serving on active duty in the U.S. Armed Forces?

XHHACTV

YES ................................................................................... 1 (GO TO BOX)

NO. ................................................................................... 2 (GO TO SX17)

REFUSED...........................................................................-7 (GO TO SX17)

DON'T KNOW......................................................................-8 (GO TO SX17)

EXPA

If SX16 = 1 (on active duty) and respondent is the only adult in

the household, autocode SX16OV to respondent's person

number. Else, go to SX16OV.

SX16OV. (Who is that?) [DISPLAY HOUSEHOLD MEMBERS WITH AGE >= 16. CODE ALL

THAT APPLY. IF RESPONDENT IS THE ONLY HOUSEHOLD MEMBER, CODE THE PERSON

NUMBER OF THE RESPONDENT.]

XACTVDUT

PERSON NUMBER........................................................... (INELIGIBLE)

REFUSED...........................................................................-7

DON'T KNOW......................................................................-8

XSCR

SX17. (Were you/Was everyone in your household) born in this country, that is,

in one of the 50 States or the District of Columbia?

XHHBORN

YES.................................................................................... 1 (GO TO SX18)

NO..................................................................................... 2 (GO TO BOX AFTER SX18)

REFUSED...........................................................................-7 (GO TO BOX AFTER SX18)

DON'T KNOW......................................................................-8 (GO TO BOX AFTER SX18)
33

SX18. (Did you/Did every member of your household) learn English as (your/their)

first language?

XHHLANG

YES.................................................................................... 1 (GO TO 1ST BOX AFTER

 SX20)

NO..................................................................................... 2 (GO TO BOX)

REFUSED...........................................................................-7 (GO TO BOX)

DON'T KNOW......................................................................-8 (GO TO BOX)

If SX17 = 2, -7, -8 (not every household member was born in

the U.S.), then ask SX19 for each person in the household;

also ask SX20 for each person where AGE >= 3, -7, -8 (age 3

or older). If SX18 = 2, -7, -8 (not every household member

learned English as their first language), ask SX20 for each

person where AGE >= 3 or -7, -8 (age 3 or older).

EXPA

SX19. In what country [were you/was (PERSON)] born?

XBORNUS

50 STATES OR THE DISTRICT OF COLUMBIA............................ 1

U.S. TERRITORIES: PUERTO RICO, GUAM, AMERICAN SAMOA,

MARIANA ISLANDS, U.S. VIRGIN ISLANDS, OR SOLOMON ISLANDS2

XBORNOS1 (SPECIFY) _____________________________________________

SOME OTHER COUNTRY ........................................................ 3

XBORNOS2 (SPECIFY) _____________________________________________

REFUSED...........................................................................-7

DON'T KNOW......................................................................-8

SX20. What was the first language [you/(PERSON)] learned to speak?

XLANG ENGLISH............................................................................. 1

SPANISH............................................................................. 2

SPANISH AND ENGLISH EQUALLY............................................ 3

OTHER LANGUAGE ............................................................ 91

XLANGOS (SPECIFY) ______________________________________________

REFUSED -7

DON'T KNOW......................................................................-8

Ask SX21 and SX22 for each person. After last person, go to

first box after SX22 (Sampling Point).
34

SX21. [Are you/Is (PERSON)] ....

[IF R GIVES RACE AND ALSO SAYS HE/SHE IS OF HISPANIC ORIGIN, CODE RACE HERE.]

EXPA.RACE

White................................................................................ 1 (GO TO SX22)

Black ................................................................................ 2 (GO TO SX22)

American Indian or Alaskan Native................................... 3 (GO TO SX22)

Asian or Pacific Islander, or.............................................. 4 (GO TO SX22)

Some other race? ............................................................. 5 (GO TO SX21A)

REFUSED...........................................................................-7 (GO TO SX21)

DON'T KNOW......................................................................-8 (GO TO SX21)

SX21A. [CODE RESPONSE IF SX21 = 5.]

EXPA.OTH

HISPANIC/LATINO/MEXICAN/SPANISH/

PUERTO RICAN.................................................................... 1 (AUTOCODE SX22=1 AND

GO TO BOX AFTER SX22)

MORE THAN ONE RACE/BIRACIAL/MULTIRACIAL ........................ 2

OTHER ............................................................................. 91

(SPECIFY) _____________________________________________

SX22. [ARE YOU/IS (PERSON)] OF HISPANIC ORIGIN?

EXPA.HISPANc

YES ................................................................................... 1

NO ................................................................................... 2

REFUSED...........................................................................-7

DON'T KNOW......................................................................-8

Sampling Point:

Select children and adults for extended interviews.

If any children are selected, go to next box.

If adults only are selected, go to second box after SX24.

If no one is selected, go to LINTRO.

Ask SX23 and SX24 for each sampled child. If there is only

one household member, other than the sampled child, who is

at least 12 years older than that child, autocode SX23 to this

adult. If the sampled child is the only or oldest person in the

household, go to THANK2.

ENUM
35

SX23. We would like to ask some questions about (your/(CHILD)'s)

(care and) education.

RESPNUM [IF SCREENER RESPONDENT IS OBVIOUSLY CHILD'S MOTHER, ASK:

 Are you (CHILD)'s mother? IF YES, ENTER HER PERSON NUMBER.]

[IF SCREENER RESPONDENT MIGHT NOT BE CHILD'S MOTHER, ASK:

 Who is the parent or guardian in this household who knows

 the most about (your/(CHILD)'s) (care and) education?] 

[DISPLAY HOUSEHOLD MEMBERS 16 YEARS OLD AND OLDER.]

PERSON NUMBER ..........................................................

If person number at SX23 NE person number of sampled child,

ask SX24 for each sampled child, and then go to HHSELECT

screen to select interview. If person number at SX23 = person

number of sampled child, display household members at least

12 years older than the sampled child and ask SX24 for each

household member listed. The respondent for the parent

interview will be selected in the following order of relationship:

mother, father, grandmother, aunt, sister, grandfather, uncle,

brother, cousin, other relative (but not husband/wife), or

nonrelative (but not boyfriend/girlfriend). If no household

member is so designated, the sampled child is ineligible; go to

LINTRO. If a respondent for the parent interview is selected, go

to HHSELECT screen to select interview.

SX24. What is [your/(CAREGIVER'S)/(PERSON'S)] relationship to [(CHILD)/you]?

[VERIFY IF KNOWN]

PARRELN MOTHER (BIRTH/ADOPTIVE/STEP/FOSTER) ............................... 1

FATHER (BIRTH/ADOPTIVE/STEP/FOSTER) ................................ 2

BROTHERS, INCLUDING STEP,

 ADOPTED, AND FOSTER....................................................... 3

SISTERS, INCLUDING STEP,

 ADOPTED, AND FOSTER....................................................... 4

GRANDPARENT..................................................................... 5

AUNT................................................................................... 6

UNCLE................................................................................. 7

COUSIN ............................................................................... 8

OTHER RELATIVE/GUARDIAN (BUT NOT HUSBAND/WIFE)............. 9

NONRELATIVE/GUARDIAN (BUT NOT BOYFRIEND/GIRLFRIEND) ... 10

HUSBAND/WIFE/BOYFRIEND/GIRLFRIEND................................ 11

REFUSED ............................................................................-7

DON'T KNOW .......................................................................-8
36

If PARRELN = 5 (grandparent) use that person's sex with

PARRELN to set RELATION and drive displays.

If sampled adult is not the screener respondent and SX7 = 1

and SX9 = 15, 16, 17 (enrolled in college, graduate school or

vocational/technical school after high school) for that person,

ask SX25. Else, go to HHSELECT (Adult CI extended

interview).

SX25. Is (ADULT) living there, in student housing, or somewhere else?

LIVENOW

HERE ................................................................................. 1 (GO TO HHSELECT)

STUDENT HOUSING [This includes all housing owned,

 sponsored, or leased by the school such as a

 dormitory or fraternity or sorority house.] .................. 2 (GO TO SX26)

OTHER PRIVATE HOME OR APARTMENT................................... 3 (INELIGIBLE, GO TO THANK2)

INSTITUTION OR GROUP QUARTERS [THIS INCLUDES

 A JAIL OR DETENTION CENTER, MEDICAL FACILITY,

 REHABILITATION CENTER, MENTAL HEALTH FACILITY,

 MILITARY BARRACKS, OR GROUP FOSTER CARE.]................... 4 (INELIGIBLE, GO TO THANK2)

REFUSED...........................................................................-7 (GO TO HHSELECT)

DON'T KNOW......................................................................-8 (GO TO HHSELECT)

SX26. Would you please give me (his/her) last name and telephone number

so that we can call (him/her) to do a brief interview about activities

related to civic involvement?

LNAME LAST NAME _______________________________________

ENUM.ENUMAREA PHONE 

EXCH REFUSED .................................................................. -7

LOCL DON'T KNOW................................................................-8

1996 Topical Component: Household Public Library Usage

LINTRO. (These next questions are about/We are interested in) public libraries. 

This does not include school or college libraries, or special research

libraries. 

L1. About how far would you say it is from your home to the closest public library?

Would you say...

LDISTANC

Less than 1 mile, .............................................................. 1

1 or 2 miles,...................................................................... 2

3 to 5 miles,...................................................................... 3

6 to 10 miles, or................................................................ 4

More than 10 miles? ......................................................... 5

REFUSED...........................................................................-7

DON'T KNOW......................................................................-8
37

L2. People use public libraries in a number of ways. In the past month, that is,

since (MONTH) (DAY), has any member of your household used a public library

in the following ways? How about...

YES NO R DK

LVISIT1 a. Going to a public library to borrow or drop off

books or tapes? ......................................................................... 1 2 -7 -8

LVISIT2 b. Going to a public library for any other purpose, such as

a lecture or story hour, or to use their equipment? ..................... 1 2 -7 -8

LCOMP c. Using a home computer to link to a public library?..................... 1 2 -7 -8

LPHONE d. Calling the public library to renew books or for

information other than library hours or directions?...................... 1 2 -7 -8

LMATLS e. Having library materials mailed or delivered to your

home? ....................................................................................... 1 2 -7 -8

LMOBILE f. Visiting a bookmobile?............................................................... 1 2 -7 -8

If L2a through L2f = 2 (no use of public libraries), ask L3. 

Else, go to L4.

L3. Has anyone in your household used a public library in any of those ways

in the past year?

LYRUSE

YES ................................................................................... 1 (GO TO BOX AFTER L4)

NO ................................................................................... 2 (GO TO BOX AFTER L4)

REFUSED...........................................................................-7 (GO TO BOX AFTER L4)

DON'T KNOW......................................................................-8 (GO TO BOX AFTER L4)

L4. In the past month, that is since (MONTH) (DAY), has any member of your

household used a public library (or bookmobile) for the following purposes? 

How about...

YES NO R DK

LSCHOOL a. For a school or class assignment?............................................. 1 2 -7 -8

LKIDSACT b. A program or activity designed for children age 6 to 12?............ 1 2 -7 -8

LKIDBOOK c. An activity for children under 6, such as story hour

 or other introduction to books and reading? ............................. 1 2 -7 -8

LRECR d. For enjoyment or hobbies, including to borrow

 books or tapes or attend activities?.......................................... 1 2 -7 -8

LJOBHELP e. To get information to help find a job?......................................... 1 2 -7 -8

LWORK f. For a work assignment or to keep up to date

 at work?................................................................................... 1 2 -7 -8

LCONSUME g. To get information for personal use, such as

 consumer or health issues, investments, and so on? ............... 1 2 -7 -8

LLRNREAD h. To work with a tutor or take a class to learn

 to read?................................................................................... 1 2 -7 -8
38

If library items are administered before matrix, go to PRE_S6. If

library items are administered in the Parent Interview, go to

SX27.

Household Characteristics XSCR

SX27. Now a few more questions about your household. Do you...

HOWNHOME Own your home, ............................................................... 1

Rent your home, or........................................................... 2

Have some other arrangement? ....................................... 3

REFUSED...........................................................................-7

DON'T KNOW......................................................................-8

SX28. Besides (PHONE NUMBER), do you have other telephone numbers

in your household?

HOTHNUM

YES ................................................................................... 1 (GO TO SX29)

NO ................................................................................... 2 (GO TO SX30)

NOT MY NUMBER [RECORD NUMBER IN COMMENTS].................. 3 (GO TO SX30)

REFUSED...........................................................................-7 (GO TO SX30)

DON'T KNOW......................................................................-8 (GO TO SX30)

SX29. How many of these additional telephone numbers are for home use?

HNUMUSE

NUMBER.......................................................................

REFUSED...........................................................................-7

DON'T KNOW......................................................................-8

SX30. So that we can group households geographically, may I have

your ZIP code?

HZIPCODE

ZIP CODE...........................................................

REFUSED...........................................................................-7

DON'T KNOW......................................................................-8

SX31. Which of these best describes the community where you live? Is it...

HCCOMMUN

A rural or farming community, .......................................... 1

A suburb of a city,............................................................. 2

A small city or town of fewer than

 50,000 people, or............................................................ 3

A city of 50,000 people or more? ...................................... 4

REFUSED...........................................................................-7

DON'T KNOW......................................................................-8
39

If SX31 = 2 (suburb), go to SX31OV. Else, if SX31 = 4 (city of

50,000 people or more), go to SX31OV2.

Else, go to box after SX31OV2.

SX31OV. Is it a suburb of...

HCSUB

A city with over 500,000 people, ....................................... 1 (GO TO BOX AFTER SX31OV2)

A city with 100,000 to 500,000, or..................................... 2 (GO TO BOX AFTER SX31OV2)

A city with 50,000 to 100,000? .......................................... 3 (GO TO BOX AFTER SX31OV2)

REFUSED...........................................................................-7 (GO TO BOX AFTER SX31OV2)

DON'T KNOW......................................................................-8 (GO TO BOX AFTER SX31OV2)

SX31OV2. Is it ...

HCCITY

A city with over 500,000 people, ....................................... 1

A city with 100,000 to 500,000, or..................................... 2

A city with 50,000 to 100,000? .......................................... 3

REFUSED...........................................................................-7

DON'T KNOW......................................................................-8

Ask SX32 if NUMKID20 (number of children age 20 or

younger) >= 1. Else, go to SX33.

SX32. In the past 12 months, has your family received funds or services

from any of the following programs? How about...

YES NO R DK

HWIC a. Women, Infants, and Children, or WIC? ........................1 2 -7 -8

HFOODST b. Food Stamps? ...............................................................1 2 -7 -8

HAFDC c. AFDC, or Aid to Families with Dependent

 Children?.....................................................................1 2 -7 -8

SX33. In studies like this, households are sometimes grouped according to income. 

What was the total income of all persons in your household over the past year,

including salaries or other earnings, interest, retirement, and so on for all

household members?
40

Was it...

HINCMRNG

$25,000 or less, or ................................................................1 (READ SET 1)

More than $25,000?..............................................................2 (READ SET 2)

REFUSED ............................................................................... -7 (GO TO THANK2)

DON'T KNOW .......................................................................... -8 (GO TO THANK2)

Was it...

HINCOME

[SET 1]

$5,000 or less .......................................................................1

$5,001 to $10,000 .................................................................2

$10,001 to $15,000 ...............................................................3

$15,001 to $20,000, or..........................................................4

$20,001 to $25,000? .............................................................5

[SET 2]

$25,001 to $30,000 ...............................................................6

$30,001 to $35,000 ...............................................................7

$35,001 to $40,000 ...............................................................8

$40,001 to $50,000 ...............................................................9

$50,001 to $75,000, or.........................................................10

Over $75,000?.....................................................................11

REFUSED ............................................................................... -7 (GO TO THANK2)

DON'T KNOW .......................................................................... -8 (GO TO THANK2)

Ask SX33OV if

(Number in HH = 2 and HINCOME < = 2) or

 (Number in HH = 3 and HINCOME < = 3) or

 (Number in HH = 4 and HINCOME < = 3) or

 (Number in HH = 5 and HINCOME < = 4) or

 (Number in HH = 6 and HINCOME < = 4) or

 (Number in HH = 7 and HINCOME < = 5) or

 (Number in HH = 8 and HINCOME < = 5) or

 (Number in HH = 9 and HINCOME < = 6) or

 (Number in HH = 10 and HINCOME < = 6) or

 (Number in HH = 11 and HINCOME < = 7) or

(Number in HH = 12 and HINCOME < = 7).

Else, go to THANK2.

SX33OV. What was your total income last year, to the nearest thousand?

HINCMEXT

AMOUNT ........................................................... $ ,

REFUSED ..............................................................................-7

DON'T KNOW .........................................................................-8

THANK1. Thank you, but we are only interviewing in private residences.

THANK2. Those are all the questions I have about your household. Thank you for your time.
41

Appendix B:

Advance Letter
42
43

August, 1995

Dear Potential Study Participant,

I am writing to strongly encourage your participation in an upcoming telephone survey, the National

Household Education Survey (NHES). One purpose of the NHES is to find out how families and schools

work together to shape the learning experiences of children. We are also interested in how both young

people and adults learn about and are involved in their communities and their country. Finally, we want to

know about how people use public libraries. Your household may be asked about one or more of these

topics.

The NHES is sponsored by the National Center for Education Statistics of the United States

Department of Education. Your telephone number was selected for the study as part of a scientific random

sample of all households in the nation, and another number cannot be substituted for yours. Your household

represents thousands of households in the United States. It is very important that you take part in our survey

so that our results show a true picture for the whole nation.

Westat, Inc., a professional research firm, will be conducting the NHES. The study will take place

from January 2 to March 31. Sometime during that time, a Westat interviewer will call you. If we happen

to call at an inconvenient time, please suggest a time that is better for you. If you would like to set an

appointment anytime before we call, contact Westat at their toll-free number (1-800-862-9452). You will

need to give your phone number and your preferred appointment time.

 Some questions and answers about the NHES are printed on the back of this letter. I hope you will

take part in this very important study.

Sincerely,

Kathryn Chandler

Project Officer

National Household Education Survey
44

Some Frequently Asked Questions about

the National Household Education Survey (NHES)

Q. How will the study results be used? What will you do with this information?

A. The information we collect will tell us about people's experiences with schools, libraries, and their

communities, and help us understand educational experiences and needs. Some information from

the study will be published in the annual report on the National Education Goals. Other findings

will be published in U.S. Department of Education reports. Those reports will be widely

distributed to educators, researchers, news organizations, and the general public.

Q. How did you get my telephone number?

A. Your telephone number was randomly selected from all of the possible telephone numbers in the

nation. We do not use telephone directories to select telephone numbers.

Q. How did you get my address?

A. After the telephone numbers were randomly selected, we sent them to a company which provided

addresses for those telephone numbers. That company gave us the file that was used to address the

envelopes for this mailing. No records were kept of the addresses after this mailing was completed,

and addresses are not on the file that contains the telephone numbers. Interviewers do not have the

names or addresses for any telephone numbers.

Q. Will you keep my information confidential?

A. All information you give to the interviewer will be kept completely confidential. All employees of

the U.S. Department of Education and Westat who are working on this study are required by law to

protect the confidentiality of respondents. Also, individual responses are never published in reports;

they are added to the responses of others and are published as combined information only.

Q. How long will the survey take?

A. All households are asked to participate in an initial interview that usually takes 5 to 7 minutes. This

includes questions about household members and about public library use. We use information

about household members to determine whether anyone in your household will be selected for an

additional interview. In more than half of the households, no one is selected for an additional

interview. In other households, we ask questions about how families and schools work together to

shape children's learning experiences and/or how young people and adults learn about and are

involved in their communities and their country. These additional interviews take about 15 to 20

minutes.