NATIONAL CENTER FOR EDUCATION STATISTICS

Technical Report December 1996

National Household Education Survey of 1993

Adjusting for Coverage Bias

Using Telephone Service

Interruption Data

m- .

National

Household

Education

Survey

J. Michael Brick

Scott Keeter

Joseph Wakaberg

Bridgett Bell

Weatat, Inc.

Kathryn Chendler

Project Offiier

Natiinal Center for Education Statiatica

U.S. Department of Education

OfHce of Educational Research and Improvement NCES 97-336
U.S. Department of Education

Richard W. Riley

Secretary

Office of Educational Research and Improvement

Sharon P. Robinson

Assisfant Secreta~

National Center for Education Statistics

Pascal D. Forgione, Jr.

Commissioner

The National Center for Education Statistics (NCES) is the primaty federal entity for collecting, analyzing,

and reporting data related to education in the United States and other nations. It fulfills a congressional

mandate to collect, collate, analyze, and reporl full and complete statistics on the condition of education in

the United States; conduct and publish reports and specialized analyses of the meaning and significance

of such statistics; assist state and local education agencies in improving their statistical systems; and

review and report on education activities in foreign countries.

NCES activities are designed to address high priority education data needs: provide consistent, reliable,

complete, and accurate indicators of education status and trends; and report timely, useful, and high

quality data to the U. S, Department of Education, the Congress, the states, other education policy makers,

practitioners, data users, and the general public.

We strive to make our products available in a variety of formats and in language that is appropriate to a

variety of audiences. You, as our customer, are the best judge of our success in communicating

information effectively. If you have any comments or suggestions about this or any other NCES product or

report, we would like to hear from you. Please direct your comments to:

National Center for Education Statistics

Office of Educational Research and Improvement

U.S. Department of Education

555 New Jersey Avenue NW

Washington, DC 20208–5574

December 1996

The NCES World Wide Web Home Page is

hnp://www.ed.govlNCESl

Suggested Citation

U.S. Department of Education. National Center for Education Statistics. Adjusting for Coverage Bias Using

Te/ephone Service hrterrupfion L3afa, NCES 97-336, by J. Michael Brick, Scott Keeter, Joseph Waksberg,

and Bridgett Bell. Kathryn Chandler, project otlcer. Washington, DC: 1996.

Contack

Kathryn Chandler

(202)219-1716

Kathryn_Chandler@ed. gov

For single cooies. cell the National Education Data Resource Center at (703) 845-3151
,.

Table of Contents

Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...................

Estimates of Interrrrptions of Telephone Sewice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Characteristics of Those With Service hmmptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Adjustment Schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Effect of Adjustments on Variarrce . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Effect of Adjustments on Mean Square Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Coverage Bias Reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Bias Ratio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .................

Consistency Across Adjustment Schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Variance Implications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Lkt of Tables

Table 1.

Table 2.

Table 3.

Table 4.

Table 5.

Table 6.

Table 7.

Table 8.

Table 9,

1

2

3

6

7

13

15

17

17

23

24

24

32

34

34

Estimated percentage of persons in the School Readhress populating with

interruptions in telephone service in last 12 months, by selected characteristics. . . . . . . . . . 8

Estimated percentage nf persons in the School Safety and Discipline population with

interruptions in telephone service in last 12 months, by selected characteristics... . . . 10

Weighting cell adjustments factors, based on length of intermption of telephone

service.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Ratios of variance inflation factor due to coverage adjustment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Estimated reduction in bias and bks ratio for selected characteristics of the School

Readiness component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Estimated reduction in bias and bias ratio for selected characteristics of the School

Safety arrd Discipline component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Estirrrated mean square error and mean square. ratio for selected characteristics of tbe

School Rerdness com~nent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Estimated mean sqrrare error and rrrcan square ratio for selected characteristics of the

School Safety arrd Discipline component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Summaries of distribution of mean square ratios for selected characteristics of

School Rea&ress arsd School Safety arrd Dkcipline components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

i

.-
. .

List of Figures

Figore 1.

Figure 2.

F@re 3.

Figure 4.

Figure 5.

F@re 6.

Figure 7.

Estimated percentage of persons with interrupted telephone service during the

previous months . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Percentage of School Readiness population with service interruption . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Estimated reduction in absolute bias for School Readhess characteristics (scheme

Al) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ..i . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ......... 22

Estimated reduction in absolute value of bias for School Safety and Discipline

characteristics (scheme Al) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Estimated mean square ratios for selected SchooI Readiness items (scheme Al) . . . . . . . . . 30

Estimated mean square ratios for selected School Safety and Dkipline items

(scheme Al) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Estimated mean square ratios for selected Schooi Safety and Discipline items,

schemes Al and A2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

List of Exhibits

Exhibit 1. Telephone interrupt items . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

-fl-
Introduction

The National Household Education Survey

(NHES) is a data collection system of the

National Center for Education Statistics (NCES),

which has as its Legislative mission the collection

and publication of data on the condition of

education in the Nation. The NHES i s

specifically designed to support this mission by

providing information on those educational

issues that are best addressed by contacting

households rather than schools or other

educational institutions. The NHES provides

descriptive data on the educational activities of

the U.S. population aod offers policymakers,

researchers, aod educators a variety of statistics

on tbe condition of education in the United

States.

The NHES is a telephone survey of the

noninstitutionalized civilian population of the

U.S. Households are selected for the survey

using random digit dialing (RDD) methods, and

data are collected using computer-assisted

telephone interviewing (CATI) procedures.

These procedures provide a cost effective means

for quickly surveying households with

telephones. Approximately 60,000 households

are screened for each administration, atrd

individuals within households who meet

predetermined criteria are sampled for more

detailed or extended interviews. The data are

weighted to permit estimates of the entire

population. Tbe NHES srrrveyfor a given year

typically consists of a Screener, which collects

household composition and demographic data,

and extended interviews on two substantive

components addressing educatiori-related topics.

In order to assess data item reliability and inform

future NHES surveys, each adrrdnistration also

includes a subsample of respondents for a

reinterview,

‘fbeprimar ypurposeof the NHES is to conduct

repeated measurements of the same phenomena

at different points m tune, although one-time

surveys on topics of interest to the Department of

Education are also conducted. This h& been

done by repeating topical components on a

rotating haais to provide comparative data across

survey years. Irraddition, each administration of

the NHES has benefited from experiences with

previous cycles, resulting in enhancements to the

srrrvey procedures and content. Thus, while the

survey affords the opportunity for tracking

phenomena across time, it is also dynamic in

addressing new issues and including conceptual

and methodological refinements.

.

A new design feature of the NHES prugram

implemented inthe NHES:96is the collection of

demographic and educational information on

members of all screened households, rather than

just those households potentially eligible for a

topicaL component. In addition, this expanded

screening feature includes a brief set of questions

“on an issue of interest to education program

administrators or policymakers. The total

Screener sample size is sufficient to produce

state estimates of household characteristics for

the NHES:96.

Full-scale implementations of the NHES have

heen conducted in 1991, 1993, 1995, and 1996.

Topics addressed by the. NHES:91 were early

childhoud education and adult education. The

NHES:93 collected information about school

readiness and school safety and discipline. The

1991 components were repeated for the

NHES :95, addressing early childhood program

participation and adult education. Both

components underwent substantial redesign to

incorporate new issues and develop new

measurement approaches. In the NI-IES:96, th e

topical components are parentlfarrrily

involvement in education and civic involvement.

The NHES:% expanded screening feature

includes a set of questions on public library use.

In addition to its topical components, the F&S

system haa also included a number of

-1
methodological investigations. These have

resulted in technical reports and working papers

covering diverse topics such as telephone

mrdercoverage bias, proxy reporting, and

sampling methods. This series of technical

reports and working papers provides valuable

information on ways of improving the NI-IES.

and may be useful to survey researchers more

generally.

This report is a continuation of research on

issues related to biases that result from the

inability to survey persons who live in

households without telephones. TWO of the

earlier NHES technical reports (Brick, Burke,

and West 1992; Brick and West 1992) addressed

this impnrtant subject. Artother bias study

involved adding certain questionnaire items to

the NHES :93 to evaluate a different method of

adjusting the estimates to reduce the bias

associated with sampling only persons living in

households with telephones. The method

involves using data on interruptions of

telephone service to adjust the weights of the

respondents to the survey. The weights for

households that report experiencing some

periods of not having telephone service during

the twelve months prior to the interview are

increased whereas households reporting no

breaks in telephone service receive their normal

weights, The assumption behind this procedure

is that households with interrupted telephone

service are more like those without telephones

than other telephone households. Although the

goal of these adjustments is to reduce the bias

due to excluding households without telephones

at the time of the survey, a consequence of tbe

adjustments is that the variances of the estimates

increase. This analysis examines the benefits of

the bias reduction in light of the variance

increases and suggests situations in which the

adjustments might be beneficial

the NHES, and previous research using data on

telephone service interruptions tO reduce

coverage bias. Subsequent sections describe the

estimates from the NHES :93 of the percentage of

persons that experienced some interruption of

telephone service, the procedures used to adjust

the survey weights using these data, and the

statistical implications of using the adjusted

weights. The final section summarizes the

findings and contains recommendations for use

of this technique.

Backgr&nd

Telephone surveys provide a relatively

economical method of data collection compared

with personrd interviewing. However, telephone

surveys are subject to art important source of bias

that does rrOt affect household surveys conducted

with face-m-face interviewing only 94’ percent

of households nationally have telephone service

at any given time. Moreover, for the children

surveyed for the two components of the

NHES:93, cnverage rates are lower than 94

percent. Indeed, persons under 6 years of age

have the lowest telephone coverage rate of all

age groups in the U.S. (Thomberry and Massey

1988).

Weighting that inchrdes poststratitlcation based

on demographic variables known to be associated

with telephone coverage is effective in mitigating

some of the consequences of coverage bias in

telephnne surveys generally, and has been shown

to do so for marry items in the NHES (Brick,

Burke, and West 1992).’ But even when

effective, weighting to known demogmpbic totals

dues not completely solve the problem of

coverage bias. lt undercompensates for some

1 Estirnate based on tabtiatirms frum the March, J~Y, ~

The next section provides background Novenrk 1992 Current Poouladon SurvcY.

information on telephone coverage bias, its 2 POstsurvey wugbting is atso used to compmsate for

implications for estimates frnm a survey such as nonmspenw and other biases.

-2-
variables (Massey and Botman 1988) an d

overcompensates for others (Brick, Burke, and

West 1992).

This report describes a study of an alternative

method for adjusting telephone survey data to

compensate for coverage bias. This method is

based on the observation that telephone

subscription not only varies across households in

the population, but also within households over

time. Keeter (1995) discusses this idea in some

depth and demonstrates that a sizable number of

U.S. households lose and gain telephone service

during a given year. Because of this

phenomenon, the telephone population at a given

time includes households that recently were in

the nontelephone population and excludes some

households that were recently in the telephone

population. Thus, weighting adjustments that

use the data from households that have

telephones only sometimes during the year might

bean improvement over the current practice.

Despite considerable information on the size and

characteristics of the nontelephone population,

little is known about its dynamics over shofler

time periods. Evidence fmm social workers,

telephone companies, and others who deal with

indigent households suggests that for marry

families, telephone subscription is episodic.

They have a telephone when they can afford i~

the telephone is turned off when times are

harder, when the bills get too large to manage, or

botb (Federal Communications Commission

1988), It is not known how many households

change their telephone status and how long they

stay in a particular status.

Keeter (1995) examined two household panel

surveys to obtain estimates of the dynamics of

telephone service subscription. Those

households that changed telephone status

(presence of a telephone in the household) from

one wave to the next of the survey are called

‘transient’ households. For data collected 12

months apart, hrdf of the 6 percent of all

households without a telephone at either time

were transient. For data collected only two

months apart, one-fourth of the 6 percent of

households without telephones at either point in

time were transient. Since these estimates were

based on observations at two points in time

rather than continuous measurement, they

underestimate the percent of households that are

transient. Nevertheless, these results show that a

substantial proportion of households without a

telephone at a specific point in time are transient.

Another important condition that must be

satisfied if the transient telephone households are

to be useful in reducing coverage bias involves

the characteristics of transient households and

households without telephones at the time of the

interview. If the two groups are not similar, then

the adjustments will not be effective. Using the

pmel ~ata and data from several Virginia

surveys, Keeter (1995) showed that th e

characteristics of the transient households are

more consistent with the nontelephone

households than telephone households.

Preliminary results on this comparison were

presented by Keeter in a paper at the 1992

meetings of the American Association of Public

Opinion Research. This presentation was the

catalyst for the inclusion of the items on the

interruption of telephone service in the

NHES:93.

Estimates of interruptions of

Telephone Service

In the NHES:93, 64,000 households completed

the screening interview and nearly 30,000

interviews were cnnducted within those screened

households. Two survey components were

included: School Readiness (SR) and Schnol

Safety and Discipline (SS&D). Approximately

11,000 parents of 3- to 7-year-olds completed

interviews on SR topics, including

developmental characteristics of preschoolers,

-3-
school adjustment and teacher feedback to

parents for kindergartners and primary students,

home activities with family members, and health

status. About 12,700 parents of children in

grades 3 through 12 and about 6,500 youth in

grades 6 through 12 were interviewed for the

SS&D component. The topics for this

component included tbe school learning

environment, safety at school, and availability

and use of tobacco, afcohnl, and other dregs at

school. For bnth components, characteristics of

the family and household background

information were collected.

The SR component included the 20 million

children between the ages of 3 and 7 years as of

December 31, 1992, and afl other children

thrnugh age 9 who were enrolled in kindergarten,

first, or second grade. The SS&D cmnponent

included the 35 million students in gmdes 3

through 12. The estimates of the population

were derived from tbe October 1992 Current

Population Survey (CPS).

For all households that completed an interview,

one parent was asked if the household had

experienced an interruption in telephone service

in the last 12 months, where an interruption is

any 24 hour period without telephone service. If

the respondent said yes, he or she was asked how

manY days. weeks, or mOnths the household was

without service. This question was asked only of

one parent in the household, even if there were

multiple interviews in tbe hnusebcdd. (see

Exhibit 1 for the interview questions.) The

responses3 to these items are the baais for the

study of the effects of adjustments for telepbnne

coverage discussed in the rest of this report.

Since tbe responses to these questinns in the

NH33S:93 were only obtained for those

bousebolds that completed either an SR or SS&D

3 The imputed responses were used for recnrds with

missing values. Only 123 of the 10,888 SR and 71 of the

12,680 SS&D values were imputed.

interview, this has implications for the analysis

of the results. The data presented below pertain

only to persons in certain households: those in

which there was at least one child from preschool

age (at least 3 years old) to tbe end of high

school. Since the two eligible populations in the

NHES:93 are nnt overlapping, the estimates are

presented separately for the SR and SS&D

children. fn addkion, the estimates are of

Chiltin rather than households, This is an

important distinction. Since tbe estimates from

the NHE.S:93 generally refer to children rather

than households, the impact of the coverage

adjustment should he measured at the person

level rather than the hnusehold level.

The estimated percentage of SR children in

households that had a telephone interruption of 1

day or more was 12 percent of all children in

telephone Jrouseholds at tbe time of interviews,

while it was only 9 percent for the SS&D

children. This estimated difference in tbe

percentage with telephone service interruptions

between the twn populations is consistent with

estimates that find lower telephone penetration

for younger children. lltomberry and Massey

(1988) reported that 12.3 percent of children

under 6 years were in nontelephone households

while only 8.5 percent of those 6 to 16 years

were in nontelephone households.

F@e 1 shows tbe estimated percentage of

persons in each population who bad a service

interruption by the length of the interruption.

llse verticrd Iines in tbe figure arc 95 percent

confidence intervals on the estimated

percentages. Intervals constmcted using these

methods include the population value in 95

percent of afl possible samples that could be

selected. Both populations exhibit roughly tbe

same pattern in the estimates by length of

service. A substantial proportion of those with

intermptions in telephone service experience

only short interruptions of less than 1 week.

4
>>

Exhibit 1

Telephone Interrupt Items

1. During the past 12 months, has your household ever been without telephone service for more than 24

hours?

YES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 (GO TO 2)

NO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

WSED . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7

DONT KNOW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -8

2. What was the total amount of time your household was without telephone service in the past 12

months?

~ER . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . m

Figure 1--- Estimated 95 percent coididence intervals of the pementage of persons with intermpted

telephone service during the previous 12 months, by length of interruption

14

12

1

n

■ SR

+ Ssw

11~

more nmre ml-e

thsn4 thsn 1 thsu 1

weeks week dsy

Lengtfs of interruption

SOURCE U.S.~tof Sducarim, National Ccmer for Educmion Stistics, Natimal Hcus+jrS Education Survey, SPI@ 1593

-5-
Characteristics of Those With Service

Interruptions

The estimated percentage of children in

households with any interruption in service is

given in table 1 for the SR population and table 2

for the SS&D population. Characteristics

collected for both populations are the first items

shown in the tables.

The purpose of examining the telephone

interruption estimates by the characteristics of

the children is to evaluate the potentisf of using

the data to adjust for nontelephone coverage bias.

If the percentages of persons in households with

telephone service interruptions are nearly the

same for all persons across the characteristics,

then little could be expected from using the items

to reduce coverage bias.

The percentage distributions of persons with

some interruption for the nine common items are

relatively consistent for the SR and the SS&D

populations. All of the items, except Census

region and community mobility in the ZIP Code

area, exhibit variation’ in the percentage with

interruptions. The characteristics associated with

lower economic status have the highest

percentage with interruptions in afl of these

common items: the percentage of black and

Hispanic children in households with

intenuptions is higher than for white children;

the percentage of households with interruptions

for those renting is higher than for those who

own; the percentage with household incomes less

than $20,000 is higher than for those with larger

incomes; the percentage for those from

households with lower parentrd education levels

(only completed high school or less) is higher

than for those from households with higher

parental education levels (college graduate or

more); and the percentage of those living in ZIP

fie statemmtr in this repni were tested at the 5 pa-cent

significance level. Bonfermni adjustnmns were mule to

Wnrpm=te fOr ~tiple levels of the mspume variables.

Code areas with median household incomes of

$15,000 or less is higher than for those in areas

with household incomes of $25,000 or more.

The remaining items in table 1 were designed to

address specific substantive issues associated

with school rearhness. For most of the items, the

percentages of children in households with

telephone intermptions are not statistically

significant for different levels of the variables.

The most striking differences are for the

estimates of those respondents participating in

Women, Infant, and Children (WIC) program

and whose children participate in the free meal or

lunch program at school. The higher estimates of

the percentage with telephone interruptions for

children participating in these programs designed

for the economically disadvantaged is consistent

with the estimates for the other economically

related it~. For other substantive items, the

differences in the percentage of persons with

some intermption in telephone service are

statistically significant, but not large enough to

be of great practical importance. For example,

the difference in the percentage of children in

kindergarten or primary school who attended a

center-based program prior to school is

statistically significant, but the estimates differ

by less thrm 3 percent. A difference of this size

may not be important for adjusting for coverage

bias.

The estimates for the SS&D population in table 2

are similar. The differences across response

categories for most of the items are either not

statistically significant or so small that they have

little practical importance. For example, the

difference between children in public and private

school is less than 2 percent and not likely to he

important for adjustment purposes.

l’be estimates in tables 1 and 2 support the

hypothesis that the chance of having telephone

service interruptions is related to the economic

situation in the househoId. Since race/ethnicity

and &onomic status are highly correlated, a

-6-
.

relevant question is whether the differences in

the percentage of persons with telephone

intermptions would be significant across

categories of race/ethnicity after controlling for

other variables related to economic status, such

as household income or parental education since

poststratification by income or education is

frequently used in RDD surveys. To examine

this, the percentage of the SR population with an

interruption in telephone service in the last year

by racdethnicity and two categories of parental

education was estimated and is shown in

Figure 2. The figure shows 95 percent

confidence intervals along with each of the point

estimates. If education level accounted for all of

the variability in the percentage with

interruptions, then all three low education

estimates should be equal and all three of h@r

education estimates should be equal within

sampling error. Clearly, this is not the situation;

the estimates for blacks and Hispanics at the high

parental education level are greater than the

estimates for the nonblack, non-Hispanics.

Race/ethnicity is art important correlate even

after controlling for parents’ education level.

Tabulations controlling for household income

instead of parental education were also prepared,

and the estimates are afso statistically significant.

The same results also hold for the SS&D

population controlling for either parental

education or household income.

Tlrese findings indicate that interruptions in

telephone service aa estimated from the

NHES:93 do vary by economic and demographic

characteristics that have been identified aa

important correlates of telephone coverage. Tfis

condition is necessary for tire telephone

interruption data to be useful in adjusting for

telephone coverage bias. The lack of impO@t

differences for marry of the substantive items,

especially in the SS&D population, suggests the

value of the adjustment may be less important for

estimates of the substantive items.

Weighting Adjustments

ISI ‘most sample surveys, the dam collected frOm

respondents are processed to make tbe estimates

more representative of the population surveyed.

A typical operation is to attach a survey weight

to each observation and use these weights in the

preparation of estimates. The weights are often

the product of several steps. A base weight that

is the reciprocal of the probability of including

the respondent in the sample is first attached to

each record. The base weight is then adjusted to

account for nonresponse and noncoverage and to

reduce the variability in the estimates by using

auxiliary data.

Kalton and Kaaprzyk ( 1986) discuss adjustments

to the base weights, classifying the adjustments

into four categories: population weighting

tijustments, sample weighting adjustments,

raking ratio adjustments, ~d mspOnse

probability adjustments. In the NHES:93,

sample weighting adjustments and raking ratio

adjustments were used. Sample weighting

adjustments were used to account for differential

nonresponse from sampled persons. Raking ratio

adjustments were then used to make the specified

rM@Od distributions of the sample correspond

to totafs from the Dctober 1992 CPS. One of the

most important benefits of the type of raking

ratio adjustment used in the NHES :93 is that it

reduces the bias associated with the

undercoverage of persons living in households

without telephones because the CPS covers

persons in both telephone and nontelephone

households.

-7-
iTable 1.– Estimated percentage of persons in the School Readiness population with intermptions in

telephone service in last 12 months, by selected characteristics

c!hsIsctmsuc !3Jimste

12.0

9.3

19.8

17.2

11.7

7.9

18,4

228

19.9

9.3

5.5

18.4

15.4

11.8

5.5

5.2

17.6

10.1

9.6

20.7

13.1

18.2

8.7

15.4

19.6

14,8

9,5

13.6

11,1

115

18.3

15.8

9.9

13.0

13.1

12.2

11.0

11.6

11.8

13.1

Standsrd error

0.4

0.5

1.5

1.5

2.6

0.5

1.0

1.3

1.4

0.8

0.5

1.8

0.8

0.7

0.8

0.7

3.5

0.7

0.9

2.2

0.7

l.1

0.4

2.6

3.4

2.6

1.2

0.7

1.0

0.9

3.0

0.9

0.5

1.9

1.3

0.9

0.8

1.4

0.5

-8-
2

Table 1.– Estimated oercentaze of oersons in the School Readiness Dooulation’ with intenuutions in

telephone s~rvice infiast Ii months, by selected chsuacteristics {c~ntinued)

Chsrsctensuc

Binh weight

5.5 pounds or lms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Omater than 5.5 pn& . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Chifd atlending center-based progm2

Ys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

No . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Chifd ever attsnded center- fxaw-dprogran?

Ys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

No . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Chifd ever attended cenrer-ksed prognnn prior w sclwu~

Ya . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

No., . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

women, 6fmt, and Chiidren program panicipan+

Ym . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

No . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Free meal at schocd or cente?

Ya . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

No . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Repsated kindcrganen’

YB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

No . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ..a . . . . . . . .

Fm”ly mm+er red to chifd in kweep

Not in last .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

once or twice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

mm or mere times . . . . . . . . . . . . . . . . . . . . . . . .

Family memfxr fau.gltt chifd feners or words in fast wse~

Not in last .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Once or twice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

‘time or mm times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Family msm6er taught chifd songs or nnuic in Law W&

Not in M k . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Once or twice . . . . . . . . . .

Three or more tim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Fare@ mem6er did arts or cmfis with chifd in last weep

Not in Isst .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Once or twice . . . . . . . .

lluee or more times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Family nwndxr visited li6mry wish chifd in fast monthS

Ya . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

No . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Fa”ly mmd%r viritcd zoo with Chifd in fat mod?

Ya . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

No . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

‘7hc SRp@admis~y20mi16L-Hl cMLdlcnfrOrn 3t07yWsohi

%Mate mtri~ m pesdwdm.

kimatea@csmaOchildmeuxppwkmlas

‘Estimue .Uiced m ddtdml in p-imary sctwol,

%imateapptiamalt chitdmcxuptdmcin@n uyscswd.

FMnste

12.0

12.0

9.3

13.7

10.5

13.0

11.7

14.4

18.2

8.0

21.1

7.6

15.7

11.7

21.9

11.9

11.5

12.7

‘ 10.5

12.6

12.3

11.1

12.4

14.9

10.9

10.4

10.2

13.2

10.6

12.3

1.6

0.4

0.7

1.1

0,7

1.2

0.6

1.2

1.3

0.6

1.2

0.5

3.5

0.6

5.4

0.8

0.9

1.8

Lo

0.6

1.0

0.9

Lo

1.0

0.9

0.9

0.8

0.7

1.3

0.6

I

SOURCE U.S. Def@mnt of IMca6m, Nationfd flnta for Edu@im Smdstics. Natimal H.admfd EducahmSwvcy, S+nin.g 1993.

-9-
=

Table 2.- Estimated percentage of persons in the School Safety and Discipline population’ with

interruptions in telephone service in last 12 months, by selected characteristics

Charactensuc

Toti . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

No . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

-1oEstimate

9.2

7.2

14.7

14.1

9.3

6.6

15.3

19.0

15.7

7.9

5.0

17.4

11.0

8.6

5.3

4.5

128

8.4

7.8

15.1

10.3

12.9

6.8

14.6

17.3

13.8

9.0

10.8

7.3

9.2

15.4

11.6

7.7

7.6

9.7

9.1

9.2

9.6

9.4

7.5

9.4

8.6

standard almr

0.3

0.3

1.1

1.1

1.5

0.3

0.8

1.3

1.1

0.6

0.3

1.6

0.8

0.5

0.8

0.6

L9

0.5

0.6

1.6

0.7

0.9

0.3

2.6

2.s

1.6

0.8

0.6

0.7

0.8

2.1

0.8

0.3

1.7

0.9

0.6

0.5

1.0

0.4

1.1

0.4

0.7
2

Table 2.- Estimated percentage of persons in the sch~l Ss.fety ~d IXscipline population’ with

interruptions in telephone service in last 12 months, by selected characteristics (continued)

Characteristic

None . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Fear of thefi or mb~ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Fear of bullying or uUl? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Fear of two or mare tyFS of ititi@3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Knowledge of crime at school

None . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Knowledge of Ihefi or mb@ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Knowledge of bullying or ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Knowledge of two or mom types of incidents] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Vicdmizadon by crime

Not vidtiA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Victim of theff or m* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Victim of bullying or asssul? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Victim of two or more types of inub*3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Witnessed crime at SC6001

None . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Witnessed robbery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wimessed bullying or .t4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Wimessed two or more Oflmti types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

mime

9.5

8.8

9.7

8.0

9.0

8.5

10.9

8.7

9.4

11.5

10.6

8.9

7.5

10.8

9.6

8.6

10.8

11.2

10.6

8,8

8,5

10.1

‘ ‘tk School Safety and Dkiplk pqmladon is appximA y35miUi0nmniausingmdc3 3 through 12

2 Only &cd for sbxkms in @cs 6 tbmugh 12.

Smndsrd error

0.8

0.5

0.6

0.8

0.7

0.4

0.7

0.4

1,0

1.2

0.9

0.5

0.8

0.9

0.5

0.4

1.0

1.2

1.2

0.5

4.1

0.5

0.9

3 f% k fw of i.cidcm knmvkdge of crime, arid riaindmf by crime vaiabtea, the seared mspnse cmegory is used if eifhu fkfi or mbba-y .s

+butnot~. W~=m~ti~ifritibtiWg mWw-butmhh.

‘This response cafcgory is wed if ark butt@g or awauh was mpomd, but nor bmb.

SOUSCE U.S. D+mrOnmt of Educadcm, Nafimu3 CmIU fcu Educatim SM6S6CS, Nadmal Hcusehold f?ducadon Survey, spring 1993.

-11-
..

F@e 2.-- FMinrated 95 pereent cotildence intervals of the percentage of School Readirma

population with service interruption

g

“g 30-

E

#25-

.-

8 20

~

~ 1]

i

~

;

15

lo-

~ ,=

% E

Z5

&go

Hispmic Black, Nenblsck,

nmrHispanic nOnHispanic

Rscdethnicity

SOUSCE U.S. Lk+nmnmt of Educatim, National Cknra for EAcn!ion Sradsdcs, Nadond Household Educadon Survey, spjng 1P93.

The data on telephone service interruptions cart

be used to make a response probability

adjustment. Response probability adjustments

are constructed by assuming that each sampled

unit has a probability of responding to the

survey, estimating that probability, and then

using the inverse of the response probability as a

weighting adjustment. The Politz and Simmons

(1949) method is probably the best known

application of the response probability

adjustment procedure.

To apply this type of adjustment with tbe

telephone service interruption data, assume that

living in a telephone household is a dynamic

phenomenon, and that a probability distribution

can be associated with this status. Conceptually,

a survey is conducted by sampling from this

distribution and observing only those members

that live in telephone households at the time of

the survey. The probability of living in a

telephone household must then be estimated for

each respondent. The inverse of the estimated

probability is the coverage adjustment. This

model assumes that each person carr be assigned

a probability of being in a household with a

telephone and that the probability is between

zero ‘md o~e (but not equrd to zero).

For this analysis, the data on whether or not a

bousebold bad an interruption in telephone

service and the length of that interruption are the

basis for an adjustment, using methods suggested

by Keeter (1995). Persons are divided into two

categories: those in households with

interruptions in service and those in households

witbout interruptions in service. The probability

is assumed to be one for persons in households

witbout interruptions and their weights are not

adjustsd. The weights of persons in households

with at least some interruptions in the last 12

months are adjusted to account for other

households that have a probability of being

covered of less thsrr one. The adjustments may

VSfY depending on the length of time they lived

in nontelepbone bousebolds and on othe r

characteristics of the household. The purpose of

having different adjustments is to account for the

fact that some persons are more likely to live in

nontelephone households than others.

Although the weighting adjustments may reduce

the undercoverage bias, introducing adjustments

-12-
also typicafly increases the vtiarrces of the

estimates. Kish (1992) discusses the reasons for

unequal weights as well as the consequences

from using them in a variety of situations. He

advocates a common statistical approach of

balancing the effect of the adjustments in

reducing the bkm of estimates against the

increases in the variances of the estimates. If the

weights reduce the bias of the estimates

significantly, then it may be worthwhile

accepting tbe variance increases. On the other

hand, small reductions in bkw associated with

large variance increases are not recormnended.

In the remainder of this section, the specific

weighting adjustment procedures examined using

the telephone service interruption data are

described. The methods for creating the

adjustments and applying them to the NHES:93

are presented in some detail. The statistical

properties of the weights developed under four

alternative adjustment scenarios are presented.

The alternative weights are applied to the

NHES :93 data and the estimated decrease in the

bias of the estimates is compared with the

increase in the variance of the estimates due to

the unequal weighting.

Adjustment Schemes

The first step was to decide how to classify the

length of interruption in telephone service.

Various lengths of interruptions were examined

to determine cut-offs which appeared to

distinguish between temporary interruptions, not

due to economic causes and others. It was

decided tn use two categories. for forming

adjustment cells: 1 week or mnre and 1 month or

more. When tables like tables 1 and 2 were

created using the 1 wwk nr mnre and 1 month or

mnre criterion rather than any intermptinn, the

estimates for the transients were still highly

related to the econnmic and demographic

variables identified in tables 1 and 2. A category

>

for interruptions of less than 1 week was not

used for adjustment because short-term

intermptions may have been caused by factors,

such as temporary weather-related outages, that

are dlffercnt from the longer term intermptions.

Within each of the length-of-service interruption

categories, the children were classified intn

adjustment cells based on either parental

education or tenure (home ownership).

Race/etfmicity was used to form cells within the

parental education and tenure categories. These

cells were chosen because the percentage of

persons’ with interruptions varied by these

characteristics and the corresponding data were

also available fmm the CPS. Four adjustment

schemes were defined using these items:

● scheme Al-children in households that had

a telephone service interruption of 1 week or

- more within categories defined by parental

.

.

.

education (less th~ high school, high schnol

d:plorna, coIlege d]ploma or above) an d

race/ethnicity (Hispanic, black/non-Hispanic,

white and other/non-Hispanic);

Scheme A2-children in households that had

a telephone service intermptinn of 1 mnnth

or mnre within categories defined by parental

education and race/ethnicity;

Scheme B1--children in households that had

a telephone service interruption of 1 week or

more within categories defined by tenure

(nwrrlother, rent) and race/ethnicity; and

Scheme B2-children in households that had

a telephone service interruption of 1 month

or more within categories defined by tenure

and racdethnicity.

In these schemes, the children classified as living

in households with intemrptions nf one month or

mnre (A2 and B2) are a subset of those classified

as having interruptions of one week or more. In

other words, if the weight for a child was

adjusted under scheme A2 or B2 it was also

adjusted under scheme Al or B 1.

-13-
. .

The adjustment factors for these schemes could

not be obtained directly from the NHES:93 data

because no data were collected from households

without telephones. Instead, the adjustments

were developed from CPS data and then applied

to the NHES :93 weights, following the idea

suggested by Keeter.

To explain the adjustment of the weights under

the response probability model assumptions,

consider petitioning the universe of persons at

the time of the interview into four components: t,

is the number of persons in tefephone households

with no telephone interruptions in the past yew

rZ is the number of wrsons in telep~ne

households with some telephone interruptions in

the past yeac t3 is the number of persons in

nontelephone households with no telephone

interruptions in the past yew, i.e., persons who

lived in nontelephone households throughout the

entire year and t4 is the number of persons in

nontelephone households with some telephone

irrterrupticms in the past year. As noted above,

the response probability modeI assumes t3-4,

i.e., no persons live in nontelephone households

for the entire year, This assumption is clearly

not true, but there are no sources to estimate the

size of t3. However, under the response

probability model assumed all Z3 persons are

included in the r4 population.

Using the March 1992 CPS it is possible to

estimate t, +t2 (but not the separate quantities)

and t4; designate these estimates as ;I +;2 and

~4, respective y. Notice that ~ includes persons

currently living in households without

telephones, regardless of whether they had an

interruption in service in the last year. Thus, it

includes the q population. From the NHES:93,

tl and t2 can be estimated separately; call these

*

estimates t; and r2, respectively. The bhs in

the NHES:93 estimates arises because they do

not include persons in nontelephone households

(t4). The goal is to reduce this bias by adjusting

the NHES :93 weights of those persons living in

telephone households with some telephone

interruption.

/

A weight adjustment of A = 1 +‘4

fz

would

result in unbiased estimates of totals under the

response probability model, since this model

assumes t4 and tz are members of the same

population but in different telephone status at the

time of the interview. However, this adjustment

involves quantities that are urdmnwn and must be

estimated. Since t2 can only be estimated

separately from the NHES :93 and t4 can only be

estimated ‘from the CPS, the adjustment is

expressed in ratios to reduce the bias due to

estimating the quantities from different surveys.

The revised weight is

where Wi is the NHES:93 weight adjusted for

nonresponse of sampled persons but not yet

raked to October 1992 CPS totals, ~i = 1 if the

person lives in a household that had an

interruption of telephone service in the last year

and is zero otherwise. The quantity in

parenthesis in (1) is the weight adjustment.

Revised weights were cnmputed separately for

the SR and SS&D components, since these were

handled as separate surveys. Rather than the

overall adjustment as given in (1), tbe weight

adjustments were computed within the cells

defined for each of the four weighting schemes

(Al, A2, B 1, and B2). Table 3 shows the

resulting adjustment factors for the SR and

SS&D components. The adjustments in the first

column am those for schemes A 1 and B 1. The

second column contains the adjustment factors

for schemes A2 and B2. The adjustment factors

for tbe schemes based on the 1 month or more

-14-
interruptions are greater than those based on the

1 week or more because the. denominator of tbe

1 ratio is smaller for this classification.

The last weighting step rakes the four alternative

weights to the same October 1992 CPS totals

used in raking the standard NHES :93 personIevel weights. The result of this process is the

standard NHES :93 weight and four alternative

weights based on different adjustment schemes.

All five of the weights conform to tbe same

marginal totals. The only difference in the

weights is the adjustment for the telephone

service interruption.

Effect of Adjuatmenta on Variance

As discussed before, the adjustment of the

weights to reduce the bias increases the

variability of the weights and the variance of the

estimates. Kish (1992) gives an approximate

expression for this increase in variance due to

having weights that are not equrd, the variance

“inflation factor (IW,). The VIF is a reasonable

approximation if the population element

variances of the persons sampled at different

rates are roughly equal. The WF can be written

as

WF= 1+ C@(weights) (2)

where CV is the coefficient of variation of the

weights.

component is broken down by the grade of the

student, because youth were selected at different

rates for these grade levels. Only one WF is

presented for the SR component because all

children were sampled at the same sampling rate.

The WF for each of the components is about 1.4,

indicating the variance is inflated by about 40

percent due to the variability in the standard

weighta. The WF for the combined SS&D file is

somewhat larger (1.5) because it includes youth

sampled at different rates. The WFS for many

subdomains of children should be well

appmximat~ by the WF for the full component.

The last four columns in table 4 are the ratios of

the VIF for the four alternative weights to the

WF for the standard weight. These ratios show

bow much greater the variances of estimates

produced using the alternative weights are

expect@ to ~ as compared tn the variances of

the standard NHES:93 weights.

Overall, the increase in variance due to tbe

telephone interruption coverage adjustment are

from 9 to 13 percent for schemes A 1 and B 1 in

the SS&D component but up to 20 percent for

the SR component. The ratios are larger for tbe

schemes A2 and B2, ranging from 24 to 35

percent, with the Iargest ratio for Scheme A2 for

the SR component. The larger ratios (hence

WFS) for the schemes based on intermptions of 1

month or more are a consequence of tbe larger

and more variable factors shown in the second

column of table 3. The ratios for the

population are bigher than the SS&D ratios.

SR

Table 4 shows the VIF for the standard NHES:93

weights for each component. The SS&D

-15-
,.

Table 3.--Weighting cell adjustments factors, based on length of interruption of telephone service

Factor

School Readiness

Cells defined by parcntsl education and racderhnicity (scheme A)

Lx than high schcol diploma Hisptic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Less than high schc-al diplonm blsck, non-Hispsnic

Less rhm high schcol diploma white and mher, non-Hispartic . . . . . . . . . . . . . . . . . . . .

High schcol diplow Hispsnk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

High schcml diplorrw black. non. Esptic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

High school diploma white snd other, non-Hispanic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Bachelor’s degree or high% Hisptic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Bschelor’s degra or highen black. non-Msptic..,.,.,,., . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Bachelor’s degmc or frigfrec white and other, nrm.Hispanic

Cells defined by tenure and mcekbtdcip’ (scherm B).

Rent Hispanic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

RrmG black, non.Hispanic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Rent white and other, non. Hisptic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

flvdofhrz Hisptic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

OwmiotherI black, non-Hispartic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

OwniOOreG white and other, non.Hisptic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

School Safeg mrdDfscipfine

Cells defined by parenfaf education and mcdefhnicity (scftanc A)

Less fhsn high schcml diploma; Hisptic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Less than high schcoi diplorrw blsck, non-Hispartic . . .

Less than high schcol diplomq white and ofhcr, norr.Hispmic . . . . . . . . . . . . . . . . . . . .

High school diplomx Hispsnic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

High school diploma black, non. Hisptic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

High schcol diplonw whifc and other, non-Hispanic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Bachelor’s degree or highec Hisptic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Bschelor’s degree or bighrm black. non-His@c.,.., . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Bachelor’s degree or high~, white snd other, non-flispmdc

Cells defined by tenure and racdetttnicity (scheme B)

Rent Hispanic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

RenG blsck, nrm-Hispmic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Renq white and other, non.Hisptic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Ownlofhq Hispmic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

@ndofh.ec black, non+fispsnic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Owm/othc& white and other, non-tfispasdc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Lmmh of service inmrnmtion

t week or more

5.75

5.10

4.98

2.31

2.65

2.16

1.34

I;;

3.74

3.23

2.43

2.(XI

2.53

2.26

4.g9

4.26

3.81

2,67

3.06

2.18

1.96

1.35

1.91

3.58

3.38

2.99

2.81

2.90

2.03

I month or more

16,35

6.72

5.37

2.76

3.73

2.79

2.33

2.64

2.09

5.15

4.5d

2.96

3.06

3.46

3.45

852

5.95

4.s6

4.51

4.71

3.OP

8.22

8.s3

3.4s

6.0S

4.95

4.IXJ

5.66

6.11

3,10

SOURCE U.S.~tof Educalion, National Cmter far Education Skadstics, Nadmal Hwxhdd Edwatim Survey, -1993

-16-
z

Table 4.–Ratios of variance inflation factor due to coverage adjustment

VIP

sfandsrd Ratio of scheme’s VIF m sfandsrd weight’s WF

Component Ssmple size weight Scheme AI Scheme A2 Scheme B I Scheme B2

Schcol Readines s 10,888 1,36 1,20 1,35 1.16 1.26

School Safefy and Oirciptine

3rd ttrmugh 5fh graders . . . . . . . . . . . . . . . . . 2,563 I .37 1:12 1 ,2S 1.13 1.26

6th through 12th graders . . . . . . . . . . . . . . . 10,117 I 39 1.13 1,27 1 SW 1.24

31d through 12th gmders . . . . . . . . . . . . . . . 12,680 I .49 1.12 1.26 1.11 I .23

● VfFisfk sIandard inflation factm. Itkticmffitiml Ofvtitim.ftiti@m _ plums,

SOURCE U.S. EqammIN of FducatioIL Nafiona2 CMIU for Education Statistics, Nadmsl H.mxSmkl Sdwadm Survey, sping 1993

Effeet of Adjustments on Mean Squstre

Error

In this section, the adjusted weights are applied

to estimate the characteristics from the SR and

SS&D components. Since four adjustments were

constructed, five different estimates are

computed: one from the standard NHES:93

weights and one for each of the four adjusted

weights. The only difference in the methods

used to compute the estimates for the five

weights is the coverage adjustment. All five sets

of weights include the final raking adjustment of

the estimates to the totals from the CPS,

Therefore, even the standard estimates are

adjusted to account for undercoverage, but they

do not have the telephone interruption

adjustment.

The dkcussion of the alternative weighting

schemes hegins by considering the reduction in

coverage bias, the difference between the

estimate and the value that would have been

obtained if households without telephones could

have been surveyed. The bias corrections am

then compared to the increase in variance

associated with the adjustment procedures.

.

Coverage Bias Reduction

If estimates of the same characteristics as those

produced from the NHES:93 were available from

an indepen~ent source and these benchmark

estimates were free of telephone coverage bkm,

then it would be pnssible to compare the five

estimates to the benchmark. The comparisons

could be used to evaluate the bias in the standard

NFIES :93 estimates and the btaa remaining in the

other estimates after the coverage adjustments.

However, benchmark estimates comparable to

the estimates fmm the two components of the

NHES:93 do not exist. Consequently, othe r

methods are needed to assess the bhs-reducing

potential of the coverage adjustments.

Due to of the lack of a benchmark, some model

assumptions are required to assess the

effectiveness of the adjustments. For this

evaluation it is assumed that the adjustment

procedures eliminate the coverage bias. As a

result of this assumption, the difference between

the standard estimate and the adjusted estimate is

an unbiased estimate of the coverage bias

resulting fkom using the procedures. In practice,

the coverage hias is not completely eliminated by

-17-
any of the adjustment procedures. Even if the

model were correct, the bias reductions estimated

from the data would still be subject to sampling

error. Despite the problems with tbk

assumption, it is necessary to obtain some idea of

tbe effectiveness of the adjustment. If the

adjustment eliminates the bias, the mean square

errors of the adjusted estimates are equal to the

variances of the estimates, with no contribution

from coverage bias. Therefore, the model

assumption is favorable to the adjustment

process, positing the adjusted estimates to be

unbiased (any difference between the standard

and adjusted estimates are attributed to bias).

llre impact of this assumption is discussed

critically after evidence of the effectiveness of

the method is presented.

The estimate from each scheme can be compared

to the standard NHES :93 estimate, and the

difference between the standard estimate and the

adjusted estimate is an estimate of the reduction

in the coverage bias. With four adjusted

estimates, four different estimates of bias

reduction are possible. The estimated reduction

in bias is

where ba is the estimated bias reduction using

adjustment scheme a (a = Al, A2, Bl, or B2),

j, is the estimate of the proportion using the

standard estimate, and ~a is the estimated

proportion using adjustment scheme a,

The estimated reductions in bias under each

adjustment weighting scheme are given in table 5

for the SR characteristics and table 6 for the

SS&D characteristics. It is important to

understand that the estimates are of the amount

of bias reduction in the standard edissrate,

assuming each adjustment scheme reduces the

coverage bias. For example, the estimated bias

reduction in the standard estimate of the

percentage of preschool children attendhg a

center-based program is 0.9 percent if scheme A 1

is the assumed benchmark, and 0.3 if scheme A2

is used instead. The standard NHES:93 estimate

of this percentage is 52.6 percent, so the scheme

Al weight results in an estimate of 51.7 percent

and the scheme A2 estimate is 52.3 percent. The

bkra reduction estimates in tables 5 and 6 are

shown for all the items in the earlier tables,

except for those items that were used in raking.’

Since the raking procedure forces the estimates

to the given marginal totals for these items, the

adjusted estimates and the standard estimate are

all equal for these characteristics.

The bias reduction estimates for most of the

items in tables 5 and 6 are less than 1 percent and

consistent across the schemes. Before

summarizing the estimates, it is important to

realize that the total number of children is

constant for all the estimates due to the raking of

the estimates to the CPS totals. The estimated

reductions in bias across different response

categories if an item, therefore, must sum tO zero

(positive bias reductions in response categories

must be balanced by a negative estimates for

other categories). As a result, the estimate and

the bias reduction for the last category of a

variable can be dednced from the estimates from

the other levels.

The fixed total number of children across

response categories has two consequences. It

creates a negative correlation in the estimated

reduction in bias across response categories

(resulting from the zero sum nature of the totaf)

and gives a false impression of the number of

independent pieces of information in the tabled

values. For example, for a dichotomous variable,

the bias estimates are perfectly negatively

correlated (the estimate in one category is the

negative of the estimate for the other category).

Thus, there is only one independent estimate for

a dichotomous item.

‘Raking was done to marginal totals by age, grxle, rurure

(own, rent), Census region, raccfuturiciry, and household

incom.

-18-
The approach taken to address this problem in

summarizing the estimates is to delete the

estimate for one of the response categories fnr

each item, For example, in the SS&D

component estimates in table 6, rather than

include both the estimate of the bias fnr the

percentage of students in public school (-O. 1 in

scheme A 1) and private school (0.1 in scheme

Al), only the public school estimate is retained

for the summary. The “no” response category for

all items with “yes” and “nn” response categories

is deleted. For other types of variables the

response category with the smallest estimate is

deleted. Of the 19 items in table 5 with estimates

for 53 response levels, only 34 (53-19) are

retained for the summaries below; of the 14

items with 48 response level estimates in table 6,

the estimates for 34 (48-14) response levels are

included in the summaries.

Figure 3 presents the reduction in bias estimated

using scheme A 1 for the SR characteristics, and

figure 4 is the same representation for the SS&D

items. The reduction in bias estimates presented

are the absolute values of the bias estimates

shown in tables 5 and 6 after deleting the

estimates for one respnnse level per item. Fnr

trnth components, the bias reductions are small.

The largest absolute bias is 1.3 percent for SR

and 0.9 percent for SS&D. The mean and

median of the bias reductions and the absolute

vafrres of the bias reductions were also computed

for each scheme and each component. For the

SR component, the mean and median of the

absnhrte value of the estimated bias reductions

are between 0.2 and 0.4 percent. For the SS&D,

the mean and median of the absolute values are

between 0.1 and 0.3.

Table 5.- Estimated redrrctinn in bias and bias ratin for selected characteristics of the School Readiness

component

standard estinmre

standard

characteristic Percent m,

P.re-nm/ educcmioncd kvd

k,, than hish school graduate . . . . . . . . . . . . . . . . . . . . . 8.6

High school graduate .r quivaient . . . . . . . . . . . . . . . 33.9

some .ollegc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57.5

Moth,, ,qdoynwnt stat.,

No mother in hmmh.ld . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4

Employed 35 hmmlweek or nmre.........,.,,..,, 34.3

Employed 1.SS rhm 35 h.xrdweek 20.9

Seeking employment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.6

Not in labor force . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35.8

Folk,,, t-mploynenl SIIIIU.

No fafher in houwhold . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26.3

Employed 35 houmfwcek cm more. . . . . . . . . . . . . . . . . 63.4

Employed less than 35 hourslwek 3.8

Seeking employ mau . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2

Not in Iahm farce . . . . . . . . . . . . . . . . . . . . . . . . . . . . ..__ . . . . 3.3

Median household inconw in ZP Code

$15.(XO or less . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.0

S15,020 to $25.000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30,1 MOX than $25,WQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . W.o

Mobilily in ZfP Code

Hig h 7.6

McdiunVhigh 17.7

Mdium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30.0

Mtiutiow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2

h . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.5

Timt since doctor visit for routine care

Less Olan I yea r 84. I

over I ya . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.9

0.3

0.s

0.7

0.2

0.5

0.5

0.4

0.6

0.5

0.6

0.3

0.3

0.2

0.4

0.8

O,g

0,5

0.6

0.7

0.s

0.5

0.4

—0.4

Estima fed rcducti.m i“ bias’

;Chcme scheme Schenw Schal

Al A2 Bl B2

-1.7 .1.9

0.4 0.3

I .3 1.6

-0.1 -0,1

0.5 0.s

-0.1 -0,2

0.0 -0.1

-0,4 -0.3

.0.4 -CI.6

0.3 0.5

0.0 -o. I

0.0 0.0

0,1 0.2

-0.2 0.0

-0,6 -0.5

0.s 0.6

0.0 0.2

-0.1 -0.2

0.0 -0.1

0.3 0.3

-0.2 -0.2

0.4 0.4

-0.4 -0.5

0. I

-0.7

0.6

.O.1

0.2

0.0

-0, I

0.0

0.0

0.1

0.0

-0.1

0.0

0,1

-0.3

0.4

0.0

-0,3

0.0

0.4

-0.1

0.2

-0.2

0. I

-1.0

0,9

.0.1

0.5

-0.2

-o. I

0.0

-0.1

0.2

0.1

-0.2

0. I

0.2

-0.5

0.4

0.2

-0.3

0.0

0.2

0.0

0. I

-o. —I

Bias mrio2

:heme Schems Sch-=lm Sciwnle

Al AZ Bl B2

-5,7 -6.3 0.3 0.3

0.5 0,4 -0.9 -1.3

1.9 2.3 0.9 1.3

-0.5 -0.5 -0.5 -0,5

I .0 1.6 0.4 1.0

-0.2 -0.4 0.0 -0,4

0.0 -0.3 -0.3 -0.3

-0,7 -0.5 0.0 0.0

-0.s -1.2 0,0 -0.2

0.5 0.s 0.2 0,3

0.0 -0,3 0,0 0,3

0.0 0.0 -0,3 -0.7

0.5 I .0 0.0 0,5

.0.5 0.0 0.3 0,5

.0.7 -0.6 -0.4 -0,6

I.o 0,7 0.5 0.5

0.0 0,4 0,0 0.4

.0.2 -03 -0.5 -0.5

0.0 -o. I 0,0 0.0

0.4 0.4 0.5 0.2

-0.4 -0.4 -0,2 0.0

1.0 I .0 0.5 0.2

.I.o -1.3 .0.5 -0.2

-19-
>

Table 5,– Estimated reduction in bias and bias ratio for selected chaacteristjcs of the .Scbool Readiness

component (continued)

SWdan2 estimate

Slandan

Chamcuxistic Pm-cent em,

93.3

6.7

52.6

47.4

62,9

37.1

73.5

26.5

33.8

66,2

35.8

64,2

5.7

94.3

4.3

16.9

78.8

12.3

27.4

60.3

33,3

30. I

36.7

32.2

35.7

32.1

39.4

60,6

16.8

83,2

0.3

0.3

0.8

0.8

0.8

0.8

0.5

0.5

1,0

1.0

0,6

0.6

0.4

0,4

0.5

0,9

0.9

0.4

0.6

0,7

0.6

0.7

0,7

0.7

0.7

0,6

0.9

0.9

0.7

0.7 —

Estimated reduction in bias’

:hew SCkme S’dmlx Scbem

Al AZ B! B2

-0.1 0,0

0. I 0.0

0.9 0.3

-0.9 -0.3

0.5 0.3

-0.5 -0.3

0.6 0.7

-0.6 -0.7

4.6 -0.1

0.6 0.1

43.9 -1.1

0.9 1.1

4.3 -o.5-

0.3 0.5

.0.3 .04

0.0 0.4

0.3 0.0

.0.4 -0.4

0.3 0.3

0.1 0.1

0.6 0.2

.0.3 -0.1

4.2 0,1

.0.3 -0.7

0.1 0.3

0.2 0,4

0.5 0.4

.0.5 -0.4

0.3 0.1

0.3 -0. I

0.0

0.8

-0.8

0,4

-0.4

0.s

-0.5

-0.8

0.8

-0.5

0.5

-0.2

0.2

0. I

0.0

-0.1

-0.3

0,3

0.0

0.3

-0.2

0.0

-0.5

-0.1

0,6

0.4

-0.4

0.2

~

0.0 0,1

-0.1

0.6

-0.6

0.3

-0.3

0,6

-0.6

-0.7

0.7

-0,5

0.5

-0,2

0.2

0.1

0,2

-0.3

-0.3

0.5

-0.2

-0.1

0,0

0,1

-1.0

0.0

1.0

0.5

-0,5

0.1

-o. —I

h31ebinm,s lbet4udaicilesmmledividdby6-eslMkdm.

k6m4temme610pzhcoks.

‘sldlmeS$. pksmau-exceplpucimkn.

%mmemsaiuedIo-inpmuy8cbwL

GiImlc@iut0and6!&ucepdauernpmRjacbd

NOTE Petremxnmy nmxUmlmtiof&

Bias ratio2

:knE Sckme Scbcme Scheme

Al AZ Bl B2

-0.3 0.0 0.0

0.3 0.0 0.0

1.1 0.4 1.0

-1.1, 4.4 -1,0

0.6 0,4 0.5

-0.6 -0,4 .0.5

1.2 1.4 1.0

-1.2 -1,4 -1.0

-0.6 -0.1 -0.8

0.6 0.1 0.8

-13 -1.8 .0.s

1.5 1.8 0.8

.0.8 -1.3 -0.5

0,7 13 0.5

4).6 -0.8 0.2

0.0 0.4 0.0

0.3 0,0 -0.1

.1.0 -1.0 -0.7

0.s 0.5 0.5

0.1 0.1 0.0

1.0 03 0.5

4,4 -0.1 -0.3

4,3 0.1 0,0

.0.4 -1.0 -0.7

0.1 0,4 -0.1

0.3 0.7 1.0

0.6 0,4 0.4

.0.6 -0.4 -0.4

0.4 0.1 0.3

.0,4 -0.1 -0.3

0.3

-0.3

0.8

-0.8

0.4

-0.4

I .2

-1.2

4.7

0,7

-0.8

0.8

-0.5

0.5

0.2

0.2

-0.3

-0.7

0.8

-0.3

-0.2

0,0

0,1

-1.4

0.0

1.7

0.6

-0.6

0.1

—-0.1

SOURCE Us. OemmmIcfSdwim, Na60ml Cnwfm-Sti,Nuie4~SdUMhCa SWq,9@ns1S93

-20-
>

Table 6.- Estimated reduction in bias ursd bias ratio for selected characteristics of the School Safety and

Discipline component

characteristic

Pam-nfal educational [<”d

!.-ss than high school @uate

High schcal gmd.are .X q.ivdent . . . . . . . . . . . . . . . . . .

some coll.ge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Mothers employmcm stanu

No mother in household . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Employed 35 hourdwek or more,..., . . . . . . . . . . . . . . .

Erqd.ycd less lhan 35 hourdweek

*king employm.w . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Not in labor face......................,..............,

Falher,s anployment SZIIIIU

No father in household_,,.,,,,,, _,,..,.. . . . . . . . . . .

Employed 35 houmhwek Or mm-e . . . . . . . . . . . . . . . . . . . .

Employed 1.SS rhm 35 harafweek

Se&ing employment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Not i“ labor face,.., . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Median household incomt in ZfP Code

S15,00+2 or less . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

S15.OW to $23.fsxl . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Mote rhan $23,13W,,...,. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Mobifify i“ ZfP Code

High

McdiurWhigh .

Mtitum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

MetiutiOw . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

School C“”td

~hlic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Mvti . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Visitors required m sign i. of school

Yes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

No . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

S!udtno infighting gnngs m schoop

Ym . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

No . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.?.ae of &raining marijuana al schd

very or fairly e.xy...,,.,, . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .._.

Har d

Nearly irnpmsibk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Had drug or alcofwi edprogmm rhis year

Ys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

NO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Far of incident of crinw ar school

Nom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Fear of fheft or mbbuy4,,,_,....._,....,...,..,..

Fear of butlying Or assault’

Fear of rvm or more types of inci&ms4.,,..,...,..

Knowledge of crime al SCbd

Nm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Fear of fhcn or rC&ry’.,_ .,....,.........,..,..,,..

F= of bullying or assault’

Fear of two w mm types of incidmfs’.,...,...,..

Victimization by crime

Not viciimiud

Victim of rhefl .x mbb5y4. . . . . . . . . . . . . . . . . . . . . . . . . . . .

Victim of bdli”g m assault’

Victim of fw m more rypcs of incickmd . . . . . . .

;randmd estimate

QIz$

9.4

32.7

57.9

3.5

46.2

20.3

4,5

25.5

26.8

63.2

3.1

2.6

4.3

4.2

31,3

64.5

5,5

17.1

29,4

33.9

14,1

91.2

8.8

79.9

20.1

22.3

77.7

39.2

29.7

31.1

68.5

31,5

66.1

11.9

8.6

13.3

38.7

14,1

15.6

31,6

73,0

10.9

8,9

7.2

0.5

0.6

0.5

0.2

0.5

0.5

0.3

0.5

0.6

0.5

0.2

0.2

0.3

0.3

0.7

0.8

0.3

0,6

0.6

0.7

0.7

0.3

0,3

0.5

0.5

0.5

0.5

0.6

0,5

0.6

0.7

0.7

0.5

0,5

0,3

0.5

0.6

0.5

0.4

0.6

0.5

0,3

0.3

Q

Estimated reduction in btas’

:helnc Schcnw scheme Schem

Al A2 B1 B2

-1.2

0.3.

0.9

0.0

0,0

0.1

-0,2

0.0

-0.2

0.6

-0,2

-0,2

-0.1

-0,1

-0.3

0.5

0.2

0.0

-0.1

-0.1

0.0

-o. I

0,1

0.1

-o. I

-0.3

0.3

-0.2

0. I

0.1

0.6

-0.6

0.0

-o. I

-o. I

o. I

0.2

0.2

-0.5

0. I

0.3

-0.2

-o. I

~

-1.3

0.0

1.3

0.0

0,1

0.0

-0,2

0.1

-0,’2

0.9

-0.2

-0.3

-0.1

-0,2

-0.4

0.6

0.1-

0,1

-o. I

-0,1

0.1

-o. I

O.1

0.4

-0.4

-0.4

0.4

-0.3

0.1

0.1

0.s

-0,8

0,0

-0.2

-0,1

0.3

0.1

0.3

-0.4

0.0

0.2

-0,1

0.0

~

-0.3

-0.2

0.5

0.0

-0.1

0.0

-0.2

0.2

-0.1

0.6

-0.2

-0.2

-0.1

0.0

4.3

0.3

0.1

0.0

-0.1

0,0

0,1

-0.1

0.1

0,0

0,0

-0,3

0,3

-0.2

0.2

0.0

0.7

-0.7

0.0

0.0

-0.1

0,1

0.2

0,2

-0.4

0,0

0,3

-0,1

-0.2

0.0

-0.6

-0.6

1.1

0.0

0.1

-0.1

4.2

0.2

-0.2

0.8

-0.2

-0.3

-0.1

-0.1

.0.4

0.5

O.t

0.0

-0.2

-0.1

0.1

-0.1

0.1

0.2

-0.2

-0.5

0.5

-0.3

0.2

0.1

0,9

-0.9

0.0

-0,2

-0.1

0,2

0.1

0.3

-0.4

0.0

0.2

0.0

-at

—-0.1

Bias ratio’

Cbemc Scherm scheme scheme

Al A2 Bl B2

-2.4

0.5

1.8

0.0

0.0

0.2

-0,7

0.0

-0.3

t ,2

-1.0

-1,0

-0.3

-0,3

-0,4

0.6

0.7

0.0

-0.2

-0.1

0.0

-0.3

0.3

0.2

-0,2

-0,6

0.6

-0.3

0,2

0.2

0,9

.0,9

0.0

-0.2

-0.3

0.2

0.3

0,4

-1,3

0.2

0.6

-0,7

-0.3

0.0

-2.6

0.0

2.6

0.0

0.2

0.0

-0.7

0.2

-0.3

1.8

-1.0

-1.5

-0.3

-0.7

-0.6

0.8

0.3

0,2

-0.2

-0,1

0.1

-0.3

0.3

0.8

-0.8

-0.8

0.8

.0.5

0.2

0.2

1.1

-1.1

0.0

-0.4

-0.3

0.6

0.2

0.6

-1.0

0.0

0.4

-0,3

0.0

~

-0.6

-0,3

1,0

0.0

-0,2

0.0

-0,7

0.4

-0.2

1,2

-1,0

-1,0

-0.3

0.0

-0,4

0,4

0.3

0.0

-0.2

0.0

0,1

-03

0,3

0.0

0,0

-0,6

0.6

-0,3

0.4

0,0

1,0

-1.0

0,0

0,0

-0,3

0.2

0.3

0.4

.1.0

0.0

0.6

.0.3

4.7

.1,2

-1,0

2.2

0,0

0.2

-0,2

-0,7

0,4

-0.3

1.6

.1,0

-1,5

.0.3

-0,3

-0.6

0.6

0.3

0.0

4.3

-0.1

0.1

-0,3

0.3

0.4

-0.4

-1,0

1.0

-0.5

0,4

0.2

1.3

-1.3

0,0

-0.4

.0,3

0,4

0.2

0.6

-1.0

0.0

0.4

0.0

4.3

0.0 -0.3

-21-
>

Table 6.- Estimated reduction in bias and bias ratio for selected chmacteristics of the School Safety and

Discipline component (continued)

Stamfard estimate I Estimated reduction in bias’ I Bias ratio]

SWKkd Sckm Scklx Sdlenlc Scheme Schem. Schcmc scheme Scfwnc

Characteristic Percent em, Al A2 Bl B2 Al A2 BI B2

I I

Witnessed crime ., SCko[

NOnc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63,S 0.8 0.2 0.2 0.2 0.2 0.2 0.2 0,2 0.2

Wirnmsed robtwy’ 0.6 0.1 0.0 .0.0 0.0 0.0 0.0 0.0 0.0 0.0

Wim.xcd bullinE or awaulr’ 24.1 0.8 -0.3 -0.3 -0.3 -0,3 -0.4 -0.4 -0.4 -0.4

Wimc.xd two ,x mmc wcs of incidents . . . . . . . . 11.4 0.4 0.0 0.1 0.0 0.0 0.0 0.2 0.0 0.0

.

Figure 3.--Estimated reduetion in absolute bias for School Readiness characteristics (scheme Al)

1.4

I .2

1.0

0.8

I

0.2

0.0

0.0 20.0 40.0 60.0 80.0 100.0

Percent of children

SOURCE U.S. Dqmrimmt of Education, Ntiond Cenfu for Fdwation Stadstics, Naficml Hcuehc4j Mucaticm Smvey, S@Ig 1593.

-22-
I .>

Figure 4.-- IMirnated reduction in absolute vafrre of bim for school Safety and Dkipline

characteriatk (scheme Al)

1.6 T

SOURCE

1.4-

~

~ .: 1.2-

~a

; .5 1.0--

4$ o.s --

●

~~ 0.6--* ● *

j< o.4--

+

d

** +

0.2-e ● ●

++% ● ’489 ● +

0.0~ I

0.0 20.0 40.0 60.0 80.0 100.0

Percent of tildrm

U.S. Ce@n&mt of )?dUdOIl, National Cmlu for Educa!ion Smdsdcs, Naricmal Household ~UC4360. Sumey, s@w 1993.

Bias Ratio

‘lIre size of the absolute reduction in bias is not a

very useful statistical measure of the impact of

the bias because it does not take the magnitude of

the standard error of tbe estimate into account.

Cochmn (1977) discusses the impact on

confidence intervafs as the ratio of tbe bias to tbe

sampling error varies. The bias ratio is defined

as the bias divided by the sampling error of the

estimate. For each scheme the bias ratio is given

by

(4)

with the sampling error of the standmd estimate

as tbe denominator. As the bias ratio increases,

the chance of covering the population value

departs significantly from “the nominal

confidence interval. For example, a bhs ratio of

0.1 baa very little impact on a 95 percent

confidence interval, but a b]ss ratio of 1.0 results

in a nomirraf 95 percent confidence interval that

only covers the population value 83 percent of

the time.

The bias ratios for all of the response categories

for the-SR items we. given in table 5, and the

ratios for the SS&D items are in table 6. The

discussion of tbe bias ratios that follows is based

on the ratios remaining after eliminating one

response level for each item.

Many of the bias ratios for the SR items are

large, even though the average and median ratios

are near zero. Nearly half of the ratios are larger

than 0.4 in absolute value. A ratio of 0.4 is large

enough to reduce a nominal confidence interval

from 95 percent to about 93 percent. For the

SS&D items, the bias ratios are smaller. Only 5

of the 34 bias ratio estimates in table 6 are

greater than 0.4.

The bias ratios show that the biases as estimated

under the assumed model could have an effect on

the inferences made from the survey estimates.

The effect on the inferences is a greater problem

for the SR componeut than for the SS&D

component. The confidence intervals based on

tbe staodard estimates for some characteristics

will not attain the nominal confidence intervals

due to the undercoverage bias.

-23-
Consistency Across Adjustment Schemes

A review of the estimates in tables 5 and 6 shows

that if the bias reduction estimated under one

adjustment scheme is large and positive, the

estimates under the other schemes tend to be

large and positive. The comelations between the

bias estimates under the four schemes are a

measure of the consistency of the reduction in

bias estimates across the schemes. In general,

the correlations’ are very high, aa might be

expected. For the SR component, the correlation

between the estimates goes from a low of 0.6

between the estimates for schemes A2 and B 1, to

a high of 0.9 for schemes Al and A2 and

schemes B 1 and B2. The correlations for the

bias reduction estimates from the characteristics

of the SS&D comparent are uniformly high, with

correlations nearly 0.9 between all the schemes.

The bias adjustments resulting from defining the

cells by educational attainment within

race.letbnicity are highly correlated with those

formed by tenure within racdethnicity.

However, the consistency of the bias estimates

does not imply that all of the adjustment schemes

are equivalent in terms of their overall statistical

properties. So far, the dkcussion has only been

about the bias reduction, Their variance

implications are presented below.

Variance Impfieations

The results above show that the standard

estimates from the NHES :93 are subject to

coverage bias under the assumed model. Since

the ordinary measure of variation for an unbiased

estimate, the variance, is not appropriate for

biased estimates, the mean square errors of the

biased estimates are considered, The mean

square error of the estimate (MSE) is a

frequently used statistic that reflects both the

variation about the average and the bias of an

estimate. Tbe MSE is the sum of the variance

and the square of the bias of the estimate.

The MSE of the standard estimate can be

computed by using the standard variance and

bias estimates presented above.’ Tbe estimated

MSE can be written as

msea = Var(j$)+ b: (5)

where the terms have been defined before. Four

different estimates of the mean square error can

be formed, one corresponding to each of the

weighting schemes. Drily the estimated mean

square errors for scheme A 1 are shown in tables

7 and 8 because the results for the other schemes

are so similar.

It is interesting to note that the mean square error

of the estimate is functionally related to the bias

ratio discussed previously. The relationship for

the standard NHES:93 estimate is given by

This relationship explains why the value of tbe

mean square error is close to the variance

estimate unless the bias ratio is large.

6.4n unbidcstirnrdc of the MSE can be found by addinS an

unbiased estimare of the varirmcc to sn unbiased estimate of

the hiss squa.rcd. The estimated bias squared is not

%e cmreladons were computed after dekdns one of the

technically an unbiavcd cstitrrare of rhe squared hiss, but the

difference is cxtrctrrdy small in this case. Consequently, the

BPOILW Catcgoriw for each itcrn, as diccrrsscd &wve. squad bias esrinram arc used for this report.

-24-
.-

Table 7--- Estimated mean square error ana mean squm ratio for selected characteristics of the School

Readiness component

stan6ar6 .stirnim

Percent Va’iancc MSEL

8.6

33.9

57.5

2.4

34.3

20.9

6.6

35.8

26.3

63.4

3.8

3.2

3.3

4.0

30. I

66.0

7.6

11.1

30.0

31.2

13.s

84.1

15.9

93.3

6.7

52.6

47.4

62.9

37,1

73,5

26.5

33.s

66,2

35,8

64.2

5.7

94.3

4.3

16.9

7s.s

12.3

27.4

60.3

0. I

0.6

0,5

0.0

0.3

0.3

0.2

0.4

0.3

0.4

0.1

0. I

0.0

0.2

0.6

0.6

0.3-

0.4

0.5

0.6

0.3

0.2

0.2

0. I

0.1

0.6

0.6

0.6

0.6

0.3

0.3

I .0

1.0

0.4

0.4

0.2

0.2

0.3

0,8

0.8

0.2

0,4

0.5

3.0

0.8

2.2

0. I

0.s

0.3

0.2

0.5

0.4

0.4

0.1

0.1

0.0

0.2

I .0

I .3

-0.3

0,4

0.5

0.7

0,3

0.3

0,3

0.1

0.1

I .5

I .5

0.9

0.9

0,6

0.6

1.4

I ,4

1,2

1.2

0.3

0,2

0.3

0.8

0.9

0.3

0.4

Q&

Mean swam rauo’

;chem scheme Scheme Schelm

Al A2 BI B2

3.6

96.0

27.0

%.0

60.0

I 15.4

120.0

83.1

73.2

%.0

120,0

120.0

%.0

%.0

76.8

60,0

120,0

116.8

120,0

105.2

103.4

60.0

60.0

108.0

108,0

53.0

53,0

86,3

S6.3

49.2

49.2

88,2

8s.2

36.9

36.9

76.8

76.8

88,2

I 20.0

108,0

60.0

%.0

117.6

4.1

108.0

30,3

108.0

67.5

129.8

135.0

93.5

82.3

108.0

135,0

135.0

108,0

N13.O

86.4

67.5

135.0

131.4

135,0

118.4

116.4

67.5

67.5

121.5

121.5

59.6

59.6

97,1

97,1

55.3

55.3

99.3

99.3

41.5

41.5

S6.4

86.4

99,3

135,0

121.5

67.5

108,0

!32.3

3.5

92,8

26.1

92.8

58.0

111.5

116.0

80.3

70.7

92.8

116.0

116,0

92.8

92.8

74.2

58.0

116.0

112.9

116.0

101,7

1 W.o

58.0

58.0

104.4

104.4

51.2

51.2

83.4

83,4

47.5

47,5

85.3

.85.3

35.7

35.7

74.2

74.2

S5.3

116,0

104.4

5s.0

928

I 13.7

3.8

!00.8

28.3

100.8

63,0

121.2

126,0

87.2

76.8

ICKI.8

126.0

126.0

IIXI.8

lIXI.8

80.6

63.0

126.0

122.6

126.0

110.5

108.6

63.0

63,0

113,4

113,4

55.6

5S.6

90.6

90,6

51.6

51.6

92.6

92.6

38.8

38.8

80.6

126.0

113.4

63,0

IIXI.8

123.5

-25-
Table 7--- Estimated mean square error and mearr square ratio for selected characteristics of the School

Readiness component (continued)

Smdard minmte Mean square mio’

scheme scheme Schmu Schmlc

characteristic Percent variance MSE1 Al A2 Bl B2

Tcwg-hl songs or music in last wed?

Notinlst w=k . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33.3 0,4 0.7 60.0 67.5 58.0 63.0

On= m twic e 30.1 0.5 0.6 101.4 114.1 98.0 106.4

tiormom tim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36.7 0.5 0.5

Did IU13 O, crofts with child in Imt WC.P

I 10.9 124.8 107.2 116.5

N.t in last week 32.2 0.5 0.6 101.4 114.1 98.0 106.4

Oncx or twice 35.7 0.5 0.5 117,6 132.3 113,7 123.5

kormEtim . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.1 0,4 0.4 108,0 121.5 to4.4 113.4

Vi~ited library wirh child in /at nwntha

Ys. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39,4 0.8 1.1 91,7 t 03.2 88.6 %.3

NO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60.6 0.8 1.1 91.7 103.2 88.6 %.3

Visited zoo with child in Iarr monthe \

Yes . . . . . . . . . . . . . . . . 16.8 0,5 0,6 101.4 114.1 98,0 11m,4

N. 83,2 0.5 0.6 101.4 114.1 98.0 106.4

‘~Etitia--Pkh_OfkhmtiwtiAl.

ti-_6kk-_aoftidjutiti&hk-_-dtiWti.

%nmQMriacdmpUdwOh.

%lmuewiitsmallckdknwpudwokm

%s6nmemsc%tedIoc6 ikSmnin@nmyschcoL

%!mIc@kstoall-excqnth@seti@umyscbmL

NOTE -myr,otddto 103 becaneofrumdr,g

Sousa: Us. ncpuUm dEdu&On N~Gncrfa Edxmion Smiks. Nadcd-!d—SW. @18 l~j.

-26-
>

Table 8.- Estirnsted mean sauare error and mean sauare ratio for selected characteristics of the School

Safety and Discipl~e component “

Standard estimate

characteristic Pmcult Vark-x MSE1

Fear of rfkfl or mbba’# .

Far of bullying or UuiC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Fern of rumor mm wcs of imidenfd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Knowledge of crime at SC6U01

Nm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Fear of fluxl or rc&x@ . . .

Fear of bullying or assaulf . .

Fear of fwo or mtypes of imidenfd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Viclim’zaion 6y crime

NOI victimized . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Victim of theft or mMery4

Victim of b.llins or =SaulP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Victim of w. or more typs ofincid.am’............, . . . . . . . . . . . . .

9.4

32.7

57.9

3.5

46.2

20.3

4.5

23.5

263

63,2

3.1

2.6

4.3

4,2

31.3

64.5

5.5

17,1

29.4

33.9

14.1

91.2

8.8

79.9

20, I

22.3

77.7

39.2

29.7

31.1

68.5

31.5

66.1

11.9

K.6

13.3

38.7

14.1

15.6

31.6

73,0

10.9

8,9

7.2

0.3

0.4

0,3

0.0

0.3

0.3

0,1

0.3

0.4

0,3

0.0

0,0

0.1

0.1

0.5

0,6

0,1

0.4

0,4

0.5

0,5

0.1

0. I

0.3

0.3

0.3

0,3

0.4

0.3

0.4

0.5

0.5

0,3

0.3

0. I

0.3

0.4

0,3

0.2

0,4

0,3

0.1

0.1

0.1

1,7

0.5

1,1

0.0

0.3

0.3

0,1

0,3

‘ 0,4

0,6

0.1

0,1

0.1

0,1

0.6

0.9

.

0.1

0.4

0,4

0.3

0,5

0.1

0.1

0.3

0.3

0.3

0,3

0.4

0.3

0.4

0.8

0.9

0.3

0,3

0.1

0,3

0.4

0.3

0.4

0.4

0.3

0.1

0.1

0.1

Mm ,q”a mtiO’

scheme scheme Scheme Scheme

Al A2 91 B2

16,6

K9.6

26.4

112.0

112.0

107,7

n.s

112.0

100.8

45.9

56.0

%.0

100.K

100.K

94.6

80.5

77,5

112.0

109,0

109.8

112.0

100.K

Iw.s

107.7

107.7

K3.1

83.1

101.7

108.7

109.9

64.6

64.6

I 12.0

107.7

100.8

107.7

100.8

96.6

43.7

109.O

82.4

n.5

lfu3.8

112.O

18.6

IIW8

29.7

126,0

126.0

121.2

K7,2

126.0

113.4

51.6

63,0

63.0

113,4

113,4

1s6,4

90.6

87,2

126,0

122.6

12s,5

126.0

113.4

113.4

121.2

121.2

93.4

93.4

114.3

122.1

123.6

72.6

72.6

126.0

121.2

I 13.4

121.2

I 13.4

108.6

49.2

122.6

92.6

K7,2

~

113.4

16.4

88.8

26.2

111.0

111.o

lM.7

76.8

111.o

99.9

45.5

55.5

55.5

9+.9

99,9

93.8

79.K

76.8

111.0

108,0

108.8

111,0

99.9

99.9

IW.7

106.7

80.1

Ko. 1

9K.1

104.8

103. I

64.0

64,0

111,0

106,7

99.9

106.7

99.9

95.7

43.3

108.O

81.6

76.8

99.9

111.o

18.5

Icm.o

29,5

123.O

125.O

I 20.2

86.5

125.0

112.5

51.2

62,5

62.5

112.5

112.5

105.6

89.9

86.5

123.0

121,6

1225

123.O

1125

112.s

120.2

120.2

91.2

91.2

111.6

119.2

120.6

72.1

72.1

125.0

120.2

112.5

lm.2

112.5

107.K

48.8

121.6

91.9

86.5

1125

~

-27-
Table 8.– Estimated man square emnr and mean square ~tlo for selected characteristics of the School

Safety and Discipline component (continued)

standard ,3Stirnale I Man square error rano’

Sdlclm Scflam Scfmlw Scilam

Cbalacm’istic Percent Varim,ce MSE1 Al A2 BI B2

Witn<ssed crime IX school

N.ne . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63.8 0.6 0.7 105.4 I 18.6 104.5 117.6

W,lnmscd mb~ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0.6 0.0 0.0 I 12.0 126.0 111.0 12s.0

Wilncmcd bulling m SSW16 . . . . . . . . . . . . . . . . . . . . . . . . . 24. I 0.6 0.7 9S.2 110.5 97.3 109.6

Wimessed two or more typa of incid3u5 . .. . . . . . . . . . . . . . . . . . . . 11.4 0.2 0.2 112.0 126,0 111,0 125,0

‘MSEiSdIC OiiIMti-PbM&eqweof 6mbim@iume miIIsscbam A1.

$rrmlupa..qOrf,sw dnfci6uhlli@g.wmilwrq .xedlmlr,mti

NOTE PcfmIIsm?ywa 4ito1COkauseof mrdimS

SOUSCE U.S. D+amcecd Of FAwaban Na&msi Ca6 * -Sm6sdcs, Nati Hmaehd6 E&aim SIKW, WEV.g 1593.

The estimated mean square errors of the

estimates can be used to contrast the bias and

variance in the standard estimate with the

variance in the adjusted estimates. As before, the

telephone service adjustments are assumed to

reduce the coverage bias. If bias were the only

factor to be considered, then the adjusted

estimates would clearly be preferred and the only

decision would be which of the four adjustments

should be implemented. However, th e

adjustments increase the variability of the

estimates. The trade-off between the bias

reduction and the variance increase associated

with the adjustments is discussed below.

The size of the variance increase from adjusting

tbe weights using the telephone service

interruption data was expressed easlier as the

variance inflation factor (V{F). The relative

VIFS for each of the four schemes are given in

table 4. Multiplying the standard variance

estimates by the relative VIF for the appropriate

adjustment factor yields an approximate variance

for the adjusted estimates. The variance

estimates for the standard estimates are shown in

.

the second column in tables 7 and 8. Multiplying

these estimates by the appropriate ratios for the

adjustment scheme give the approximate

variances of the estimates for each adjustment

scheme.

To aid in comparing the weighting procedures,

the ratio of the estimated variance of the adjusted

estimate to the estimated mean square error for

the standard estimate was tabulated. This

estimate is called tbe mean square ratio. It can

be expressed as

msra (~) = 100 x “[ativevr? x ‘ar(p’) (7)

mse~l (p)

Note that the mean square error is derived using

the bias estimated from scheme Al only, but it is

used to compute the mean square ratios for afl

four schemes. This is done to make the ratios

comparable across the schemes. The estimates

were also computed using scheme B2 and the

results are not sensitive to the scheme used for

the bias.

-28-
The mean square ratios are useful in assessing

the effectiveness of the adjustments since they

include contributions from both the reduction in

bias (in the mean square error estimates) and the

variance (in the VIF). When the mean square

ratio is 100, the variance of the adjusted estimate

is exactly equal to the mean square error of the

biased, standard estimate. This is the break-even

point. A ratio less than 100 indicates that the

bias reduction of the adjustment is greater than

the variance increase so the adjusted estimate has

a smaller mean square error than the standard

estimate. A mean square ratio over 100 means

that the variance increase associated with the

adjustment is greater than the bias reduction and

the unadjusted estimator has the smafler mean

square error.

The mean square ratios for the selected items for

the SR and SS&D components we given in tables

7 and 8, respectively. These ratios are

summarized below. As before, the estimate for

one response category for each item was deleted

before Summarizing the estimates. To give a

more complete representation of the

dktributions, figure 5 displays the mean square

ratios for the SR items and figure 6 displays the

mean square ratios for the SS&D items. For both

of these figures, the horizontal axis is the

estimated percentage of children in the category,

The break-even line, when the mean square ratio

is 100, is shown on the chasts.

The distributions of mean square ratios for both

components are very similar. The medians for

schemes A 1 and B 1 (those based on interruptions

of 1 week or more) arc near the break-even point

of 100. The means for these schemes are close to

90 and the figures confirm that this is due to the

skewed @distributions of the mean square ratios.

Table 9.– Summaries of dkribution of mean squart ratios for selected characteristics of School Readiness

and School Safety and Discipline components

Scheme

Al AZ B1 B2

School Readiness

mea n I 89.8 101.0 86.8 94.2

median 96.0 108.0 92.8 100.8

minimum 27.0 30.3 26.1 28.3

maximum 120.0 135.0 116.0 126.0

School Safety and Discipline

mean 93.3 104.9 92.2 103.9

median 100.8 113.4 99.9 112.5

minimum 26.4 29.7 26.2 29.5

maximum 112.0 126.0 111.0 125.0

SOURCE: U.S. Depamrrent of Education, National Center for Education Statistics, National Household Education

Survey, spring 1993.

-29-
,>

Figure 5.--Estimated mean square ratios for selected School Readiness items (scheme Al)

140.0-

120.0- -m ● m ■ . w

■ -m

■ ■

.2 100.0 - ..

s

~ 80,0.-

w :

3

■ ■ ■

$ 60.0- 9 ■ ■

m

1 40.0--

w

■

■

20.0-

0.0.

0.0 20.0 40.0 60.0 80.0 100.0

Percsnt of children

SOURCE U.S. Dqamnmt of Educmion. N.Amal CQUU for Fducaim Statistics, Nalioml Household Edu@ion Survey, $pring 1993.

Figure 6.--Esfimated mean square ratios for selected School Safety and D~iplirse itesns (scheme Al)

■

20.0

0.0

0 20 40 60 80 lCO

Percent of children

SOURCE: U.S. Eepnmmnt of Ed.catim, National Cmla f. Fducmicm Sudsdcs. Nasicaal HOIWSICM IHucation Survey, spins 1993,

-30-
Figure 7.--Estitnated mean square ratios for s&xted Schmi&#@and Discipline items, schemes

and A2

140, i

I

i

● SChemc Al

OScheme Al

04 ‘1

o 20 40 do 80 lo@

Peqent of ctriidrsn

U.S. Ueparmbxt .f Fducmiom Nmicmd Cenrer for Education .$ra!istics. Nadcmd H.usch.ld Education Survey, spring 1993.

Another important observation about the

distributions of the mean square ratios for

schemes Al and B 1 is the size of the ratios at the

extremes of the distribution. The maximum

mean square ratios for both components is 120,

while some ratios are as small as 26. This means

tbe maximum increase in the mean square error

of the estimates is 20 percent, while the

reductions in mean square error for other

estimates are large. Thus, the penalty sssnciated

with adjusting even when the estimate is not

biased is modest, but the benefits of adjusting

when it is needed can be quite large.

The distributions for the mean square ratios for

schemes A 1 and B 1 are approximately equal, and

the choice of which of these schemes should be

used may be determined by nonstatistical issues,

such as availability of data and the other types of

adjustments required in the survey. The mean

square ratios show that the adjusted weights

reduce the mean square error for about half the

estimates considered when compared with the

standard weights.

The distributions of the mean square ratios for

schemes A2 and B2 (those based on interruptions

Al

of 1 month or more) have medians and means

tha~ are ~ater than 100. Essentially, these mean

square ratios are shifted upward when compared

with those of schemes Al and B 1. This is clear

from figure 7, which displays the ratios for

schemes A 1 and A2 for the SS&D component

estimates. Because estimates with smaller mean

square ratios have lower mean square errors, the

upward shift indicates that adjusting using the

shorter telephone interruption period of 1 week

or more (schemes Al and B 1) is preferable to the

longer period (schemes A2 and B2). This result

shows that in the trade-off between the variance

inflation (the longer time period bas a larger

variance) and the bias reduction (the longer time

period has smaller bias), the variance inflation

has a bigger impact on the mean square error.

Since the mean square ratios were computed

using the estimated bias from scheme A 1 (see

equation 7), this might favor those schemes with

sbnrter telephone interruption periods. As

mentioned abnve, the mean square ratios were

slsn computed using the bias estimates from

scheme B2. The distributions of the mean square

ratios using the scheme B2 b[as estimates are

very similar to those using the Al bias estimates.

-31-
For example, in the SR component the means of

the distributions of mean square ratios using

scheme B2 hias estimates are 95.7 for scheme

Al, 107.7 for scheme A2, 92.5 for scheme B 1,

and 100.5 for scheme B2. The adjustments using

the shorter time periods (Al and B 1) still

dominate those using the longer time periods (A2

and B2).

Conclusions

In most surveys conducted solely by telephone,

the potential bias introduced by exchsding

persons living in nontelephone households is a

major concern. If the percentage of the target

population living in nontelephnne households is

relatively large and the characteristics of thnse

persons are different from those who live in

telephone households, then the estimates may be

susceptible to significant coverage bka.v.

One method of addressing this problem without

resorting to other modes of data collection is to

adjust tbe estimates using data collected from the

responding households with telephones. Since

having a telephone in the housebold is not a

static phenomenon, a reasonable response

propensity model leads to adjustments based on

data on interruptions in telephone service. In

essence, households that have bad intermptions

in service are assumed to be similar to

households witbout telephones at the time of the

survey. Actually, the model only assumes that

the relationship between persnns Iiving in

households with intermptions in service and

those in households without telephones is closer

than between persons in all telephone households

and those in households without telephones. The

weights for persons in households reporting an

intermption in telephone service are increased to

adjust for those without telephones.

In tbe Nf-tES :93, households were asked about

interruptions in telephone service during the past

12 months. Estimates of the percentage of

children living in households with any

intermptions in telephone service were computed

separately for the two NHES:93 components.

The estimated percentage of children differed

somewhat for the two populations, with 12

percent of the younger children (the SR

component) living in households with some

intermption in service and 9 percent of the older

children living in households with this

characteristic. The response to the questions

about intenuptions in telephone service were

then used to adjust the standard weights. Four

alternative weights were created depending on

the length nf the telephone interruption (at least 1

week or at least 1 month) and demographic

characteristics of the household. The alternative

weights were then used to produce estimates of

the bias reduction in the standard estimates.

The bias reduction estimates computed under the

assumed - model showed that the coverage

adjustments for the SR component could have an

impnrtant effect on the inferences frnm the

survey estimates. The estimates of the ratio of

the bias to the standard error of the estimate

demonstrated that confidence intervafs for some

of the estimates based on the standard estimates

were nnt likely to attain the nominal confidence

intervals due to the undercoverage bias. On the

other hand, many of the bias ratios were small

and not substantively important.

Although the adjustments reduced bias, they afso

increased the variability of tbe estimates. The

trade-off between the bias reduction and the

variance increase was examined by comparing

the estimated mean square error of the standard

estimates (which were assumed to be biased

because of the coverage problem) to the variance

estimates from the alternative weights (assumed

to be unbiased). These ratios are referred to as

mean square ratios.

The alternative weighting schemes performed

differently with respect tn the mean square ratios,

even though they were consistent in terms of bka.v

reduction. The schemes baaed on intermptions

-32-
of telephone service of 1 week or more (schemes

Al and B 1 ) were better than the schemes based

on interrrrptions of 1 month or more (A2 and

B2). The bias adjustments resulting from using

educational attainment by race/ethnicity

categories (A 1 ) were roughly equivalent to those

using tenure by radethnicity (B 1 ). Either

scheme Al orBl could beusedif the adjustment

procedure were adopted.

The distributions of the mean square ratios show

that about half the estimates could be improved

using the telephone service intermption

adjustments. Furthermore, even for those

estimates that were less accurate due to the

variance increases associated with the

differential weights, the magnitude of the

increases were not large. In other words, the

penalty for adjusting when it did not reduce the

coverage bias was not very great. These tindlngs

suggest that the adjustments should be seriously

considered.

The size of the sample is another factor that

should be considered when evaluating the use of

the telephone service interruption adjustment.

Bias ratios increase with the sample size because

the bias is not affected while the sampling error

of the estimate (the denominator of the biaa ratio)

decreases. Thus, the adjustments should be more

beneficial in surveys with large sample sizes

where the bias ratios might be expected to be

large.

While the results of this study suggest that the

adjustments could be usefil for many estimates

from telephone surveys, other stsrdies are needed

before the adjustments are recommended. As

discussed earlier, the estimates nf the mean

square errors in this study were based on the

assumption that the adjusted estimates reduced

the bias of the estimates. This model assumption

could not be verified because of the lack of

benchmark data for comparison. The assumed

model may be beneficial to the adjusted

estimates in the sense that it results in lower

,.

estimates of the mean square errors for the

adjusted estimates. Thus, the findings of this

study should be taken as an indication that

adjustment using data on interruptions in

telephone service is a feasible rmthod that

requires further study and evaluation.

The questions about intermptions in telephone

service were included in the NHES :95 tn frrrther

evaluate this methnd of adjustment for coverage

bias. The NHES:95 has a survey compmrent on

adult education so that data on service

intermptions will be obtained for virtually all

types of households rather than being restricted

to households with children as was the case with

the NHES :93. In addition, the questions recently

were added to the National Health Interview

Survey, a survey conducted by the Census

Bureau for the National Center for Health

Statistics. The findings from this survey shnrdd

he ‘even-more useful in evaluating the method

because the survey covers households without

telephones by in-person interviews, eliminating

the need for the critical model assumption used

in this study.

In summary, the tindings of this study are:

.

.

.

.

The coverage bias associated with

households without telephones could k

impormrrt for some statistics, even after the

ordinary poststratification adjustments;

Data collected on telephone intermptimts can

be used to reduce this bias by using a

response probability type of adjuatrrren~

‘lIre benefits of the bias reduction appear to

he large enough to offset the variance

increases due to increased variability in the

weights in this strrdy, although the results

may differ for different size samples; and

The findings are tentative became they rely

on a variety of assumptions, and some of

them are favorable to the adjustment.

Further research is needed hefore the

prncedure can be recommended.

-33-
Acknowledgments

The authors and the staff of NCES would like to

thank the reviewers and editors who helped to

improve the content and presentation of the

material in the report. In particular, we would

like to thank the panel of peer reviewers who

provided valuable advice on the report, They

are: Charles Alexander, Demographic and

Statistical Methods Divisinn, Bureau of the

Census; Audrey Brinkley, Demographic and

Statistical Methods Division, Bureau of the

Census; Boh Burton, Statistical Standards and

Services Group, National Center for Education

Statistics; Ralph Lee, Data Development and

Longitudinal Studies Group, National Center for

Education Statistics; Marilyn McMillen, Surveys

and Cooperative Systems Group, National Center

for Education Statistics; and, Mike Cohen,

Statistical Standards and Services Group,

National Center for Education Statistics.

References

Brick, J., Burke, J., and West, J. (1992). Telephone Undercoverage Bias of 14- to 21-year-ofds and 3-to 5-

year-olds. National Household Education Survey Technical Repat No. 2. Washington, DC: U.S.

Department of Education, ~lce of Educational Reserrrcb and Isrrproverrrent, N(2ES 92-101

Brick, J., and West, J. (1992). Multiplicity Sampling for Dropouts in the NHES Field Test. National

Household Education Survey Technical Report No. 3. Washington, DC: U.S. Department of Education,

Office of Educational Research arrd frnprnvernent, NCES 92-102.

Cnchran, W. (1977). Sampling Techniques. New York John Wtley and Sons,

FederaJ Cornrnunications Commission (1988). J.frmitoting Repo~: CC l)rrcket hr.. 87-339. Prepared by

the Staff of the Federal-State Joint Board in CC Dncket No. 80-286, Washington, DC.

Kakon, G., and Kasprzyk, D. (1986). The treatment of missing survey data, Survey Methodology, 12, 1.

Keeter, S, (1995). Estimating telephone rrancoverage bias with a telephone survey, Paper presented at the

annual meeting of the American Association for Public Opinion Research, St. Petersburg, FL.

Kish, L. (1992). Weighting for unequal Pi, Journal of Ojiciaf Statistics, 8.

Massey, J., and Botman, S. (1988). Weighting adjustments for random digit dialed surveys. In Telephone

Survey Jfethockrkrgy, Groves, Biemer, Lyberg, Massey, Nicholls, and Waksberg (Ed.). New York

John Wiley arrd Sons.

Politz, A., and Simmons, W. (1949). An attempt to get the ‘not at homes’ into the sample without

caflbacka, Journal of the American Statistical Association, 44.

Thomberry, O., arrd Massey, J. (1988). Trends in United States telephone coverage across time and

subgroups. In Telephone Survey Methaddogy, Groves, Bierner, Lyherg, Massey, Nicholls, and

Waksberg (EA.). New York: Jnhn Wiley and Sons.

-34-
tideby !ke U.S. Gvwmment WnliWm>ce

SUFUM_, .fti.n~. M.@J s[w SSOP. WSMWIO.. GC 21MOZ9328

ISBN 0-16 -048917-2