Series 2, Number 156 June 2012

Nonresponse in the National

Survey of Children’s Health,

2007
Copyright information

All material appearing in this report is in the public domain and may be

reproduced or copied without permission; citation as to source, however, is

appreciated.

Suggested citation

Skalland BJ, Blumberg SJ. Nonresponse in the National Survey of Children’s

Health, 2007. National Center for Health Statistics. Vital Health Stat 2(156).

2012.

Library of Congress Cataloging-in-Publication Data

Nonresponse in the National Survey of Children’s Health, 2007.

p. ; cm.— (Vital and health statistics. Ser. 2 ; no. 156) (DHHS publication ;

no. (PHS) 2012-1356)

‘‘June 2012.’’

Includes bibliographical references.

ISBN 0-8406-0651-6

I. National Center for Health Statistics (U.S.) II. National Survey of Children’s

Health, 2007. III. Series: DHHS publication; no. (PHS) 2012-1356. IV. Series:

Vital and health statistics. Series 2, Data evaluation and methods research ;

no. 156.

[DNLM: 1. Child Welfare—United States—Statistics. 2. Bias (Epidemiology)—

United States—Statistics. 3. Child Health Services—United States—Statistics.

4. Data Collection—United States—Statistics. W2 A N148vb no.156 2012]

362.1989200973—dc23 2012010879

For sale by the U.S. Government Printing Office

Superintendent of Documents

Mail Stop: SSOP

Washington, DC 20402–9328

Printed on acid-free paper.
Series 2, Number 156

Nonresponse in the National

Survey of Children’s Health, 2007

Data Evaluation and Methods Research

U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES

Centers for Disease Control and Prevention

National Center for Health Statistics

Hyattsville, Maryland

June 2012

DHHS Publication No. (PHS) 2012–1356
National Center for Health Statistics

Edward J. Sondik, Ph.D., Director

Jennifer H. Madans, Ph.D., Associate Director for Science

Division of Health Interview Statistics

Jane F. Gentleman, Ph.D., Director
Contents
 

Abstract .................................................................................................. 1 
 

Introduction ............................................................................................... 1 
 

The National Survey of Children’s Health, 2007.................................................................. 1 
 

Unit Nonresponse in the 2007 NSCH .......................................................................... 2 
 

Nonresponse Bias .......................................................................................... 2 
 

Information Available on Nonrespondents ....................................................................... 3 
 

Key Survey Estimates ....................................................................................... 3 
 

NSCH Weighting ........................................................................................... 4 
 

Assessing Nonresponse Bias in the 2007 NSCH.................................................................. 4 
 

Comparing Response Rates Across Subgroups ................................................................. 4 
 

Using Rich Sampling Frame Data or Supplemental Matched Data ................................................. 4 
 

Studying Variation Within the Existing Survey ................................................................. 6 
 

Comparing Similar Estimates From Other Sources.............................................................. 8 
 

Conclusions ............................................................................................... 8 
 

Children in Excellent or Very Good Health.................................................................... 8 
 

Children With Consistent Insurance in the Past 12 Months ....................................................... 9 
 

Children With One or More Preventive Medical Care Visits in the Past 12 Months ................................... 9 
 

Children With a Medical Home ............................................................................. 9 
 

Children Whose Families Ate a Meal Together Every Day in the Past Week ......................................... 9 
 

Children Usually or Always Safe in the Community or Neighborhood .............................................. 9 
 

Limitations.............................................................................................. 9 
 

References ............................................................................................... 10 
 

Detailed Tables (Tables 1–16)................................................................................ 11 
 

Text Figure

Stages and Types of Nonrespondents in the 2007 National Survey of Children’s Health ........................... 2 
 

List of Detailed Tables

1.	 National weighted response rates....................................................................... 11 
 

2.	 Information available for both respondents and nonrespondents .............................................. 11 
 

3.	 National response rates by frame variables using base weights and nonresponse-adjusted weights .................. 12 
 

4.	 Use of frame information to compare respondents and nonrespondents at each stage ............................. 13 
 

5.	 Observed and expected means of frame variables for respondents through the interview stage ..................... 15 
 

6.	 Estimates of nonresponse bias in key survey variables attributable to biases in frame information .................. 15 
 

7.	 Comparison of nonrefusals and converted refusals......................................................... 16 
 

8.	 Comparison of non-HUDIs and converted HUDIs......................................................... 16 
 

9.	 Comparison of low-call-attempt respondents and high-call-attempt respondents ................................. 17 
 

10.	 Use of frame information to compare nonrespondents and respondents, and nonrefusals and converted refusals, at
 

each stage ......................................................................................... 18 
 

11. Use of frame information to compare nonrespondents and respondents, and non-HUDIs and converted HUDIs, at
 

each stage ......................................................................................... 19 
 

iii
12.	 Use of frame information to compare nonrespondents and respondents, and low-call-attempt respondents and
 

high-call-attempt respondents, at each stage .............................................................. 20 
 

13.	 Estimates of nonresponse bias in the key survey varibles based on comparison of all respondents and respondents
 

with five or more calls............................................................................... 21 
 

14.	 Percentage of children in excellent or very good health: Comparison of estimates from the National Survey of
 

Children’s Health and the National Health Interview Survey ................................................ 21 
 

15.	 Percentage of children with consistent insurance coverage in past 12 months: Comparison of estimates from the
 

National Survey of Children’s Health and the National Health Interview Survey ................................ 22 
 

16.	 Estimates of nonresponse bias in key survey variables, based on method used to estimate bias .................... 22 
 

iv
Objectives

For random-digit-dial telephone

surveys, the increasing difficulty in

contacting eligible households and

obtaining their cooperation raises

concerns about the potential for

nonresponse bias. This report presents

an analysis of nonresponse bias in the

2007 National Survey of Children’s

Health, a module of the State and Local

Area Integrated Telephone Survey

conducted by the Centers for Disease

Control and Prevention’s National

Center for Health Statistics.

Methods

An attempt was made to measure

bias in six key survey estimates using

four different approaches: comparison

of response rates for subgroups, use of

sampling frame data, study of variation

within the existing survey, and

comparison of survey estimates with

similar estimates from another source.

Results

Even when nonresponse-adjusted

survey weights were used, the

interviewed population was more likely

to live in areas associated with higher

levels of home ownership, lower home

values, and greater proportions of

non-Hispanic white persons when

compared with the nonresponding

population. Bias was found (although

none greater than 3%) in national

estimates of the proportion of children

in excellent or very good health, those

with consistent health insurance

coverage, and those with a medical

home. However, the level and direction

of the bias depended on the approach

used to measure it. There was no

evidence of significant bias in the

proportion of children with preventive

medical care visits, those with families

who ate daily meals together, or those

living in safe neighborhoods.

Keywords: survey error • bias • 

evaluation • SLAITS

Nonresponse in the National

Survey of Children’s Health,

2007

by Benjamin J. Skalland, M.S., NORC at the University of Chicago;

and Stephen J. Blumberg, Ph.D., Division of Health Interview

Statistics, National Center for Health Statistics

Introduction
 

Nonresponse in telephone surveys

occurs when eligible sample members

(e.g., selected households) are not

measured, either in their entirety (‘‘unit

nonresponse’’) or for particular items

(‘‘item nonresponse’’). Unit nonresponse

occurs if contact cannot be established

with eligible sample members, if eligible

sample members refuse to participate, or

if there is a language or other barrier

that prevents the interviewer from

conducting the survey with an eligible

sample member (1). Of these causes, the

first two (noncontact and

noncooperation) are particularly

troubling for random-digit-dial (RDD)

telephone surveys.

Technological impediments to

making contact with a household are

one of the primary causes of unit

nonresponse in telephone surveys (2).

These impediments include answering

machines and call-waiting, caller ID,

and call-blocking features. Each of these

services allows potential respondents to

avoid contact with unknown callers and

to be selective about which calls are

answered. If contact is made with a

household, respondent refusals also

result in nonresponse. An individual’s

propensity to refuse cooperation (either

directly or by avoiding contact) can be

related to his or her personal

characteristics and how those

characteristics interact with the

perceived cost or benefit of answering

the telephone and participating in the

survey (3).

If these personal characteristics are

also related to the substantive topics of

the survey, bias can occur. This

nonresponse bias can vary by survey

topic because different topics may be

more or less strongly related to the

personal characteristics that influence

telephone survey response propensity.

This report presents an analysis of

unit-nonresponse bias for selected

national estimates from the 2007

National Survey of Children’s Health

(NSCH).

The National Survey

of Children’s Health,

2007

According to its vision statement,

the Maternal and Child Health Bureau

(MCHB) of the U.S. Department of

Health and Human Services’ Health

Resources and Services Administration

strives ‘‘for a society where children are

wanted and born with optimal health,

receive quality care, and are nurtured

lovingly and sensitively as they mature

into healthy, productive adults’’ (4,5).

This effort is fostered by block grants to

states, which are matched by state

funds. NSCH was conducted by the

Centers for Disease Control and

Prevention’s (CDC) National Center for

Health Statistics (NCHS) to assess how

well individual states, and the nation as

a whole, are meeting MCHB’s strategic

plan goals and national performance

measures. The results from NSCH

Page 1
Page 2 [ Series 2, No. 156

support these goals by providing a basis

for federal and state program planning

and evaluation efforts.

The content of NSCH is broad,

addressing a variety of physical,

emotional, and behavioral health

indicators and measures of children’s

health experiences with the health care

system. The survey includes an

extensive battery of questions about the

family, including parental health, stress

and coping behaviors, and family

activities. NSCH also asks respondents

for their perceptions of the child’s

neighborhood. No other survey provides

this breadth of information about

children, families, and neighborhoods

with sample sizes sufficient for

state-level analyses in every state,

collected in a manner that allows

comparison among states and nationally

(6). Maternal and child health programs

in each state, and MCHB at the federal

level, use data from NSCH to

characterize children’s health status,

understand their families and

communities, and identify the challenges

they face in navigating the health care

system. Federal and state Title V

programs find the data invaluable for

planning and evaluating programs.

Researchers and public policy analysts

at the state and federal levels also use

these data to assess issues such as the

prevalence of uninsured children, the

relationship of family health to

children’s health, and the impact of state

programs on children’s health and

well-being. Finally, the data provide

baseline estimates for several MCHB

companion objectives for the Healthy

People 2020 initiative (7).

The 2007 NSCH was conducted as

part of the State and Local Area

Integrated Telephone Survey (SLAITS)

program (8), which is sponsored by

NCHS. SLAITS is a broad-based,

ongoing surveillance system available at

the national, state, and local levels for

tracking and monitoring the health and

well-being of children and adults.

SLAITS modules use the same sampling

frame as CDC’s National Immunization

Study (NIS) and immediately follow

NIS in selected households, using the

NIS sample for efficiency and economy.

In the course of identifying households

with children aged 19–35 months, NIS

uses a landline RDD sample and

computer-assisted telephone interview

(CATI) technology to screen

approximately 1 million households

each year. The process of identifying

this large number of households—most

of which are ultimately age-ineligible

for NIS—offers an opportunity to

administer other surveys on a range of

health- and welfare-related topics in an

operationally seamless, cost-effective,

and statistically sound manner.

Unit Nonresponse in

the 2007 NSCH

The stages of the 2007 NSCH and

the types of nonrespondents are shown

in the Figure. A list-assisted (9) RDD

sample of landline telephone numbers is

drawn in each state, and an attempt is

made to identify and interview

households containing children under

age 18 years. To contribute to the

survey estimates, a telephone number

that is part of the initial sample must

first be ‘‘resolved’’; that is, it must be

determined whether the telephone

number belongs to a household. If a

household is identified, it must then be

screened for the presence of children

under age 18. If the household contains

such children, a child is selected

randomly, a detailed interview about that

child is administered, and survey

estimates are produced from the

resulting data (8).

Nonresponse can occur at any of

the three stages. For some telephone

numbers, it is never determined whether

the number belongs to a household. That

is, some numbers remain unresolved.

Some households that have been

identified do not complete the

age-eligibility screener, and some

households that are identified as

containing children under age 18 do not

complete the detailed interview. This

report explores the effects of the three

types of nonrespondents—nonresolved,

non-age-screened, and noninterviewed—

on key national survey estimates.

Nonresponse Bias

Nonresponse bias in a survey – estimate (yr

) can be expressed in two

forms (10). The first formulation

NOTE: RDD is random-digit dial. 

SOURCE: CDC/NCHS, National Survey of Children’s Health, 2007. 

RDD sample 

Resolution 

Age screener 

Interview Survey

estimates 

Noninterviewed 

nonrespondents 

Non-age-screened

nonrespondents 

Nonresolved 

nonrespondents 

Figure. Stages and types of nonrespondents in the 2007 National Survey of Children’s

Health
Series 2, No. 156 [ Page 3

assumes that each unit in the target

population is, a priori, either a

respondent or a nonrespondent:

B – M – – ias(yr

) = (Yr – Ym) N

where M is the number of

nonrespondents in the population, N is

the total number of units in the target – population, Yr is the mean for

respondents in the target population, and –

Ym is the mean for nonrespondents in

the target population.

The second formulation assumes

that each unit (i) in the target population

has a propensity (ρi

) to respond:

where σyρ is th

σ

Bias(–yr

) ≈ y ρ 

–

ρ

e correlation between the

survey variable and the response

propensity (ρ), and ρ

– is the mean

response propensity in the population. In

either formulation then, the bias is

related to both the response rate and the

degree to which the respondents differ

from the nonrespondents with respect to

the survey variable.

The response rate is known, or at

least estimated, from the results of the

survey data collection operation. Table 1

presents the national weighted response

rate and its components. The response

rate was calculated in accordance with

the American Association for Public

Opinion Research standards for

Response Rate 4 (11). This response rate

calculation recognizes that some cases

of unknown eligibility (e.g., telephone

lines that rang with no answer, or

households in which the person

answering the phone refused to say

whether the household included

children) were in fact eligible. In

accordance with Council of American

Survey Research Organizations

guidelines, the proportion of eligible

cases among those with unknown

eligibility was assumed to be the same

as the proportion of eligible cases

among those with known eligibility.

Although this response rate is on the

upper end of the expected range for an

RDD survey, 50%–60% nonresponse

represents a potential for substantial

nonresponse bias. However, this is only

a potential. A meta-analysis of

nonresponse bias studies (10) revealed

little to no relationship between the

nonresponse rate and nonresponse bias.

In fact, there was more variation in

nonresponse bias between estimates

from the same survey than between

estimates from different surveys with

differing response rates.

The more important factor

contributing to nonresponse bias is the

degree to which respondents differ from

nonrespondents in regard to the survey

variables. This quantity is generally

unknown, and nonresponse bias analyses

attempt to measure this difference in

either a direct or an indirect way. From

a review of the nonresponse bias

literature, Groves (10) identified the

following five nonresponse bias study

designs and discussed the strengths and

weaknesses of the design alternatives:

+	 Comparing response rates across

subgroups.

+	 Using rich sampling frame data or

supplemental matched data.

+	 Studying variation within the

existing survey.

+	 Comparing similar estimates from

other sources.

+	 Contrasting alternative post-survey

adjustments for nonresponse.

The present report gives the results

of studies based on four of these five

designs. (Alternative post-survey

adjustments for nonresponse are not

available for the 2007 NSCH.) Each of

these approaches has its weaknesses

(10). Although there was no guarantee

of the outcome, it was hoped that using

several different approaches would

overcome the weaknesses of any

individual approach and would yield an

accurate picture of nonresponse bias.

Information Available

on Nonrespondents

Several of the approaches to

assessing nonresponse bias rely on the

availability of information on both

respondents and nonrespondents.

Because NSCH is an RDD survey, the

information available on nonrespondents

is very limited. Table 2 shows the

information known for both respondents

and nonrespondents in the 2007 NSCH.

Because this information is available on

the sampling frame and is not collected

during the survey itself, it is referred to

here as the ‘‘frame information.’’ The

first two variables—residential listed

status and advance letter status—are

case-specific. The remaining variables

are ecological; that is, they contain

information not about each case

specifically but about the telephone

exchange containing the case’s

telephone number. (A telephone

exchange is the area code plus the first

three digits of the telephone number.)

For example, although the income of

each case is unknown, the median

income for households sharing the

case’s telephone exchange is known.

This ecological information is based on

census-tract-level data, aggregated to the

telephone-exchange level. Note that

telephone exchanges vary widely in

terms of the number of people they

contain, from fewer than 10 to tens of

thousands, and so there can be

significant individual variation within a

telephone exchange.

Key Survey Estimates

In assessing nonresponse bias, this

report will focus on six selected survey

estimates that represent the six major

content areas for the survey: health,

insurance coverage, health care

utilization, health care quality, child and

family well-being, and neighborhood

characteristics. The following estimates

were selected from among the key

national indicators for children of all

ages presented in MCHB’s The National

Survey of Children’s Health 2007 (12):

+	 The proportion of children in

excellent or very good health.

+	 The proportion of children with

consistent insurance coverage (i.e.,

with no periods of uninsurance)

during the past 12 months.

+	 The proportion of children who have

had one or more medical preventive

care visits in the past 12 months.
Page 4 [ Series 2, No. 156

+	 The proportion of children who

receive coordinated, ongoing,

comprehensive care within a

medical home.

+	 The proportion of children whose

families ate a meal together every

day in the past week.

+	 The proportion of children usually

or always safe in their community

or neighborhood.

The survey respondent was a parent

or guardian who lived in the household

and who knew about the health and

health care of the child. Data collected

represent the experiences and

perceptions of those respondents, and

estimates may be subject to

measurement errors (such as respondent

memory, classification, and reporting

errors) that are not considered in this

nonresponse report.

NSCH Weighting

This report seeks to answer two

questions:

+	 What level of bias would be present

in the key survey estimates if no

post-survey adjustments for

nonresponse were performed? That

is, what is the effect of nonresponse

on the raw estimates?

+	 How well do the post-survey

adjustments for nonresponse

mitigate the raw nonresponse bias?

To answer these questions, each analysis

presented in the next section is

preformed twice: first using only the

base weights (i.e., the weights that

reflect the probabilities of telephone

number selection but do not reflect

post-survey adjustments) and then using

either the nonresponse-adjusted weights

(the weights that have been adjusted for

nonresponse at each stage) or the final

weights that have been both adjusted for

nonresponse at each stage and raked to

population control totals. For a full

description of the weighting procedures,

see ‘‘Design and Operation of the

National Survey of Children’s Health,

2007’’ (8).

Assessing

Nonresponse Bias in

the 2007 NSCH

Comparing Response Rates

Across Subgroups

A comparison of response rates

across subgroups could reveal the

presence of nonresponse bias in a

survey. If the response rate is lower for

a particular subgroup relative to that of

other subgroups, that could indicate that

the subgroup is underrepresented in the

final sample and, to the extent that the

key survey estimate is different for that

particular subgroup than for other

subgroups, there would be bias in the

overall survey estimate. Similarly, if the

response rate is higher for a particular

subgroup relative to other subgroups,

that would indicate that the subgroup is

overrepresented in the final sample, and,

to the extent that the key survey

estimate is different for that particular

subgroup than for other subgroups, there

would be bias in the overall survey

estimate. On the other hand, if the

response rate is the same across

subgroups, or if the key survey estimate

does not differ among subgroups, the

key survey estimate could still be

biased, but unequal response rates across

these subgroups will have been ruled

out as a source of bias.

Table 3 presents the national

response rates for various subgroups.

The response rates are presented first

using only the base weights and then

using the weights that have been

sequentially adjusted for nonresponse at

each stage. The subgroups were formed

based on the frame information listed in

Table 2; for each of the continuous

variables in Table 2, cases were

classified into two subgroups: those with

values above and those with values

below the median value of the variable

for all sampled cases.

These tables show that it was more

difficult to interview households in

urban areas, in wealthier areas, and in

areas with larger nonwhite populations.

The response rates were more than

5 percentage points higher for cases

outside of metropolitan statistical areas

(MSAs) than for cases inside MSAs,

and about 3 to 4 percentage points lower

for areas with higher household density.

The response rates were lower in areas

that were above the median in terms of

measures associated with wealth (e.g.,

household income, home value, rental

costs) and higher in areas with a

relatively older population. Finally, the

response rates were 5 to 6 percentage

points higher in areas above the median

in terms of percentage of the population

that is white, and lower in areas above

the median in terms of percentage of the

population that is Hispanic, black, or

Asian. As can be seen when comparing

the base-weighted response rates with

those using the adjusted weights, the

weighting adjustments for nonresponse

did little to remove these response rate

differences.

There are two limitations to this

approach. First, in order to form

subgroups each continuous sampling

frame variable in Table 2 had to be

categorized into groups, resulting in a

loss of some of the information

contained in these variables. Second, the

‘‘adjusted’’ response rates presented in

Table 3 necessarily reflect only the

weighting adjustments for nonresponse

at each stage and not the final raking of

the weights to population control totals;

the extent to which this final raking

reduced the under- or overrepresentation

of a particular subgroup in the final

weighted sample is not captured by this

analysis. The next section presents a

similar approach that is not subject to

the first limitation.

Using Rich Sampling

Frame Data or

Supplemental Matched

Data

In the previous section, response

rates were compared among subgroups

defined using sampling frame

information (i.e., the variables listed in

Table 2). The converse of that analysis

is presented here. The frame information
Series 2, No. 156 [ Page 5

is used to compare the respondents at

each stage of the survey with all cases

eligible for the stage. With the frame

information for both respondents and

nonrespondents at each stage, the

stage-specific nonresponse bias in these

variables can be measured directly.

Next, the overall nonresponse bias in

each frame variable for the survey is

estimated. For this second step, the

stage-specific measures of bias in the

frame variables are used to estimate the

total nonresponse bias in each frame

variable across the stages of the survey.

Finally, statistical models are employed

to translate the estimated overall biases

in the frame variables into estimates of

bias in the key survey estimates. In this

way, the transition is made from

nonresponse bias in the frame variables

to estimates of nonresponse bias in the

key survey estimates.

For each stage of the survey,

Table 4 shows a comparison of the

frame information for the entire sample

eligible for the stage and for the

respondents to the stage, first using the

base weights only and then using the

weights that have been sequentially

adjusted for nonresponse at each stage.

An example will be useful. Looking

at the ‘‘listed’’ variable in Table 4,

using the base weights reveals that

40.84% of the entire sample of

telephone numbers are residential-listed,

and among the resolved cases (i.e., the

respondents to the resolution stage),

36.50% are residential-listed. That is,

using the unadjusted base weights, the

resolved cases are 10.62% less

residential-listed than they would be

under full response to the resolution

stage of the survey; after the resolution

stage, without any adjustment for

nonresolution, the sample is biased

downward 10.62% in terms of

residential-listed status. However, using

the weights that have been adjusted for

nonresolution, 40.84% of the resolved

cases are residential-listed; that is, all of

the bias in residential-listed status due to

nonresolution has been removed by the

nonresponse adjustment. (This is to be

expected because residential-listed status

was one of the variables used to form

the nonresponse adjustment cells.)

Moving to the age-screener stage

and using only the unadjusted base

weights, among all resolved households

86.39% are residential-listed, and among

age-screener respondents 87.30% are

residential-listed. That is, the

age-screener respondents are 1.05%

more residential-listed than they would

be if there were full response at the

age-screener stage, meaning that an

upward bias of 1.05% was introduced in

residential-listed status at the

age-screener stage. However, using the

nonresolution-adjusted weights, 88.29%

of resolved households are listed and,

using the weights that were adjusted for

nonresponse to the age-screener, 88.29%

of age-screened households are listed.

Thus, the weighting adjustment for

non-age-screening removed all the bias

introduced by nonresponse to the

age-screener stage.

Finally, moving to the interview

stage and using only the base weights,

among households with an age-eligible

child 84.39% are residential-listed and

86.34% of the completed interviews are

residential-listed; that is, households

completing the interview were 2.31%

more residential-listed than all

households that screened as eligible to

complete the interview, indicating an

upward bias of 2.31% at the interview

stage. Using the weights adjusted for

non-age-screening, 85.45% of the

age-eligible households are listed and,

using the weights that were adjusted for

nonresponse to the interview, 85.84% of

interviewed households are listed. Thus,

the interview nonresponse adjustment

lowered, but did not completely

eliminate, the residential-listed bias

introduced due to interview

nonresponse.

Multiplying together the biases at

the resolution, age-screener, and

interview stages calculated using only

the base weights, it was estimated that

the eligible household population

identified and interviewed is 7.59% less

residential-listed than the eligible

household population as a whole. In

making this multiplication, it is assumed

(a) that the proportion residential-listed

among unresolved cases that are really

households, is equal to the proportion

residential-listed among the resolved

households, and (b) that the proportion

residential-listed among the non-agescreened households that are really

age-eligible is equal to the proportion

residential-listed among the agescreened eligible households. (These are

the same types of assumptions that were

made when calculating the response

rates in this report.) By doing the same

calculation but using the weights that

were sequentially adjusted for

nonresponse to each stage, it was

estimated that the eligible household

population identified and interviewed is

0.46% more residential-listed than the

eligible household population as a

whole. That is, although it was

estimated that a bias of about 7%–8% in

residential-listed status was introduced

due to nonresponse at the resolution,

age-screener, and interview stages, the

weighting adjustments for nonresponse

eliminated nearly all of that bias.

As shown in Table 4, this is

generally the case for the other frame

variables as well—although nonresponse

introduced biases, the nonresponse

adjustments substantially reduced those

biases. The variables with the largest

biases remaining after the nonresponse

adjustments are advance letter status

(−1.25%), the percentage of the

population that is Hispanic in the

telephone exchange (−2.25%), and the

percentage of the population that is

non-Hispanic black in the telephone

exchange (−2.09%).

Table 5 shows the observed means

of the frame variables for respondents

and the means that would be expected

under full response. For example, using

the base weight, the median household

income in the telephone exchange for

respondents who completed the

interview is $55,940. Table 4 shows the

estimated median income to be 0.65%

less than would be expected under full

response; that is, the median household

income in the telephone exchange is

expected to be $56,305 under full

response:

$55,940 $56,305 = (1 − 0.0065) .

These biases in the frame

information translate into biases in the

key survey estimates only to the extent

that the frame information is related to

the key survey estimates. To examine

these relationships, for each key survey
Page 6 [ Series 2, No. 156

estimate a logistic regression model was

filled on the respondents of the form:

′

β eXi

pi = , ′

1 + β eXi

where pi is the probability that the ith

respondent’s child is positive for the key

survey variable (e.g., is in excellent or

very good health and had consistent

insurance coverage in the past 12

months), Xi

′ is a vector containing the

frame information for the ith child, and

β is a vector of unknown parameters to

be estimated.

Evaluating the fitted model first at

the observed means of the frame

information and then at the expected

means of the frame information from

Table 5 yields an estimate of the bias in

each key survey estimate that can be

attributed to biases in the frame

variables due to nonresponse. These

estimates of biases in the key survey

estimates are shown in Table 6, first

using the base weights only and then

using the weights that have been

sequentially adjusted for nonresponse at

each stage.

As these tables show, the biases in

the frame information translate into

smaller biases in the key survey

estimates. It is estimated that the largest

bias when the base weights are used is

in the proportion of children whose

families ate a meal together every day

in the past week (1.05% bias), but this

bias is reduced to −0.10% when the

nonresponse-adjusted weights are used.

The largest absolute bias when the

nonresponse-adjusted weights are used

is in the proportion of children with a

medical home (0.35% bias).

Although these results suggest that

differences between respondents and

nonrespondents in terms of the frame

information lead to very little bias in the

key survey estimates, this does not

necessarily mean that the key survey

estimates are biased very little. It is

possible that there are differences

between the respondents and

nonrespondents that are not reflected in

the frame information. Additionally, the

results in this section do not reflect the

final raking of the nonresponse-adjusted

weights to population control totals.

This final raking could have reduced or

increased bias, but if it did, that

reduction or increase was not captured

in the analysis in this section. The next

section presents an analysis that makes

use of the final, raked weights.

Studying Variation Within

the Existing Survey

In a level-of-effort analysis, those

respondents who respond only after a

great deal of interviewing effort has

been applied are assumed to resemble

nonrespondents. Given this assumption,

a difference in a survey estimate

between ‘‘high-effort’’ respondents and

‘‘low-effort’’ respondents would indicate

that a difference exists between the

respondents and nonrespondents, and

therefore the survey estimate is biased.

This ‘‘interviewing effort’’ is

measured in three ways: verbal refusal

status, nonverbal refusal status [i.e.,

whether the respondent ‘‘hung up during

the introduction’’ (HUDI)], and the

number of calls placed. It is assumed

that respondents who verbally refused at

least once, who nonverbally refused at

least once, or who required more calls

before completing the interview are

high-effort respondents and resemble the

nonrespondents with respect to the key

survey variables.

Table 7 compares the key survey

estimates for converted verbal-refusal

cases with those for cases that

completed the interview without

verbally refusing. The comparison is

made first using the base weights and

then using the final weights that have

been adjusted for nonresponse and raked

to population control totals. Table 8

compares converted HUDIs with cases

that completed without an HUDI, and

Table 9 compares households completing

the interview in five or more calls with

those completing in four or fewer calls.

If high-effort respondents resemble

nonrespondents, then a difference in the

survey estimate between converted

refusals and nonrefusals, between

converted HUDIs and non-HUDIs, or

between those completing in five or

more calls and those completing in four

or fewer calls would suggest the

presence of nonresponse bias.

The following summarizes the

findings of the level-of-effort analyses

for each of the key survey estimates

presented in the tables:

+	 The percentage of children in

excellent or very good health is

significantly higher for converted

refusals and significantly lower for

converted HUDIs and households

completing in five or more calls.

+	 The percentage of children with

consistent insurance in the past 12

months is significantly higher for

converted refusals and significantly

lower for converted HUDIs and

households completing in five or

more calls.

+	 The percentage of children with one

or more medical preventive care

visits in the past 12 months is not

significantly different for converted

refusals, converted HUDIs, or

households completing in five or

more calls.

+	 The percentage of children with a

medical home is significantly higher

for converted refusals and

significantly lower for converted

HUDIs and households completing

in five or more calls.

+	 The percentage of children whose

families ate a meal together every

day in the past week is not

significantly different for converted

refusals but is significantly lower for

converted HUDIs and for

households completing in five or

more calls.

+	 The percentage of children usually

or always safe in the community or

neighborhood is significantly higher

for converted refusals and

significantly lower for converted

HUDIs and households completing

in five or more calls.

Conclusions that could be drawn

from this level-of-effort analysis rely on

the assumption that high-effort

respondents resemble nonrespondents

with respect to the survey variables. The

validity of this assumption is highly

questionable, and some studies have

found that it does not hold (13,14). To

test the assumption, the level-of-effort

analyses were repeated using the frame

information shown in Table 2.
Series 2, No. 156 [ Page 7

Ideally, the same analyses would

have been conducted, but instead of

using the key survey variables (the

values of which were lacking for

nonrespondents), the frame information

(which was available for both

respondents and nonrespondents) would

be used. That is, low-effort and

high-effort respondents would be

compared with nonrespondents.

However, the definition of

‘‘nonrespondent’’ must be based on the

definition of ‘‘respondent.’’ If

respondents are defined as all

interviewed cases (as they were in the

level-of-effort analyses above), then by

the fact that they were interviewed it is

known that they are households with

children. To compare them fairly with

nonrespondents, the nonrespondents

would have to be defined in the same

way; that is, nonresolved

nonrespondents would have to be

defined as households with children

whose telephone number was never

resolved; non-age-screened

nonrespondents would have to be

defined as households with children who

were never age-screened; and

noninterviewed nonrespondents would

have to be defined as households with

children who were never interviewed.

Yet if the telephone number was never

resolved or never age-screened, there is

no way to know whether the number

belongs to a household with children.

Therefore, if respondents are defined as

all interviewed households, the

corresponding nonrespondents cannot be

identified at the resolution and screener

stages.

Therefore, in testing the

assumptions, respondents and

nonrespondents were defined at each

stage separately; that is, at the resolution

stage, respondents are all resolved

telephone numbers and nonrespondents

are all nonresolved telephone numbers;

at the age-screening stage, respondents

are all age-screened households and

nonrespondents are telephone numbers

that have been resolved as households

but have not been age-screened; and at

the interviewing stage, respondents are

all age-eligible interviewed households

and nonrespondents are all age-eligible

households that were not interviewed.

This test of the assumptions, then, is not

a full test of the level-of-effort analyses

described above. Nevertheless, in

defining nonrespondents and

respondents differently at each stage, it

is still possible to test the assumption

that high-effort respondents resemble

nonrespondents within each stage.

In testing the assumption, low-effort

respondents at each stage are defined in

three ways: as those cases completing

the stage without refusing, those

completing the stage without an HUDI,

and those completing the stage in four

or fewer calls. High-effort respondents

are correspondingly defined as those

cases completing the stage after refusing

during the stage, those completing the

stage after an HUDI during the stage,

and those completing the stage in five

or more calls.

Tables 10–12 show, for the frame

variables, the percentage difference

between nonrespondents and

respondents at each stage and the

percentage difference between high- and

low-effort respondents at each stage,

where ‘‘effort’’ is defined based on

refusal status, HUDI status, and the

number of calls for the stage. The tables

also indicate which of the differences

are significant at the 0.05, 0.01, and

0.001 levels.

Table 10 suggests that the difference

between converted refusals and

nonrefusals is not indicative of the

difference between nonrespondents and

respondents. For the frame variables, the

refusal/nonrefusal difference and the

nonrespondent/respondent difference

disagree in sign or magnitude for the

majority of the comparisons. In fact, the

correlation between the refusal/

nonrefusal differences and the

nonrespondent/respondent differences is

actually negative (−0.49).

The difference between HUDI and

non-HUDI is a better indicator of the

nonrespondent/respondent difference.

Table 11 shows that the sign of the

HUDI/non-HUDI difference is the same

as the sign of the nonrespondent/

respondent difference for 25 of the 34

comparisons. The correlation between

the HUDI/non-HUDI differences and the

nonrespondent/respondent differences is

0.72, indicating fairly good agreement.

The high-call-attempt/low-callattempt difference is the best predictor

of the nonrespondent/respondent

difference. Table 12 shows that the signs

of the differences agree for 46 of the 51

comparisons. The correlation between

the high-call-attempt/low-call-attempt

differences is very high at 0.98.

This test of the assumptions, then,

supports the idea that high-effort

respondents resemble nonrespondents

when effort is defined in terms of the

number of call attempts. (But note that

just because this assumption holds for

the frame variables, it need not hold for

the key survey variables.) Returning to

the analysis of the key survey variables

by the number of calls needed to

complete the survey (Table 9), and

accepting the assumption that

respondents requiring five or more calls

to complete resemble nonrespondents, it

would appear that the final estimates of

the percentage of children in excellent

or very good health, the percentage with

consistent insurance coverage in the past

12 months, the percentage with a

medical home, the percentage whose

families ate a meal together every day

in the past week, and the percentage

usually or always safe in the community

or neighborhood are all too high (i.e.,

they are biased upward).

To turn the differences between

those completing in five or more calls

and those completing in four or fewer

calls into numerical estimates of bias for

each key survey estimate, the

five-or-more-calls respondent mean of

the key survey estimate is assigned to

all nonrespondents. The results are

presented in Table 13. For example,

when the base weights are used, the

percentage of children in excellent or

very good health based on all

respondents is 87.27%, and Table 9

shows that the rate for respondents

completing in five or more calls is

86.04%. According to Table 1, the

response rate using base weights is

46.6% (and therefore the nonresponse

rate is 53.4%). Assigning a weight of

0.466 to the 87.27% estimate for

respondents, and assuming an estimate

of 86.04% for the nonrespondents and

assigning them a weight of 0.534,

results in an overall estimate for both

respondents and nonrespondents of the

percentage of children in excellent or

very good health of 86.61%.
Page 8 [ Series 2, No. 156

With this method, the largest

estimated bias across the key survey

estimates was in the estimate of the

percentage of children with a medical

home (1.56% using base weights; 1.86%

using final weights). Since the estimates

of the biases are similar when the base

weights and final weights are used, the

weighting adjustments seem to have had

little effect on the bias.

Comparing Similar

Estimates From Other

Sources

The National Health Interview

Survey (NHIS) produces national-level

estimates of health outcomes based on

personal household interviews. Because

NHIS is a face-to-face survey, the

response rate is much higher than that

of NSCH; in 2007, the overall response

rate for the child component of NHIS

was 76.5%, compared with 46.7% for

the 2007 NSCH. In addition, NHIS

covers households that do not have

landline telephone service, whereas

NSCH does not. NHIS is thus a higher

quality source of national-level estimates

of the health of children. By taking the

NHIS estimates as ‘‘truth’’ and

comparing NSCH estimates with

corresponding estimates from NHIS, the

bias in the NSCH estimates due to

noncoverage and nonresponse can be

estimated. This comparison is done for

the estimates of the percentage of

children in excellent or very good health

and the percentage of children with

consistent insurance in the past 12

months. (NHIS estimates are not

available for the other key NSCH

estimates.)

Table 14 shows a comparison of the

national estimates of the percentage of

children reported to be in excellent or

very good health from the 2007 NSCH

and the 2007 NHIS for all children and

for age, gender, race, and household

education subgroups. Table 15 shows the

same comparisons for the national

estimates of the percentage of children

with consistent insurance in the past 12

months. The NSCH estimates are

presented using both the NSCH base

weights and the NSCH final weights;

the NHIS estimates are presented using

the final NHIS weights.

Examination of Table 14 reveals

that when the base weights are used, the

NSCH estimate of the percentage of

children in excellent or very good health

is somewhat higher than the

corresponding NHIS estimate. The

NSCH weighting adjustments moved the

estimate closer to the NHIS estimate,

but the final weighted NSCH estimate

remains 1.76 percentage points higher

than the NHIS estimate, a difference

that is statistically significant. This

result is consistent with the level-ofeffort analysis, which found evidence of

upward bias in the NSCH estimate (see

Table 13).

The final NSCH estimates are also

significantly higher than the NHIS

estimates for several of the subgroups in

Table 14 (children aged 0–4 years,

children aged 12–17 years, males,

non-Hispanic white children, and

children whose mother has more than a

high school education). The NSCH

estimate is significantly lower than the

NHIS estimate for Hispanic children and

for children whose mother has less than

a high school education.

Table 15 shows that the overall

NSCH estimate of the percentage of

children with consistent insurance in the

past 12 months is similar to the

corresponding NHIS estimate when the

NSCH base weights are used; however,

the NSCH weighting adjustments moved

the final NSCH estimates lower: the

NSCH estimate is 2.5 percentage points

lower than the NHIS estimate when the

final NSCH weights are used. The final

NSCH estimate is also significantly

lower than the NHIS estimate for most

of the subgroups (children aged 0–4

years, children aged 5–9 years, males,

females, Hispanic children, nonHispanic black children, children in each

mother’s education category, and

children whose father’s education level

is high school graduate or beyond).

The finding that the NSCH estimate

of the percentage of children with

consistent insurance in the past 12

months is significantly lower than the

NHIS estimate is surprising. Based on

the frame information analysis, finding

bias in this estimate was unexpected;

and based on the level-of-effort analysis,

the NSCH estimate was expected to be

biased upward, not downward. It should

be noted that these analyses measured

nonresponse bias and not bias due to

noncoverage, so the differences seen

between the NSCH and NHIS estimates

could be due to NSCH’s noncoverage of

no-phone and cell-phone-only

households. Another explanation may be

that although the concept of ‘‘consistent

insurance’’ was the same in both NSCH

and NHIS, the survey questions on

which this estimate is based differed

somewhat between the two surveys.

Conclusions

Assessing the extent to which

nonresponse produces biased survey

estimates is difficult, particularly in a

multistage RDD survey where little is

known about the nonrespondents. In this

report, the most commonly used

methods were applied; each has its

shortcomings, but multiple approaches

were taken with the hope of drawing

reasonably accurate conclusions about

the level of nonresponse bias in key

survey estimates.

In general, it was found that the

interviewed population was more likely

to live in rural and other areas with

lower household density when compared

with the nonresponding population. The

interviewed population was also more

likely to live in areas associated with

higher levels of home ownership, lower

home values, and a greater percentage

of non-Hispanic white persons. Even

when the nonresponse-adjusted weights

were used, minor differences by home

ownership, home values, and race

remained. Table 16 presents the resulting

estimates of bias for each key NSCH

estimate. These findings are summarized

below, and some possible limitations are

discussed.

Children in Excellent or

Very Good Health

The reported national estimates of

the percentage of children in excellent

or very good health are likely too high.

The final, national estimate is 84.37%,
Series 2, No. 156 [ Page 9

with a 95% confidence interval of

83.67%–85.03%. Based on the frame

information analysis and the level-ofeffort analysis, it is estimated that this

percentage is biased by 0.12% and

0.98%, respectively. (Note that the

biases are presented here in percentage

terms, not absolute terms, so that a

0.98% bias in an estimate of 84.37%

means that the reported estimate is

0.98% higher than the true value; that

is, the true value is 84.37%/1.0098 =

83.55%.) Similarly, if the corresponding

NHIS estimate is taken as the true

value, the NSCH estimate is found to be

too high (1.76 percentage point bias, or

2.13% bias).

Children With Consistent

Insurance in the Past 12

Months

Inconsistent measures were obtained

for the bias in the estimates of

percentage of children with consistent

insurance in the past 12 months. The

final, national estimate is 84.90%

(84.23%–85.54%), and the estimates of

bias are 0.06% (from the frame analysis)

and 0.42% (from the level-of-effort

analysis). Both of these bias estimates

imply that the true value is within the

reported 95% confidence interval.

However, when compared with the

corresponding estimate from NHIS, the

NSCH estimate was found to have a

statistically significant bias of

−2.50 percentage points, or −2.86% bias.

This inconsistency between the

measures of bias may be due to the fact

that the comparison with the NHIS

estimate is measuring both noncoverage

and nonresponse bias, whereas the frame

analysis and level-of-effort analysis are

measuring only nonresponse bias.

Additionally, because the survey

questions used to define ‘‘consistent

insurance’’ differed between NSCH and

NHIS, the estimates produced from the

two surveys may not be measuring the

same construct.

Children With One or

More Preventive Medical

Care Visits in the Past 12

Months

There was no evidence of

significant bias in the percentage of

children with one or more preventive

medical care visits in the past 12

months. The final, national estimate is

88.50% (87.98%–89.02%). The

estimated bias is 0.01% from the frame

analysis and −0.10% from the

level-of-effort analysis.

Children With a Medical

Home

The estimate of the percentage of

children with a medical home is likely

too high. The final, national estimate is

57.52% (56.68%–58.37%), and the bias

estimates are 0.35% (frame analysis)

and 1.86% (level-of-effort analysis).

Children Whose Families

Ate a Meal Together Every

Day in the Past Week

Measures of the bias in the

estimates of percentage of children

whose families ate a meal together

every day in the past week were

inconsistent. The final, national estimate

is 45.78% (44.96%–46.61%), and the

estimates of bias are −0.10% (frame

analysis) and 0.80% (level-of-effort

analysis).

Children Usually or

Always Safe in the

Community or

Neighborhood

The final, national estimate of the

percentage of children usually or always

safe in the community or neighborhood

is 86.05% (85.45%–86.66%). The

estimates of bias are 0.16% (frame

analysis) and 0.40% (level-of-effort

analysis), indicating that the final

estimate is slightly too high.

Limitations

This report focused on six survey

estimates. Each estimate was selected to

represent its associated content area:

health, insurance coverage, health care

utilization, health care quality, child and

family well-being, and neighborhood

characteristics. However, evidence of

nonresponse bias (or lack thereof) for

one estimate does not indicate the

presence (or absence) of nonresponse

bias for all other estimates within the

content area. Nonresponse bias can and

does vary for every survey estimate.

Still, the scope of any nonresponse bias

analysis must be limited to selected

survey estimates, and there is no reason

to believe that the selected survey

estimates are more or less susceptible to

nonresponse bias than any others.

As with any nonresponse bias

analysis, the findings are limited by the

information that is available about the

nonrespondents. Throughout, models

were used and assumptions were made,

some or all of which may be inaccurate

or incomplete. In transforming the

measured bias in the frame information

into bias in the key survey estimates,

models were used to relate the frame

information to the key survey estimates;

however, because the frame variables

(which are nearly all at the telephoneexchange level and not at the case level)

are not strongly related to the key

survey estimates, the models may not

have had much power to detect bias in

those estimates. The level-of-effort

analysis relied on the assumption that

those responding only after five or more

call attempts resemble nonrespondents

with respect to the key survey variables.

Although this was shown to be true with

respect to the frame variables, it need

not be true for the key survey variables.

Finally, comparison of the key survey

estimates with those obtained from

NHIS relied on the assumption that the

NHIS estimates are accurate, which may

not be the case if NHIS suffers from

nonresponse or other forms of bias.

Moreover, the NHIS estimates were

available for only two of the six key

survey variables. To the extent that the
Page 10 [ Series 2, No. 156

models and assumptions used in the

present analyses are not valid, the

conclusions may be incorrect.

Still, use of four different

approaches consistently revealed no

evidence of significant bias in the

proportion of children with preventive

medical care visits, with families who

ate daily meals together, or those living

in safe neighborhoods. Bias was found

(although none greater than 3%) in

national estimates of the proportion of

children in excellent or very good

health, with consistent health insurance

coverage, and with a medical home.

However, the level and direction of the

bias depended on the approach used to

measure it. Thus, no consistent evidence

was found of significant bias in six

survey estimates that represent the six

major content areas of the 2007

National Survey of Children’s Health.

References

1.Groves RM, Lyberg LE. An overview

of nonresponse issues in telephone

surveys. In: Groves RM, Biemer PP,

Lyberg LE, Massey JT, Nicholls WL,

Waksberg J, eds. Telephone survey

methodology. New York, NY: John

Wiley and Sons, 191–212. 1988.

2.Groves RM, Couper MP. Nonresponse

in household interview surveys. New

York, NY: John Wiley and Sons. 1998.

3.Nicoletti C, Peracchi F. Survey

response and survey characteristics:

Microlevel evidence from the European

Community Household Panel. J Royal

Stat Soc A 168(4):763–81. 2005.

4.Ireys HT, Nelson RP. New federal

policy for children with special health

care needs: Implications for

pediatricians. Pediatrics 90(3):321–7.

1992.

5.Maternal and Child Health Bureau.

Strategic plan: FY 2003–2007.

Rockville, MD: Health Resources and

Services Administration, U.S.

Department of Health and Human

Services. 2003. Available from:

http://mchb.hrsa.gov/research/

documents/mchbstratplan0307.pdf.

6.van Dyck P, Kogan MD, Heppel D,

Blumberg SJ, Cynamon ML,

Newacheck PW. The National Survey

of Children’s Health: A new data

resource. Matern Child Health J

8(3):183–8. 2004.

7.U.S. Department of Health and Human

Services. Healthy People Initiative

(ongoing). Available from: http://

healthypeople.gov/2020/default.aspx.

8.Blumberg SJ, Foster EB, Frasier AM,

et al. Design and operation of the

National Survey of Children’s Health,

2007. National Center for Health

Statistics. Vital Health Stat 1(55). 2012.

Available from: http://www.cdc.gov/

nchs/data/series/sr_01/sr01_055.pdf.

9.Lepkowski JM. Telephone sampling

methods in the United States. In:

Groves RM, Biemer PP, Lyberg LE,

Massey JT, Nicholls WL, Waksberg J,

eds. Telephone survey methodology.

New York, NY: John Wiley and Sons,

73–98. 1988.

10.Groves RM. Nonresponse rates and

nonresponse bias in household surveys.

Public Opin Q 70(5):646–75. 2006.

11.American Association for Public

Opinion Research (AAPOR). Standard

definitions: Final dispositions of case

codes and outcome rates for surveys.

5th ed. Lenexa, KS: AAPOR. 2008.

12.U.S. Department of Health and Human

Services (HHS), Health Resources and

Services Administration, Maternal and

Child Health Bureaus. The National

Survey of Children’s Health 2007.

Rockville, MD: HHS. 2009. Available

from: http://mchb.hrsa.gov/nsch/

07main/moreinfo/pdf/nsch07.pdf.

13.Fitzgerald R, Fuller L. I hear you

knocking but you can’t come in: The

effects of reluctant respondents and

refusers on sample survey estimates.

Sociol Methods Res 11(1):3–32. 1982.

14.Lin I-F, Schaeffer NC. Using survey

participants to estimate the impact of

nonparticipation. Public Opin Q

59(2):236–58. 1995.
Series 2, No. 156 [ Page 11

Table 1. National weighted response rates

Resolution Screener Interview CASRO1

Weights used rate completion rate completion rate response rate

Percent

Base . . . . . . . . . . . . . . . . . . . . . 81.9 86.3 66.0 46.6

Adjusted .................. 81.9 86.4 66.0 46.7

1

CASRO is Council of American Survey Research Organizations. The CASRO response rate is the product of the resolution rate, the age-screener completion rate, and the interview completion rate.

Table 2. Information available for both respondents and nonrespondents

Variable name Description

Listed. . . . . . . . . . . . . . . . . . . . . . . . . . . . Indicator of residential listed status.
 

Advance_letter ...................... Indicator of advance letter sent status.
 

MSA ............................ Indicator of metropolitan statistical area (MSA) status.
 

Median_HH_income................... Median household (HH) income in the telephone exchange.
 

Median_home_val .................... Median home value in the telephone exchange.
 

Median_rent........................ Median rent in the telephone exchange.
 

Median_years_educ ................... Median years of education of the population in the telephone exchange.
 

College_graduate .................... Percentage of the population in the telephone exchange that are college graduates.
 

Approx_median_age .................. Approximate median age of the population in the telephone exchange.
 

Hispanic_p ........................ Percentage of the population in the telephone exchange that is Hispanic.
 

White_p .......................... Percentage of the population in the telephone exchange that is non-Hispanic white.
 

Black_p .......................... Percentage of the population in the telephone exchange that is non-Hispanic black.
 

Asian_pacif_p....................... Percentage of the population in the telephone exchange that is non-Hispanic Asian or Pacific Islander.
 

Household_density ................... Household density in the telephone exchange.
 

Percent_listed ...................... Percentage of telephone numbers in the telephone exchange that are residential-listed.
 

Owner_occupied_p ................... Percentage of homes in the telephone exchange that are owner-occupied.
 

Rent_other_p ....................... Percentage of homes in the telephone exchange that are rented or otherwise not owner-occupied.
 
Page 12 [ Series 2, No. 156

Table 3. National response rates by frame variables using base weights and nonresponse-adjusted weights

Frame variable1 Value

Using base

weights

Using

nonresponseadjusted weights

Percent

Listed ..............................

Advance_letter ........................

MSA...............................

Median_HH_income .....................

Median_home_val ......................

Median_rent ..........................

Median_years_educ .....................

College_graduate.......................

Approx_median_age .....................

Hispanic_p ...........................

White_p. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Black_p .............................

Asian_pacif_p .........................

Household_density ......................

Percent_listed .........................

Owner_occupied_p......................

Rent_other_p .........................

Not listed

Listed

Not sent

Sent

Outside of MSA

In MSA

Below median

Above median

Below median

Above median

Below median

Above median

Below median

Above median

Below median

Above median

Below median

Above median

Below median

Above median

Below median

Above median

Below median

Above median

Below median

Above median

Below median

Above median

Below median

Above median

Below median

Above median

Below median

Above median

40.89

43.10

41.48

42.33

51.25

45.71

47.60

45.72

49.12

44.28

49.43

44.01

46.59

46.66

46.89

46.39

45.43

48.01

49.44

43.84

43.80

49.22

47.86

45.31

48.73

44.64

49.00

45.37

45.56

47.01

45.23

47.74

47.73

45.26

40.85

43.03

41.53

42.37

51.38

45.79

47.72

45.76

49.26

44.33

49.58

44.06

46.73

46.67

47.03

46.39

45.53

48.05

49.56

43.95

43.91

49.30

48.00

45.29

48.73

44.80

48.97

45.52

45.65

47.08

45.15

47.92

47.92

45.18

1

See Table 2 for description of each variable name.
Series 2, No. 156 [ Page 13

Table 4. Use of frame information to compare respondents and nonrespondents at each stage

Using base weights Using nonresponse-adjusted weights

All cases All cases

eligible Respondents Percent eligible for Respondents Percent

Frame variable1 Stage2 for the stage at the stage difference2 the stage at the stage difference2

Percent

Listed .................... 1. Resolution 40.84 36.50 –10.62 40.84 40.84 0.00

2. Age screener 86.39 87.30 1.05 88.29 88.29 0.00

3. Interview 84.39 86.34 2.31 85.45 85.84 0.46

Overall . . . . . . –7.59 . . . . . . 0.46

Advance_ letter .............. 1. Resolution 33.51 29.01 –13.43 33.51 31.88 –4.84
 

2. Age screener 79.14 80.01 1.10 79.67 80.20 0.66

3. Interview 78.03 80.90 3.68 78.12 80.53 3.09

Overall . . . . . . –9.25 . . . . . . –1.25

MSA . . . . . . . . . . . . . . . . . . . . . 1. Resolution 81.72 81.24 –0.59 81.72 81.83 0.14
 

2. Age screener 81.97 81.57 –0.49 82.12 82.13 0.01

3. Interview 83.58 82.81 –0.93 84.27 84.22 –0.05

Overall . . . . . . –1.99 . . . . . . 0.09

College_graduate ............. 1. Resolution 26.27 26.15 –0.44 26.27 26.28 0.06
 

2. Age screener 25.74 25.78 0.14 25.81 25.84 0.14

3. Interview 26.12 26.24 0.46 26.11 26.26 0.61

Overall . . . . . . 0.17 . . . . . . 0.81

Hispanic_p ................. 1. Resolution 12.80 12.58 –1.75 12.80 12.78 –0.13
 

2. Age screener 12.38 11.99 –3.15 12.54 12.49 –0.45

3. Interview 13.06 12.46 –4.58 13.85 13.61 –1.68

Overall . . . . . . –9.20 . . . . . . –2.25

White_p . . . . . . . . . . . . . . . . . . . 1. Resolution 67.85 68.03 0.27 67.85 67.85 0.00
 

2. Age screener 69.72 70.40 0.98 69.68 69.80 0.19

3. Interview 69.40 70.44 1.49 68.70 69.14 0.63

Overall . . . . . . 2.76 . . . . . . 0.81

Black_p ................... 1. Resolution 12.23 12.36 1.04 12.23 12.26 0.25
 

2. Age screener 11.23 11.04 –1.70 11.06 11.01 –0.49

3. Interview 10.78 10.50 –2.62 10.61 10.41 –1.85

Overall . . . . . . –3.27 . . . . . . –2.09

Asian_pacif_p ............... 1. Resolution 4.37 4.28 –2.18 4.37 4.36 –0.29
 

2. Age screener 4.05 3.97 –2.03 4.10 4.09 –0.34

3. Interview 4.10 3.97 –3.11 4.17 4.18 0.16

Overall . . . . . . –7.15 . . . . . . –0.47

Percent_listed ............... 1. Resolution 65.60 65.26 –0.51 65.60 65.47 –0.19
 

2. Age screener 70.13 70.32 0.27 70.13 70.14 0.02

3. Interview 69.85 70.19 0.48 69.67 69.75 0.11

Overall . . . . . . 0.25 . . . . . . –0.05

Owner_occupied_p ............ 1. Resolution 65.88 65.91 0.04 65.88 65.90 0.02
 

2. Age screener 68.70 68.90 0.29 68.71 68.72 0.01

3. Interview 69.32 69.64 0.46 69.29 69.46 0.26

Overall . . . . . . 0.79 . . . . . . 0.29

Rent_other_p. . . . . . . . . . . . . . . . 1. Resolution 34.12 34.09 –0.08 34.12 34.10 –0.04
 

2. Age screener 31.30 31.10 –0.63 31.29 31.28 –0.02

3. Interview 30.68 30.36 –1.05 30.71 30.54 –0.58

Overall . . . . . . –1.75 . . . . . . –0.64

See footnotes at end of table.
Page 14 [ Series 2, No. 156

Table 4. Use of frame information to compare respondents and nonrespondents at each stage—Con.

Using base weights Using nonresponse-adjusted weights

Frame variable1 Stage2

All cases

eligible

for the stage

Respondents

at the stage

Percent

difference2

All cases

eligible for

the stage

Respondents

at the stage

Percent

difference2

Value (dollars) Value (dollars)

Median_HH_income ........... 1. Resolution

2. Age screener

3. Interview

Overall

$53,584

54,353

55,964

. . .

$53,306

54,304

55,940

. . .

–0.52

–0.09

–0.04

–0.65

$53,584

54,497

56,271

. . .

$53,601

54,503

56,405

. . .

0.03

0.01

0.24

0.28

Median_home_val............. 1. Resolution

2. Age screener

3. Interview

Overall

224,262

218,615

219,596

. . .

220,427

216,971

215,737

. . .

–1.71

–0.75

–1.76

–4.16

224,262

220,847

222,574

. . .

223,967

220,923

222,085

. . .

–0.13

0.03

–0.22

–0.32

Median_rent ................ 1. Resolution

2. Age screener

3. Interview

Overall

573

569

577

. . .

568

566

573

. . .

–0.90

–0.50

–0.82

–2.20

573

571

582

. . .

573

571

582

. . .

–0.01

–0.03

0.05

0.01

Median (years) Median (years)

Median_years_educ............

Approx_median_age ...........

1. Resolution

2. Age screener

3. Interview

Overall

1. Resolution

2. Age screener

3. Interview

Overall

13.17

13.15

13.17

. . .

37.23

37.18

36.60

. . .

13.17

13.16

13.18

. . .

37.21

37.25

36.64

. . .

–0.05

0.05

0.10

0.10

–0.04

0.18

0.12

0.26

13.17

13.15

13.16

. . .

37.23

37.20

36.47

. . .

13.18

13.16

13.17

. . .

37.22

37.22

36.49

. . .

0.01

0.02

0.10

0.13

–0.03

0.04

0.06

0.08

Number of residents Number of residents

Household_density ............ 1. Resolution

2. Age screener

3. Interview

Overall

2.53

2.57

2.63

. . .

2.52

2.56

2.62

. . .

–0.07

–0.34

–0.40

–0.82

2.53

2.57

2.65

. . .

2.53

2.57

2.64

. . .

0.07

–0.08

–0.16

–0.17

0.00 Quantity more than zero but less than 0.005.
 

. . . Category not applicable.
 

1

See Table 2 for description of each variable name.
 

2

(Respondent mean at this stage − All eligible cases mean)/All eligible cases mean.
 

3

The overall percentage is equal to the product of the percent difference across the resolution, age-screener, and interview stages. This provides an estimate of the percent difference in the frame
 

variable between the interview respondents and the nonrespondents (at any stage) who are eligible for the interview (i.e., households with children); that is, it is an estimate of the over- or
 

underrepresentation of the interviewed households compared with the eligible population as a whole. This technique assumes that the mean of the frame variable for the eligible nonrespondents is
 

equal to the observed mean of the frame variable for the respondents. Using residential ‘‘Listed’’ as an example, it assumes that, among the nonresolved numbers that are actually households, the
 

proportion listed is equal to proportion listed among the resolved households; and it assumes that, among the non-age-screened households that actually contain children, the proportion listed is equal
 

to the proportion listed among the age-screened-eligible households.
 
Series 2, No. 156 [ Page 15

Table 5. Observed and expected means of frame variables for respondents through the interview stage

Using base weights Using nonresponse-adjusted weights

Frame variable1 Observed Expected Observed Expected

Percent

Listed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86.34 93.44 85.84 85.45 
 

Advance_letter ........................ 80.90 89.15 80.53 81.55 
 

MSA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.81 84.49 84.22 84.15 
 

College_graduate....................... 26.24 26.20 26.26 26.05 
 

Hispanic_p ........................... 12.46 13.72 13.61 13.93 
 

White_p. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70.44 68.54 69.14 68.58 
 

Black_p ............................. 10.50 10.85 10.41 10.64 
 

Asian_pacif_p ......................... 3.97 4.28 4.18 4.20 
 

Percent_listed ......................... 70.19 70.02 69.75 69.79 
 

Owner_occupied_p...................... 69.64 69.10 69.46 69.26 
 

Rent_other_p ......................... 30.36 30.90 30.54 30.73 
 

Value (dollars)

Median_HH_income ..................... $55,940 $56,305 $56,405 $56,247
 

Median_home_val ...................... 215,737 225,110 222,085 222,790
 

Median_rent .......................... 573 585 582 582 
 

Years

Median_years_educ ..................... 13.18 13.17 13.17 13.16 
 

Approx_median_age ..................... 36.64 36.55 36.49 36.46 
 

Number of residents

Household_density ...................... 2.62 2.64 2.64 2.65 
 

1

See Table 2 for description of each variable name.

Table 6. Estimates of nonresponse bias in key survey variables attributable to biases in frame information

Using base weights Using nonresponse-adjusted weights

Model evaluated Model evaluated Model evaluated Model evaluated

at observed at means of at observed at means of

respondent frame information respondent frame information

means of frame expected under Estimated means of frame expected under Estimated

Key survey variable information1 full response bias2 information1 full response bias2

Percent

Percentage of children in excellent or very good health . . 88.24 88.39 –0.17 86.43 86.32 0.12

Percentage of children with consistent insurance

coverage in the past 12 months ................ 88.74 89.00 –0.30 86.73 86.68 0.06

Percentage of children with one or more medical

preventive care visits in the past 12 months ......... 89.04 89.05 –0.01 89.19 89.19 0.01

Percentage of children with a medical home......... 61.75 61.67 0.13 59.30 59.10 0.35 
 

Percentage of children whose families ate a meal

together every day in the past week.............. 42.99 42.55 1.05 45.00 45.04 –0.10

Percentage of children usually or always safe in the

community or neighborhood................... 90.84 90.86 –0.03 89.45 89.30 0.16

1

Although the logistic regression models were evaluated at the observed means of the frame information, the results are not the observed means of the key survey variables (e.g., the final estimates of
 

the proportion of children in excellent or very good health, or the proportion of children with a medical home), as would be the case for linear regression models.
 

2

(Model evaluated at observed means − Model evaluated at expected means)/Model evaluated at expected means.
 
Page 16 [ Series 2, No. 156

Table 7. Comparison of nonrefusals and converted refusals

Using base weights Using final weights

p value for Estimate for p value for

Estimate for Estimate for Percent test of no Estimate for converted Percent test of no

Key survey variable nonrefusals converted refusals difference1 difference nonrefusals refusals difference1 difference

Percent Percent

Percentage of children in excellent or very good health . . 86.92 88.57 1.90 < 0.01 83.72 86.90 3.80 < 0.01

Percentage of children with consistent insurance

coverage in the past 12 months ................ 87.30 89.72 2.78 < 0.01 84.33 87.14 3.33 < 0.01

Percentage of children with one or more medical

preventive care visits in the past 12 months ......... 88.48 88.40 –0.10 0.87 88.57 88.22 –0.39 0.59

Percentage of children with a medical home......... 61.04 63.47 3.97 < 0.01 56.84 60.22 5.95 < 0.01

Percentage of children whose families ate a meal

together every day in the past week.............. 43.15 42.81 –0.79 0.68 45.95 45.12 –1.82 0.41

Percentage of children usually or always safe in the

community or neighborhood................... 88.76 90.15 1.57 < 0.01 85.60 87.85 2.63 < 0.01

1

(Converted refusal respondent mean − Nonrefusal respondent mean)/Nonrefusal respondent mean.

Table 8. Comparison of non-HUDIs and converted HUDIs

Using base weights Using final weights

Key survey variable

Estimate for

non-HUDIs

Estimate for

converted

HUDIs

Percent

difference1

p value for

test of no

difference

Estimate for

non-HUDIs

Estimate for

converted

HUDIs

Percent

difference1

p value for

test of no

difference

Percent Percent

Percentage of children in excellent or very good health . . 88.85 83.60 –5.91 < 0.01 86.41 80.13 –7.28 < 0.01

Percentage of children with consistent insurance

coverage in the past 12 months ................ 88.68 85.80 –3.25 < 0.01 86.16 82.29 –4.49 < 0.01

Percentage of children with one or more medical

preventive care visits in the past 12 months ......... 88.61 88.13 –0.54 0.33 88.74 88.00 –0.83 0.21

Percentage of children with a medical home......... 63.68 56.63 –11.08 < 0.01 60.31 51.74 –14.22 < 0.01

Percentage of children whose families ate a meal

together every day in the past week.............. 43.71 41.60 –4.84 < 0.01 46.29 44.74 –3.34 0.09

Percentage of children usually or always safe in the

community or neighborhood................... 90.00 86.85 –3.50 < 0.01 87.19 83.68 –4.02 < 0.01

1

(Converted HUDI respondent mean − Non-HUDI respondent mean)/Non-HUDI respondent mean.

NOTE: HUDI is hung up during the introduction.
Series 2, No. 156 [ Page 17

Table 9. Comparison of low-call-attempt respondents and high-call-attempt respondents

Using base weights Using final weights

Estimate for Estimate for Estimate for Estimate for p value

respondents respondents p value for respondents respondents for test

with 4 or with 5 or Percent test of no with 4 or with 5 or Percent of no

Key survey variable fewer calls more calls difference1 difference fewer calls more calls difference1 difference

Percent Percent

Percentage of children in excellent or very good health . . 89.06 86.04 –3.39 < 0.01 86.72 82.83 –4.49 < 0.01

Percentage of children with consistent insurance

coverage in the past 12 months ................ 88.51 87.34 –1.32 0.01 85.92 84.23 –1.96 0.01

Percentage of children with one or more medical

preventive care visits in the past 12 months ......... 88.09 88.72 0.72 0.17 88.25 88.66 0.47 0.45

Percentage of children with a medical home......... 64.11 59.79 –6.74 < 0.01 60.53 55.55 –8.22 < 0.01

Percentage of children whose families ate a meal

together every day in the past week.............. 44.01 42.43 –3.59 0.02 46.83 45.10 –3.69 0.04

Percentage of children usually or always safe in the

community or neighborhood................... 89.98 88.42 –1.74 < 0.01 87.04 85.40 –1.88 < 0.01

1

(5-or-more-call respondent mean − 4-or-fewer-call respondent mean)/(4-or-fewer-call respondent mean).
Page 18 [ Series 2, No. 156

Table 10. Use of frame information to compare nonrespondents and respondents, and nonrefusals and converted refusals, at each stage

Frame variable1 Stage2

Nonrespondent/

respondent

High-/low-effort

respondents3

Percent difference4,5

Listed .............................. Age screener

Interview

–6.59 ***

–6.15 ***

–0.40

1.86 *

Advance_letter ........................ Age screener

Interview

–7.65 ***

–10.43 ***

–0.68

6.47 ***

MSA............................... Age screener

Interview

3.55 ***

2.69 ***

2.77 ***

0.27

Median_HH_income ..................... Age screener

Interview

0.68 *

0.06

3.94 ***

2.52 **

Median_home_val ...................... Age screener

Interview

5.53 ***

5.52 ***

5.78 ***

1.04

Median_rent .......................... Age screener

Interview

3.68 ***

2.37 ***

3.60 ***

1.10

Median_years_educ ..................... Age screener

Interview

–0.35 ***

–0.31 **

0.50 ***

0.41 *

College_graduate....................... Age screener

CSHCN interview6

–1.05 **

–1.41 *

3.28 ***

2.29 *

Approx_median_age ..................... Age screener

Interview

–1.30 ***

–0.35 *

–0.09

1.00 ***

Hispanic_p ........................... Age screener

Interview

23.75 ***

13.50 ***

–3.59 *

–15.81 ***

White_p. . . . . . . . . . . . . . . . . . . . . . . . . . . . . Age screener

Interview

–7.08 ***

–4.42 ***

1.98 ***

4.59 ***

Black_p ............................. Age screener

Interview

12.35 ***

8.66 ***

–11.28 ***

–6.29 **

Asian_pacif_p ......................... Age screener

Interview

15.26 ***

8.43 **

6.66 **

–4.45

Household_density ...................... Age screener

Interview

2.53 ***

1.20 ***

0.71 ***

–1.24 **

Percent_listed ......................... Age screener

Interview

–1.93 ***

–1.36 ***

0.51 **

0.65

Owner_occupied_p...................... Age screener

Interview

–2.06 ***

–1.64 ***

1.27 ***

1.45 **

Rent_other_p ......................... Age screener

Interview

4.57 ***

3.76 ***

–2.80 ***

–3.30 **

* p < 0.05 
 

** p < 0.01 
 

*** p < 0.001
 

1

See Table 2 for description of each variable name.
 

2

For this analysis, it is not possible for a case to refuse at the resolution stage.
 

3

High-effort respondents are those who refused at the stage before completing the stage. Low-effort respondents completed the stage without refusing.
 

4

The percent difference for nonrespondent/respondent was calculated as follows: (Nonrespondent mean − Respondent mean)/Respondent mean.
 

5

The percent difference for high-/low-effort respondents was calculated as follows: (High-effort respondent mean − Low-effort respondent mean)/Low-effort respondent mean.
 

6

CSHCN is children with special health care needs.
 
Series 2, No. 156 [ Page 19

Table 11. Use of frame information to compare nonrespondents and respondents, and non-HUDIs and converted HUDIs, at each stage

Frame variable1 Stage2

Nonrespondent/

respondent

High-/low-effort

respondents3

Percent difference4,5

Listed .............................. Age screener

Interview

–6.59 ***

–6.15 ***

–2.05 ***

–1.36

Advance_letter ........................ Age screener

Interview

–7.65 ***

–10.43 ***

–1.37 ***

1.98

MSA............................... Age screener

Interview

3.55 ***

2.69 ***

–1.69 ***

–0.80

Median_HH_income ..................... Age screener

Interview

0.68 *

0.06

–4.53 ***

–4.52 ***

Median_home_val ...................... Age screener

Interview

5.53 ***

5.52 ***

–3.85 ***

–1.60

Median_rent .......................... Age screener

Interview

3.68 ***

2.37 ***

–3.55 ***

–2.87 *

Median_years_educ ..................... Age screener

Interview

–0.35 ***

–0.31 **

–1.29 ***

–1.19 ***

College_graduate....................... Age screener

Interview

–1.05 **

–1.41 *

–7.36 ***

–6.51 ***

Approx_median_age ..................... Age screener

Interview

–1.30 ***

–0.35 *

–0.94 ***

–0.26

Hispanic_p ........................... Age screener

Interview

23.75 ***

13.50 ***

23.63 ***

15.05 **

White_p. . . . . . . . . . . . . . . . . . . . . . . . . . . . . Age screener

Interview

–7.08 ***

–4.42 ***

–4.91 ***

–5.70 ***

Black_p ............................. Age screener

Interview

12.35 ***

8.66 ***

5.21 ***

17.95 ***

Asian_pacif_p ......................... Age screener

Interview

15.26 ***

8.43 **

2.90

5.77

Household_density ...................... Age screener

Interview

2.53 ***

1.20 ***

2.22 ***

1.10 *

Percent_listed ......................... Age screener

Interview

–1.93 ***

–1.36 ***

–0.96 ***

–0.72

Owner_occupied_p...................... Age screener

Interview

–2.06 ***

–1.64 ***

–1.49 ***

–1.81 ***

Rent_other_p ......................... Age screener

Interview

4.57 ***

3.76 ***

3.32 ***

4.19 ***

* p < 0.05 
 

** p < 0.01 
 

*** p < 0.001
 

1

See Table 2 for description of each variable name.
 

2

For this analysis, it is not possible for a case to HUDI at the resolution stage.
 

3

High-effort respondents are those who had an HUDI at the stage before completing the stage. Low-effort respondents completed the stage without an HUDI.
 

4

The percent difference for nonrespondent/respondent was calculated as follows: (Nonrespondent mean − Respondent mean)/Respondent mean.
 

5

The percent difference for high-/low-effort respondents was calculated as follows: (High-effort respondent mean − Low-effort respondent mean)/Low-effort respondent mean.
 

NOTE: HUDI is hung up during interview.
Page 20 [ Series 2, No. 156

Table 12. Use of frame information to compare nonrespondents and respondents, and low-call-attempt respondents and high-call-attempt

respondents, at each stage

Frame variable1 Stage

Nonrespondent/

respondent

High-/low-effort

respondents2

Percent difference3,4

Listed .............................. Resolution

Age screener

Interview

65.76 ***

–6.59 ***

–6.15 ***

68.73 ***

–3.24 ***

–2.25 **

Advance_letter ........................ Resolution

Age screener

Interview

85.85 ***

–7.65 ***

–10.43 ***

89.46 ***

–3.34 ***

2.03 *

MSA............................... Resolution

Age screener

Interview

3.31 ***

3.55 ***

2.69 ***

4.61 ***

3.49 ***

2.47 ***

Median_HH_income ..................... Resolution

Age screener

Interview

2.88 ***

0.68 *

0.06

4.34 ***

–0.03

0.45

Median_home_val ...................... Resolution

Age screener

Interview

9.63 ***

5.53 ***

5.52 ***

8.54 ***

4.48 ***

5.60 **

Median_rent .......................... Resolution

Age screener

Interview

5.01 ***

3.68 ***

2.37 ***

5.72 ***

2.71 ***

2.83 **

Median_years_educ ..................... Resolution

Age screener

Interview

0.26 ***

–0.35 ***

–0.31 **

0.68 ***

–0.49 ***

–0.26

College_graduate....................... Resolution

Age screener

Interview

2.44 ***

–1.05 **

–1.41 *

5.12 ***

–1.28 **

–0.45

Approx_median_age ..................... Resolution

Age screener

Interview

0.19 ***

–1.30 ***

–0.35 *

–0.08

–1.42 ***

–0.33 *

Hispanic_p ........................... Resolution

Age screener

Interview

9.86 ***

23.75 ***

13.50 ***

2.96 **

30.71 ***

11.13 **

White_p. . . . . . . . . . . . . . . . . . . . . . . . . . . . . Resolution

Age screener

Interview

–1.51 ***

–7.08 ***

–4.42 ***

–0.55 **

–7.19 ***

–4.37 ***

Black_p ............................. Resolution

Age screener

Interview

–5.70 ***

12.35 ***

8.66 ***

–2.63 ***

7.73 ***

12.46 ***

Asian_pacif_p ......................... Resolution

Age screener

Interview

12.32 ***

15.26 ***

8.43 **

6.68 ***

16.33 ***

11.86 **

Household_density ...................... Resolution

Age screener

Interview

0.41 ***

2.53 ***

1.20 ***

0.42 ***

2.84 ***

1.25 ***

Percent_listed ......................... Resolution

Age screener

Interview

2.82 ***

–1.93 ***

–1.36 ***

2.12 ***

–1.92 ***

–1.12 ***

Owner_occupied_p...................... Resolution

Age screener

Interview

–0.24 **

–2.06 ***

–1.64 ***

0.11

–2.70 ***

–1.45 ***

Rent_other_p ......................... Resolution

Age screener

Interview

0.47 **

4.57 ***

3.76 ***

–0.22

6.08 ***

3.39 ***

* p < 0.05 
 

** p < 0.01 
 

*** p < 0.001
 

1

See Table 2 for description of each variable name.
 

2

High-effort respondents are those who completed the stage in five or more calls. Low-effort respondents completed the stage in four or fewer calls.
 

3

The percent difference for nonrespondent/respondent was calculated as follows: (Nonrespondent mean − Respondent mean)/Respondent mean.
 

4

The percent difference for high-/low-effort respondents was calculated as follows: (High-effort respondent mean − Low-effort respondent mean)/Low-effort respondent mean.
 
Series 2, No. 156 [ Page 21

Table 13. Estimates of nonresponse bias in the key survey variables, based on comparison of all respondents and respondents with five or

more calls

Using base weights Using final weights

Respondents Respondents Respondents Respondents

All with 5 or and Percent All with 5 or and Percent

Key survey variable respondents more calls nonrespondents1 bias2 respondents more calls nonrespondents1 bias2

Percent

Percentage of children in excellent or very good health . . 87.27 86.04 86.61 0.76 84.37 82.83 83.55 0.98

Percentage of children with consistent insurance

coverage in the past 12 months ................ 87.81 87.34 87.56 0.29 84.90 84.23 84.54 0.42

Percentage of children with one or more medical

preventive care visits in the past 12 months ......... 88.47 88.72 88.61 –0.15 88.50 88.66 88.59 –0.10

Percentage of children with a medical home......... 61.56 59.79 60.62 1.56 57.52 55.55 56.47 1.86 
 

Percentage of children whose families ate a meal

together every day in the past week.............. 43.07 42.43 42.73 0.80 45.78 45.10 45.42 0.80

Percentage of children usually or always safe in the

community or neighborhood................... 89.06 88.42 88.72 0.39 86.05 85.40 85.71 0.40

1

(Estimate for all respondents * Response rate) + (Estimate for respondents with 5 or more calls * Nonresponse rate).

2

(Estimate for all respondents – Estimate for respondents and nonrespondents)/Estimate for respondents and nonrespondents.

Table 14. Percentage of children in excellent or very good health: Comparison of estimates from the National Survey of Children’s Health

and the National Health Interview Survey

NSCH1 NHIS2

Using base weights Using final weights Using final weights Comparison using final weights

Characteristic Estimate

Lower

95%

limit

Upper

95%

limit Estimate

Lower

95%

limit

Upper

95%

limit Estimate

Lower

95%

limit

Upper

95%

limit

Percent

difference

Lower

95%

limit

Upper

95%

limit

Percent Percentage points

Overall ................. 87.27 86.78 87.76 84.37 83.67 85.03 82.61 81.53 83.64 1.76 0.50 3.02

Child’s age

0–4 years ............... 90.08 89.22 90.94 87.76 86.48 88.93 82.99 81.16 84.67 4.77 2.63 6.91

5–11 years ............... 86.67 85.82 87.53 83.31 82.12 84.42 84.10 82.51 85.57 –0.79 –2.71 1.13

12–17 years .............. 85.97 85.16 86.78 82.84 81.65 83.97 80.66 78.83 82.37 2.18 0.07 4.29

Child’s gender

Male................... 86.73 86.05 87.42 83.87 82.91 84.80 82.25 80.76 83.64 1.62 –0.10 3.34

Female ................. 87.85 87.15 88.55 84.92 83.91 85.88 82.99 81.52 84.37 1.93 0.20 3.66

Child’s race/ethnicity

Hispanic ................ 71.68 69.60 73.76 68.38 65.97 70.70 72.13 69.68 74.46 –3.75 –7.12 –0.38

Non-Hispanic white only....... 91.94 91.53 92.34 91.00 90.38 91.59 87.73 86.40 88.94 3.27 1.87 4.67

Non-Hispanic black only....... 81.76 80.31 83.21 80.43 78.71 82.04 77.59 74.93 80.04 2.84 –0.21 5.89

Non-Hispanic other.......... 88.08 86.52 89.64 86.54 84.52 88.34 82.92 78.80 86.37 3.62 –0.61 7.85

Mother’s education level

Less than high school ........ 65.26 62.52 68.00 63.03 59.99 65.98 68.03 64.85 71.05 –5.00 –9.31 –0.69

High school graduate ........ 83.13 81.85 84.41 81.08 79.50 82.56 80.36 78.13 82.41 0.72 –1.91 3.35

More than high school ........ 91.97 91.53 92.41 90.71 90.04 91.34 88.85 87.66 89.94 1.86 0.55 3.17

Father’s education level

Less than high school ........ 67.71 64.64 70.78 64.76 61.17 68.19 69.25 65.67 72.61 –4.49 –9.43 0.45

High school graduate ........ 86.71 85.58 87.85 84.55 82.98 86.00 83.01 80.86 84.97 1.54 –1.01 4.09

More than high school ........ 92.84 92.35 93.32 91.92 91.19 92.60 90.98 89.78 92.05 0.94 –0.39 2.27

1

NSCH is National Survey of Children’s Health.

2

NHIS is National Health Interview Survey.
Page 22 [ Series 2, No. 156

Table 15. Percentage of children with consistent insurance coverage in the past 12 months: Comparison of estimates from the National

Survey of Children’s Health and the National Health Interview Survey

NSCH1 NHIS2

Using base weights Using final weights Using final weights Comparison using final weights

Lower Upper Lower Upper Lower Upper Lower Upper

95% 95% 95% 95% 95% 95% Percent 95% 95%

Characteristic Estimate limit limit Estimate limit limit Estimate limit limit difference limit limit

Percent Percentage points

Overall ................. 87.81 87.34 88.29 84.90 84.23 85.54 87.40 86.42 88.31 –2.50 –3.65 –1.35

Child’s age

0–4 years ............... 87.83 86.84 88.83 85.01 83.61 86.31 89.84 88.23 91.24 –4.83 –6.85 –2.81

5–11 years ............... 87.19 86.35 88.02 84.56 83.42 85.62 87.08 85.46 88.54 –2.52 –4.41 –0.63

12–17 years .............. 88.37 87.67 89.06 85.19 84.17 86.17 85.76 84.18 87.20 –0.57 –2.38 1.24

Child’s gender

Male................... 87.82 87.17 88.47 84.89 83.96 85.78 87.96 86.66 89.15 –3.07 –4.61 –1.53

Female ................. 87.79 87.09 88.48 84.89 83.91 85.81 86.81 85.39 88.11 –1.92 –3.58 –0.26

Child’s race/ethnicity

Hispanic ................ 74.84 72.82 76.85 71.73 69.41 73.95 79.80 77.74 81.71 –8.07 –11.09 –5.05

Non-Hispanic white only....... 91.20 90.79 91.62 89.63 89.02 90.22 89.68 88.40 90.83 –0.05 –1.41 1.31

Non-Hispanic black only....... 85.25 83.76 86.75 83.07 81.23 84.77 90.37 88.39 92.05 –7.30 –9.84 –4.76

Non-Hispanic other.......... 89.55 88.35 90.75 88.21 86.43 89.79 85.46 80.24 89.47 2.75 –2.13 7.63

Mother’s education level

Less than high school ........ 72.15 69.60 74.71 71.02 68.18 73.71 76.61 73.70 79.29 –5.59 –9.52 –1.66

High school graduate ........ 83.76 82.50 85.02 80.90 79.30 82.41 86.31 84.25 88.14 –5.41 –7.90 –2.92

More than high school ........ 91.47 91.02 91.93 89.75 89.07 90.39 91.75 90.69 92.70 –2.00 –3.20 –0.80

Father’s education level

Less than high school ........ 72.59 69.76 75.42 71.79 68.67 74.72 75.96 72.61 79.01 –4.17 –8.58 0.24

High school graduate ........ 84.29 82.94 85.63 80.95 79.12 82.65 84.96 82.67 87.00 –4.01 –6.80 –1.22

More than high school ........ 92.54 92.08 93.01 91.01 90.27 91.70 92.71 91.48 93.77 –1.70 –3.05 –0.35

1

NSCH is National Survey of Children’s Health.

2

NHIS is National Health Interview Survey.

Table 16. Estimates of nonresponse bias in key survey variables, based on method used to estimate bias

Method2

Key survey Comparison

estimate1 Frame Level-of­ with NHIS3

(95% confidence information effort estimate

Key survey variable interval) analysis analysis analysis

Percent

Percentage of children in excellent or very good health . . 84.37 0.12 0.98 2.13

(83.67, 85.03)

Percentage of children with consistent insurance

coverage in the past 12 months ................ 84.90 0.06 0.42 –2.86

(84.23, 85.54)

Percentage of children with one or more medical

preventive care visits in the past 12 months ......... 88.50 0.01 –0.10 --­

(87.98, 89.02)

Percentage of children with a medical home......... 57.52 0.35 1.86 --­

(56.68, 58.37)

Percentage of children whose families ate a meal

together every day in the past week.............. 45.78 –0.10 0.80 --­

(44.96, 46.61)

Percentage of children usually or always safe in the

community or neighborhood................... 86.05 0.16 0.40 --­

(85.45, 86.66)

- - - Data not available (not collected in NHIS). 
 

1

Key survey estimates use final weights that have been adjusted for nonresponse and raked to population control totals.
 

2

Estimates of bias use nonresponse-adjusted or raked weights, depending on the analysis. Here, the biases are presented as percentages, not absolute terms, so that a 0.98% bias in an estimate of
 

84.37 means that the reported estimate is 0.98% higher than the true value (i.e., the true value is 84.37/1.0098 = 83.55).

3

NHIS is National Health Interview Survey.
Vital and Health Statistics

Series Descriptions

ACTIVE SERIES

Series 1.	 Programs and Collection Procedures—This type of report

describes the data collection programs of the National Center

for Health Statistics. Series 1 includes descriptions of the

methods used to collect and process the data, definitions, and

other material necessary for understanding the data.

Series 2.	 Data Evaluation and Methods Research—This type of

report concerns statistical methods and includes analytical

techniques, objective evaluations of reliability of collected

data, and contributions to statistical theory. Also included are

experimental tests of new survey methods, comparisons of

U.S. methodologies with those of other countries, and as of

2009, studies of cognition and survey measurement, and final

reports of major committees concerning vital and health

statistics measurement and methods.

Series 3.	 Analytical and Epidemiological Studies—This type of

report presents analytical or interpretive studies based on vital

and health statistics. As of 2009, Series 3 also includes

studies based on surveys that are not part of continuing data

systems of the National Center for Health Statistics and

international vital and health statistics reports.

Series 10.	 Data From the National Health Interview Survey—This

type of report contains statistics on illness; unintentional

injuries; disability; use of hospital, medical, and other health

services; and a wide range of special current health topics

covering many aspects of health behaviors, health status, and

health care utilization. Series 10 is based on data collected in

this continuing national household interview survey.

Series 11.	 Data From the National Health Examination Survey, the

National Health and Nutrition Examination Surveys, and

the Hispanic Health and Nutrition Examination Survey—

In this type of report, data from direct examination, testing,

and measurement on representative samples of the civilian

noninstitutionalized population provide the basis for (1)

medically defined total prevalence of specific diseases or

conditions in the United States and the distributions of the

population with respect to physical, physiological, and

psychological characteristics, and (2) analyses of trends and

relationships among various measurements and between

survey periods.

Series 13.	 Data From the National Health Care Survey—This type of

report contains statistics on health resources and the public’s

use of health care resources including ambulatory, hospital,

and long-term care services based on data collected directly

from health care providers and provider records.

Series 20.	 Data on Mortality—This type of report contains statistics on

mortality that are not included in regular, annual, or monthly

reports. Special analyses by cause of death, age, other

demographic variables, and geographic and trend analyses

are included.

Series 21.	 Data on Natality, Marriage, and Divorce—This type of

report contains statistics on natality, marriage, and divorce

that are not included in regular, annual, or monthly reports.

Special analyses by health and demographic variables and

geographic and trend analyses are included.

Series 23.	 Data From the National Survey of Family Growth—These

reports contain statistics on factors that affect birth rates,

including contraception and infertility; factors affecting the

formation and dissolution of families, including cohabitation,

marriage, divorce, and remarriage; and behavior related to

the risk of HIV and other sexually transmitted diseases.

These statistics are based on national surveys of women and

men of childbearing age.

DISCONTINUED SERIES

Series 4.	 Documents and Committee Reports—These are final

reports of major committees concerned with vital and health

statistics and documents. The last Series 4 report was

published in 2002. As of 2009, this type of report is included

in Series 2 or another appropriate series, depending on the

report topic.

Series 5.	 International Vital and Health Statistics Reports—This

type of report compares U.S. vital and health statistics with

those of other countries or presents other international data of

relevance to the health statistics system of the United States.

The last Series 5 report was published in 2003. As of 2009,

this type of report is included in Series 3 or another series,

depending on the report topic.

Series 6.	 Cognition and Survey Measurement—This type of report

uses methods of cognitive science to design, evaluate, and

test survey instruments. The last Series 6 report was

published in 1999. As of 2009, this type of report is included

in Series 2.

Series 12.	 Data From the Institutionalized Population Surveys—

The last Series 12 report was published in 1974. Reports

from these surveys are included in Series 13.

Series 14.	 Data on Health Resources: Manpower and Facilities—

The last Series 14 report was published in 1989. Reports on

health resources are included in Series 13.

Series 15.	 Data From Special Surveys—This type of report contains

statistics on health and health-related topics collected in

special surveys that are not part of the continuing data

systems of the National Center for Health Statistics. The last

Series 15 report was published in 2002. As of 2009, reports

based on these surveys are included in Series 3.

Series 16.	 Compilations of Advance Data From Vital and Health

Statistics—The last Series 16 report was published in 1996.

All reports are available online, and so compilations of

Advance Data reports are no longer needed.

Series 22.	 Data From the National Mortality and Natality Surveys—

The last Series 22 report was published in 1973. Reports

from these sample surveys, based on vital records, are

published in Series 20 or 21.

Series 24.	 Compilations of Data on Natality, Mortality, Marriage, and

Divorce—The last Series 24 report was published in 1996.

All reports are available online, and so compilations of reports

are no longer needed.

For answers to questions about this report or for a list of reports published

in these series, contact:

Information Dissemination Staff

National Center for Health Statistics

Centers for Disease Control and Prevention

3311 Toledo Road, Room 5412

Hyattsville, MD 20782

1–800–232–4636

E-mail: cdcinfo@cdc.gov

Internet: http://www.cdc.gov/nchs
U.S. DEPARTMENT OF

HEALTH & HUMAN SERVICES

Centers for Disease Control and Prevention

National Center for Health Statistics

3311 Toledo Road

Hyattsville, MD 20782

OFFICIAL BUSINESS

PENALTY FOR PRIVATE USE, $300

MEDIA MAIL
 

POSTAGE & FEES PAID
 

CDC/NCHS
 

PERMIT NO. G-284
 

CS230450

DHHS Publication No. (PHS) 2012–1356, Series 2, No. 156