Assessment of Potential Bias in Telephone Survey Estimates due to

Noncontact and Noncoverage where Respondents Primarily use Wireless

Telephones or do not have Landline Telephones 

Meena Khare and Abera Wouhib 

Meena Khare, National Center for Health Statistics, 3311 Toledo Road, Room #3112, Hyattsville, MD 20782, mkhare@cdc.gov 

Abstract1

The quality of survey estimates depends on coverage of the target population, response rates, and measurement errors. 

Landline telephone surveys exclude households that do not have landline telephones including wireless-only households. 

Among children under 18 years of age, the prevalence of living in wireless-only households increased from 3.7% in 2003 

(January-June) to 19% in 2008 (July-December) according to the National Health Interview Survey (NHIS). In addition, 

households with mixed telephone service where members primarily use wireless telephones for most calls (wireless-mostly) 

may not answer their landline telephones which results in survey noncontact. In 2007, wireless-mostly and wireless-only 

households accounted for 15% and 13% of children under 18 years of age, respectively. In the U.S., the prevalence of the 

wireless-only and wireless-mostly population has been consistently increasing and may affect the validity of estimates from 

telephone surveys such as the National Immunization Survey (NIS). Telephone status and interview data for children under 

18 years of age from the NHIS-Sample-Child file are analyzed to compare alternative methods to adjust sampling weights 

and evaluate potential bias in weighted estimates. We used interview data to compare characteristics of children and 

estimates of selected outcome measures that are associated with health conditions and vaccinations. The purpose of this paper 

is to evaluate potential bias in RDD survey estimates after adjusting for noncoverage of households that do not have access to 

landline telephone. 

Keywords: Weighting, propensity score models, RDD telephone survey 

Introduction 

The quality of survey estimates depends on coverage of the target population, response rates, and extent of measurement 

errors. Response rates for household-based telephone surveys have been continuously declining for the last decade (Battaglia 

et al., 2007, Curtin et al., 2005). With rapidly changing technology and telephony, households (i.e., persons) are substituting 

residential landline telephones with wireless (aka cell-phone or mobile) telephones, and as a result, noncoverage of 

residential households in landline telephone (LT) surveys has been also increasing (Blumberg and Luke, 2009). Despite all 

challenges, random-digit-dialing (RDD) landline telephone surveys are still the quickest and cost-effective ways to collect 

data and investigate emerging public health issues. The goal of the household-based RDD surveys is to produce unbiased 

national estimates but high noncoverage and nonresponse increase the potential for bias in survey estimates. 

Blumberg and Luke (2009a) used data from the National Health Interview Survey (NHIS), which covers both telephone and 

nontelephone households, to show that trends in wireless substitution among adults increased from 2.9% in 2003 to 18.4% in 

2008 while the prevalence of phoneless households remained constant and ranged from 1.6-2.1%. Among households with 

children under 18 years of age, the wireless substitution increased from 2.9% in 2003 to 18.7% in 2008 while prevalence of 

phoneless households ranged from 1.5-2.4%. The authors also showed that the prevalence of wireless substitution is higher 

among certain socio-demographic subgroups: adults aged 25-29 years; men; those living in poverty, renting their home, 

living alone or living with unrelated roommates; those living in south, and of Hispanic or non-Hispanic black race/ethnicity. 

Renting home and living alone or living with unrelated roommates are the strongest predictors of wireless substitution. Rates 

of landline telephone coverage also show substantial variation by state and other geography (Blumberg et al., 2009b). In 

addition, households with mixed telephone service where members primarily use wireless telephones for most calls (wirelessmostly) may not answer their landline telephones which results in survey noncontact. Adults from wireless-mostly 

households are more likely to be college graduates, living with children, having higher income, and living in metropolitan 

1 “The findings and conclusions in this paper are those of the authors and do not necessarily represent the views of the Centers for Disease Control and

Prevention.” 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

0

10

20

30

40

50

60

70

Jan–June

2003

July–Dec

2003

Jan–June

2003

July–Dec

2004

Jan June

2005

July Dec

2005

Jan June

2006

July-Dec

2006

Jan June

2007

July-Dec

2007

Jan June

2008

June Dec

2008

Landline households with a wireless telephone Landline households without a wireless telephone

Wireless only households Phoneless households

Cell phone questions 

changed in 2007

areas (Blumberg and Luke, 2009a). The following chart shows 6-monthly trends in the prevalence household telephone status 

among households with children under 18 years of age from the 2003-2008 NHIS. 

Figure 1: Trends in the 6-Monthly Prevalence of Household Telephone Status among Households with Children Aged <18 

Years, 2003-2008 NHIS (source: Blumberg and Luke, 2009a) 

- - - - - -

-

0

10

20

30

40

50

60

70

Jan–June

2003

July–Dec

2003

Jan–June

2003

July–Dec

2004

Jan-June

2005

July-Dec

2005

Jan-June

2006

July-Dec

2006

Jan-June

2007

July-Dec

2007

Jan-June

2008

June-Dec

2008

Landline households with a wireless telephone Landline households without a wireless telephone

Wireless-only households Phoneless households

Cell phone questions 

changed in 2007 

In 2007, 15.2%, 13.3% and 13.8% of children under 18 years of age were living in non-landline (NLT), wireless-only 

(WPO), and wireless-mostly households, respectively. The prevalence of children living in WPO households decreased with 

increasing age of child and was 20.6%, 11.7 %, and 8.7% among children aged 0-4 years, 5-12 years, and 13-17 years, 

respectively. The corresponding prevalence of children from NP households was 3.3%, 1.4%, and 1.3% for the three age 

groups. The prevalence of wireless-mostly did not vary much by age (12.7%, 13.0%, 14.6% for the three age groups, 

respectively). Since the characteristics of persons living in landline telephone (LT) households is considerably different from 

the population living in NLT households, estimates from RDD surveys are subject to potential bias due to noncoverage and 

adjustments are required to compensate for this noncoverage and to reduce bias. The increasing prevalence of the WPO and 

wireless-mostly populations may affect validity of estimates from telephone surveys such as the National Immunization 

Survey (NIS; Smith et al., 2005). We used data from the NHIS for the age groups 0-4 years and 13-17 years to assess 

potential bias in vaccination estimates for age-eligible children from the NIS-Child (19-35 months) and the NIS-Teen (13-17 

years) surveys assuming similar nonresponse and noncoverage characteristics for the two surveys. 

Data 

The target population for the NHIS is the US civilian non-institutionalized resident population (with or without access to 

telephones) and data are collected through in-person interviews. In 2003, a question about access to wireless telephone was 

added to the NHIS and in 2007 another question was added on usage pattern of wireless phones (sometime or most-of the 

time). Data from the 2007 NHIS-child sample (n=9417, http://www.cdc.gov/NCHS/nhis/nhis_2007_data_release.htm ) are 

used to model and compare characteristics of children living in phoneless (NP) households, wireless-only households (WPO), 

with children living in landline telephone (LT) households or in households with interruptions (LTI) in landline telephone 

service. We created six groups based on household telephone status: 

1) LTI contains children from households which had only LT service at the time of the survey and had interruptions 

in LT service (I) for one week or more during the previous 12 months;

2) LTWP-some contains children from mixed households which had LT service, had no interruption in LT service 

and also report using wireless phones sometime; 

3) LTWP-mostly contains children from mixed households which had LT service, had no interruption in LT 

service, and also report using wireless phones most of the time; 

4) LT-only contains children from households which had only LT service, had no interruption in LT service, and 

had no access to wireless phones in the household; 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

5) WPO contains children from households without any LT service during the previous year and which had at least 

one household member with access to a wireless telephone during that time; 

6) NP contains children from households with no access to LT or wireless telephone service in the household during 

the previous year (i.e. phoneless households). 

To assess bias, we used variables common to both NIS and NHIS and compared the prevalence of selected health related 

variables such as Influenza vaccinations and Asthma among children aged 0-4 years, 5-12 years, and 13-17 years. We 

assumed that the LT sample from the NHIS was similar to a sample from a telephone survey like the NIS; households and 

children are selected randomly within sampling domains. 

Methods 

Generally, to reduce bias in survey estimates, sampling weights are adjusted and post-stratified within homogeneous 

weighting classes to account for interview nonresponse and noncoverage of the target population. Keeter (1995), Brick et al.

(1996), Frankel et al. (2003), and Srinath et al. (2002) previously showed that the socioeconomic characteristics of persons 

who live in households with interruptions of one week or more in landline telephone service within the past 12 months (~4%) 

are similar to those who live in nontelephone households (~3%); households with interruption of less than one week were 

assumed to be similar to those with continuous landline service. This is based on the observation that had the survey been 

conducted at some point in time, when the household had interruptions, the household would have been considered as part of 

the population of nontelephone households (regardless of access to a wireless telephone). Therefore, persons living in 

households with an interruption in landline telephone service can be used to represent persons living in nontelephone 

households in RDD surveys. However, with the recent increase in WPO households, Keeter’s interruption method may not 

be effective when >90% of the noncoverage is due to wireless phone substitution and characteristics of the small LTI 

population (~4%) might not be the same as the WPO population. Khare et al. (2009) and Chowdhury et al., (2007) 

concluded from their analyses that although some of the characteristics of the LTI or NP household members are similar to 

those of the WPO household members (e.g., renters or under 200% poverty level), interruption method was not effective in 

reducing noncoverage bias for exclusion of WPO households. Also, interruption method introduced substantial variation in 

the final sample weights due to large adjustment factors (e.g., 4% LTI children representing 20% children in the WPO 

population may increase weights of LTI children by 5folds). 

We selected the sample of children under 18 years of age living in LT households from the 2007 NHIS-Child sample to 

approximate a RDD type sample and evaluated alternate adjustment methods using ratio- and propensity-based weighting 

B methods. We used the nonresponse adjusted interim NHIS weights WTIA_SC (=W ) for the children in the 2007 NHIS- i

Child LT sample to adjust for noncoverage. To assess bias in the resulting weighted estimates, mean-squared errors (MSE= 

Bias2 + SE2

) are computed with respect to the national estimates from the 2007 NHIS-Child sample and compared ratios of 

MSEs between methods. The weighting method with smaller MSEs is expected to perform better in reducing bias in 

estimates. 

Method M1 is similar to the interruption method used in the NIS to compensate for the noncoverage of nontelephone 

B

households (Frankel et al., 2003; Smith et al., 2005). In method M1, initial weights W for children from households with i

interruptions (LTI) are directly ratio-adjusted to the [(LTI) + (NP+WPO)] population control totals (Ni) within the 

demographic [(age (3), race/ethnicity (3)] weighting cells; no noncoverage adjustments were applied to the sampling weights 

of children from the LTWP-some, LTWP-mostly, and LT-only households. Thus, the new noncoverage adjusted sampling 

weight M1 for the unit i in the LTI group is defined as 

M 1,LTI B

(NLTI , ps  NNP, ps  NWPO , ps ) 

Wi i B W * .  Wi

i LTI  , ps

Method M2 used a different direct ratio-adjustment procedure. Based on the findings from Khare et al. (2009) showed that 

the characteristics of children living in LT-only households are similar to those who live in WPO households, method M2 

separately adjusts sampling weights for children from LT-only households to represent children from WPO households. 

However, because the proportions and sample sizes of the LTI and NP groups are very small and the children from the two 

groups had somewhat similar characteristics as the children from the WPO and the LT-only groups, method M2 

simultaneously adjusts weights for the children from the combined LTI and LT-only groups for those from the NLT (WPO 

+NP) group. Hence, weights of the children in the (LT-only + LTI) groups are adjusted to the (LT-only + LTI + WPO + NP) 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

estimated population control totals within weighting cells based on characteristics [house tenure (2), <200% poverty level (2), 

and race/ethnicity (3)] associated with WPO and NP phone status; no noncoverage adjustments were applied to the sampling 

weights of children from the LTWP-some and LTWP-mostly households. Thus, the new sampling weight M2 for the unit i

in the (LT-only + LTI) group is defined as 

M 2,( LT only  LTI ) B N( LT only  LTI ), ps  NWPO , ps  NNP, ps W  W * . i i B  Wi

i( LT only  LTI ), ps

An alternative approach to the direct ratio-adjustment method is to use logistic regression to model response propensities of 

nontelephone status for the LT sample and use the predicted propensities to create weighting classes and adjust sampling 

weights for noncoverage (Battaglia et al., 1995; Khare and Chowdhury, 2006) within those classes. Method M3 uses data 

from the 2007 NHIS-Child sample to develop a logistic regression model of response propensities for NLT households 

consisting of (WPO + NP) households and (LT-only + LTI + LT-mostly) households. The final NLT propensity score model 

used child’s age group(3), housing status(2), region (4), mother’s age group (3), mother’s marital status(4), and poverty 

level(4) as covariates. To account for the complex sample design we used normalized sampling weights WTIA_SC in the 

logistic regression model. Next, WTIA_SC weights for the children from the telephone sample are adjusted within weighting 

classes based on the quintiles of the predicted propensity scores and selected socio-demographic characteristics [age(3), 

house tenure(2), and <200% poverty level(2)]; no noncoverage adjustments were applied to the sampling weights of children 

from the LTWP-some households. 

Finally, a poststratification step adjusted the total weighted cell counts for children from the LT sample using methods M1--

M3 to the total target US population under 18 years of age within demographic cells [age(3), sex (2), and race/ethnicity(4)]. 

These final poststratified weights are then used to compute the new noncoverage adjusted estimates which are compared with 

the overall 2007 NHIS-Child sample estimates. 

To compare the three adjustment methods, weighted estimates of selected health related and socio-demographic 

characteristics are compared. We selected household respondent-reported information on child’s access to health insurance, 

receiving influenza vaccinations, ever having Asthma, and ever having chickenpox to assess bias by taking the difference 

between the overall 2007 NHIS-Child sample estimates and the estimates from the LT sample using the sampling weights 

from methods M1--M3. To account for the complex sample design of the NHIS, SUDAAN (RTI, 2008) was used to 

compute standard errors. Differences in estimates and ratios of estimated MSEs are used to compare weighting methods and 

evaluate bias. 

Results

Characteristics and the prevalence by the type of telephone status 

Table 1 shows the prevalence of having or not having access to landline and/or wireless telephones defined by six telephone 

groups (LTI, LTWP-some, LTWP-mostly, LT-only, WPO, and NP) and by selected child’s characteristics. In 2007, among 

children under 18 years of age, 84.8% lived in LT household, 13.3% lived in WPO household, and 1.9% had no access to 

phone; 4.1% children lived in households with interruptions of >1 week in telephone service in the past 12 months. The WPO 

group represented 92% of the noncoverage in a landline telephone survey of children. Children from the WPO household are 

more likely to: be younger (0-4 years); live in rented house; be of Hispanic or non-Hispanic black race/ethnicity; live in the 

south; have a less educated (<high school) young mother (<29 years) who is a single parent (household size=2) and is either 

never married or widowed/separated/divorced; have no health insurance (i.e., uninsured); and have household income below 

200% of the poverty level. Children from LT-only, LTI, and NP households, in general, have similar characteristics as 

children from WPO households with few exceptions. Children from LTWP-mostly households (~14%) appear to have more 

overlapping characteristics with LTWP-some households (~52%) than with NLT or LT-only households. It is difficult to 

estimate the proportion of LTWP-mostly households that do not respond to ringing landline telephone resulting in a 

noncontact. The LTWP-mostly household respondents are more likely to be home owners, with high education, and have 

high income. We used these characteristics with high prevalence to develop a propensity score model for noncoverage using 

a stepwise logistic regression model. The final propensity score model, significant correlates of noncoverage, and parameter 

estimates are presented in Table 3. 

Figure 2 shows variation in the prevalence of selected vaccination related health characteristics by the three age groups and 

six telephone groups. The prevalence of children with influenza vaccination (i.e., flu shots), asthma, chickenpox, and 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

without health insurance (i.e., uninsured) shows different association with LTI, NP, LT-only, and WPO telephone status in 

comparison to LTWP-some and LTWP-mostly telephone status by the three age groups. 

Propensity score model for NLT households

Table 3 shows the results of the logistic regression model predicting propensity of being a NLT household among LTI, LTonly and LTWP-mostly households. A subset of the variables listed in Tables 1 with large percentages or differences 

between the NLT and other telephone groups is selected to develop the propensity models; most of these are also available in 

the NIS. Method M3 used the quintiles of the predicted propensity score to create homogeneous weighting cells and adjusted 

for noncoverage among the LTWP-mostly, LT-only, and LTI households. It is expected that including the LTWP-mostly 

group in the model and using house tenure status, mother’s education and poverty level may simultaneously adjust for the 

residual noncontact with the LTWP-mostly households. The first column of Table 3 lists the set of covariates selected from 

the stepwise logistic regression models (in the order variables entered the model) to predict propensity scores for the NLT 

status; beta coefficients and estimates of Odds Ratios for the covariates are shown in columns 2 and 3 of the table. Table 3 

also shows some model diagnostics. The concordance (69%) and estimated value of ‘c’, the area under the ROC curve 

(69%), indicate a moderate association between predicted propensities and observed responses (i.e., indicators of NLT 

status). 

Comparison of adjustment factors and weighted estimates 

Table 4 shows the distribution of the overall adjustment factors that are applied to the WTIA_SC weights using methods M1-

-M3. It shows that methods M2 and M3 performed slightly better than the method M1 (Keeter’s method). The maximum 

adjustment factor using method M1 was ~3 times larger than the maximum factors using methods M2 or M3; The CV from 

method M1 is also ~2 times larger than other two methods. The ratio of methods M3 to M2 shows that overall both methods 

performed equally well, however, method M3 performed slightly better than M2 with smaller mean, standard deviation, CV, 

and inter-quartile range (i.e., with ratios M3/M2<1). 

Comparison of weighted estimates and MSEs 

Table 5 and 6 present a comparison of the prevalence of influenza vaccinations (i.e., flu shots) among children, 95% 

confidence interval for the NHIS-Child sample estimates and MSEs (=Bias**2 + se**2) using the 2007 NHIS-Child sample 

estimates as true population estimates. Table 5 shows that after adjusting sampling weights for noncoverage of NLT 

households, the bivariate prevalence of influenza vaccinations are not statistically different from the NHIS-Child estimate. 

The prevalence estimates of influenza vaccinations by the three age groups are also not statistically different from the NHISChild estimates. The estimates of MSEs are smaller for methods M2 and M3 than method M1. Although overall weighted 

estimates using methods M2 and M3 are very close, the ratios of MSE (M3/M2) in Table 5 are slightly smaller for method 

M2, however, this varies by the three age groups and the characteristics of interest. Table 6 shows a comparison of the ratios 

of MSEs by three age groups. Method M2 appears to perform better among children of ages 0-4 years (the overall ratio 

M3/M2>1.0) while method M3 appears to perform better among teens 13-17 years and children aged 5-12 years (the overall 

ratio M3/M2<1.0) in reducing noncoverage bias. 

Conclusion

Our analysis shows that children living in LT-only, LTI, WPO, or NP households have different socio-demographic 

characteristics than children living in LTWP-some and LTWP-mostly households. The prevalence of children from WPO 

households decreased with increasing age among children under 18 years of age and also with increasing age of their mother. 

Other factors that are highly correlated with WPO, LT-only, and NLT status are house tenure, household size, race/ethnicity, 

poverty level, mother’s age, education, and marital status, and geographic region. After adjusting for differential 

noncoverage, the difference in estimates of household-reported influenza vaccinations with respect to the overall 2007 NHISChild sample estimates were 0.08%, -0.01%, and -0.06% (the corresponding maximum differences by subcategories were 

2.64%, 1.16%, 1.15%) using weighting methods M1, M2, and M3, respectively. Among the three age groups of children, 

these differences in influenza vaccinations were 1.68%, 0.99%, and 0.69% among 0-4 year olds, -0.50%, -0.81%, and -0.74% 

among 5-12 year olds, and -0.58%, 0.23%, and 0.24% among teens 13-17 year olds using the three weighting methods, 

respectively; none of adjusted estimates are statistically different from the overall NHIS-Child sample estimates. 

With increasing trends in the prevalence of wireless only households, using separate adjustments for wireless-only and 

phoneless household may be desirable and may control potentially larger bias in population estimates that are correlated to 

characteristics of wireless-only households. Adjustments based on interruption in landline telephone service reduced the 

noncoverage bias, especially for those characteristics that are highly correlated with the absence of landline telephone or 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

presence of wireless services but increased variances due to small sample sizes and large NLT adjustment factors. Methods 

M2 and M3 appear to perform somewhat better than the Keeter’s interruption method M1. The ratios of MSE (M3/M2) are 

>1.0 (Table 6) suggesting slightly greater reduction in bias with method M2 (a hybrid of Keeter’s ratio adjustment method). 

Because method M3 can use more covariates and categories associated with telephone status of households, it may be 

desirable to use the propensity score method when feasible, to reduce noncoverage bias in RDD estimates. To use methods 

M2 or M3, RDD surveys need to collect information on household tenure (the most significant characteristic of NP, WPO, 

LT-only, LTWP-mostly households) and access to wireless telephones by household members. 

Annual evaluation of potential noncoverage bias in the NIS is needed as the prevalence of wireless-only households 

continues to increase. A sensitivity analysis indicates that with a prevalence of 25% NLT households, and a 10% difference 

between landline and NLT group estimates, the estimated landline noncoverage bias may be as much as 2.5% [= NLT 

prevalence* (difference in LT and NLT estimates)]. The NHIS provides a unique source of data for this evaluation and could 

be used in a similar way for other RDD surveys. Findings from the analysis of the NHIS data by telephone status are 

reassuring, however, direct assessment of bias in RDD surveys like NIS are needed. In 2009, a provider-record-check study 

has been added to the NHIS for children under 5 years of age and teens of age 13-17 years to directly evaluate noncoverage 

and nonresponse bias in vaccination estimates from the NIS. 

To continue further evaluation of reduction in noncoverage bias with increasing substitution of landline telephones with 

wireless phones, and to adjust for the noncoverage of wireless-only households in telephone surveys, the 2007 NIS added a 

question on access to wireless telephones during the interruption in landline service. Also, a pilot study was conducted 

whereby sample of wireless phone numbers are hand-dialed to interview households with access to wireless phones. In 2008, 

in addition to socio-demographic and geographic information, NIS has added questions to collect information on household 

tenure status and access to wireless telephones to identify LT-only households and for direct assessment of potential bias due 

to noncoverage of wireless-only households; this data collection continued in 2009 NIS. Furthermore, for direct assessment, 

NIS is also conducting several experiments using dual frame and address-based sampling to cover telephone and nontelephone households in 2009. NIS is also conducting a small wireless telephone survey to compare vaccination coverage 

estimates from a landline and a wireless telephone survey. NIS is also re-evaluating the household interview weighting 

procedure to reduce noncoverage bias. Findings from some of these experiments will be available in 2010. 

Finally, our analyses had a few limitations. First, we assumed that nonresponse and noncoverage patterns were similar for a 

traditional RDD sample and the NHIS LT sample. Second, the NHIS conducts in-person face-to-face interviews while RDD 

surveys collect data through computer-assisted telephone interviews which may cause plausible mode effect in responses. 

Also, NHIS collects limited information related to vaccinations and therefore it is difficult to do a direct noncoverage bias 

assessment for the NIS estimates. Lastly, it is difficult to benchmark the NIS estimates to the NHIS estimates due to 

differences in weighting and estimation procedures. 

References 

Blumberg SJ, Luke JV. Wireless substitution: Early release of estimates from the National Health Interview Survey, JulyDecember 2008. National Center for Health Statistics. May 2009a. Available from: http://www.cdc.gov/nchs/nhis.htm. 

Blumberg SJ, Luke JV, Davidson G, Davern ME, Yu T, Soderberg K. Wireless substitution: State-level estimates from the 

National Health Interview Survey, January-December 2007. National health statistics report; no 14. Hyattsville, MD: 

National Center for Health Statistics. 2009b. 

Battaglia, Michael P., Malec, Donald J., Spencer, Bruce D., Hoaglin, David C., and Sedransk, J. 1995. Adjusting for 

noncoverage of nontelephone households in the National Immunization Survey. 1995 Proceedings of the Section on Survey 

Research Methods, Alexandria, VA: American Statistical Association, pp. 678-683. 

Brick J, Waksberg J, Keeter S. Using data on interruptions in telephone service as coverage adjustments. Survey 

Methodology 1996; 22:185–197. 

Chowdhury, S.R., Montgomery, R., and Smith, P.J. (2008). Adjustment for Noncoverage of Non-landline Telephone 

Households in an RDD Surveys. 2008 Proceedings of the Section on Survey Research Methods [CD-ROM], Alexandria, 

VA: American Statistical Association: pp 1438-1443. 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

Curtin, Richard, Presser, Stanley, and Singer, Eleanor. 2005. Changes in Telephone Survey Nonresponse Over the Past 

Quarter Century. Public Opinion Quarterly. Vol. 69: 87-98. 

Frankel, Martin R., Srinath, K.P., Hoaglin, David C., Battaglia, Michael P., Smith, Philip J., Wright, Robert A., and Khare, 

Meena. 2003. Adjustments for non-telephone bias in random-digit-dialling surveys. Statistics in Medicine, Volume 22, pp. 

1611-1626. 

Keeter S. Estimating non-coverage bias from a phone survey. Public Opinion Quarterly 1995; 59:196–217. 

Khare, M, Chowdhury, S, Wolter, K, and Singleton, JA. “An Assessment of Bias due to Noncoverage of Wireless Only 

Households in RDD Surveys”, 2007 Proceedings of the American Statistical Association, Survey Research Methods Section, 

Alexandria, VA: American Statistical Association (CD-ROM), 2008, pp 3911- 3918 

Khare, M., Wouhib, A., and Singleton, J.A. (2009). Assessment of Potential Bias in the National Immunization Survey Using 

Data on Telephone Status from the 2007 NHIS. Presented at the 2009 Conference of the American Association of Public 

Opinion Research, Hollywood, FL, May 2009. 

Research Triangle Institute (RTI). SUDAAN Language Manual, Release 10.0. Research Triangle Park, NC: Research 

Triangle Institute, 2008. 

Smith, P.J., Hoaglin, D.C., Battaglia, M.P., Khare, M., and Barker, L.E. (2005), Statistical Methodology of the National

Immunization Survey: 1994-2002. National Center for Health Statistics. Vital Health Stat 2(138). 

Srinath, K.P., Battaglia, Michael P., Frankel, Martin R., Khare, Meena, and Zha, Wenxing. 2002. Evaluation of a procedure 

based on interruptions in telephone service for reducing coverage bias in RDD surveys. 2002 Proceedings of the Annual 

Meeting of the American Statistical Association [CD-ROM], Alexandria, VA: American Statistical Association, pp. 3323-

3329. 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Table 1: Prevalence of telephone coverage among US children (age <18 years) by selected characteristics, 

2007 NHIS-Child sample (numbers in blue show categories with high proportions) 

Characteristics 

NHIS-Child 

sample 

Distribution

Prevalence by Telephone Status (Children of Age <18 years)

LTI 

(>1wk)

LTWPsome LTWPMostly LT-only WPO NP 

n % % % % % % % 

All 9417 100.0 4.1 51.8 13.8 15.1 13.3 1.9 

Age: 0 - 4 Years 2813 27.9 4.2 43.8 12.7 15.5 20.6 3.3 

5 - 12 Years 3778 43.6 3.7 54.0 13.9 15.3 11.7 1.4 

13 - 17 Years 2826 28.5 4.6 56.3 14.6 14.5 8.7 1.3 

House tenure: Owned 5785 65.9 2.3 62.1 15.2 12.0 7.5 0.8 

 Rented 3632 34.1 7.5 32.0 10.9 21.1 24.6 3.9 

Sex: Male 4865 51.1 4.6 51.1 13.4 15.6 13.7 1.5 

 Female 4552 48.9 3.5 52.6 14.1 14.6 12.9 2.3 

Race/Ethnicity: Hispanic 2870 20.8 4.5 40.7 11.9 21.1 18.2 3.6 

NH- White 4282 57.6 3.3 59.6 14.5 11.5 10.1 0.9 

NH- Black 1656 15.8 6.6 37.3 13.0 20.9 19.5 2.6 

NH- Others 609 5.8 4.0 53.6 15.3 13.4 10.2 3.6 

Region: Northeast 1616 16.8 2.8 59.9 11.6 16.9 7.7 1.1 

Midwest 1862 23.7 5.5 53.2 12.4 14.4 12.7 1.8 

 South 3548 36.9 4.3 47.5 15.7 13.1 17.1 2.3 

West 2391 22.7 3.2 51.5 13.7 17.8 12.0 1.9 

MSA: MSA 8037 83.8 4.1 52.5 14.2 14.5 13.1 1.7 

Not MSA 1380 16.2 4.1 48.5 11.6 18.6 14.3 3.0 

Mother_educ: < Highschool 1740 15.6 6.7 34.5 7.9 25.6 20.5 4.9 

Highschool 2333 24.4 5.1 45.1 12.6 18.6 16.5 2.1 

> Highschool 4931 56.2 3.0 60.3 16.1 9.8 10.1 0.9 

Mother_Age: <25 Years 760 5.9 7.9 22.4 11.6 17.4 35.1 5.6 

25 - 29 Years 1239 13.0 4.3 34.2 14.6 17.5 26.8 2.7 

30+ Years 7418 81.1 3.8 56.8 13.8 14.6 9.6 1.5 

Mother Mar_Stat: Married 6420 70.8 3.2 56.9 14.2 13.3 11.0 1.4 

 Wid, Div, or Sep 1046 10.8 6.1 40.7 12.1 17.0 21.2 2.8 

 Never married 967 8.5 8.4 29.8 10.8 22.7 24.2 4.1 

HH _size: 2 724 4.0 5.1 35.3 15.8 17.8 23.9 2.2 

 3 TO 4 5902 53.2 3.8 51.6 14.2 14.4 14.0 1.9 

 GE 5 2791 42.8 4.3 53.6 13.0 15.8 11.5 1.8 

Poverty_Level: <100% 1747 18.3 8.3 29.3 7.8 25.0 24.0 5.6 

100-199% 2288 24.3 5.3 41.1 12.0 19.8 19.6 2.1 

200-399% 2817 31.0 2.6 59.9 14.6 12.4 9.8 0.8 

400%+ 2565 26.4 1.8 67.9 18.6 7.1 4.3 0.4 

Uninsured: Yes 935 8.7 4.3 36.0 15.5 19.4 19.8 5.0 

NO 8482 91.3 4.1 53.3 13.6 14.7 12.7 1.6 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Figure 2: Prevalence of selected health characteristics among children by age group and six type of telephone status,

2007 NHIS-Child sample 

0

10

20

30

40

50

60

70

80

All <18 Yr 0-4 Yr 5-12 Yr 13-17 Yr All <18 Yr 0-4 Yr 5-12 Yr 13-17 Yr All <18 Yr 0-4 Yr 5-12 Yr 13-17 Yr All <18 Yr 0-4 Yr 5-12 Yr 13-17 Yr

Flu Shot Asthma Chickenpox Uninsured

NHIS-Child LTI (>1 w k) LTWP-some LTWP-mostly LT-only WPO NP

Table 3: Significant covariates, Beta coefficients, and estimates of Odds Ratios from stepwise logistic regression 

models for predicting propensity scores of combined wireless-only and phoneless status among children (age<18

years) from LTI, LT-only, LT-mostly households, 2007 NHIS-Child sample

Covariates used in the final propensity model 

(NLT) 

Beta 

coefficient 

Odds Ratio 

(OR)

95% OR

LCL 

95% OR

UCL 

Intercept 0.7631 

House tenure: Owner vs Rented 0.3036 1.835 1.569 2.146 

Mother_age: 30 + Years vs < 25 Years 0.3784 1.840 1.440 2.351 

Mother_age: 25 - 29 Years vs < 25 Years -0.1470 1.088 0.852 1.389 

Mother_Mar_Stat: Married vs Never 0.0875 0.864 0.701 1.065 

Mother_Mar_Stat: Wid, Div, and Sep vs Never -0.3211 0.574 0.447 0.737 

Region: Northeast vs West 0.3856 1.330 1.034 1.709 

Region: Midwest vs West -0.1190 0.803 0.654 0.986 

Regon: South vs West -0.3674 0.626 0.523 0.750 

Poverty_level: <100% vs 400%+ -0.3374 0.404 0.310 0.527 

Poverty_level: 100-199% vs 400%+ -0.2728 0.431 0.337 0.553 

Poverty_level: 200-399% vs 400%+ 0.0420 0.591 0.459 0.761 

Age: 0 - 4 Years vs 13 - 17 Years -0.3069 0.597 0.480 0.741 

Age: 5 - 12 Years vs 13 - 17 Years 0.0971 0.894 0.737 1.083 

# Quintiles of propensities and age groups are used to create weighting cells and adjust WTIA_SC weights for children in 

LTI, LT-only, and LT-mostly groups for non-landline status in the method M3 

Association of Predicted Probabilities and Observed Responses 

Percent Concordant 68.8 Somers' D 0.382 

Percent Discordant 30.5 Gamma 0.385 

Percent Tied 0.7 Tau-a 0.170 

Pairs 3994551 c 0.691 
 

 

 

 

 

Table 4: Distribution of adjustment factors to compensate for noncoverage of Children from WPO and NP

households using M1--M3 adjustment methods, 2007 NHIS-Child LT sample 

Statistics Adjustment Method Ratio of adjustment factors 

M1 M2 M3 M2/M1 M3/M1 M3/M2 

Minimum

Maximum

Median 

Mean 

Standard Deviation 

Coefficient of Variation (CV)

Inter Quartile Range 

1.1339 

11.942 

1.2935 

1.5440 

1.2262 

79.4072 

0.1195 

1.1032 

3.8183 

1.3328 

1.5462 

0.5123 

33.1343 

0.5423 

1.187 

4.1538 

1.401 

1.5447 

0.4632 

29.9868 

0.4668 

0.9729 

0.3197 

1.0304 

1.0014 

0.4178 

0.4173 

4.5381 

1.0468 

0.3478 

1.0831 

1.0005 

0.3778 

0.3776 

3.9063 

1.0760 

1.0879 

1.0512 

0.9990 

0.9042 

0.9050 

0.8608 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Table 5: Comparison of the prevalence for Influenza vaccinations among all children aged <18 years using weighting methods M1--M3, 2007 NHIS-Child sSample 

(numbers in blue show categories with Ratio of MSEs<1) 

Characteristics 

NHIS-Child sample: All < 18 Years Weighting Methods M1--M3

Influenza 

Vaccination 

Estimate 

95% Confidence Limits LT Sample estimates MSE* Ratio of MSE** 

LCL UCL M1 M2 M3 M1 M2 M3 M2/M1 M3/M1 

All children <18 years 22.95 21.81 24.08 22.92 22.96 23.01 0.519 0.417 0.433 0.795 0.815 

0-4 years 33.13 30.96 35.38 31.55 32.14 32.43 5.021 2.699 2.282 0.537 0.454 

5-12 years 21.69 20.09 23.38 22.20 22.50 22.42 1.275 1.538 1.420 1.206 1.113 

13-17 years 14.90 13.26 16.70 15.56 14.66 14.69 1.633 0.965 0.908 0.591 0.556 

Asthma: Yes 33.79 30.49 37.09 32.82 34.14 34.3 5.576 3.807 3.829 0.683 0.687 

No 21.33 20.15 22.5 21.38 21.33 21.36 0.519 0.429 0.438 0.827 0.845 

Chickenpox: Yes 16.58 14.70 18.46 16.22 16.31 16.39 1.293 1.132 1.096 0.875 0.847 

No 25.41 24.05 26.78 25.52 25.46 25.54 0.758 0.606 0.615 0.800 0.811 

Uninsured: Yes 12.39 9.65 15.14 13.3 12.46 12.78 4.298 2.169 2.578 0.505 0.600 

No 23.95 22.75 25.16 23.76 23.9 23.94 0.600 0.473 0.473 0.789 0.789 

House tenure: Rented 23.98 22.15 25.8 23.41 23.15 23.46 2.234 1.871 1.52 0.837 0.680 

Race/ethnicity: Hispanic 22.39 20.45 24.33 22.18 22.5 22.25 1.964 1.383 1.369 0.704 0.697 

NH-White 21.65 20.05 23.25 21.65 21.79 22.09 0.912 0.794 0.998 0.870 1.094 

NH- Black 26.07 23.35 28.79 26.29 25.45 25.63 3.794 3.063 2.744 0.807 0.723 

NH- Other 29.39 24.58 34.21 28.95 29.42 27.87 8.748 7.879 9.438 0.901 1.079 

Poverty_level: < 100% 24.57 21.98 27.16 23.83 24.04 24.41 4.354 2.594 2.381 0.596 0.547 

100 - 199% 21.65 19.39 23.9 21.89 22.05 22.29 2.136 1.996 2.335 0.934 1.093 

200 - 399% 20.91 19.03 22.79 20.77 21.21 21.12 1.235 1.246 1.222 1.009 0.989 

400% + 25.42 23.33 27.52 25.65 25.11 24.94 1.444 1.298 1.429 0.899 0.989 

Region: Northeast 27.47 24.71 30.22 27.74 28.01 28.08 2.882 2.747 2.783 0.953 0.966 

Midwest 22.43 19.74 25.13 22.11 22.57 22.23 2.691 2.252 2.359 0.837 0.877 

South 23.2 21.35 25.05 23.31 23.25 23.53 1.594 1.129 1.277 0.708 0.801 

West 19.74 17.84 21.64 19.6 19.13 19.37 1.378 1.526 1.268 1.108 0.921 

Mother_educ: 

<HighSchool 21.99 19.48 24.5 23.33 22.55 22.69 5.976 2.953 2.891 0.494 0.484 

HighSchool 21.83 19.63 24.04 20.94 21.13 21.2 2.9 2.022 1.973 0.697 0.68 

> HighSchool 23.93 22.34 25.53 23.9 24.17 23.9 0.866 0.831 0.743 0.96 0.858 

Mother_Mar_stat:

Married 23.38 22.05 24.71 23.6 23.5 23.37 0.657 0.573 0.572 0.872 0.87 

Wid Div Sep 20.28 16.99 23.57 22.46 21.12 20.83 13.043 5.122 4.486 0.393 0.344 

Never Married 27.2 23.6 30.81 24.88 26.68 27.71 12.682 5.179 5.493 0.408 0.433 

Mother _age: <25 Years 31.2 27.11 35.29 32.95 32.99 32.48 18.254 12.414 10.068 0.68 0.552 

25 - 29 Years 27.33 24.17 30.5 24.99 26.17 24.24 9.564 4.72 12.251 0.494 1.281 

30+ Years 21.65 20.41 22.89 21.88 21.86 21.87 0.619 0.518 0.528 0.837 0.853 

*Smaller MSE means higher reduction in bias, **MSE Ratio <1 means that M2 or M3 method performed better than the method M1 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Table 6: Comparison of MSE ratios for influenza vaccinations for selected characteristics and weighting methods by

age groups, 2007 NHIS-Child Telephone Sample

2007 NHIS:

Prevalence of Influenza 

vaccinations 

All Children <18 years Age: 0-4 years Age: 5-12 years Age: 13-17 years 

M3/M1 M3/M2* M3/M1 M3/M2* M3/M1 M3/M2* M3/M1 M3/M2* 

Total 0.809 1.025 0.425 0.837 1.130 0.932 0.602 0.947 

Asthma: Yes 0.659 0.998 0.283 0.892 1.017 1.129 0.767 0.959 

No 0.847 1.024 0.566 0.850 0.912 0.846 0.595 0.945 

Chickenpox: Yes 0.834 0.972 1.204 1.066 0.749 1.026 0.922 0.937 

No 0.825 1.015 0.450 0.835 1.017 0.881 0.274 0.928 

Uninsured: Yes 0.588 1.268 0.912 1.302 0.998 0.991 0.287 0.836 

No 0.759 1.001 0.383 0.826 1.128 0.956 0.803 0.955 

House tenure: Rented 0.644 0.851 0.580 0.781 0.898 0.887 0.583 0.876 

Race/ethnicity: Hispanic 0.695 0.995 0.647 1.192 1.866 1.348 0.687 1.019 

NH- White 1.108 1.264 0.756 1.215 1.008 0.998 0.774 0.968 

NH- Black 0.762 0.937 0.538 0.821 0.689 0.952 0.480 0.941 

NH- Other 1.028 1.187 1.105 1.353 0.770 0.795 0.987 0.859 

Poverty_level: < 100% 0.531 0.906 0.413 0.947 0.796 1.091 0.603 0.921 

100 - 199% 1.145 1.180 0.919 1.316 0.900 0.805 0.925 0.958 

200 - 399% 0.994 0.982 0.708 1.084 1.253 0.971 0.629 0.990 

400% + 0.992 1.088 0.893 0.872 0.902 1.004 0.634 0.988 

Region: Northeast 0.919 0.964 0.592 0.980 1.238 0.788 0.765 0.907 

Midwest 0.859 1.054 0.524 1.011 0.892 0.795 0.844 1.024 

South 0.854 1.199 0.490 1.146 1.091 1.463 0.372 0.918 

West 0.936 0.860 1.102 0.866 0.761 0.931 0.957 0.944 

Mother_Educ: < HighSchool 0.514 1.025 0.650 0.986 0.622 1.342 0.502 0.628 

HighSchool 0.614 0.867 0.638 0.772 0.616 1.037 0.727 0.914 

> HighSchool 0.912 0.950 0.402 1.037 0.774 0.824 0.811 0.909 

Mother _Mar_stat: Married 0.883 1.003 0.780 0.948 0.739 0.770 0.709 0.963 

Wid Div Sep 0.353 0.847 0.450 0.960 0.396 0.797 0.616 0.925 

Never Married 0.382 1.060 0.443 1.101 0.454 1.124 0.474 0.830 

Mother_Age: Less than 25 Yrs 1.122 1.490 1.083 1.429 0.886 1.235 na na 

25 - 29 Yrs 0.468 0.997 0.991 0.745 1.102 0.903 0.974 1.033 

30+ Yrs 0.749 0.892 0.399 1.017 0.641 0.888 0.567 0.943 

*MSE Ratio <1 means method M3 performed better then M2