National Immunization Survey-Child 

Error Profile for the 2022 NIS-Child

Centers for Disease Control and Prevention

National Center for Immunization 

and Respiratory Diseases

Presented by:

NORC at the University of Chicago

November 2023
Acknowledgments

The development and production of the Error Profile for the 2022 NIS-Child is a team effort 

that has included contributions from many individuals (listed in alphabetical order) in two 

organizations: 

National Center for Immunization and Respiratory Diseases, CDC – Laurie D. Elam-Evans, 

Holly A. Hill, Sean Hu, and James A. Singleton. 

NORC at the University of Chicago – Zachary H. Seeskin, Benjamin Skalland, Kirk Wolter, 

and Yuhei Koshino
Error Profile for the 2022 NIS-Child i

Table of Contents

1. Introduction........................................................................................................1 

2. Part I: Comparisons of NIS-Child Data to External Sources..............................2 

2.1 Demographic Distributions: Comparison of NIS-Child Distributions to Population Distributions...2 

2.2 Comparison of NIS-Child and IISAR Vaccination Coverage.........................................................5 

What is IISAR?..............................................................................................................................5 

Visual Comparison of the Vaccination Coverage Estimates .........................................................6 

Child Participation Rate (CPR)......................................................................................................7 

Negative Relationship Between Difference in Vaccination Coverage and CPR ...........................8 

2.3 Comparison of Children by Type of Health Insurance Coverage..................................................9 

2.4 Comparison of Vaccination Coverage Estimates: NIS-Child v. State Immunization Surveys.....11

3. Part II: Assessment of Total Survey Error for NIS-Child Vaccination 

Coverage Estimates ............................................................................................22

3.1 Sampling-Frame Coverage Error................................................................................................24

3.2 Nonresponse Error......................................................................................................................28

Components of Nonresponse in NIS-Child .................................................................................28

Weight Adjustment for Nonresponse Error .................................................................................29

Assessment of Nonresponse Error .............................................................................................29

NIS-Child Item Nonresponse Rates ............................................................................................32

3.3 Measurement Error.....................................................................................................................32

3.4 Sampling Error............................................................................................................................34

3.5 Total Survey Error Distribution....................................................................................................35

3.6 Assessment of the Change in Bias Using the Bridging Cohort Method......................................38

4. Summary .........................................................................................................39

References ..........................................................................................................43
Error Profile for the 2022 NIS-Child ii

List of Figures 

Figure 2.1: Scatterplot of NIS-Child (in %) v. IISAR (in %) Vaccination Coverage Estimates 

for the Combined 7-Vaccine Series: 56 Estimation Areas, 2021 ......................................6

Figure 2.2: Scatterplot of NIS-Child (in %) v. IISAR (in %) Vaccination Coverage Estimates 

for the Combined 7-Vaccine Series: Six 2013-2017 Sentinel Sites, 2021 ........................7

Figure 2.3: Scatterplot of IISAR (in %) Vaccination Coverage Estimate for the 

Combined 7-Vaccine Series v. CPR (in %): 56 Estimation Areas, 2021...........................8

Figure 2.4: Scatterplot of the Difference (NIS-Child Minus IISAR) for the Combined 7-Vaccine 

Series (in %) v. CPR (in %) with Regression Line: 56 Estimation Areas, 2021 ................9

Figure 2.5.1: Comparison of Vaccination Coverage Estimates for Children 19-35 Months: 

Kansas Retrospective Vaccination Coverage Survey, 2014-2015 v. NIS-Child, 2015....14

Figure 2.5.2: Comparison of Vaccination Coverage Estimates for Children at 24 Months: 

Tennessee Immunization Status Survey, 2022 v. NIS-Child, 2022................................. 15

Figure 2.5.3: Comparison of Vaccination Coverage Estimates for Children at 24 Months: 

Virginia Annual Immunization Survey, 2021 v. NIS-Child, 2021 .....................................16

Figure 2.5.4: Comparison of Vaccination Coverage Estimates for Children at 24 Months: 

Florida Immunization Survey, 2020 v. NIS-Child, 2020...................................................17

Figure 2.5.5: Comparison of Vaccination Coverage Estimates for Children 19-35 Months: 

Georgia Immunization Study, 2022 v. NIS-Child, 2022...................................................18

Figure 2.5.6: Comparison of Vaccination Coverage Estimates for Children 19-35 Months: 

Michigan Care Improvement Registry, 2022 v. NIS-Child, 2022.....................................19

Figure 2.5.7: Comparison of Vaccination Coverage Estimates for Children 19-35 Months: 

Washington Immunization Information System, 2022 v. NIS-Child, 2022.......................20

Figure 2.5.8: Comparison of Vaccination Coverage Estimates for Children at 24 Months: 

Wisconsin Immunization Registry, 2022 v. NIS-Child, 2022 ...........................................21
Error Profile for the 2022 NIS-Child iii

List of Tables 

Table 2.1: One-Way Demographic Distributions Among Children with Adequate Provider 

Data vs. Population Distributions: NIS-Child, 2022 ...........................................................4

Table 2.2: Comparison of Alternative Estimates of Health Insurance Coverage Among 

Child: NIS-Child, ACS, CPS ASEC, and NHIS for 2021 and NIS-Child, CPS 

ASEC, and NHIS for 2022...............................................................................................10

Table 2.3: Sample Sizes of Children from State Immunization Surveys in 2015, 2020, 2021, 

and 2022 .........................................................................................................................13

Table 3.1: Percentage of Age-Eligible Children in the Population by Telephone Status by Year: 

NIS-Child, United States, 2012-2021 ..............................................................................25

Table 3.2: Vaccination Coverage Estimates and Standard Errors of Select Vaccines and Vaccine 

Series for the Cell-Phone and LLO Domains: NIS-Child, United States, 2017 ...............26

Table 3.3: Vaccination Coverage Estimates and Standard Errors of Select Vaccines and 

Vaccine Series for Children 19-35 Months in the Cell-Phone and Phoneless 

Domains: NHIS-PRC, United States, 2012 .....................................................................27

Table 3.4: Preliminary Assessment of Sampling-Frame Coverage Error and Mean True 

Vaccination Coverage (in %): NIS-Child, United States, 2022........................................28

Table 3.5: Estimates of Nonresponse Error for Vaccination Coverage (%) Derived from 

Estimates Based on NHIS Imputations and NIS-Child Estimates ...................................31

Table 3.6: Item Nonresponse Rates, NIS-Child Household Interview, United States, 2022 ............32

Table 3.7: Estimated Under-Reporting Error by Vaccine Series: NIS-Child, United States, 

2017 and 2019 ................................................................................................................34

Table 3.8: Vaccination Coverage Estimates and Standard Errors Using Design Weights and 

Final Weights: NIS-Child, United States, 2022................................................................35

Table 3.9: Total Survey Error Model Inputs by Stages: NIS-Child, United States, 2022..................36

Table 3.10: Mean and 95% Credible Interval for the Estimated TSE Distribution and Component 

Error Distributions: National Immunization Survey-Child, United States, 2022...............36

Table 3.11: Difference between the Estimates* for the Bridging Birth Cohort† by Age 19 Months: 

NIS-Child, United States, 2021 vs. 2022.........................................................................39
Error Profile for the 2022 NIS-Child 1

1. Introduction 

Total survey error (TSE) is the difference between a survey estimate and the true value of the 

corresponding population parameter. TSE is the net effect of sampling error and all forms of 

nonsampling error, including sample-frame coverage error, error due to survey nonresponse, and 

errors of measurement (such as reporting, record checking, coding, and other processing errors). TSE 

excludes conceptual errors committed in deciding what should be measured in the survey and 

judgmental errors made in interpreting the survey findings or in making public policy based on the 

survey data.

The main aim of this report is to provide a well-rounded but brief discussion of what is known about 

TSE for the 2022 NIS-Child. A recent report describes TSE for the 2022 NIS-Teen (CDC, 2023b). 

The statistics and methodology of the NIS have been described by Smith et al. (2005) and Wolter et al. 

(2017).

The report is written in two parts. The first part, which appears in Section 2, compares NIS-Child 

statistics to corresponding benchmarks derived from censuses or large reference surveys, such as the 

National Health Interview Survey and the American Community Survey. A large difference between an 

NIS-Child statistic and its corresponding benchmark is likely a signal of error in the NIS-Child or of 

definitional differences between the NIS-Child and benchmark concepts. A small difference may be a 

signal of good accuracy in the NIS-Child or simply an indicator that the NIS-Child statistic and its 

benchmark are consistent with one another. This part of the report examines demographic statistics, 

vaccination coverage, and health insurance statistics.

The second part of the report, set forth in Section 3, focuses attention on the NIS-Child estimated 

vaccination coverage. The material presents what is known from special evaluation studies about the 

component errors and the total error in vaccination coverage estimates. The section culminates with 

discussion of the distribution of TSE in the 2022 NIS-Child and of the change in TSE between the 2021

and 2022 NIS-Child. 

Both parts of the report are conducted at the national level. The report closes in Section 4 with a 

summary of findings and limitations.

Throughout the report, we analyze the following vaccines and vaccine series:

• diphtheria, tetanus, and acellular pertussis vaccine (DTaP), ≥4 doses (4+ DTaP);

• poliovirus vaccine, ≥3 doses (3+ Polio);

• measles, mumps, and rubella vaccine (MMR), ≥1 dose (1+ MMR);
Error Profile for the 2022 NIS-Child 2

• the full series of Haemophilus influenzae type b conjugate vaccine (Hib), ≥3 doses or ≥4 doses 

depending on brand (Hib-FS);1

• hepatitis B vaccine (HepB), ≥3 doses (3+ HepB);

• hepatitis B dose within 3 days of birth (HepB birth dose);

• varicella vaccine, ≥1 dose (1+ Var);

• pneumococcal conjugate vaccine (PCV), ≥4 doses (4+ PCV); and

• the combined 7-vaccine series (≥4 doses of DTaP, ≥3 doses of poliovirus vaccine, ≥1 dose of 

measles-containing vaccine, the full series of Hib, ≥3 doses of HepB, ≥1 dose of varicella 

vaccine, and ≥4 doses of PCV). 

For the analysis of TSE, we focus on 4+ DTaP, 1+ MMR, HepB birth dose, and the combined 7-vaccine 

series.

2. Part I: Comparisons of NIS-Child Data to 

External Sources 

We begin by comparing NIS-Child demographic distributions (child’s age, child’s sex, mother’s race and 

ethnicity, mother’s education, and mother’s age) to benchmark distributions derived from National Vital 

Statistics System (NVSS) natality data and other sources. Second, we compare NIS-Child vaccination 

coverage estimates to estimates provided by the Immunization Information Systems Annual Report. 

Third, we compare health insurance distributions derived from the NIS-Child Health Insurance Module 

(HIM) to corresponding distributions obtained from (i) the American Community Survey (ACS), (ii) the 

National Health Interview Survey (NHIS), and (iii) the Current Population Survey Annual Social and 

Economic Supplement (CPS ASEC). Finally, we compare NIS-Child vaccination coverage estimates to 

corresponding coverage estimates produced by state immunization surveys.

2.1 Demographic Distributions: Comparison of NIS-Child 

Distributions to Population Distributions

A direct method of estimating survey error is to compare the survey estimates to benchmark estimates 

from other, higher quality sources. While high-quality benchmark estimates of national-level vaccination 

coverage are not available, we can compare the survey estimates of demographic distributions to those 

1 At times in this report, we also refer to the Hib primary series (Hib-PS) which is ≥2 doses or ≥3 doses depending on brand. 

We also refer to ≥3 doses of Hib vaccine as 3+ Hib.
Error Profile for the 2022 NIS-Child 3

derived from NVSS natality data. These data yield population counts for several important 

characteristics of children: mother’s race and ethnicity, mother’s education, mother’s age, child’s sex, 

and child’s age.

To create benchmark population demographic distributions for children aged 19 to 35 months in 2022, 

we began by obtaining the counts of live births from the 2019 and 2020 NVSS natality data for children 

that would be 19 to 35 months old as of July 1, 2022, the midpoint of the reference year. These counts 

were obtained overall and by mother’s race and ethnicity, mother’s education, mother’s age, child’s sex, 

and child’s age as of July 1, 2022. The raw NVSS counts were then adjusted to account for infant 

mortality and immigration into the United States to produce counts of children aged 19 to 35 months 

living in the United States on July 1, 2022. These adjustments were applied separately by race and 

ethnicity group to account for differences in mortality and immigration across these groups.

We produced 2022 NIS-Child national-level demographic distribution estimates first using design 

weights and then using final weights. The design weights reflect the sample design but do not include 

any adjustments for sampling-frame noncoverage or interview nonresponse and are not calibrated to 

population control totals. Final weights are the design weights, with adjustments for noncoverage, 

nonresponse, and calibration to population control totals. (See the footnotes to Table 2.1 for the 

demographic variables used in this calibration.)

Table 2.1 compares 2022 NIS-Child national-level survey estimates of demographic distributions for 

children with adequate provider data to benchmark distributions for child’s age, sex, mother’s race and 

ethnicity, mother’s education, and mother’s age. We observe that the survey distribution for child’s age 

and child’s sex is close to the population distribution, even when using the design weights. The 

differences are zero when the final weights are used due to calibration based on child’s age and child’s 

sex. The design-weighted distribution of mother’s race and ethnicity differs somewhat from the 

population distribution, with the largest differences being for non-Hispanic White only mothers (59.4

percent in survey, 51.0 percent in population), Hispanic mothers (18.7 percent in survey, 24.4 percent 

in population), and non-Hispanic Black only children (11.3 percent in survey, 14.7 percent in 

population). The differences are small when the final weights are used due to calibration using race and 

ethnicity. 

Differences between the respondent set and the population are also observed for mother’s education 

and mother’s age. For mother’s education, the respondent set over-represents children whose mothers 

have a four-year college degree when the design weights are used (53.0 percent in survey, 33.7 

percent in population) and under-represents children whose mothers do not have a four-year college 

degree. When the final weights are used, the survey still somewhat over-represents children whose 

mothers have a four-year college degree (40.2 percent in survey, 33.7 percent in population) and 

under-represents children whose mothers have some college but not a four-year degree (21.2 percent 

in survey, 27.8 percent in population). The survey also over-represents children whose mothers are age 

30 or older when the final weights are used (67.2 percent in survey, 62.5 percent in population) and 

under-represents children whose mothers are age 20 to 29 (31.8 percent in survey, 36.5 percent in 

population). 
Error Profile for the 2022 NIS-Child 4 

Table 2.1: One-Way Demographic Distributions Among Children with Adequate Provider Data vs. 

Population Distributions: NIS-Child, 2022

Demographic Domain

Design-Weighted 

Estimates

Final-Weighted 

Estimates*

Population 

Distribution 

(%)

Survey 

Distribution 

(%)

Survey – 

Population 

Survey 

Distribution 

(%)

Survey – 

Population 

Child’s Age

19-23 months 29.3 29.3 ± 1.3 0.0 ± 1.3 29.3 ± 1.3 0.0 ± 1.3

24-29 months 33.9 32.0 ± 1.7 -1.9 ± 1.7 33.9 ± 1.4 0.0 ± 1.4

30-35 months 36.8 38.7 ± 1.6 1.9 ± 1.6 36.8 ± 1.4 0.0 ± 1.4

Child’s Sex

Male 51.1 52.1 ± 1.7 0.9 ± 1.7 51.1 ± 1.4 0.0 ± 1.4

Female 48.9 47.9 ± 1.7 -0.9 ± 1.7 48.9 ± 1.4 0.0 ± 1.4

Mother’s Race and 

Ethnicity

Hispanic 24.4 18.7 ± 1.7 -5.6 ± 1.7 24.1 ± 1.5 -0.3 ± 1.5

White 51.0 59.4 ± 1.7 8.4 ± 1.7 50.8 ± 1.5 -0.2 ± 1.5

Black 14.7 11.3 ± 1.1 -3.5 ± 1.1 15.0 ± 1.1 0.3 ± 1.1

Other 9.9 10.6 ± 0.9 0.7 ± 0.9 10.0 ± 0.9 0.1 ± 0.9

Mother’s Education

Less than high school 12.1 6.4 ± 0.9 -5.7 ± 0.9 10.6 ± 1.1 -1.5 ± 1.1

High school 26.5 15.4 ± 1.1 -11.0 ± 1.1 27.9 ± 1.5 1.5 ± 1.5

Some college 27.8 25.2 ± 1.6 -2.6 ± 1.6 21.2 ± 1.2 -6.6 ± 1.2

4-year college graduate 33.7 53.0 ± 1.7 19.3 ± 1.7 40.2 ± 1.4 6.6 ± 1.4

Mother’s Age

< 20 years 1.0 0.5 ± 0.2 -0.5 ± 0.2 0.9 ± 0.4 0.0 ± 0.4

20-29 years 36.5 26.2 ± 1.6 -10.3 ± 1.6 31.8 ± 1.5 -4.7 ± 1.5

>= 30 years 62.5 73.3 ± 1.6 10.8 ± 1.6 67.2 ± 1.5 4.8 ± 1.5

NOTE: Excludes U.S. territory samples. Table presents survey estimate and 95% confidence interval and 

presents the difference between the survey and the population percentages, along with a 95% confidence 

interval for the difference, assuming no error in the population proportion.

* Final provider-phase weights are calibrated within each geographic estimation area to marginal totals for 

child’s age (19-23, 24-29, 30-35), child’s sex (male, female), child’s race and ethnicity (Hispanic, Black, other), 

mother’s education (high school or less, more than high school), household telephone status (cell-phone only, 

other), and quintile of the estimated propensity to have adequate provider data for the child, given the 

household interview was completed for the child. The marginal totals for child’s race and ethnicity are estimated 

by summing the final household-phase weight within each category; the final household-phase weight itself had 

been calibrated to marginal control totals for mother’s race and ethnicity.

Comparisons of demographic distributions were made between survey estimates and population values 

for all two-way combinations of child’s age, child’s sex, mother’s race and ethnicity, and mother’s 

education, first using design weights and then using final weights. While final weights are controlled to 

marginal population totals for these characteristics individually, the weights are not controlled to totals 
Error Profile for the 2022 NIS-Child 5 

for cross-classifications of these characteristics. Differences between survey estimates and population 

values for these cross-classifications are all less than 5.0 percentage points when final weights are 

used.

2.2 Comparison of NIS-Child and IISAR Vaccination Coverage 

This subsection compares NIS-Child vaccination coverage Immunization Information System Annual 

Report (IISAR) data in 2021.2 The comparison is given for the combined 7-vaccine series using the 

data available from IISAR, recognizing that the findings may not apply to other vaccine series. 

Agreement between the vaccination coverage estimates signals consistency between NIS-Child and 

IISAR, and it may signal that both sources provide an accurate measurement of the true vaccination 

coverage in the age-eligible child population (19- to 35-month-old children). Lack of agreement between 

the vaccination coverage estimates signals inconsistency and that one or both sources are less 

accurate.

Our work in this subsection is divided into four parts. First, we describe the IISAR and some definitions 

we will use in this analysis. Second, we compare visually the vaccination coverage estimates in NISChild and IISAR using scatterplots. Third, we introduce the concept of the IIS (Immunization Information 

System)3 Child Participation Rate (CPR). Finally, we demonstrate through regression analysis that the 

difference between the combined 7-vaccine series vaccination coverage estimates in NIS-Child and 

IISAR is related to the CPR. 

What is IISAR?

The IISAR is an annual assessment of IIS activity among the 64 immunization program awardees, 

which include the 50 states, 6 cities (Chicago, District of Columbia, Houston, New York City, 

Philadelphia, and San Antonio), and 8 U.S. territories (American Samoa, Guam, Marshall Islands, 

Micronesia, Northern Mariana Islands, Palau, Puerto Rico, and Virgin Islands). To evaluate each 

awardee’s performance, the immunization program manager in the awardee area is asked to complete 

a self-administered, web-based questionnaire asking for demographic and immunization information, 

public and private provider site participation levels, and information about fulfillment of IIS functional 

standards. Because the questionnaire is self-administered and web-based, some awardees may report 

partial data or no data at all. 

NCIRD provided competitive supplemental funds to grantees that have achieved consistently high 

standards. During the period 2013-2017, six grantees have been recognized as sentinel sites, including 

2 https://www.cdc.gov/vaccines/programs/iis/annual-report-iisar/overview.html

3 State IIS are computer databases that aspire to contain information about all of the doses of all vaccines administered to all 

children residing within the state. It is known that different state IIS vary in their completeness of both children and the doses 

they received. https://www.cdc.gov/vaccines/programs/iis/about.html
Error Profile for the 2022 NIS-Child 6 

Michigan, Minnesota, North Dakota, New York City, Oregon, and Wisconsin.4 Because of the higher 

standards they achieved regarding participation rates and other indicators of IIS quality, vaccination 

coverage estimates reported in IISAR by sentinel sites are thought to be relatively more accurate than 

vaccination coverage estimates reported by non-sentinel sites. In this analysis, the sentinel sites are 

referred to as 2013-2017 sentinel sites.

In what follows, we compare 2021 NIS-Child vaccination coverage estimates to 2021 IISAR coverage 

estimates. Because 2022 IISAR vaccination coverage estimates are not available as of this writing, the 

2021 comparison will serve as the most current information available about the relative accuracy of the 

2022 NIS-Child. 

Visual Comparison of the Vaccination Coverage Estimates

Figures 2.1 and 2.2 display plots of the NIS-Child vaccination coverage estimate versus the IISAR 

vaccination coverage estimate for the combined 7-vaccine series in year 2021. Figure 2.1 includes 56 

of the estimation areas used in the NIS-Child; it does not include points corresponding to 8 U.S. 

territories. Figure 2.2 includes only the six 2013-2017 sentinel sites.

Figure 2.1: Scatterplot of NIS-Child (in %) v. IISAR (in %) Vaccination Coverage Estimates for the 

Combined 7-Vaccine Series: 56 Estimation Areas, 2021 

4 https://www.cdc.gov/vaccines/programs/iis/activities/sentinel-sites.html 
Error Profile for the 2022 NIS-Child 7 

Figure 2.2: Scatterplot of NIS-Child (in %) v. IISAR (in %) Vaccination Coverage Estimates for the 

Combined 7-Vaccine Series: Six 2013-2017 Sentinel Sites, 2021 

In all plots, the straight line through the origin reflects the yy = xx line. Points above the line represent 

areas in which the NIS-Child vaccination coverage estimate is greater than the IISAR estimate, and 

points below the line represent areas in which the IISAR estimate is greater. The line itself represents 

complete agreement between the NIS-Child and IISAR estimates. In addition, the color and symbol for

the point signify the magnitude of the difference between the NIS-Child and IISAR estimates. 

The plots reveal that in most areas the NIS-Child vaccination coverage estimates are greater than the 

IISAR estimates. There is reasonably good agreement between the two estimates for the six 2013-

2017 sentinel sites. Generally, IISAR estimates tend to be lower in non-sentinel areas. For IIS that 

achieve high standards, NIS-Child and IISAR vaccination estimates are reasonably similar.

Child Participation Rate (CPR)

To test the hypothesis that increasing quality of a state’s IIS data implies increasing agreement 

between the NIS-Child and IISAR vaccination coverage estimates, we introduce the CPR and evaluate 

it as a possible measure of the quality of the IIS. The CPR is defined as the proportion of children in the 

area who have two or more doses of any vaccine recorded in the IIS relative to a U.S. Census Bureau 

count of children living in the area. Note that the use of two data sources can result in some IISAR CPR 

values being greater than 100 percent. In Figure 2.3, we plot the IISAR vaccination coverage estimate

for the combined 7-vaccine series versus the CPR for the year 2021, including 56 core estimation 

areas. Points for the six 2013-2017 sentinel sites are labeled, and these sites are generally located in 
Error Profile for the 2022 NIS-Child 8 

the upper right corner of the plot, corresponding to higher values of both the CPR and vaccination 

coverage estimates. 

We fit a linear regression model to the points in Figure 2.3 and the corresponding fit is represented by 

the solid line depicted in the figure. The association of CPR with the IISAR vaccination coverage 

estimate is positive and statistically significant. The Pearson correlation is 0.527 with a 95% confidence 

interval of [0.306, 0.694]. Therefore, we conclude that CPR is positively associated with the 

completeness of the vaccination histories of the children. CPR appears to be a reasonable, but not 

necessarily comprehensive, measure of the quality of data in the IIS databases.

Figure 2.3: Scatterplot of IISAR (in %) Vaccination Coverage Estimate for the Combined 7-Vaccine 

Series v. CPR (in %): 56 Estimation Areas, 2021 

Negative Relationship Between Difference in Vaccination Coverage and CPR

We conduct further evaluation of the hypothesis that increasing quality of state IIS data is associated 

with increasing agreement between the NIS-Child and IISAR vaccination coverage estimates. 

Specifically, taking CPR to be the measure of IIS quality, we calculated the difference between the 

combined 7-vaccine series coverage estimates in NIS-Child and IISAR and fit a simple linear 

regression model relating the difference to the CPR. Figure 2.4 presents the scatterplot of the 

difference versus the CPR for the set of 56 core estimation areas in 2021, and the straight line depicted 

in the figure is the regression line. The CPR has a statistically significant relationship with the difference 

for all years. The coefficient on the CPR is negative (-0.64 percentage points with a 95% confidence 

interval of [-0.95, -0.33]), which implies that the difference declines with increasing CPR, or in other 
Error Profile for the 2022 NIS-Child 9

words, the difference between NIS-Child and IISAR vaccination coverage estimates declines as IIS 

quality increases. As the CPR, as an indicator of IIS data quality, increases, IISAR vaccination 

estimates tend to converge towards NIS-Child vaccination estimates, thus potentially supporting the 

accuracy of the NIS-Child vaccination coverage estimates.

Figure 2.4: Scatterplot of the Difference (NIS-Child Minus IISAR) for the Combined 7-Vaccine Series

(in %) v. CPR (in %) with Regression Line: 56 Estimation Areas, 2021 

We conducted an additional analysis pooling the data over the period 2016 to 2021 to achieve greater 

precision and regressed the difference between NIS-Child and IISAR vaccination coverage estimates 

on CPR, dummy variables for year, and state dummy variables. Similar to the results for 2021 in Figure 

2.4, the pooled results showed that CPR has a statistically significant, negative relationship with the 

difference, with a regression slope coefficient of -0.15 percentage points with a 95% confidence interval 

of [-0.27, -0.03].

Summarizing, we have presented evidence in this section that the CPR is a reasonable, though not 

comprehensive, measure of the quality of IIS data for children. As state IIS data quality indicators 

improve, the difference between the NIS-Child and IISAR vaccination coverage estimates declines.

2.3 Comparison of Children by Type of Health Insurance 

Coverage 

In this subsection, we compare NIS-Child health insurance estimates to those from the ACS, CPS 

ASEC, and the NHIS. We discuss the percentages of children with any private insurance coverage, any 
Error Profile for the 2022 NIS-Child 10 

public insurance coverage, and no insurance coverage. Before reviewing the results of the table, we 

provide an overview of each data source.

Conducted by the U.S. Census Bureau, the ACS is an ongoing survey that provides essential 

information about the population of the United States on a yearly basis, including statistics related to 

social, housing, economic, and demographic characteristics of the population. Table 2.2 contains 

information on the health insurance status of the child population 12 to 35 months based on the 2021

ACS5 at the national level. ACS interviews are conducted throughout the calendar year, and the ACS 

instrument assesses health insurance status as of the date of the interview. 

Table 2.2: Comparison of Alternative Estimates of Health Insurance Coverage Among Child: NISChild, ACS, CPS ASEC, and NHIS for 2021 and NIS-Child, CPS ASEC, and NHIS for 2022

Type of 

Health 

Insurance 

Coverage

2021 2022

ACS CPS 

ASEC NHIS NISChilda

CPS 

ASEC NHIS NISChilda

Any privateb 54.5% 58.5% 52.4% 52.3% 58.1% 47.3% 53.7%

Any publicc 47.1% 42.1% 46.2% 57.5% 42.7% 51.0% 56.1%

Uninsuredd 4.2% 5.0% 2.8% 1.9% 4.7% 4.3% 1.9%

aNIS-Child estimates were produced among children with adequate provider data using the final NISChild weight, which has been adjusted for noncoverage and nonresponse and calibrated to 

demographic population control totals.

bPrivate: Includes coverage provided through an employer or union or purchased directly from an 

insurance company that helps pay for both doctor visits and hospital stays.

c

Public: Includes Medicaid, CHIP, Indian Health Service, TRICARE, CHAMPUS and CHAMP-VA.

dUninsured: Defined as an uninsured if they do not have private insurance that helps pay for both 

doctor visits and hospital stays and do not have any other form of health insurance.

CPS ASEC is conducted in March of every year. While CPS is a monthly household survey conducted 

by the U.S. Census Bureau and the Bureau of Labor Statistics and designed mainly for measuring 

employment and unemployment, CPS ASEC provides additional detailed statistics related to household 

income, poverty, health insurance status, and other topics. The CPS ASEC asks current health 

insurance coverage status as of the time of the interview. Based on data from the March 2021 and 

2022 CPS ASEC6, national-level estimates of the health insurance distribution in 2021 and 2022 among

children aged 12 to 35 months are shown in Table 2.2.

The NHIS is a cross-sectional household interview survey, conducted by the National Center for Health 

Statistics, that covers the civilian noninstitutionalized population in the United States. The objective of 

the NHIS is to monitor the health status of the U.S. population. In addition to collecting variables related 

5 U.S. Census Bureau. 2021 ACS PUMS Data. Retrieved from https://www.census.gov/programssurveys/acs/microdata/access/2021.html

6 U.S. Census Bureau. 2021 and 2022 ASEC dataset. Retrieved from https://www.census.gov/data/datasets/timeseries/demo/cps/cps-asec.html
Error Profile for the 2022 NIS-Child 11 

to health status, the survey collects many demographic and socioeconomic characteristics of 

household members. NHIS7 national-level estimates of the health insurance distribution in 2021 and 

2022 for children aged 12 to 35 months are shown in Table 2.2. In the NHIS, health insurance status is 

assessed as of the time of the interview.

In reviewing Table 2.2, we find NIS-Child estimates, which are based on 19- to 35-month-old children

as of the date of the interview, of the size of the privately-insured population to be lower than those 

from the ACS, CPS ASEC, comparable to the NHIS in 2021, and higher than the NHIS in 2022. We find 

the estimates for public health insurance in NIS-Child to be larger than the corresponding estimates 

from ACS, CPS ASEC, and NHIS. Finally, we find NIS-Child estimates of the size of the uninsured 

population tend to be less than corresponding estimates from ACS, CPS ASEC, and NHIS. The 

differences in estimates between the NIS-Child and the other three sources could be due to differential 

sample-frame coverage error, nonresponse error, or measurement error; to differences in the ages of 

the children targeted by the estimates; or to definitional differences (questionnaire differences) in how 

health insurance status is measured.

2.4 Comparison of Vaccination Coverage Estimates: NIS-Child

v. State Immunization Surveys

In this subsection, we compare vaccination coverage estimates from NIS-Child and state immunization 

surveys. Agreement between the estimates signals consistency between the two sources and may 

indicate the accuracy of the sources. Disagreement between the estimates signals inconsistency and 

that at least one of the sources may be inaccurate. Eight state immunization surveys have been 

identified and included in the analysis, including surveys in Florida, Georgia, Kansas, Michigan, 

Tennessee, Virginia, Washington, and Wisconsin. As of this writing, we are unaware of immunization 

surveys sponsored or conducted by other states.

The Kansas Retrospective Vaccination Coverage Survey (KRS)8 and Virginia Annual Immunization 

Survey (VAIS)9 are based on random samples of students selected from public and private 

kindergartens and retrospective collection of students’ vaccination statuses from school records. The 

Tennessee Immunization Status Survey (TIS)10 and Florida Immunization Survey (FIS)11 are based on 

random samples of children selected from state listings of registered births. The Georgia Immunization 

7 NHIS - Data, Questionnaires and Related Documentation. Retrieved from https://www.cdc.gov/nchs/nhis/dataquestionnaires-documentation.htm 

8 https://www.kdhe.ks.gov/ArchiveCenter/ViewFile/Item/1642

9 https://www.vdh.virginia.gov/content/uploads/sites/11/2016/04/VAISAgRates.pdf

10 https://www.tn.gov/content/dam/tn/health/documents/cedep-weeklyreports/2022-24-Month-Old-Survey.pdf

11 https://www.floridahealth.gov/programs-and-services/immunization/resources/surveys/_documents/2yo2020.pdf
Error Profile for the 2022 NIS-Child 12 

Study (GIS)12, Michigan Care Improvement Registry (MCIR)13, Washington Immunization Information 

System (WAIIS)14, and Wisconsin Immunization Registry (WIR)15 prepare estimates of vaccination 

coverage by sampling children from the state IIS and aggregating IIS vaccination histories for the 

selected children. 

We compare estimates from the state surveys to corresponding estimates from the NIS-Child. The year 

of the comparison varies from state to state, and we always conduct the comparison for the most recent 

year data from the state survey are available. Table 2.3 gives the sample sizes of the state 

immunization surveys, where we find the available sample sizes of each state survey is much larger 

than the NIS-Child sample size in the state.

Figures 2.5.1 to 2.5.8 depict the vaccination coverage estimates for each of the eight states. The age 

range of the comparison also varies from state to state, depending on what range each state reports. 

For four of the states, the vaccination coverage estimates are compared at 24 months, and for these 

states the NIS-Child estimates are calculated in terms of vaccination status at 24 months for the set of 

children with adequate provider data who were 24 to 35 months at the time of the interview. For the 

remaining four states, the vaccination coverage estimates are compared for children 19 to 35 months.

The estimates for Hib vaccine and the combined 7-vaccine series are not always strictly comparable. 

For all eight state figures, we report the NIS-Child Hib vaccination coverage estimate and the combined 

7-vaccine series vaccination coverage estimate in terms of Hib full series, which requires 3 or 4 doses 

of Hib, depending on the brand of Hib vaccine used. The Tennessee and Florida surveys also report in 

terms of Hib full series, yet the Kansas, Virginia, Georgia, Michigan, Washington, and Wisconsin 

surveys appear to report the Hib and the combined 7-series estimates in terms of 3+ Hib, although 

state documentation is not fully clear. NIS-Child vaccination coverage estimates for Hib may appear to 

be lower than some of the corresponding state Hib vaccination coverage estimates due to this 

noncomparability.

12 https://immunizationstudyreports.s3.amazonaws.com/Child+Report/Child_Report_2022.html#summary

13 https://www.michigan.gov/mdhhs/adult-child-serv/childrenfamilies/immunization/localhealthdepartment/county-immunizationreport-card

14 https://doh.wa.gov/data-and-statistical-reports/washington-tracking-network-wtn/immunization-data/county-public-healthmeasures-dashboard

15 https://www.dhs.wisconsin.gov/publications/p02003a.pdf
Error Profile for the 2022 NIS-Child 13 

Table 2.3: Sample Sizes of Children from State Immunization Surveys in 2015, 2020, 2021, and 2022

2015 2020 2021 2022

Survey

Children 

19-35 

Months

Children 

24-35 

Months

Children 

24-35 

Months

Children 

24-35 

Months

Children 

19-35 

Months

Florida Immunization Survey (FIS) 5,5181)

NIS-Child in Florida, Children with Adequate Provider Data 335

Georgia Immunization Survey (GIS) 337,8032)

NIS-Child in Georgia, Children with Adequate Provider Data 304

Kansas Retrospective Vaccination Coverage Survey (KRS) 7,8813)

NIS-Child in Kansas, Children with Adequate Provider Data 228

Michigan Care Improvement Registry (MCIR) 165,7634)

NIS-Child in Michigan, Children with Adequate Provider Data 457

Tennessee Immunization Status Survey (TIS) 1,3995)

NIS-Child in Tennessee, Children with Adequate Provider 

Data 347

Virginia Annual Immunization Survey (VAIS) N/A6)

NIS-Child in Virginia, Children with Adequate Provider Data 520

Washington Immunization Information System (WAIIS) 129,8607)

NIS-Child in Washington, Children with Adequate Provider 

Data 394

Wisconsin Immunization Registry (WIR) N/A8)

NIS-Child in Wisconsin, Children with Adequate Provider 

Data 278

Source:

1) https://www.floridahealth.gov/programs-andservices/immunization/resources/surveys/_documents/2yo2020.pdf

2) https://immunizationstudyreports.s3.amazonaws.com/Child+Report/Child_Report_2022.html#summary

3) https://www.kdhe.ks.gov/ArchiveCenter/ViewFile/Item/1642 

4) https://www.michigan.gov/mdhhs/adult-childserv/childrenfamilies/immunization/localhealthdepartment/county-immunization-report-card

5) https://www.tn.gov/content/dam/tn/health/documents/cedep-weeklyreports/2022-24-Month-Old-Survey.pdf

6) https://www.vdh.virginia.gov/content/uploads/sites/11/2016/04/VAISAgRates.pdf

7) https://doh.wa.gov/data-and-statistical-reports/washington-tracking-network-wtn/immunization-data/countypublic-health-measures-dashboard

8) https://www.dhs.wisconsin.gov/publications/p02003a.pdf

The Kansas KRS and NIS-Child vaccination coverage estimates are compared in Figure 2.5.1. 

Because KRS vaccination coverage estimates are assessed at 35 months of age, one might expect 

them to be somewhat higher than NIS-Child estimates, which are based on children aged 19 to 35

months. Nevertheless, we find general agreement between the KRS and NIS-Child estimates. While 

the smallest absolute difference is 0.7 percentage points for 3+ Polio in Figure 2.5.1, the largest is 6.2

percentage points for 4+ PCV. Note that the KRS Hib and combined 7-vaccine series estimates may be 

based on a 3+ Hib definition rather than the Hib full series definition, limiting comparability between the 

two sources for these estimates. 
Error Profile for the 2022 NIS-Child 14 

Figure 2.5.1: Comparison of Vaccination Coverage Estimates for Children 19-35 Months: Kansas 

Retrospective Vaccination Coverage Survey, 2014-2015 v. NIS-Child, 2015

88.8

94.2 95.2

88.8

94.9 94.0

80.0

74.1

86.8

94.9

92.3

84.6

95.7

92.8

86.2

75.2

50

55

60

65

70

75

80

85

90

95

100

4+DTaP 3+Polio 1+MMR Hib full series 3+Hep B 1+Var 4+PCV Combined 7-

vaccine

series

Kansas Retrospective Vaccination Coverage Survey 2014-2015 (Children 35 Months)

NIS-Child Kansas 2015 (Children 19-35 Months)
Error Profile for the 2022 NIS-Child 15 

The Tennessee TIS and NIS-Child estimates are compared for children at age 24 months in Figure 

2.5.2. We find a reasonable degree of agreement between the two sets of estimates. While the smallest 

absolute difference is 0.1 percentage points for 4+ PCV, the largest absolute difference is 9.3

percentage points for the combined 7-vaccine series.

Figure 2.5.2: Comparison of Vaccination Coverage Estimates for Children at 24 Months: Tennessee 

Immunization Status Survey, 2022 v. NIS-Child, 2022 

81.3

92.9

91.0

79.6

93.9

90.3

82.1

77.1

81.1

93.8

91.9

76.0

95.8

92.0

82.0

67.8

50

55

60

65

70

75

80

85

90

95

100

4+DTaP 3+Polio 1+MMR Hib full

series

3+Hep B 1+Var 4+PCV Combined

7-vaccine

series

Tennessee Immunization Status Survey 2022 (Children 24 Months)

NIS-Children Tennessee 2022 (Children 24-35 Months)
Error Profile for the 2022 NIS-Child 16

The Virginia VAIS and NIS-Child estimates are compared for children at age 24 months in Figure 2.5.3. 

We find good agreement between the two sets of estimates, except for the Hib full series; the Hib full 

series vaccination coverage is estimated to be 9.1 percentage points higher in the VAIS than in the 

NIS-Child. However, note that the VAIS Hib estimate may be based on a 3+ Hib definition rather than 

the Hib full series definition, limiting comparability between the two sources.

Figure 2.5.3: Comparison of Vaccination Coverage Estimates for Children at 24 Months: Virginia 

Annual Immunization Survey, 2021 v. NIS-Child, 2021 

80.2

93.5

91.4

90.1 89.6 90.5

86.5

95.8

92.4

80.7

96.3

90.8

50

55

60

65

70

75

80

85

90

95

100

4+DTaP 3+Polio 1+MMR Hib full series 3+Hep B 1+Var

Virginia Immunization Survey 2021 (Children 24 Months)

NIS-Child Virginia 2021 (Children 24-35 months)
Error Profile for the 2022 NIS-Child 17

Figures 2.5.4 presents the comparisons for children at 24 months for the Florida FIS. The FIS uses 

vaccination history data from the state IIS, and in the event the child is not up-to-date according to this 

history, makes an effort to contact the child’s parent or guardian to obtain any vaccinations missing 

from the IIS history. Note that the FIS Hib estimate may be based on a 3+ Hib definition rather than the 

Hib full series definition, limiting comparability between the two sources. 

Figure 2.5.4: Comparison of Vaccination Coverage Estimates for Children at 24 Months: Florida 

Immunization Survey, 2020 v. NIS-Child, 2020

Figures 2.5.5, 2.5.6, 2.5.7 and 2.5.8 present vaccination coverage estimates in Georgia, Michigan, 

Washington, and Wisconsin, respectively. In these states, the vaccine-to-vaccine pattern revealed in 

the states’ estimates are generally similar to NIS-Child estimates. The differences between state and 

NIS-Child estimates are smaller in Georgia and Wisconsin than in Michigan and Washington. For all 

four states, NIS-Child estimates tend to be larger than the state estimates. Note that the Hib and 

combined 7-vaccine series estimates for these four state surveys may be based on a 3+ Hib definition 

rather than the Hib full series definition, limiting comparability to the NIS-Child for these estimates.

86.2

93.4 92.9 92.9 92.8 92.5

83.0 83.2

87.2

94.9 93.9

83.9

93.5 94.0

82.6

70.9

50

55

60

65

70

75

80

85

90

95

100

4+DTaP 3+Polio 1+MMR Hib full

series

3+Hep B 1+Var 4+PCV Combined

7-vaccine

series

Florida Immunization Survey 2020 (Children 24 Months)

NIS-Child Florida 2020 (Children 24-35 Months)
Error Profile for the 2022 NIS-Child 18

Figure 2.5.5: Comparison of Vaccination Coverage Estimates for Children 19-35 Months: Georgia 

Immunization Study, 2022 v. NIS-Child, 2022

80.1

90.8

88.4

86.6

89.6 8...

91.5

75.5

86.3

94.8 93.4

76.9

94.7

93.1

85.3

73.6

50

55

60

65

70

75

80

85

90

95

100

4+DTaP 3+Polio 1+MMR Hib full

series

3+Hep B 1+Var 4+PCV Combined

7-vaccine

series

Georgia Immunization Study 2022 (Children 19-35 Months)

NIS-Child Georgia 2022 (Children 19-35 Months)
Error Profile for the 2022 NIS-Child 19

Figure 2.5.6: Comparison of Vaccination Coverage Estimates for Children 19-35 Months: Michigan 

Care Improvement Registry, 2022 v. NIS-Child, 2022

70.2

76.4

67.5

87.6 87.3

80.5

50

55

60

65

70

75

80

85

90

95

100

4+DTaP 4+PCV Combined 7-vaccine series

Michigan Care Improvement Registry 2022 (Children 19-35 Months)

NIS-Child Michigan 2022 (Children 19-35 Months)
Error Profile for the 2022 NIS-Child 20

Figure 2.5.7: Comparison of Vaccination Coverage Estimates for Children 19-35 Months: Washington 

Immunization Information System, 2022 v. NIS-Child, 2022

64.4

76.7

74.9 75.4

76.9

74.1

63.4

57.4

87.1

94.7

92.6

85.4

97.4

92.2

86.9

75.9

50

55

60

65

70

75

80

85

90

95

100

4+DTaP 3+Polio 1+MMR Hib full

series

3+Hep B 1+Var 4+PCV Combined 7-

vaccine

series

Washington Immunization Information System 2022 (Children 19-35 Months)

NIS-Child Washington 2022 (Children 19-35 Months)
Error Profile for the 2022 NIS-Child 21

Figure 2.5.8: Comparison of Vaccination Coverage Estimates for Children at 24 Months: Wisconsin 

Immunization Registry, 2022 v. NIS-Child, 2022 

In summary, there is a reasonable agreement in vaccination coverage estimates between NIS-Child 

and KRS, TIS, VAIS, and FIS. Because of follow-up interviews conducted with parents of children who 

are not up-to-date according to IIS vaccination histories, GIS vaccination estimates may be differentially 

biased upwards relative to NIS-Child estimates. On the other hand, NIS-Child vaccination coverage 

estimates are relatively higher than state estimates in Michigan, Washington, and Wisconsin, which rely 

on IIS vaccination histories as the source of their coverage estimates. Definitional differences used in 

accounting for Hib vaccine cloud the comparisons between state and NIS-Child estimates.

72.1

83.3 81.9

80.7

83.8

80.4

78.6

69.0

80.4

90.8

93.9

82.2

92.6 92.1

81.8

69.5

50

55

60

65

70

75

80

85

90

95

100

4+DTaP 3+Polio 1+MMR Hib full

series

3+Hep B 1+Var 4+PCV Combined

7-vaccine

series

Wisconsin Immunization Registry 2022 (Children 24 Months)

NIS-Child Wisconsin 2022 (Children 24-35 Months)
Error Profile for the 2022 NIS-Child 22 

3. Part II: Assessment of Total Survey Error for 

NIS-Child Vaccination Coverage Estimates 

In this second part of the report, we assess the total survey error in NIS-Child vaccination coverage 

estimates using the framework developed and implemented in Molinari et al. (2011) and 

Wolter et al. (2017). We decompose TSE into components of sampling and nonsampling error, and 

then assemble the best information available about the magnitude of each component error from 

specialized evaluation studies. We view each component error as a random variable subject to a 

conditional distribution, given the outcome of the NIS-Child. The mean of the conditional distribution is 

estimated from numerical evidence obtained in the corresponding evaluation study, and the variance of 

the distribution, reflecting both variability in the evaluation survey samples and other uncertainties in our 

knowledge about the component error, is estimated from internal evidence within the evaluation study 

and additional professional judgment when necessary. After assembling our best information about 

each of the component errors, we combine the information to produce a total survey error distribution, 

using a Monte Carlo method.

Before proceeding to consider the component errors, we introduce some notation that will be helpful in 

this section. Let μμ0 denote the true but unknown vaccination coverage in the age-eligible population of 

children and let μμ̂denote the NIS-Child estimate of the vaccination coverage. The TSE in vaccination 

coverage estimate is then given by

(1) 

We use a three-stage model for TSE, where Stage 1 represents error due to the sampling frame’s 

undercoverage of the population of age-eligible children, Stage 2 represents error due to nonresponse 

among sampled units, and Stage 3 represents measurement error among the responding units. The 

model for the first stage (sampling-frame coverage) is

(2)

where μμ1 is the true vaccination coverage for the age-eligible children covered by the sampling frame, 

μμ1AA is the true vaccination coverage for the age-eligible children not covered by the sampling frame, 

and pp1AA is the proportion of the age-eligible population not covered by the sampling frame. The model 

at the second stage (response) is

(3)
Error Profile for the 2022 NIS-Child 23 

where μμ2 is the true vaccination coverage for children who respond to NIS-Child, μμ2AA is the true 

vaccination coverage for children who do not respond, and pp2AA is the proportion of children who do not 

respond. Finally, the model at the third stage (measurement) is

(4)

where μμ3 is the true vaccination coverage of children for whom accurate response is given to the 

survey, μμ3AA is the true vaccination coverage of children for whom inaccurate response is given to the 

survey, and pp3AA is the proportion of children for whom inaccurate response is given.

Combining all three stages together, the true vaccination coverage can be written as

(5)

We can also write the TSE as 

(6)

where qq1 = μμ1 − μμ0 is the error due to noncoverage, qq2 = μμ2 − μμ1 is the error due to nonresponse, and 

qq3 = μμ̂ − μμ2 is the error due to inaccurate reporting by survey respondents.

The seven parameters on the right side of (5) are . Estimates of 

the values of these seven parameters, based on the analyses to be presented in Sections 3.2 

through 3.4, are denoted by . Let denote the estimated variancecovariance matrix of . We assume the seven parameters are independently distributed and that 

, where each is our estimate of the variance of the 

corresponding parameter. 

We assume our knowledge about the true parameters , can be acceptably represented by a 

probability distribution, with parameters and We assess TSE by making random draws of from 

its distributions. For each draw, we use (5) to produce a draw from the distribution of true vaccination 

coverage in the overall age-eligible population (say, μμ0

∗) and we compute qq∗ = μμ̂ − μμ0∗ as a draw from 

the distribution of TSE. We obtain the distribution of TSE through 10,000 such draws.

Having established our model and notation, we now consider sampling-frame coverage error in the 

NIS-Child, which arises because the sampling frame omits direct representation of the phoneless and 

landline only (LLO) populations. Second, we consider nonresponse error in the NIS-Child, which comes 

about due to nonresponse in the random digit dial (RDD) telephone survey of households, failure of the 

parental respondent to give consent to contact the children’s immunization providers, and missing 
Error Profile for the 2022 NIS-Child 24

vaccination histories in the Provider Record Check given consent. Third, we consider response or 

measurement error in the provider reporting of vaccination histories. This component of error has also 

been referred to as under-ascertainment of vaccination histories. Fourth, we consider error in the NISChild due to sampling, i.e., error because the survey observes only approximately 1 out of 300 children 

in the age-eligible population. Fifth, we combine the foregoing component error distributions, resulting 

in the TSE distribution of the vaccination coverage estimate for the 2022 NIS-Child. Finally, we close 

this part of the report by examining the change in the TSE from the 2021 NIS-Child to the 2022 NISChild using the bridging cohort method, developed and implemented by Yankey et al. (2015).

3.1 Sampling-Frame Coverage Error

Sampling-frame coverage errors arise in a survey when the sampling frame does not include the entire 

target population. In 2018, the NIS-Child began using a single-frame cell-phone RDD design, which 

omits direct representation of children in LLO and phoneless households. To account for the excluded 

population groups, the NIS-Child weighting methodology makes adjustment to the weights by raking the 

weights to select demographic characteristics of the population of children 19 to 35 months. The 

assumption embedded in this procedure is that, after controlling for these characteristics, the 

vaccination coverage in the population not represented on the sampling frame equals the coverage in 

the population represented on the frame. However, it is possible that estimated vaccination coverage of 

children in the omitted domains experience different vaccination coverage than the domains of children 

included in the survey, namely, cell-phone-only (CPO) children and dual-user children, which may 

introduce bias into the estimator of vaccination coverage. 

In this subsection, we attempt to measure the bias in estimated vaccination coverage introduced by 

sampling-frame coverage error. Table 3.1 displays the proportion of 19- to 35-month-old children in the 

population by telephone status for 2012 to 2021 based on estimates from the NHIS. The proportion of 

children with CPO status is increasing throughout this period and the proportion of children with dualuser status is decreasing. The estimated proportion in cell-phone households (i.e., CPO and dual-user 

combined), was relatively steady until 2018, before increasing from 93.3% in 2018 to 98.2% in 2019. 

This estimate remained at a similar level in 2020 (98.7%) and 2021 (97.8%). LLO children have 

decreased in prevalence over time. Estimates of the phoneless population increased between 2012 

and 2018 before lower estimates were found in 2019, 2020, and 2021 (0.7%, 0.9%, and 1.2%

respectively). 
Error Profile for the 2022 NIS-Child 25 

Table 3.1: Percentage of Age-Eligible Children in the Population by Telephone Status by Year: NISChild, United States, 2012-2021

Year Cell-Phone Only Dual-User Landline Only Phoneless 

2012 51.3 43.4 4.0 1.3

2013 57.6 36.8 3.0 2.6

2014 64.1 30.8 2.6 2.5

2015 68.9 26.5 2.2 2.4

2016 71.8 24.6 1.3 2.4

2017 73.3 20.9 2.5 3.3

2018 74.0 19.3 1.6 5.1

2019 80.9 17.3 1.1 0.7

2020 82.5 16.2 0.4 0.9

2021 87.2 10.6 1.0 1.2

Source: Produced using the methods of Blumberg, Ganesh, Luke, and Gonzales (2013) applied to data from 

the 2012-2021 National Health Interview Survey sponsored by CDC’s National Center for Health Statistics 

(https://www.cdc.gov/nchs/nhis/index.htm).

The 2022 NIS-Child did not directly measure LLO or phoneless children, and to assess vaccination 

coverage in these domains and determine whether they differ from estimates in the combined cellphone domain, we must turn to other sources. Specifically, the 2012-2017 NIS-Child sampled and 

directly represented LLO children, thus permitting comparison of the vaccination coverage by telephone 

status. Table 3.2 displays the vaccination coverage estimates for 2017, the closest such year to 2022. 

We observe that vaccination coverage estimates are generally higher in the cell-phone domain than in 

the LLO domain, and four of the nine differences in estimates of vaccination coverage are statistically 

significant. 

Since NIS-Child has never included direct sampling of phoneless children, we study vaccination 

coverage in the phoneless domain using estimates from the 2012 National Health Interview SurveyProvider Record Check (NHIS-PRC).16 Table 3.3 shows the vaccination coverage estimates for select 

vaccines and vaccine series for children in the cell-phone domain compared with those in the 

phoneless domain. For most of the vaccines presented, we observe higher vaccination coverage 

estimates from the cell-phone domain compared to the phoneless domain; however, none of the 

differences are statistically significant, likely due to the small number of phoneless children in the NHIS.

16 2012 was the last, and therefore most recent, year for which the NHIS-PRC was conducted, and thus for which a direct 

measurement was obtained of the vaccination status of phoneless children.
Error Profile for the 2022 NIS-Child 26 

Table 3.2: Vaccination Coverage Estimates and Standard Errors of Select Vaccines and Vaccine 

Series for the Cell-Phone and LLO Domains: NIS-Child, United States, 2017

Cell-Phone Domain Landline-Only Domain Difference

Variance/Series Estimate Standard 

Error 

Estimate Standard 

Error 

Estimate Standard 

Error 

4+DTaP 83.4 0.59 58.7 10.65 24.6 10.66 * 

3+Polio 92.9 0.39 67.6 11.28 25.3 11.28 * 

1+MMR 91.6 0.43 80.0 7.82 11.5 7.83

Hib-FS 80.9 0.66 58.7 10.68 22.1 10.70 * 

3+HepB 91.5 0.46 81.1 7.77 10.4 7.78

HepB Birth Dose 73.7 0.80 70.1 8.07 3.6 8.12

1+Var 91.2 0.44 72.6 8.64 18.6 8.66 * 

4+PCV 82.6 0.64 62.0 10.83 20.6 10.85

7-Vaccine Series 70.6 0.75 54.0 10.23 16.6 10.26

∗ pp ≤ 0.05. 

Note that 2012 NHIS-PRC estimates for HepB birth dose are not available. To assess coverage error in 

the HepB birth dose estimate, we used a conservatively high estimate of the difference between the 

vaccination coverage for the cell-phone and phoneless domains. We used this approach to 

demonstrate in subsequent analysis that coverage error is small even if the difference in vaccination 

coverage between the covered and non-covered domains is large. Specifically, for HepB birth dose, we 

used the largest estimated difference available for the eight other vaccine series. 

The foregoing tables can be translated into an assessment of sampling-frame coverage error in the 

2022 NIS-Child estimated vaccination coverage. We can also write the true vaccination coverage as 

μμ00 = μμ11 − BB , (7)

where BB = pp1AA(μμ1 − μμ1AA) equates to sampling-frame coverage error. To fully assess the distribution of 

total survey error in the NIS-Child, we will require estimates of the parameters, pp̂1AA, μμ̂1AA, and μμ̂1, and 

their standard errors, which we present in Section 3.5. Here we simply observe that pp̂1AA is obtained 

from the landline only and phoneless columns on the right side of Table 3.1 for the most recent year 

available 2021, and μμ̂1AA is obtained from the results of the 2022 NIS-Child and the Difference columns 

on the right side of Tables 3.2 and 3.3. The estimate of vaccination coverage in the sampling-frame 

covered population, μμ̂1, is obtained from the results of the 2022 NIS-Child and from analyses presented 

in Sections 3.2 and 3.3 on nonresponse error and measurement error.
Error Profile for the 2022 NIS-Child 27 

Table 3.3: Vaccination Coverage Estimates and Standard Errors of Select Vaccines and Vaccine 

Series for Children 19-35 Months in the Cell-Phone and Phoneless Domains: NHIS-PRC, United 

States, 2012

Cell-Phone Domain Phoneless Domain Difference

Vaccine Estimate Standard 

Error Estimate Standard Error Estimate Standard Error 

4+DTaP 83.87 1.36 87.40 6.53 -3.53 6.67

3+Polio 93.90 0.82 91.10 5.61 2.80 5.67

1+MMR 92.24 1.08 92.70 4.44 -0.46 4.57

Hib-PS 94.72 0.89 91.10 5.61 3.62 5.68

3+HepB 91.06 1.03 85.30 7.81 5.76 7.87

HepB Birth Dose 6.47# 5.68#

1+Var 91.69 0.98 92.70 4.44 -1.01 4.55

4+PCV 85.37 1.32 78.90 8.27 6.47 8.37

7-Vaccine Series 72.53 1.71 67.80 9.85 4.73 9.99

Notes: Estimates for the Hib full series (Hib-FS) are not available from the NHIS-PRC. Instead, we use the 

estimate for the Hib primary series (Hib-PS), which is ≥2 doses or ≥3 doses depending on brand. The estimates 

for the combined 7-vaccine series in this table are also based on Hib-PS rather than Hib-FS. 

∗ pp ≤ 0.05. 

# Estimates for HepB birth dose are not available from the NHIS-PRC Analysis Report. As inputs for estimating 

coverage error for the HepB birth dose, we used the largest difference in vaccination coverage estimates 

between the cell-phone and phoneless domains available from the eight other vaccines. This approach was 

chosen to be conservative and overestimate the extent of potential coverage error. For the standard error of the 

difference, we used the same standard error as for the corresponding estimate for 3+ Hib, as 3+ Hib and HepB 

birth dose have similar 2022 vaccination coverage estimates.

As a preliminary assessment of the effect of sampling-frame coverage error, we estimate the true 

vaccination coverage ignoring the effects of nonresponse and measurement error. In this circumstance, 

the NIS-Child vaccination coverage estimate is essentially μμ1. Then, Table 3.4 presents estimates of BB

and of the true vaccination coverage, μμ0. We find that sampling-frame coverage error is 0.3 percentage 

points or less and the estimated error is less than the standard error of μμ0.
Error Profile for the 2022 NIS-Child 28 

Table 3.4: Preliminary Assessment of Sampling-Frame Coverage Error and Mean True Vaccination 

Coverage (in %): NIS-Child, United States, 2022

Vaccine

(2022 NISChild

Vaccination 

Coverage 

Estimate) 

(Mean of ,

the True 2022 

Vaccination 

Coverage) 

Standard 

Error of 

4+DTaP 84.5 0.2 84.3 0.6

3+Polio 93.5 0.3 93.2 0.4

1+MMR 93.0 0.1 92.9 0.4

Hib-FS 81.3 0.3 81.0 0.6

3+HepB 92.9 0.2 92.7 0.4

HepB Birth Dose 81.6 0.1 81.5 0.6

1+Var 92.6 0.2 92.4 0.4

4+PCV 83.8 0.3 83.6 0.6

7-Vaccine Series 72.1 0.2 71.9 0.7

a The estimated sampling-frame coverage error, , is obtained by combining information in Table 3.2 about the 

landline-only population in 2017 with information in Table 3.3 about the phoneless population in 2012.

3.2 Nonresponse Error 

There are two types of nonresponse error impacting NIS-Child, unit nonresponse error due to not 

obtaining responses (or completed interviews) for all children sampled and item nonresponse error due 

to missing questionnaire items among survey respondents. This section focuses on assessing survey 

error due to unit nonresponse. We conclude with a review of 2022 NIS-Child item nonresponse rates. 

NIS-Child vaccination coverage estimates are based on provider-reported vaccination histories; 

incomplete (or missing vaccination) information on these histories is a form of measurement error or 

under-reporting error, which is assessed in Section 3.3. 

Components of Nonresponse in NIS-Child

Unit nonresponse error in NIS-Child estimates of vaccination coverage is the error arising because

responses are not obtained for all children sampled. Nonresponse arises at four steps in the survey 

process, as follows: (1) failure to resolve the selected telephone number as an occupied household or 

some other known entity, (2) failure to screen the household for the presence of one or more ageeligible children, (3) failure to complete the interview of an eligible household, and (4) failure to obtain 

consent to contact the child’s vaccination providers or failure to obtain sufficient information from 

providers to determine the child’s vaccination status, given consent. We do not observe the vaccination 
Error Profile for the 2022 NIS-Child 29 

statuses of children for whom the household interview or the Provider Record Check is missing. This 

subsection assesses the extent of nonresponse error in the 2022 NIS-Child estimates of vaccination 

coverage for four vaccines or vaccine series, including 4+DTaP, 1+MMR, HepB birth dose, and the 

combined 7-vaccine series.

Weight Adjustment for Nonresponse Error

NIS-Child addresses error due to nonresponse by using weight adjustments that correct for known 

differences between children in responding and nonresponding households based on observable 

characteristics. Specifically, weighting cells are defined based on sample-frame information known for 

both respondents and nonrespondents, and weights are adjusted by a factor inversely proportional to 

the response rates within each cell. Calibration of the weights to demographic population totals also 

serves to adjust for differences between the responding sample and the population. The NIS-Child 

weighting methodology is described in detail in Wolter et al. (2017).

The weighting adjustment method assumes that nonresponse is a missing at random process 

(Rubin 1976), or that the conditional distribution of vaccination coverage on the characteristics used to 

form the weighting cells and calibration dimensions is the same whether or not the data are missing. 

This assumption, while widely used for weighting nonresponse adjustments, is generally untestable 

since we do not observe vaccinations statuses for the nonrespondents. Thus, further methods are 

needed to assess the extent of nonresponse error after conducting weighting adjustments.

Assessment of Nonresponse Error

To inform our TSE models, an estimate of the proportion of children with adequate provider data among 

children in households corresponding to the sampled telephone numbers is needed. The 2022 NISChild realization rate17 of children with adequate provider data was 2.7 percent with a standard error of 

0.04 percentage points. Dividing the realization rate by the coverage rate of the sampling frames as 

estimated in Section 3.1 yields an estimate of the proportion of children with adequate provider data 

among those covered by the sampling frame of 2.8 percent, or 97.2 percent without adequate provider 

data, with a standard error of 0.5 percentage points. These two numbers (97.2% with a standard error 

of 0.5 percentage points) serve as model inputs pp̂2AA and for the TSE analysis.

We now assess the extent of nonresponse error both before conducting nonresponse weighting 

adjustments as well as the residual error after accounting for such adjustments. It is common in TSE 

analyses to compare estimates derived from the survey under study, NIS-Child in this instance, with 

those from leading reference surveys (Biemer, 2010). A reasonable benchmark for the NIS-Child is the 

NHIS, because it provides representation of the same population of children as the NIS-Child and is 

17 The realization rate (Skalland, 2011) is calculated as the ratio of the unadjusted survey estimate of the size of the target 

population to an external estimate of the true size of the target population and can be interpreted as the product of the 

coverage rate of the sampling frame and the response rate.
Error Profile for the 2022 NIS-Child 30 

known to be a premier health survey of the general population in the United States. The NHIS is 

conducted using face-to-face interviewing methods, and has a relatively high response rate, with the 

final response rates for the Sample Child component in 2021 and 2022 being 49.9% and 45.8%.18,19

We combined two years of NHIS data to improve the accuracy of estimates and increase the sample 

size for young children. 

Comparing NIS-Child estimates to those based on the NHIS enables estimation of nonresponse error. 

If nonresponse error is minimal in the NHIS, then the comparison to the NIS-Child can be taken as a 

measure of nonresponse error in the NIS-Child. To ensure comparability with the population covered by 

the NIS-Child, we examine NHIS children who are in the corresponding age range and have a working 

cell-phone in the family.

While the NHIS does not collect vaccination data for children and thus does not produce direct 

estimates of vaccination coverage, we can compare indirect vaccination coverage estimates derived 

from the NHIS to the direct estimates of vaccination coverage derived from the NIS-Child. We take 

advantage of the range of variables that are common to both the 2022 NIS-Child and the 2021 and 

2022 NHIS and produce estimates of nonresponse error for four vaccination coverage estimates: 

4+DTaP, 1+MMR, HepB birth dose, and the combined 7-vaccine series. Specifically, we estimate 

logistic regression models for vaccination statuses in the NIS-Child employing variables common to 

both surveys as independent variables and then use these models to produce multiple imputations of 

vaccination statuses for the NHIS case set, using the 2021 and 2022 NHIS Public Use Files (PUFs). 

Then, we estimate vaccination coverage using the NHIS data after pooling the survey-weighted 

estimates across the multiply-imputed datasets. We treat the estimates based on imputations of the 

NHIS as the true vaccination coverage among the population covered by the NIS-Child and estimate 

nonresponse error in the NIS-Child estimates by taking the difference between the NIS-Child and the 

pooled NHIS estimates.

We note that a common method for nonresponse bias analysis is to apply modeling, including logistic 

regression, to develop predictions or imputations of key variables among nonrespondents to develop 

full-response key estimates and compare to estimates based on respondents alone (U.S. Census 

Bureau, 2019). The method we employ in this study extends this concept to applying predictions or 

imputations to a reference survey. The 2009 report of the NCES/NISS Task Force on Nonresponse 

Bias Analysis recommends producing multiple imputations when employing such methods to account 

for imputation modeling variance in estimates of nonresponse bias.20

Table 3.5 compares the estimates of vaccination coverage based on the NHIS imputed data and NISChild provider-reported data. Two NIS-Child estimates are presented, one based on applying design 

weights and another based on applying the final weights that reflect adjustments for noncoverage and 

nonresponse. Presenting both sets of estimates shows how the NIS-Child estimates, before and after 

18 See p. 26 of https://ftp.cdc.gov/pub/Health_Statistics/NCHS/Dataset_Documentation/NHIS/2021/srvydesc-508.pdf

19 See p. 26 of https://ftp.cdc.gov/pub/Health_Statistics/NCHS/Dataset_Documentation/NHIS/2022/srvydesc-508.pdf

20 Details of our methods for estimating nonresponse error are available from the authors.
Error Profile for the 2022 NIS-Child 31 

weighting adjustments that account for nonresponse, compare to the estimates based on NHIS 

imputations. The table further shows the percentage point difference between the NHIS-based estimate 

and the NIS-Child estimate based on design weights. It includes a tt-statistic for testing the difference 

between the two vaccination coverage estimates, accounting for the uncertainty in both estimates.

Table 3.5: Estimates of Nonresponse Error for Vaccination Coverage (%) Derived from Estimates 

Based on NHIS Imputations and NIS-Child Estimates

Statistics 4+ DTaP 1+ MMR HepB Birth 

Dose 

7-Vaccine 

Series

(a) Estimate Based on NHIS Imputations 84.1 92.4 81.8 73.5

Standard Error 1.6 1.3 1.7 2.1

(b) NIS-Child Estimate (Design Weighted) 84.3 92.2 80.2 73.7

Standard Error 0.8 0.6 0.8 0.8

(c) NIS-Child Estimate (Final Weighted) 84.5 93.0 81.6 72.1

Standard Error 0.6 0.4 0.6 0.7

(d) Difference, (b) – (a) 0.2 -0.1 -1.6 0.2

Standard Error of Difference 1.8 1.4 1.9 2.2

t-statistic 0.09 -0.09 -0.83 0.10

p-value (2-sided) 0.93 0.93 0.41 0.92

(d) Difference, (c) – (a) 0.4 0.6 -0.2 -1.4

Standard Error of Difference 1.7 1.3 1.8 2.2

t-statistic 0.23 0.45 -0.13 -0.62

p-value (2-sided) 0.81 0.65 0.90 0.54

The NIS-Child estimates based on design weights are 0.2 percentage points higher than the estimate 

based on NHIS imputations for 4+ DTaP, 0.1 percentage points lower for 1+ MMR, 1.6 percentage 

points lower for HepB birth dose, and 0.2 percentage point higher for the combined 7-vaccine series. 

None of these differences are statistically significant. For the TSE model, we base our estimates of μμ̂2AA

and on these differences by taking (b) – (a) as the difference in estimates between a full-response 

dataset and the respondent set and then using our estimate of pp̂2AA above to derive μμ̂2AA as an estimate 

of the vaccination coverage among nonrespondents.

When applying final weights, the NIS-Child estimates are 0.4 percentage higher than the estimate 

based on NHIS imputations for 4+ DTaP, 0.6 percentage points higher for 1+ MMR, 0.2 percentage 

points lower for HepB birth dose, and 1.4 percentage points lower for the combined 7-vaccine series. 

Once again, none of these differences are statistically significant. Overall, when viewing the estimates 

based on NHIS imputations as the full-response estimate of vaccination coverage, these results 

indicate that nonresponse error in these four NIS-Child estimates is modest, although the standard 

errors reflect uncertainty in our knowledge about the extent of nonresponse error.

One caveat is that the results depend on the fit of the models used for imputation and the assumption 

that the conditional distributions of the NHIS case and NIS-Child case vaccination statuses on the 
Error Profile for the 2022 NIS-Child 32 

model variables are the same. The goodness-of-fit for the imputation models is not particularly strong 

(pseudo-RR2 ≤ 0.10), resulting in fairly large standard errors for estimates based on NHIS imputations. 

This fact further illuminates the extent of uncertainty in our estimates of nonresponse error.

NIS-Child Item Nonresponse Rates 

Thus far, this section has focused on assessing error in vaccination coverage rates due to unit 

nonresponse to NIS-Child. Here, we present item nonresponse rates for the household interview 

portion of the survey in Table 3.6, focusing on socio-demographic variables used in raking procedures 

for survey weighting. The item nonresponse rates in this table for variables used in raking, all variables 

except income, are low and less than five percent. This indicates low risk for impact on vaccination 

coverage estimates. We also include exact family income which has a higher item nonresponse rate of 

22.9%, although the majority of exact-income nonrespondents completed the follow-up cascade of 

income questions, which establish tight income bounds.

Table 3.6: Item Nonresponse Rates, NIS-Child Household Interview, United States, 2022

Variable

2022 Item 

Nonresponse 

Rate (Percent)*

Sex of child 0.5

Hispanic ethnicity of child 0.7

Race of child 4.6

Education of mother 0.9

Household phone status – landline-only, cell-phone-only, or landline- and cell-phone 0.4

Family income 22.9

Exact income not reported but income cascade completed 15.7

Exact income not reported and income cascade not completed 7.2

* Unweighted percent of "don't know" or "refused" responses among respondents asked the question. For race 

of child, percent also includes "other" responses that could not be back-coded into one or more of the race 

categories presented in the questionnaire. Rates presented in this table exclude U.S. territories.

3.3 Measurement Error

In this subsection, we assess provider under-reporting of the child’s vaccination status. Throughout, we 

assume that if a provider reported a vaccination, it was given. We consider a child to have underreported vaccination status if the child is truly up-to-date for the vaccine but the child is classified as not

up-to-date based on the vaccination history reported by the child’s provider(s). That is, children with 

under-reporting are up-to-date but are reported as not up-to-date; children without under-reporting are 

either both truly up-to-date and reported as up-to-date, or are truly not up-to-date and are reported as 

not up-to-date. Note that all children with under-reporting are, by definition, truly up-to-date. 
Error Profile for the 2022 NIS-Child 33

To assess under-reporting in provider-reported vaccination histories, we rely on projects sponsored by 

CDC in which the 2017 and 2019 NIS-Child samples of children in selected states were matched to the 

state or local IIS. For each of these projects, the NIS-Child interview requested parental consent to 

contact both the child’s vaccination providers and the local IIS. Children for whom consent was 

obtained were matched to their respective IIS databases. Then, for the set of matched children, we 

compared each child’s vaccination status based on the provider report(s) to the child’s vaccination 

status when both the provider(s) and the IIS reports are included in a combined vaccination history. 

We take the combined history to offer the best available information about the child’s true vaccination 

status, and we view the NIS-Child provider-reported history to be possibly subject to an under-reporting 

mechanism. This mechanism, often called under-ascertainment, can arise if some but not all of the 

child’s providers were nominated by the household respondent, if the nominated provider’s contact 

information was reported incorrectly by the household respondent, if not all nominated providers 

responded to the mailed Provider Record Check, or if respondent providers did not have complete 

vaccination records (such as when a child moved out-of-area and vaccination records were not 

forwarded).

From these studies, we estimated the proportion of children with under-reported vaccination status. For 

each given vaccine, we determined the subset of matched children for whom measured vaccination 

status (i.e., up-to-date or not up-to-date) from the combined (provider and IIS) vaccination history was 

equivalent to the vaccination status from the NIS-Child provider-reported vaccination history alone.

Then we made the reasonable assumption that equivalence of the measured vaccination statuses is a 

sign of accurate reporting in the NIS-Child Provider Record Check. In other words, if the IIS did not add 

information about vaccination status beyond that already embodied within the NIS-Child providerreported data, then we took the NIS-Child data to be accurate (not under-reported). If the child was upto-date based on the combined (provider and IIS) vaccination history but not up-to-date based on the 

NIS-Child vaccination history alone, then we classified the child as having their vaccination status 

under-reported in the NIS-Child. Of the children with adequate NIS-Child provider data that were 

located in the IIS and had two or more doses in the IIS, we estimated the NIS-Child under-reporting 

rate for the vaccine as the design-weighted proportion classified as having under-reported vaccination 

status for the vaccine. 

Recent sources of information for assessing under-reporting in the NIS-Child are the match projects 

completed in 2017 and 2019. In 2017, match projects were conducted in 21 jurisdictions: Arkansas, 

Connecticut, Georgia, Idaho, Iowa, Louisiana, Maine, Michigan, Mississippi, Nevada, New Mexico, New 

York City, North Carolina, North Dakota, Oklahoma, Rhode Island, South Dakota, Vermont, 

Washington, Wisconsin, and Wyoming. In 2019, match projects were conducted in 7 jurisdictions: 

Arkansas, Kansas, Louisiana, Missouri, Nevada, New York City, and Vermont. Because only a subset 

of jurisdictions participated in match projects, we estimated the standard error of the under-reporting 

rate by treating each selected jurisdiction as a cluster sampled from the population of jurisdictions.

Table 3.7 presents the estimated under-reporting error for the vaccines and vaccine series under study.
Error Profile for the 2022 NIS-Child 34 

Table 3.7: Estimated Under-Reporting Error by Vaccine Series: NIS-Child, United States, 2017 and 

2019

Under-Reporting Error

Vaccine Estimate 

(percentage points)

Standard Error

(percentage points)

Combined 7-vaccine series 8.3 0.57 

≥4 DTaP 4.7 0.38 

≥3 Polio 2.1 0.25 

≥1 MMR 2.7 0.28 

Hib-FS 5.0 0.46 

≥3 HepB* 2.8 0.31 

HepB birth dose 3.3 0.31 

≥1 VRC at or after age 12 months 2.5 0.27 

≥4 PCV 3.7 0.35 

Note: National-level under-reporting in NIS-Child provider-reported vaccination status was estimated using data 

from the 2017 and 2019 IIS-NIS Match Projects. Among children with adequate provider data found in the IIS 

database with two or more IIS doses, those classified as up-to-date based on the combined IIS-NIS vaccination 

history but not up-to-date based on the NIS-Child vaccination history alone were considered to have underreported NIS-Child vaccination status.

*Estimates for ≥3 HepB do not incorporate data from North Dakota, as data were not available. 

3.4 Sampling Error

Sampling error is the error arising since we do not observe the entire population, only a random sample 

of the population. Table 3.8 presents estimated vaccination coverage and their standard errors for 2022

NIS-Child, using the Taylor series method. The standard errors are calculated first for the designweighted vaccination coverage estimate and then for the final-weighted vaccination coverage estimate. 

The design weights reflect the sample design but do not include adjustments for noncoverage, 

nonresponse, or calibration to population control totals. Final weights are the design weights adjusted

for noncoverage, nonresponse, and calibration to population control totals.

The national-level standard errors are estimated to be small, ranging from approximately 0.4 to about 

0.7 percentage points for final-weighted estimates.
Error Profile for the 2022 NIS-Child 35 

Table 3.8: Vaccination Coverage Estimates and Standard Errors Using Design Weights and Final 

Weights: NIS-Child, United States, 2022

Vaccine

Design Weighted Final Weighted

Estimate 

(%)

Standard 

Error 

(percentage 

points) 

Estimate 

(%)

Standard 

Error 

(percentage 

points) 

7-Vaccine Series 73.7 0.84 72.1 0.70

4+ DTaP 84.3 0.84 84.5 0.59

3+ Polio 92.0 0.77 93.5 0.41

1+ MMR 92.2 0.58 93.0 0.38

Hib-FS 81.6 0.79 81.3 0.61

3+ HepB 91.5 0.72 92.9 0.39

HepB birth dose 80.2 0.82 81.6 0.58

1+ VRC at or after age 12 months 91.8 0.58 92.6 0.39

4+ PCV 84.7 0.79 83.8 0.60

Note: Excludes U.S. territory samples.

3.5 Total Survey Error Distribution

This subsection consolidates the component assessments of sampling-frame coverage error (Section 

3.1), nonresponse error (Section 3.2), measurement error (Section 3.3), and sampling error (Section 

3.4), resulting in an assessment of the total survey error in the 2022 NIS-Child estimated vaccination 

coverage. We focus the overall assessment on four estimates of vaccination coverage corresponding to 

4+DTaP, 1+MMR, HepB birth dose, and the combined 7-vaccine series. The subsection culminates 

with the presentation of total survey error distributions, constructed using the methodology described in 

Molinari, Wolter, Skalland, et al. (2011) and Wolter, Pineau, Skalland, et al. (2017). For each estimate, 

we review the distribution of total survey error across 10,000 Monte Carlo simulations, treating the 

mean of the distribution as the point estimate of total error and the interval between the 2.5th percentile 

and the 97.5th percentile as the 95% credible interval of total error. 

At the beginning of Part II of this report, we presented our TSE model and its seven parameters. Table 

3.9 contains the values of these seven parameters and their standard errors we used in the model for 

TSE in the 2022 NIS-Child. These values arise from the analyses described in Sections 3.2 through 3.4 

above.21 We assume the logit transformations of the inputs are normally distributed and independent 

(i.e., no covariance between inputs).

21 μμ̂1AA and μμ̂2AA, which were estimated based on the NHIS-PRC and models built from NIS-Child vaccination data, respectively, 

have been adjusted upwards to account for provider under-reporting error in those surveys, assuming the same level of underreporting error as was estimated in Section 3.4.
Error Profile for the 2022 NIS-Child 36 

Table 3.9: Total Survey Error Model Inputs by Stages: NIS-Child, United States, 2022

Parameter 4+ DTaP 1+ MMR HepB Birth Dose 7-Vaccine

Series

Stage 1: Sampling-Frame Coverage Error (LLO and Phoneless Households)

2.2% 2.2% 2.2% 2.2% 

0.5% 0.5% 0.5% 0.5% 

80.0% 90.7% 79.7% 70.3% 

7.3% 4.9% 5.2% 7.6% 

Stage 2: Nonresponse Error

97.2% 97.2% 97.2% 97.2% 

0.5% 0.5% 0.5% 0.5% 

88.8% 94.9% 85.2% 81.8% 

1.7% 1.3% 1.8% 2.2% 

Stage 3: Measurement Error

4.7% 2.7% 3.4% 8.3% 

0.4% 0.3% 0.3% 0.6%

100.0% 100.0% 100.0% 100.0%

0.0% 0.0% 0.0% 0.0%

88.5% 94.8% 83.0% 80.4% 

0.9% 0.7% 0.9% 1.0%

Table 3.10 presents the total survey error distribution and component error distributions based on 

10,000 Monte Carlo draws from the input parameter distributions and application of the TSE model. 

The means of the estimated TSE distributions are -4.0, -1.7, -3.3, and -9.2 percentage points for 4+ 

DTaP, 1+ MMR, HepB birth dose, and the combined 7-vaccine series, respectively, suggesting that the 

2022 NIS-Child may have underestimated the true vaccination coverage. The largest estimated 

component of error in absolute value for all four vaccine series is measurement error, i.e., provider 

under-reporting error. 

Table 3.10: Mean and 95% Credible Interval for the Estimated TSE Distribution and Component Error 

Distributions: National Immunization Survey-Child, United States, 2022

Vaccine or Series Component 

Mean TSE

(percentage 

points) 

95% Credible Interval

(percentage points)

4+ DTaP TSE (final weighted) -4.0 (-6.9, -0.6)*

TSE (design weighted) -4.2 (-7.1, -0.8)*

Noncoverage error 0.2 (-0.1, 0.7)

Nonresponse error 0.2 (-3.2, 4.1)

Measurement error -4.7 (-6.3, -2.8)*

Sampling error 0.0 (-1.7, 2.0)
Error Profile for the 2022 NIS-Child 37 

Vaccine or Series Component 

Mean TSE

(percentage 

points) 

95% Credible Interval

(percentage points)

1+ MMR TSE (final weighted) -1.7 (-3.8, 1.3)

TSE (design weighted) -2.4 (-4.5, 0.6)

Noncoverage error 0.1 (-0.1, 0.4)

Nonresponse error 0.1 (-2.4, 3.3)

Measurement error -2.6 (-3.8, -1.3)*

Sampling error 0.1 (-1.2, 1.5)

HepB Birth Dose TSE (final weighted) -3.3 (-6.4, 0.3)

TSE (design weighted) -4.6 (-7.8, -1.1)*

Noncoverage error 0.1 (-0.1, 0.4)

Nonresponse error -1.5 (-5.1, 2.5)

Measurement error -3.3 (-4.9, -1.6)*

Sampling error 0.0 (-1.7, 1.8)

Combined 7-vaccine series TSE (final weighted) -9.2 (-13.1, -4.8)*

TSE (design weighted) -7.7 (-11.5, -3.2)*

Noncoverage error 0.3 (0.0, 0.7)

Nonresponse error 0.3 (-4.1, 5.1)

Measurement error -8.3 (-10.1, -6.3)*

Sampling error 0.0 (-2.0, 2.2)

* 95% credible interval does not include 0.
Error Profile for the 2022 NIS-Child 38

3.6 Assessment of the Change in Bias Using the Bridging 

Cohort Method

The previous section assessed TSE in the 2022 NIS-Child estimated vaccination coverage, while the 

current section assesses the change in TSE between 2021 and 2022. Change is measured using the 

bridging cohort method introduced by Yankey et al. (2015). Each survey quarter includes children born 

in 20 monthly birth cohorts. Every pair of adjacent survey quarters spans 23 monthly birth cohorts, of 

which 17 are in common and 6 are not in common. In turn, every survey year represents 29 monthly 

birth cohorts. Every pair of adjacent survey years spans 39 monthly birth cohorts, of which 17 are in 

common and 22 are not in common. We shall call the 17 common months the bridging cohort, and for 

2021 and 2022, the bridging cohort extends from children born in January 2019 through children born in 

May 2020. 

Consider a vaccination coverage estimate for the bridging cohort as of a given child age, such as 19 

months or 24 months. Two estimates are possible, one using the sample of children in the bridging 

cohort within the 2021 NIS-Child sample and the second using the corresponding sample of children 

within the 2022 NIS-Child sample. Ideally, the two estimators should exhibit the same expected value. 

A large difference between the two estimates may signal a change in the expectation from one survey 

year to the next, which could result from a change in the distribution of sampling-frame coverage error, 

nonresponse error, or measurement error. Differences may also result simply from the effects of 

random sampling error.

Table 3.11 presents the two estimates of vaccination coverage for children as of 19 months of age for 

the 2021-2022 bridging cohort. Estimates and standard errors are presented for several vaccines and 

vaccine series and for the proportion of unvaccinated children in the population. The columns on the 

right side of the table reveal the differences between the 2022 and 2021 estimates for the bridging 

cohort, the estimated standard errors of the differences, and the p-values associated with statistical 

tests of the hypothesis that the expectations of the two estimators are the same. For example, for the 

4+ DTaP vaccination coverage estimate by 19 months, the difference is -0.2 percentage points with a 

standard error of 1.3; given the p-value of 0.87, the hypothesis of no change in expectation is not 

rejected. Summarizing, we do not observe any statistically significant differences between the 2022 and 

2021 vaccination coverage estimates for the 2021-2022 bridging cohort. Overall, the results do not 

provide evidence of a change in the expectation of total survey error between 2021 and 2022. 
Error Profile for the 2022 NIS-Child 39 

Table 3.11: Difference between the Estimates* for the Bridging Birth Cohort† by Age 19 Months: NISChild, United States, 2021 vs. 2022

 2021 2022 Difference

Description Est 

Std 

Error Est 

Std 

Error Est 

Std 

Error 

pp-value for 

Test of No 

Difference

3+ DTaP/DTP/DT by 19 months 93.5 0.46 93.1 0.59 -0.4 0.75 0.596

4+ DTaP/DTP/DT by 19 months 72.5 0.94 72.3 0.95 -0.2 1.34 0.873

3+ Polio by 19 months 92.5 0.49 91.7 0.63 -0.8 0.80 0.321

1+ MMR by 19 months 88.7 0.66 89.8 0.62 1.1 0.91 0.220

3+ Hib by 19 months 90.7 0.56 90.1 0.64 -0.7 0.86 0.445

Hib-PS by 19 months 93.1 0.49 93.0 0.57 -0.1 0.75 0.903

Hib-FS by 19 months 74.2 0.91 74.4 0.92 0.2 1.30 0.885

1+ Varicella by 19 months, excluding shots before 

12 months 88.0 0.67 88.4 0.68 0.3 0.96 0.718

3+ Hepatitis B by 19 months 92.1 0.48 91.4 0.62 -0.7 0.78 0.377

3+ Pneumococcal by 19 months 92.3 0.54 92.2 0.63 -0.1 0.83 0.873

4+ Pneumococcal by 19 months 80.0 0.85 80.1 0.86 0.2 1.21 0.883

1+ Hepatitis A by 19 months 84.3 0.74 83.9 0.79 -0.4 1.08 0.729

2+ Hepatitis A by 19 months 29.9 0.96 28.1 0.88 -1.9 1.30 0.148

2+ or 3+ Rotavirus depending on type by 19 months 78.8 0.85 77.6 0.89 -1.2 1.23 0.331

7-series by 19 months 62.2 1.02 62.0 1.02 -0.1 1.44 0.920

1+ Hepatitis B-containing on day of birth or on day 

1, 2 or 3 following birth 82.5 0.78 82.9 0.71 0.5 1.05 0.660

Unvaccinated children 0.78 0.13 0.75 0.11 0.0 0.17 0.871

2+ Flu by 19 months, doses at least 24 days apart 61.6 1.01 60.9 1.01 -0.7 1.42 0.611

* National-level estimates computed among children with adequate provider data, excluding children from U.S. territories.

† The bridging birth cohort used for this analysis of the 2021 and 2022 NIS-Child includes children born between 

January 2019 and May 2020.

4. Summary 

We profiled the sources of error in 2022 NIS-Child statistics at the national level (excluding U.S. 

territories) for children aged 19 to 35 months. We compared NIS-Child statistics to corresponding 

values from benchmark surveys and other external sources (Part I) and assessed component and total 

error in vaccination coverage estimates through a series of specialized evaluation studies (Part II).

Wherever possible, we used 2022 sources and studies to assess error in the 2022 NIS-Child. Where 

2022 sources were not available, we reported information from prior year sources as the best 

information available for understanding error in the 2022 NIS-Child.
Error Profile for the 2022 NIS-Child 40

In Part I, we compared NIS-Child demographic distributions (child’s age, child’s sex, mother’s race and 

ethnicity, mother’s education, mother’s age) to benchmark values derived from natality data supplied by 

the National Vital Statistics System. When using design weights that have not been calibrated to 

external population totals, demographic distributions as estimated in the survey are generally close to 

the population distributions. Before calibration of the weights to external population totals, the NIS-Child 

somewhat over-represented children whose mothers are college graduates, non-Hispanic White, or age 

30 or greater. The survey somewhat under-represented children whose mothers are not college 

graduates, are Hispanic or non-Hispanic Black, or age 20 to 29. When using the final weights that have 

been calibrated to external population totals, the differences between survey and population narrowed, 

but the 2022 NIS-Child still over-represented children whose mothers are college graduates or are age 

30 or older.

We compared NIS-Child vaccination coverage estimates to IISAR vaccination coverage estimates and 

generally found that NIS-Child estimates are higher. For the six 2013-2017 sentinel sites, we found 

good agreement between NIS-Child and IISAR estimates. Further, we determined that the child 

participation rate is a reasonable indicator of the quality of the corresponding IIS database. We learned 

that the difference between NIS-Child and IISAR vaccination coverage declines as the child 

participation rate increases (i.e., as the quality of the IIS increases). The findings are consistent with the 

hypothesis that IIS vaccination coverage estimates converge towards NIS-Child vaccination coverage 

estimates as the quality of the IIS increases.

We compared NIS-Child health insurance distributions to similar distributions produced by the ACS, 

CPS, and NHIS. The three surveys use somewhat different definitions of insurance status and different 

age ranges of children. Nevertheless, we found the three distributions to be broadly similar, but with 

some differences. The NIS-Child estimate of percent of children with any public insurance was higher 

than the corresponding estimates from ACS, CPS, and NHIS, and the NIS-Child estimate of the percent 

of children uninsured was lower than the estimates from the benchmark surveys.

We compared NIS-Child vaccination coverage estimates at the state level to corresponding estimates 

obtained from eight state immunization surveys. Some aspects of NIS-Child and state survey practices 

are hard to compare and limit the importance of the results. That said, we believe it is safe to conclude 

that there is reasonable agreement between NIS-Child and state vaccination coverage estimates in 

many of the states. Where NIS-Child and state results were comparable, we found similar vaccination 

coverage estimates. For state surveys that rely on IIS vaccination histories, we found NIS-Child 

estimates were generally higher than the corresponding states’ estimates. While this work did not 

provide specific measures of error in NIS-Child statistics, we believe it did provide general support for 

the accuracy of the vaccination coverage estimates.

In Part II of the report, we assessed NIS-Child with respect to sampling-frame coverage error, 

nonresponse error, measurement error, sampling error, and total survey error. We also assessed the 

change in total survey error from 2021 to 2022. 
Error Profile for the 2022 NIS-Child 41

As the NIS-Child cell-phone RDD sampling frame fails to include the landline only and phoneless 

populations, we assessed vaccination coverage estimates in the former using data collected in the 

2017 NIS-Child and in the latter using data collected in the 2012 NHIS Provider Record Check. The 

vaccination coverage estimates in the population covered by the sampling frame were found to be 

higher than the vaccination coverage estimates in the uncovered population. Because the samplingframe uncovered population is so small relative to the covered population, however, we found mean 

sampling-frame coverage error to be minimal, namely 0.3 percentage points or less, for all nine

vaccines series studied.

We used the 2021 and 2022 NHIS in assessing nonresponse error in the 2022 NIS-Child. The NHIS 

does not offer direct estimates of vaccination coverage. Instead, we used a model-based technique to 

multiply impute NHIS vaccination status, and then compared the resulting NHIS vaccination coverage 

estimates (treated as estimates void of nonresponse error) to NIS-Child vaccination coverage 

estimates, with the difference treated as nonresponse error in the NIS-Child. Incorporating all sources 

of missing data, including (1) nonresolution of telephone numbers, (2) nonresponse to the screener, 

(3) failure to complete the interview, (4) non-consent to contact providers, and (5) nonresponse from 

providers, we estimated that over 90% of the sample failed to respond to the NIS-Child. Despite this 

large percentage, we found mean nonresponse error in vaccination coverage estimates to be modest

and not statistically significant for all of the vaccine series examined. 

We used 28 IIS-NIS match studies from 2017 and 2019 to assess measurement error, or underascertainment, in NIS-Child vaccination coverage estimates. In this work, the standard of truth for a 

given child is taken to be the synthesis of the NIS-Child and IIS vaccination histories. We found 

measurement error depressed observed vaccination coverage estimates by about two to eight

percentage points. Under-ascertainment of a child’s vaccination history may arise due to the failure of 

the household respondent to nominate all of the child’s vaccination providers, failure of the nominated 

vaccination providers to respond, or failure of the responding providers to report all of the vaccinations 

that the child has received.

We combined the component errors and assessed the distribution of total error in the NIS-Child 

vaccination coverage estimates using a Monte Carlo technique. For the 4+DTaP vaccination coverage 

estimate, we found the mean of the total survey error distribution to be -4.0 percentage points with a 

95% credible interval of (-6.9, -0.6) percentage points. That is, the NIS-Child vaccination coverage 

estimate was on average about 4.0 percentage points too low. For the 1+MMR vaccination coverage 

estimate, we found the mean of the TSE distribution to be -1.7 percentage points with a 95% credible 

interval of (-3.8, -1.3) percentage points. For the estimate of HepB birth dose vaccination coverage, the 

mean of the TSE distribution was -3.3 percentage points while the 95% credible interval of (-6.4, 0.3) 

percentage points includes 0. Finally, for the combined 7-vaccine series, we found the mean of the TSE 

distribution to be -9.2 percentage points with a 95% credible interval of (-13.1, -4.8) percentage points. 

Again, under-ascertainment of the provider-reported vaccination history dominated total survey error. 

Estimates of nonresponse error have wider 95% credible intervals because those estimates have 

greater uncertainty than estimates of other error components.
Error Profile for the 2022 NIS-Child 42

Finally, we conducted a bridging cohort analysis to assess the possibility of a change in expected value

between the 2021 and 2022 NIS-Child by comparing vaccination coverage estimates between the two 

survey years for children born during a 19-month period included for both survey years. Among 

eighteen vaccine series examined, we found no statistically significant differences and no evidence of a 

change in mean TSE between 2021 and 2022.

Our results for the 2022 NIS-Child are subject to various limitations. The comparisons to benchmark 

distributions in Part I are flawed because the benchmark source usually uses somewhat different 

concepts or definitions than the NIS-Child. Our comparison of NIS-Child and IISAR vaccination 

coverage estimates is limited to the 7-vaccine series, and the findings may not apply to other vaccine 

series. In Part II, the results are based on input distributions for the component errors as estimated 

using our best available information from external sources and studies, but these inputs may not be 

accurate. While large-sample theory motivates our choice of the normal family of distributions, we have 

not validated this choice. Two key external sources of the information on component errors are the 

NHIS and state IIS. The NHIS is based on a smaller sample size than the NIS-Child, its NHIS Provider 

Record Check (used in the study of sampling-frame coverage error) is likely subject to many of the 

same measurement issues as the NIS-Child Provider Record Check, and it is subject to its own 

nonresponse and sampling-frame coverage errors. To study nonresponse error in the NIS-Child, we 

utilized imputed vaccination statuses in the NHIS rather than provider-reported statuses, because the 

NHIS Provider Record Check was terminated in 2013. IIS may underestimate vaccination coverage to 

some extent (e.g., to miss some resident children and some vaccine doses within included children), 

and completeness may vary substantially from one state or local area to the next. Our results are based 

on work with IIS in a subset of states that have conducted IIS-NIS match projects. Our results are also 

based on an assumption of independence of the component errors and this assumption might not be 

accurate. We conducted the TSE analysis for selected national-level vaccination coverage estimates, 

and the results do not necessarily extend to other vaccines, states or estimation areas, or sociodemographic domains.
Error Profile for the 2022 NIS-Child 43 

References

Biemer, P. P. (2010). Total survey error: Design, implementation, and evaluation. Public Opinion 

Quarterly, 74(5), 817-848.

Lackmore, C. (2022). Results of the Survey of Immunization Levels in 2-Year-Old Children (2020). 

http://www.floridahealth.gov/programs-andservices/immunization/resources/surveys/_documents/2yo2020.pdf

Blumberg, S.J., Ganesh, N., Luke J.V., and Gonzales G. (2013). Wireless substitution: State-level 

estimates from the National Health Interview Survey, 2012. National health statistics reports; no 70. 

Hyattsville, MD: National Center for Health Statistics. Available from: 

https://www. cdc.gov/nchs/data/nhsr/nhsr070.pdf

CDC (2022). National Health Interview Survey: 2021 Survey Description. Division of Health Interview 

Statistics, National Center for Health Statistics, U.S. Department of Health and Human Services. 

https://ftp.cdc.gov/pub/Health_Statistics/NCHS/Dataset_Documentation/NHIS/2021/srvydesc-508.pdf

CDC (2023a). National Health Interview Survey: 2022 Survey Description. Division of Health Interview 

Statistics, National Center for Health Statistics, U.S. Department of Health and Human Services. 

https://ftp.cdc.gov/pub/Health_Statistics/NCHS/Dataset_Documentation/NHIS/2022/srvydesc-508.pdf

CDC (2023b). National Immunization Survey-Teen: Error profile for the 2022 NIS-Teen. National 

Center for Immunization and Respiratory Diseases, Centers for Disease Control and Prevention, 

Atlanta, GA. https://www.cdc.gov/vaccines/imz-managers/coverage/teenvaxview/downloads/errorprofile-2022-nis-teen.pdf

Georgia Department of Public Health (n.d.). GA immunization report for children 19-35 Months, 2022. 

https://immunizationstudyreports.s3.amazonaws.com/Child+Report/Child_Report_2022.html#summary

Geslois, J. (2022). Retrospective Vaccination Coverage Survey 2014-2015 results (School year 2018-

2019). Kansas Department of Health and Environment, Topeka, KS. 

https://www.kdheks.gov/immunize/download/Retrospective_2017-2018.pdf

Michigan Department of Health & Human Services (n.d.). County immunization report cards. 

https://www.michigan.gov/mdhhs/adult-childserv/childrenfamilies/immunization/localhealthdepartment/county-immunization-report-card 
Error Profile for the 2022 NIS-Child 44 

Molinari, N.M., Wolter, K., Skalland, B., Montgomery, R., Khare, M., Smith, P.J., Barron, M.L., 

Copeland, K., Santos, K., and Singleton, J.A. (2011). Quantifying bias in a health survey: Modeling total 

survey error in the National Immunization Survey. Statistics in Medicine, 30, 505-514.

National Institute of Statistical Sciences. (2009). NCES/NISS task force on nonresponse bias analysis: 

Final report.

Rubin, D. B. (1976). Inference and missing data. Biometrika, 63(3), 581-592.

Skalland, B. (2011). An alternative to the response rate for measuring a survey’s realization of the 

target population. Public Opinion Quarterly, 75(1), 89-98.

Tennessee Department of Health (2022). Results of the 2022 Immunization Status Survey of 24-monthold children in Tennessee. https://www.tn.gov/content/dam/tn/health/documents/cedepweeklyreports/2022-24-Month-Old-Survey.pdf

U.S. Census Bureau. (2019). 2017 National Survey of Children’s Health: Nonresponse Bias Analysis.

Virginia Department of Health. (n.d.). Virginia Immunization Survey, SY 2021 Single antigen coverage 

rates (%) at 24 Months of age for PubKG, PrivKG, combined KG & DC. 

https://www.vdh.virginia.gov/content/uploads/sites/11/2016/04/VAISAgRates.pdf

Washington State Department of Health (n.d.). Immunization measures by county dashboard. 

https://doh.wa.gov/data-and-statistical-reports/washington-tracking-network-wtn/immunizationdata/county-public-health-measures-dashboard

Wisconsin Department of Health Services (n.d.). Vaccination coverage among Wisconsin children by 

age 24 months, by vaccine or series, region of residence and year.

https://www.dhs.wisconsin.gov/publications/p02003a.pdf

Wolter, K.M., Pineau, V.J., Skalland, B., Zeng, W., Singleton, J.A., Khare, M., Zhao, Z., Yankey, D., and 

Smith, P.J. (2017). Total survey error assessment for sociodemographic subgroups in the 2012 U.S. 

National Immunization Survey. In Total Survey Error in Practice, edited by Biemer, P.P., de Leeuw, E., 

Eckman, S., Edwards, B., Kreuter, F., Lyberg, L.E., Tucker, N.C., and West, B.T. John Wiley & Sons, 

Hoboken, New Jersey.

Wolter, K. M., Smith, P.J., Khare, M, Welch, B., Copeland, K. R., Pineau, V. J., and Davis, N. (2017). 

Statistical methodology of the National Immunization Survey, 2005-2014. Vital and Health Statistics 

1(61). National Center for Health Statistics, Hyattsville, MD.

Yankey, D., Hill, H.A., Elam-Evans, L.D., Khare, M., Singleton, J.A., Pineau, V., and Wolter, K. (2015). 

Estimating change in telephone survey bias in an era of declining response rates and transition to 

wireless telephones – evidence from the National Immunization Survey (NIS), 1995-2013. Presented at 

the annual conference of the American Association for Public Opinion Research, Hollywood, FL.