Department of Health and Human Services 


OFFICE OF 

INSPECTOR GENERAL 

SCHIP: STATES’ PROGRESS IN 


REDUCING THE NUMBER OF 


UNINSURED CHILDREN


Inspector General 

August 2004 

OEI-05-03-00280 
Office of Inspector General 
 http://oig.hhs.gov 

The mission of the Office of Inspector General (OIG), as mandated by Public Law 95-452, 

as amended, is to protect the integrity of the Department of Health and Human Services 

(HHS) programs, as well as the health and welfare of beneficiaries served by those 

programs. This statutory mission is carried out through a nationwide network of audits, 

investigations, and inspections conducted by the following operating components: 

Office of Audit Services 

The OIG's Office of Audit Services (OAS) provides all auditing services for HHS, either by 

conducting audits with its own audit resources or by overseeing audit work done by others. 

Audits examine the performance of HHS programs and/or its grantees and contractors 

in carrying out their respective responsibilities and are intended to provide independent 

assessments of HHS programs and operations in order to reduce waste, abuse, and 

mismanagement and to promote economy and efficiency throughout the department. 

Office of Evaluation and Inspections 

The OIG's Office of Evaluation and Inspections (OEI) conducts short-term management 

and program evaluations (called inspections) that focus on issues of concern to the 

department, the Congress, and the public. The findings and recommendations contained 

in the inspections reports generate rapid, accurate, and up-to-date information on the 

efficiency, vulnerability, and effectiveness of departmental programs. The OEI also 

oversees State Medicaid fraud control units, which investigate and prosecute fraud and 

patient abuse in the Medicaid program. 

Office of Investiga ions t

The OIG's Office of Investigations (OI) conducts criminal, civil, and administrative 

investigations of allegations of wrongdoing in HHS programs or to HHS beneficiaries 

and of unjust enrichment by providers. The investigative efforts of OI lead to criminal 

convictions, administrative sanctions, or civil monetary penalties. 

Office of Counsel to the Inspector General 

The Office of Counsel to the Inspector General (OCIG) provides general legal services to 

OIG, rendering advice and opinions on HHS programs and operations and providing all 

legal support in OIG's internal operations. The OCIG imposes program exclusions and 

civil monetary penalties on health care providers and litigates those actions within the 

department. The OCIG also represents OIG in the global settlement of cases arising 

under the Civil False Claims Act, develops and monitors corporate integrity agreements, 

develops model compliance plans, renders advisory opinions on OIG sanctions to the 

health care community, and issues fraud alerts and other industry guidance. 
 ABSTRACT 

The Balanced Budget Refinement Act of 1999 requires that every 

3 years the Office of Inspector General assess the progress made by 

States in reducing the number of uninsured low-income children. This 

is our second inspection to fulfill this mandate. 

As of June 1, 2003, 46 States submitted State Children’s Health 

Insurance Program (SCHIP) 2002 Annual Reports. Of these, 44 

provided some response to the Centers for Medicare & Medicaid 

Services’s (CMS) requirement that all States describe their progress in 

reducing the number of uninsured low-income children in their Annual 

Reports. However, only 22 of these States directly addressed the CMS 

regulation to report a change in the number of uninsured children in the 

State. Of these 22 States, 17 reported a reduction in the number of 

uninsured children, 3 reported an increase, and 2 reported no change. 

National data indicate that the rate of uninsured children nationally 

has declined. 

Instead of measuring changes in insurance among children, 19 other 

States responded to this requirement by reporting on SCHIP 

enrollment. Three States reported on something other than the number 

of uninsured children or SCHIP enrollment. Two States submitted 

Annual Reports that did not address their progress in reducing the 

number of uninsured low-income children. 

We recommend that CMS resolve the inconsistency between the 

requirement that States report on changes in the number of uninsured 

children and the practice of accepting enrollment data as a proxy. We 

also recommend that CMS ensure the integrity, validity, and usefulness 

of the SCHIP Annual Report and SCHIP enrollment data. 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS i 
 EXECUTIVE SUMMARY 

OBJECTIVE 

3⁄4 To fulfill the congressional mandate under the State Children’s 

Health Insurance Program (SCHIP) to assess States’ progress in 

reducing the number of uninsured, low-income children 

3⁄4 To describe States’ self-assessment methods 

3⁄4 To review the Centers for Medicare & Medicaid Services’s (CMS) 

oversight of States’ assessments 

BACKGROUND 

Enacted in 1997, SCHIP made approximately $40 billion in Federal 

matching funds available to States over 10 years to provide coverage to 

uninsured children below 200 percent of the Federal poverty level. 

The Balanced Budget Refinement Act of 1999 requires that every 

3 years the Office of Inspector General (OIG) (1) evaluate whether 

States are enrolling Medicaid eligible children in SCHIP, and (2) assess 

the progress made by States in reducing the number of uninsured lowincome children, including their progress in meeting the strategic 

objectives and performance goals included in the State child health 

plan. This report addresses the second mandate. The first mandate 

will be addressed in a separate report. This is the second time OIG has 

conducted these inspections to fulfill the mandate. 

Congress also mandated and CMS regulated that States annually 

assess and report on their progress in reducing the number of uninsured 

low-income children. 

To assess States’ progress in reducing the number of uninsured lowincome children, we reviewed the fiscal year (FY) 2002 Annual Reports 

submitted by 46 States. We determined that a State directly addressed 

CMS’s regulations if the State reported on a change, or lack of change, 

in the number and/or rate of uninsured children measured at two or 

more points in time. In addition, we conducted case studies of six States 

and consulted national data sources to further inform Congress on 

national progress in reducing the rate of uninsured children. 

FINDINGS 

In FY 2002, 22 States described changes in the number of uninsured 

children, and 17 of these States reported a decline. Forty-six States 

submitted FY 2002 SCHIP Annual Reports to CMS. These reports are 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS ii 
EXECUTIVE SUMMARY 

required to provide assessments of State’s progress in reducing the 

number of uninsured low-income children. However, only 22 of these 

States directly addressed CMS’s requirement to provide an estimate of 

changes in the number of uninsured low-income children. The other 

24 States did not report sufficient information to determine whether the 

number of uninsured children had changed. 

Of the 22 States that provided data on change in the number of 

uninsured children, 17 States reported a decrease, 3 reported an 

increase, and 2 reported no change in the number of uninsured children. 

Several national data sources show a reduction in the national number 

of uninsured children. 

In response to CMS’s requirement, States provided Current 

Population Survey, State survey, and enrollment data. The 22 States 

that measured changes in the number of uninsured children relied on 

the Census Bureau’s Current Population Survey (CPS) or State survey 

data. 

Instead of providing data on change in the number of uninsured 

children, 19 States provided enrollment data in response to the 

requirement to describe their progress in reducing the number of lowincome uninsured children. There are several reasons why an increase 

in enrollment does not directly correspond to a decrease in the number 

of uninsured. Some children who enroll in SCHIP had a previous 

source of insurance, and so their SCHIP enrollment does not change the 

number of uninsured children. For example, some children transition to 

SCHIP from Medicaid. Likewise, a small percentage of SCHIP enrollees 

may have had prior private health coverage, despite SCHIP provisions 

to minimize shifts from private to public insurance. Further, external 

factors that result in loss of private coverage, such as a rise in 

unemployment, could cause uninsurance rates for children to increase 

despite increases in SCHIP enrollment. 

Three additional States provided responses that measured something 

other than insurance or enrollment, and two States submitted Annual 

Reports that did not provide any response to CMS’s requirement. 

States’ challenges to addressing their progress to reduce the number of 

low-income uninsured children include limitations associated with CPS 

estimates and costs of conducting a State survey. CPS limitations 

include small sample sizes for some States and underestimates of 

enrollment in public programs. Congress has funded annual 

improvements to the CPS through FY 2006. 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS iii 
EXECUTIVE S EXECUTIVE SU UMMAR MMARY Y 

CMS has not ensured that States describe changes in the number of 

uninsured children. As the Federal administrator of SCHIP, CMS’s 

responsibilities include holding States accountable for directly 

addressing SCHIP regulations. CMS has not ensured that States report 

data on changes in the number of uninsured low-income children, as 

evidenced by the 28 of 50 States that did not directly address this 

requirement in 2002. 

Though CMS regulations require States to provide an annual estimate 

of changes in the number of uninsured in the State, in practice, CMS 

accepts enrollment data as a proxy measure of States’ progress in 

reducing the number of uninsured children. CMS recognizes that 

enrollment data does not measure changes in insurance coverage but 

reports that it accepts an enrollment proxy because of challenges States 

face in assessing their progress in reducing the number of uninsured 

children. 

RECOMMENDATIONS 

CMS should resolve the inconsistency between the requirement that 

States report on changes in the number of uninsured children and 

CMS’s practice of accepting enrollment data as a proxy. We present 

two options for resolving this inconsistency. 

Option 1: Enforce exisiting regulations that require States to report 

changes in the number of uninsured children. 

Option 2: Broaden the requirements for States’ reporting of progress 

toward reducing the number of uninsured children to include changes in 

SCHIP enrollment as an acceptable measure of this progress. 

CMS should ensure the integrity, validity, and usefulness of the 

SCHIP Annual Report and SCHIP enrollment data. Toward this end, 

regardless of the option chosen above, CMS should work with States in 

the following three specific areas. 

(1) CMS should encourage States to obtain the most precise and 

reliable State-level estimates of number of children with health 

insurance. 

(2) CMS should improve its oversight to ensure that States submit 

relevant and timely information on progress toward reducing the 

number of uninsured children. 

(3) CMS should ensure the accuracy of SCHIP enrollment data. 

O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS iv 
EXECUTIVE SUMMARY

AGENCY COMMENTS 

CMS provided comments on the draft report. CMS agreed with several 

of our recommendations and described steps the agency has 

implemented to improve the integrity of the States’ SCHIP Annual 

Reports. The complete text of CMS’s comments can be found in 

Appendix B. 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS v 
 T A B L E O F C O N T E N T S 

ABSTRACT ....................................................... i 

EXECUTIVE SUMMARY ........................................ ii 

INTRODUCTION ................................................ 1 

FINDINGS ....................................................... 7 

Insurance Trends .................................................................................7 

 States’ Data Sources ............................................................................9 

 CMS Oversight...................................................................................14 

RECOMMENDATIONS ......................................... 18

AGENCY COMMENTS ......................................... 22

APPENDICES 

A : SUMMARY OF STATE RESPONSES ..................... 24 

B : CMS COMMENTS ........................................... 25 

ACKNOWLEDGMENTS ......................................... 30 

END NOTES .................................................... 31
 INTRODUCTION 

OBJECTIVE 

3⁄4 To fulfill the congressional mandate under the State Children’s 

Health Insurance Program (SCHIP) to assess States’ progress in 

reducing the number of uninsured, low-income children 

3⁄4 To describe States’ self-assessment methods 

3⁄4 To review the Centers for Medicare & Medicaid Services’ (CMS) 

oversight of States’ assessments 

BACKGROUND 

Congressional Mandates to Evaluate SCHIP 

Office of Inspector General (OIG). Section 703 of the Balanced Budget 

Refinement Act (BBRA) of 1999 requires that every 3 years OIG 

(1) evaluate whether States are enrolling Medicaid eligible children in 

SCHIP, and (2) assess the progress made by States in reducing the 

number of uninsured low-income children, including their progress in 

meeting the strategic objectives and performance goals included in the 

State child health plan (State plan). BBRA requires OIG to conduct the 

assessment in a sample of States that administer SCHIP separately 

from their Medicaid programs. 

Government Accountability Office (GAO). BBRA also requires GAO to 

monitor OIG evaluations. In response to OIG’s 2001 evaluation, 

Assessment of State Evaluations Reports (OEI-05-00-00240), GAO 

released a March 2002 report entitled Inspector General Reviews 

Should be Expanded to Further Inform Cong s res . The GAO report 

found that increasing the number of States under review and including 

those that administer Medicaid expansion programs and MedicaidSCHIP combination programs would better inform Congress. Further, 

GAO suggested that OIG identify and evaluate States that use more 

rigorous methods to measure their progress and to review other sources 

of information that assist States in improving their evaluations. 

OIG concurred with GAO’s recommendation to expand the scope of 

future reviews of SCHIP. In addition, OIG agreed to consider other 

data sources to assess progress, while continuing to focus on what 

States currently do to measure their program performance. 

States. Through section 2108(a) of the Social Security Act (the Act), 

Congress also requires States to annually assess their own SCHIP 

progress, including the progress made in reducing the number of 

uninsured low-income children, and report to the Secretary by 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 1 
INTROD UCTION 

January 1 on the results of their assessment.1 Section 2107(a) of the 

Act requires that each SCHIP State plan include a description of the 

program’s strategic objectives and performance goals, including those 

related to increasing health insurance coverage for low-income children, 

and methods for assessing progress toward these goals.2 

CMS also issued regulations implementing this mandate of Section 

2108(a) of the Act. Title 42 C.F.R. § 457.750(b)(1) requires that, in their 

Annual Reports, States must “describe the State’s progress in reducing 

the number of uncovered, low-income children.” The regulations also 

outline States’ options for estimating the number of uninsured, lowincome children, using the Bureau of the Census’s Current Population 

Survey (CPS), State surveys, or “another appropriate source.” 

Whichever source of data is chosen, according to 

42 C.F.R. § 457.750(c)(2), “the State must provide an annual estimate of 

changes in the number of uninsured in the State.” 

The State Children’s Health Insurance Program: Overview 

Enacted in 1997 as Title XXI of the Act, SCHIP legislation made 

approximately $40 billion in Federal matching funds available to States 

over a 10-year period to provide coverage to uninsured children below 

200 percent of the Federal poverty level.3 The program’s goal is to 

expand coverage to uninsured children with incomes that exceed the 

States’ Medicaid eligibility levels, but remain too low to purchase 

private health insurance coverage.4 Within Federal guidelines, States 

set eligibility standards for the program; determine the type, amount, 

duration, and scope of services offered; and set payment rates for 

providers.5 

SCHIP is a State and Federal partnership in which all States and the 

District of Columbia have approved State plans, and 49 States and the 

District of Columbia have enrolled children (herein referred to as the 

50 States).a Unlike Medicaid, SCHIP is not an entitlement program, 

and States have the discretion to cap enrollment, create waiting lists of 

eligible children, or not offer a SCHIP at all. Title XXI requires States 

with separate programs to screen SCHIP applicants for Medicaid 

eligibility before enrolling them in SCHIP. 

States have three options for covering uninsured children under 

Title XXI. They can (1) institute a separate children’s health insurance 

a Tennessee does not have any children enrolled in its Title XXI program. 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 2 
INTRODUCTION 

program, (2) expand Medicaid eligibility, or (3) institute both a separate 

SCHIP and a Medicaid expansion, known as a combination program. 

In fiscal year (FY) 2002, 5,315,229 children were enrolled in SCHIP 

under Title XXI.6 

The Current Population Survey 

The Current Population Survey (CPS), sponsored by the Bureau of the 

Census and the Bureau of Labor Statistics, is a primary source of data 

on the number of uninsured children. CPS’s main purpose is to collect 

labor force statistics, but it also collects data on health insurance 

coverage through the Annual Demographic Supplement survey 

conducted each March. 

CPS provides a consistent source of longitudinal data on the number 

and rate of uninsured children each year.7 CPS is the only source of 

both national and State-level estimates of the number and rate of 

uninsured children for all 50 States. The State-level CPS estimates are 

used to determine each State’s annual SCHIP-funding allotment. 

Some States have concerns about the precision and reliability of CPS 

data due to small State sample sizes. In 1999, Congress responded in 

Public Law 106-113 to these concerns by allocating $10 million annually 

through FY 2006 to the Bureau of the Census to 

“make appropriate adjustments to the annual Current Population 

Survey...in order to produce statistically reliable annual State 

data on the number of low-income children who do not have health 

insurance coverage, so that real changes in the uninsured rates of 

children can reasonably be detected.”8 

Adjustments include increasing sample sizes to improve estimates of 

the number of uninsured low-income children. 

Other National Data on Insurance Rates for Children 

The National Health Interview Survey conducted annually by the 

Centers for Disease Control and Prevention (CDC) collects nationallevel data on a broad range of health topics, including the health 

insurance status for children. 

The Behavioral Risk Factor Surveillance System conducted annually by 

CDC collects State-level health insurance data on adults. States may 

choose to fund additional State specific questions on the health 

insurance status of children. 

The Urban Institute, a non-profit organization, conducts the National 

Survey on America’s Families to collect information on a variety of 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 3 
INTROD UCTION 

issues, including children’s current health insurance status and SCHIP 

enrollment. This survey provides national data and State-level data for 

13 States. Surveys were conducted in 1997, 1999, and 2002. 

State Fiscal Constraints and Impact on SCHIP 

According to The Fiscal Survey of States, conducted by the National 

Governors Association and the National Association of State Budget 

Officers, States continue to face budgetary pressures. This report found 

that “budget gaps are lingering as spending pressures persist, 

particularly from Medicaid and other health care.”9 States have 

curtailed spending, and FY 2004 expenditures are expected to rise only 

by 0.2 percent over FY 2003, representing the smallest nominal increase 

since 1979.10 In FY 2004, 13 States enacted decreased budgets.11 

As States encounter difficulty producing required Federal matching 

funds for Title XXI programs, some States have elected to cap SCHIP 

enrollment, increase premiums, increase the rigor of the enrollment 

process, and lower the upper limit for income eligibility to adjust to the 

increasing budget deficits.12 As of November 2003, six States had 

frozen their SCHIP enrollment.13 

Related Work by the Office of Inspector General 

In February 2001, OIG released two separate reports on SCHIP to fulfill 

the congressional mandate to OIG. In Ensuring Medicaid Eligibles are 

not Enrolled in SCHIP, (OEI-05-00-00241), OIG found that between 97 

and 99 percent of SCHIP participants were correctly enrolled in SCHIP 

in FY 1999. In A ent of State E aluations R p rts ssessm v e o ,

(OEI-05-00-00240), OIG found that questionable evaluations by the 

States undermine the reliability of reported success in reducing the 

number of uninsured children. OIG also found that State evaluations 

have conceptual and technical weaknesses. 

SCOPE 

This evaluation addresses the second directive of the BBRA mandate, to 

assess States’ progress in reducing the number of uninsured low-income 

children. As required by Congress, OIG is concurrently conducting a 

separate evaluation to determine if Medicaid-eligible children are being 

incorrectly enrolled in SCHIP. 

To assess State progress, this report evaluates States’ assessments of 

their progress toward reducing the number of uninsured children. We 

also assessed the methods States use to determine this progress, as 

described in their FY 2002 SCHIP Annual Reports. In addition, we 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 4 
INTRODUCTION 

provide national level data on changes in the rate of uninsured children. 

We included all States and the District of Columbia (herein referred to 

as a “State”) that submitted FY 2002 Annual Reports by June 1, 2003. 

Annual Reports were due by January 1, 2003. Connecticut, Hawaii, 

Minnesota, and Nevada had not submitted reports by June 1, 2003, and 

Tennessee was exempt from the Annual Report requirement because 

the State had no children enrolled in its SCHIP program. Therefore, we 

reviewed Annual Reports from 46 States. 

We analyzed States’ progress and methods to the extent that sufficient 

data and information was provided in their Annual Reports. We did not 

conduct an independent evaluation of the reported outcomes of 

individual States’ performance objectives. 

METHODOLOGY 

To assess States’ progress in reducing the number of uninsured lowincome children, we reviewed the FY 2002 Annual Reports submitted by 

46 States. From these reports we determined whether States directly 

addressed CMS’s regulations to describe the State’s progress toward 

reducing the number of uninsured low-income children by providing an 

annual estimate of changes in the number of uninsured in the State. 

We determined that a State directly addressed CMS’s requirement if 

the State reported on a change, or lack of change, in the number and/or 

rate of uninsured children measured at two or more points in time. 

Though Congress and CMS-specified low-income children, we 

considered States that provided information on progress toward 

reducing the number of all uninsured children as having addressed 

CMS’s requirement. Data limitations prevented more detailed analyses, 

including comparisons across States. 

We also undertook in-depth analyses and conducted site visits in six 

case study States: California, Florida, Massachusetts, North Carolina, 

Rhode Island, and Utah. These States were identified by CMS and 

SCHIP stakeholders as having notable evaluation expertise or 

experience. Diversity in size, geography, program administration, and 

measurement methodologies were additional criteria for selection. 

These States encompass all three types of SCHIP program design 

(i.e., separate child health program, SCHIP Medicaid expansion, and 

combination). These States are not representative of the universe of 

States. 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 5 
INTRODUCTION 

In addition to their Annual Reports, we further reviewed the methods 

by which the six case study States evaluated their outcomes through 

interviews with their SCHIP staff, researchers, and advocates. 

We interviewed CMS staff to gain further understanding of how they 

oversee States’ Annual Reports and their role in monitoring SCHIP 

progress toward reducing the number of uninsured children. 

We also obtained information on the national trends in insurance 

coverage for children from (1) the CPS, (2) the National Health 

Interview Survey, and (3) the Urban Institute’s National Survey of 

America’s Families. 

This inspection was conducted in accordance with the Quality 

Standar s for Inspecti d ons issued by the President’s Council on Integrity 

and Efficiency. 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 6 
 FINDINGS

While 44 of the 46 States provided 

some response to CMS in their 

In FY 2002, 22 States described changes in the 

number of uninsured children, and 17 of these 

2002 Annual Reports, only 22 of States reported a decline. 

these States directly addressed 

CMS’s regulations requiring States to describe progress in reducing the 

number of uninsured low-income children by estimating changes in the 

number of uninsured in the State.b We determined that a State 

directly addressed CMS regulations if the State reported a change, or 

lack of change, in the number and/or rate of uninsured children 

measured at two or more points in time. For additional information on 

States’ responses, see Appendix A. 

Seventeen of these twenty-two States reported a reduction in the number or 

rate of uninsured children. 

Of the 22 States that directly addressed CMS’s requirement to measure 

insurance coverage changes, 17 reported a decrease in the number of 

uninsured children. Three States reported an increase in the number of 

uninsured children and two States measured no change. Table 1 below 

summarizes these States’ results. 

Table 1. State-Reported Changes in Insurance for Children 

States reporting a decrease in the number of uninsured children 17 

States reporting an increase in the number of uninsured children 3 

States measuring no change in the number of uninsured children 2 

Total State responses 22 

Source: States’ SCHIP Annual Reports to CMS, FY 2002 

Of the 17 States measuring a decrease in uninsured children, 5 

specifically reported a reduction in the number of uninsured children 

over time; 9 States reported a decrease in the rate of uninsured 

children; and 3 States reported reductions in both the number and rate 

of uninsured children. 

State-reported reductions in the number of uninsured children range 

from a 5 to 37 percent decline. For example, 1 State’s estimate of 

uninsured children declined from 311,000 to 197,000 children, a 

decrease of 37 percent. State-reported changes in insurance rates 

ranged from 1.5 percentage points (representing an 18 percent 

b Though Congress and CMS specified “low-income” children, we include States that 

provided information on progress toward reducing the number of all uninsured children. 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 7 
FINDING S 

reduction in the rate of uninsured children) to 6.3 percentage points 

(representing a 66 percent reduction). For example, one State reported 

its estimated rate of uninsured children dropped from 9.5 percent in 

1996 to 3.2 percent in 2001. This represents a 66 percent reduction in 

the rate for uninsured children. 

State-reported progress in reducing the number of uninsured children is 

not comparable across these 17 States. States measure change over 

different periods of time and for different populations. For example, 

three States reported change over a 1-year period, while other States 

compared current estimates of the number of uninsured children to 

estimates from up to 5 years earlier. Some States also provided 

estimates from more than two points in time. 

Further, 12 of the 17 States reported changes in the number of 

uninsured children overall, while 5 reported specifically on children in 

certain income ranges (e.g., children up to 200 percent of poverty). Only 

2 of the 17 States provided information on the statistical significance of 

their reported decreases in the number of uninsured children. 

Five of the twenty-two States that directly addressed CMS requirements 

reported an increase or no change in the number of uninsured children. 

Specifically, three States reported an increase in the number of 

uninsured children, and two States reported no change. Two of these 

five States attributed their lack of measurable insurance progress to 

factors related to their measurement of the number of uninsured 

children. Specifically, one State’s estimate of the number of uninsured 

children decreased, but this change was not statistically significant, so 

we categorized this State as measuring no change. Another State 

reported that their increase in the number of uninsured children was 

due, in part, to a change in the weighting of its latest CPS estimates. 

National data show a decline in the number of uninsured children. 

To further inform Congress on the national progress toward reducing 

the number of uninsured children, we consulted several data sources on 

national insurance coverage trends for children. These estimates are 

not directly comparable to one another because they provide data on 

different populations (e.g., all children versus low-income children) or 

different periods of time. However, all three sources provide evidence 

that the national rate of uninsured children has declined. 

Specifically, CPS data demonstrate a decline in the rate of uninsured 

children nationally.14 In 1998, CPS estimated that 15.4 percent of all 

children under 18 years old lacked health coverage. By 2002, this 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 8 
FINDINGS 

estimate dropped to 11.6 percent, which represents a 25 percent 

reductionc in the rate of uninsured children.15 

The National Health Interview Survey, a survey administered by CDC 

which collects health data from a random sample of individuals 

nationwide, shows a statistically significant reduction in the rate of 

uninsured children from 1997 to the first half of 2003. In 1997, this 

survey estimated that nationally, 13.9 percent of all children lacked 

health insurance at the time of the survey. By 2003, this rate had 

dropped to 9.4 percent.16 

The Urban Institute found that the rate of uninsured low-income 

children below 200 percent of the poverty level fell from 22.5 percent in 

1999 to 16.8 percent in 2002. 17 This decrease was associated with 

increased Medicaid and SCHIP coverage.18 

To Respond to CMS’s requirement, States use 

CPS, State Survey, and Enrollment Data. The 22 States that directly 

addressed CMS’s requirement 

relied on the CPS or State survey data to measure a change in the 

number of uninsured children over time. Instead of measuring SCHIP 

progress in reducing the number of uninsured children, 19 additional 

States provided enrollment data in response to CMS’s requirement. 

Three additional States provided responses that measured something 

other than children’s insurance or enrollment progress, and two States 

submitted Annual Reports that did not provide any response to this 

requirement. Table 2 summarizes States’ responses from their 2002 

Annual Reports. 

Table 2. States’ Annual Report Assessments of Changes in the Number of Uninsured Children 

States whose responses measured changes in the number of uninsured children 22 

States whose responses measured SCHIP enrollment 19 

States whose responses measured something other than children’s insurance or enrollment 3 

States whose Annual Reports did not provide any response to CMS’s requirement 2 

Total 46 

 Source: States’ SCHIP Annual Reports to CMS, FY 2002 

c The CPS revised its survey question during this time, which may affect the magnitude of this change. 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 9 
FINDING S 

Of the 22 States that directly addressed CMS’s requirement, 12 relied on 

CPS data. 

Twelve States, including one case-study State detailed below, used CPS 

data to measure the outcome of their efforts to reduce the number or 

rate of uninsured children in their CMS Annual Report. Annually, CPS 

provides a free, easily accessible source of longitudinal data on the 

number and rate of uninsured children. CPS provides State-level 

insurance estimates for all children and specific estimates for children 

below 200 percent of the poverty level. For many States, CPS is the 

only data source available for estimating changes in the number of 

uninsured children.19 

All States, as well as Congress, CMS, OIG, and the public, have access 

to annual, State-level CPS estimates of uninsured low-income children. 

However, some States face limitations by relying solely on CPS data for 

State-level estimates of uninsured children.20 For States with smaller 

populations, CPS sample sizes are relatively small.21 The smaller the 

sample size, the less precise the estimate. These estimates may not 

detect incremental changes in insurance rates over time. For example, 

one State noted that its CPS estimates showed a reduction in the 

number of uninsured children, but this reduction was not statistically 

significant because the State’s CPS sample size was too small. Another 

State reported that its total CPS-estimated eligible population was less 

than its current Medicaid and SCHIP enrollment. 

Beyond the concerns about sample size, CPS is an employment survey 

that includes health insurance questions at the end of the survey. Some 

researchers believe that a survey focused on health issues is more likely 

to elicit accurate responses about health insurance status.22 Further, 

CPS asks respondents in March about their health insurance coverage 

at any time during the past calendar year, rather than their current 

insurance status.23 Researchers expressed concern that CPS 

respondents may erroneously report on current insurance status.24 

Finally, CPS data tend to underestimate the number of children 

enrolled in Medicaid, and these children may be incorrectly counted as 

uninsured.25 One State noted that its CPS estimate of SCHIP coverage 

was 83 percent lower than the State’s actual SCHIP coverage. 

Rhode Island. Rhode Island finds CPS data valuable to track State 

health insurance trends and progress in reducing the number of 

uninsured children, despite its limitations. Though State SCHIP staff 

acknowledged the imprecision of CPS point estimates, they expressed 

confidence in the overall direction of change indicated by CPS data. The 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 10 
FINDING S 

exact number of uninsured children is less important to the State than 

knowing whether the rate of uninsured children is declining. 

To confirm CPS insurance trends for children, State staff consult 

several additional data sources. Rhode Island added a question on 

children’s insurance status for its State sample in the CDCadministered Behavioral Risk Factor Surveillance System. The State 

also collects health insurance status data through a State household 

survey usually conducted every 5 years. Finally, the State consults 

private coverage data from BlueCross BlueShield, which accounts for 69 

percent of the State’s health insurance market, and data on the use of 

uncompensated care provided by safety net providers. 

The other 10 States that directly addressed CMS’s requirement used State 

surveys. 

Ten States conducted statewide surveys that enabled them to measure 

and report on change in the number or rate of uninsured children over 

time. By developing their own surveys States are able to tailor their 

questions to overcome some of the limitations associated with CPS 

insurance rate estimates and address State-specific concerns. Three of 

these ten States were included in our case studies, and their efforts are 

detailed below. 

Florida. While Florida SCHIP staff expressed confidence in their CPS 

insurance estimates, the State sought more detailed information on its 

uninsured population. State staff believed that estimates of the rate of 

uninsured children for different regions and demographic groups would 

help Florida better understand and more effectively address uninsured 

subpopulations. Florida fielded telephone surveys in 1998 and 2002 

designed to produce estimates of insurance specific to 17 regions and 

various demographic groups. The surveys also sought to determine if 

uninsured children were likely to be eligible for Medicaid or SCHIP. 

Massachusetts. According to Massachusetts’s staff, their State survey 

has several advantages over CPS for estimating the State’s rate of 

uninsured children, including a larger sample size which allows a more 

precise estimate. And, unlike CPS, this survey focuses exclusively on 

health insurance. Further, Massachusetts tested its survey questions 

extensively using cognitive interviews and focus groups to ensure that 

the survey would yield valid and reliable results. 

Massachusetts’s survey results support its contention that CPS 

overestimates its number of uninsured children. In 2002 the State 

survey estimated that 3 percent of all children were uninsured, while 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 11 
FINDING S 

CPS estimated 7.8 percent. Estimates for Massachusetts from the 

National Survey of America’s Families align more closely with the 

results of the State survey than the CPS for this State. 

Utah. From 1986 to 2001 Utah conducted a telephone survey every 

5 years to collect comprehensive information on children’s health, 

including their health insurance status. In its FY 2002 Annual Report, 

Utah compared its estimated rates of uninsured children from 1996 to 

2001 and found that the rate for uninsured children decreased from 8.5 

percent to 7 percent. However, the State noted a limitation of this 

comparison. In 2001, Utah changed its health insurance questions. The 

State cannot determine how much of the change in it’s estimated 

insurance rates since 1996 should be attributed to the change in the 

survey questions, rather than an actual change in insurance coverage 

during that time. 

In response to CMS’s requirement, 19 additional States reported on 

enrollment instead of progress in reducing uninsured children. 

In their Annual Reports 19 additional States responded to CMS by 

reporting on SCHIP enrollment, without providing information on the 

changes in the number or rate of uninsured children over time. These 

States reported a variety of enrollment information, including 

enrollment at a point in time; changes in enrollment over time; 

comparisons of SCHIP enrollment to an estimate of uninsured children 

at a point in time, known as a penetration rate; or some combination of 

these. Fourteen States reported changes in SCHIP enrollment over 

time. Of these, 13 States reported enrollment increases, and 1 State 

reported enrollment had remained stable. 

While SCHIP enrollment increases provide one measure of program 

success, these numbers do not indicate changes in States’ rates or 

number of uninsured children. Some States interpret increases in 

enrollment, an output, as net gains in health insurance coverage, an 

outcome. However, enrollment increases are not analogous to insurance 

increases for several reasons. Some children transition to SCHIP from 

Medicaid, and so their SCHIP enrollment does not change the number 

of uninsured children. Likewise, a small percentage of SCHIP enrollees 

may have had prior private health coverage, despite SCHIP provisions 

to minimize the shift from private to public sector insurance, which is 

known as “crowd out.” 

Further, external factors may affect the insurance rate for low-income 

children. For example, if a large number of children lose their private 

health coverage due to a rise in unemployment or a rise in private 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 12 
FINDING S 

insurance premiums, the number of uninsured children may increase 

despite concurrent increases in SCHIP enrollment. 

States offered technical and economic reasons for using enrollment data. 

Eleven States, including three States that used enrollment data, cited 

data limitations associated with CPS estimates in their Annual Reports. 

States expressed concerns that CPS produces unreliable and imprecise 

estimates for their States. As mentioned above, this is primarily due to 

small State sample sizes, the fact that CPS is a lengthy labor-related 

survey with health insurance questions at the end, and historical CPS 

undercounts of public health insurance participation. For example, one 

State explained that the CPS estimate of the total number of children 

under 100 percent of poverty in the State was lower than the actual 

number of children under 100 percent of poverty that were enrolled in 

the State’s Medicaid program. Further, the State’s enrollment in 

SCHIP is 128 percent of the CPS baseline estimate of all uninsured 

children in the State. Another State also considered the CPS 

measurement of insurance coverage at any time during the prior year, 

rather than at a point in time, a disadvantage. 

While a State survey could overcome CPS precision and reliability 

concerns, a State survey may be cost-prohibitive. In interviews, SCHIP 

staff from four States pointed to recent economic downturns and fiscal 

constraints to explain why they did not focus on assessing their progress 

in reducing the number of uninsured. Respondents explained that their 

priorities have shifted from reducing the number of uninsured children 

and increasing program enrollment to maintaining current enrollment 

levels and preserving SCHIP benefits. One State’s SCHIP staff 

acknowledged their inability to reliably measure their progress in 

reducing the number of uninsured children but viewed this as a 

necessary tradeoff, stating that they would “rather spend their money to 

cover kids than to count them.” 

North Carolina. North Carolina illustrates these explanations for using 

enrollment data. Their performance objective related to reducing the 

number of uninsured children is “to enroll as many children as possible 

that can be covered within available funds.” This State estimates that 

there are 35,000 more SCHIP-eligible children than the State’s target 

enrollment goal, which is the maximum number of children the budget 

will cover. According to SCHIP staff, they have insufficient funds to 

cover all eligible children, in part, because CPS underestimated their 

number of eligible children, and they have exhausted their Federal 

SCHIP allotment. 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 13 
FINDINGS 

In their Annual Report North Carolina provided CMS with a baseline 

insurance rate derived from an adjustment of CPS data. However, 

neither the State staff nor the researchers who calculated this baseline 

have confidence in the accuracy of this estimate. This lack of confidence 

in the CPS estimate, combined with a lack of funding for a State survey, 

has motivated a continuing reliance on enrollment numbers to assess 

and report success. 

In the Annual Report North Carolina specifies its performance goal as 

its current number of enrollees. In effect, the States' goal is to maintain 

current enrollment in the program, not to reduce the overall insurance 

rate. State staff report that State funding availability and the 

assumption that SCHIP is enrolled to capacity are the factors used to 

determine their SCHIP goal. State SCHIP staff explain that because 

they lack reliable data to demonstrate that there are additional eligible 

but uninsured children, they must set their goals within budgeted 

funds. 

CMS has not ensured that States describe CMS has not held States

accountable for directly changes in the number of uninsured children. 

addressing the CMS requirement 

that States describes their changes in the number of uninsured lowincome children. As detailed above, 24 of the 46 States that submitted 

Annual Reports to CMS did not describe their progress in reducing the 

number of uninsured children. Four additional States had not 

submitted Annual Reports by June 1, 2003, despite a deadline of 

January 1, 2003. 

CMS accepts enrollment data as a proxy measure of SCHIP progress. 

Six years after SCHIP enactment, less than half of States report 

directly on their progress in reducing the number of uninsured as 

required by CMS. In lieu of this direct assessment, CMS accepts and 

uses enrollment data from States as an alternative measure of program 

success. CMS’s staff explained that States’ enrollment numbers provide 

a reliable indicator of States’ SCHIP progress, as long as States 

continue to take measures to prevent “crowd out” (i.e., substitution of 

SCHIP for private health insurance). In addition, CMS consults CPS 

data available from the Census Bureau to track national health 

insurance trends for children as well as State-level CPS data for each 

State. 

There is incongruity between CMS’s requirement that States describe 

their progress in reducing the number of uninsured children and CMS’s 

acceptance of enrollment data from States as a substitute for a measure 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 14 
FINDING S 

of change in the number of uninsured children. Title XXI states that 

the main long-term goal of SCHIP is to expand health assistance to 

uninsured low-income children.26 The mandates that States assess and 

report their progress in reducing the number of uninsured low-income 

children, and that OIG assess States’ progress toward this goal 

demonstrate Congress’s interest in this specific outcome. CMS 

regulations also require States to provide an annual estimate of changes 

in the number of uninsured children.27 

However, CMS sets its SCHIP Government Performance and Results 

Act (GPRA) goal around increasing enrollment. In its assessment of 

SCHIP, the Office of Management and Budget (OMB) noted that CMS’s 

GPRA goals do not measure the impact of SCHIP on the rate of 

uninsured children, and OMB recommended that CMS develop goals to 

measure this impact.28 

CMS reported to us that it sets its GPRA goal around increasing SCHIP 

enrollment because of the difficulties associated with measuring 

changes in insurance rates, particularly in the context of changes in 

economic conditions and private insurance markets. CMS also 

emphasized the limitations of CPS data for State-level insurance 

estimates, particularly for smaller States, and recognized that many 

States cannot afford to field their own State insurance coverage surveys. 

Further, CMS respondents pointed to studies that have shown minimal 

“crowd out” to demonstrate the link between enrollment and insurance 

increases. 

Accuracy of SCHIP enrollment data raises additional concerns. 

CMS emphasized limitations of available health insurance data. 

However, we also found limitations with one of the primary enrollment 

data elements for SCHIP -- the annual number of children “ever 

enrolled” by State for each FY. CMS produces an annual summary of 

the number of children “ever enrolled” in SCHIP by State, based on 

unverified, State-submitted data. The Annual Report template also 

asks States for the number of children “ever enrolled” in SCHIP. For 10 

States, we found discrepancies between the numbers of “ever enrolled” 

children reported in the Annual Reports and those reported in the CMS 

enrollment summary. For instance, between FYs 2001 and 2002, one 

State’s Annual Report showed an increase in enrollment of 7,000 

children, while the CMS summary reported a decrease of 15,000 

children for the same State. CMS described a new data reporting 

system that it has implemented since our review that it expects will 

improve the accuracy of State-reported enrollment data. 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 15 
FINDING S 

In addition, CMS’s enrollment summary double counts children who 

transfer from one SCHIP program to another during the year. This 

duplicate count occurs for children in the 20 States with both Medicaid 

expansion and separate programs (combination States) who move from 

one program to the other within the same State. It also occurs for 

children who move from one State’s to another State’s SCHIP. CMS 

recognized this limitation of its data on “ever enrolled” children. 

However, CMS pointed out that it collects other types of unpublished 

enrollment data, such as quarterly point-in-time enrollment counts, 

that are not subject to this problem. 

CMS has not ensured that States submit relevant, clear, and timely 

information in their Annual Reports. 

The law and CMS regulations require States to submit SCHIP Annual 

Reports to describe their progress toward meeting the strategic 

objectives and performance goals described in their State plans.29 CMS 

regulations for State plans, which reflect Section 2107(a) of the Act, set 

forth that States must specify one or more performance goals for each 

strategic objective and must describe how performance will be measured 

through objective, independently verifiable means and compared 

against performance goals.30 SCHIP law specifically requires States to, 

“assess the operation of the State plan, including the progress made in 

reducing the number of uncovered low-income children.”31 And CMS 

regulations require States to provide an annual estimate of changes in 

the number of uninsured.32 

CMS reports that its project officers are in contact with the States to 

provide technical assistance that States need to complete their Annual 

Reports and to check on States’ progress. CMS also reported that it 

discusses content and deadlines of the Annual Report on monthly 

SCHIP Technical Advisory Group calls. 

Yet, these efforts have not ensured that States establish and report 

strategic objectives, performance goals, and methodologies that are 

relevant to measuring change in the number of uninsured children. For 

example, one State described its performance goal related to reducing 

the number of uninsured children as expanded capacity of the State’s 

data systems and additional field staff training. Another State 

described increased administrative and outreach capacity as its 

performance goals and the hiring of 47 outstationed eligibility workers 

as its progress related to reducing the number of uninsured children. 

While these may be creditable goals for achieving other SCHIP 

objectives, they are not performance goals that lead to measurement of 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 16 
FINDING S 

changes in the number of uninsured children. In comparison, an 

example of a relevant, objective, and measurable performance goal 

related to reducing the number of uninsured children was one State’s 

goal that “the percent of children with creditable coverage for the entire 

year, whose family income is between 150% and 200% of the Federal 

Poverty Level, will be increased from 89.7% in 1998 to 95% in 2003.” 

CMS has not required States to resubmit sections of the Annual Report 

when they provide unclear information or outdated goals. In their 2002 

Annual Reports, seven States submitted outdated goals related to 

reducing the number of uninsured children. For example, one State 

continued to cite a performance goal related to reducing the number of 

uninsured children that the State had already achieved more than 

2 years earlier. 

CMS has not held States responsible for timely State Annual Report 

submissions. Twelve States’ Annual Reports were submitted after the 

January 1, 2003 deadline.d Late submissions ranged from 2 days to 

5 months past this deadline. In addition, four States had not submitted 

2002 Annual Reports at the time of our analysis in June 2003. CMS has 

indicated that those States have since submitted 2002 Annual Reports, 

but these reports were more than 5 months late. 

d Eight additional States’ reports did not include a submission date. 

O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 17 
 R E C O M M E N D A T I O N S 

In response to our congressional mandate, we found that 22 States 

reported a change in the number or rate of uninsured children in their 

FY 2002 Annual Report. The remaining 28 States did not report 

sufficient information to allow us to assess whether they had reduced 

their numbers of uninsured children.e CMS has not ensured that 

States comply with CMS’s requirement that States’ describe changes 

in the number of uninsured children. As such, we offer two options 

for CMS to select in order to increase States’ compliance. In addition, 

we offer a three-prong approach to assist CMS in working with States to 

ensure the integrity, validity, and usefulness of all data submitted by 

States in their Annual Report. 

CMS should resolve the inconsistency between the requirement that States 

report on changes in the number of uninsured children and CMS’s practice 

of accepting enrollment data as a proxy. 

Currently, there is inconsistency between CMS’s requirement that 

States report on changes in the number of uninsured and CMS’s 

acceptance of enrollment data as a proxy for progress toward reducing 

the number of uninsured children. As CMS recognizes, enrollment data 

does not measure changes in insurance coverage. We present two 

options for resolving this inconsistency. 

Option 1: Enforce exisiting regulations that require States to report changes in 

the number of uninsured children. 

As the agency responsible for overseeing SCHIP, CMS should hold 

States accountable to directly address its requirement that States 

describe their progress in reducing the number of uninsured low-income 

children by providing an annual estimate of changes in the number of 

uninsured in the State. If a State’s response does not demonstrate 

change in the number of uninsured children, CMS should follow up with 

that State to provide further clarification and technical assistance as 

needed. 

e Tennessee is excluded from this count because the State has no children enrolled in 

SCHIP. 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 18 
RECOMMENDATIONS 

Option 2: Broaden the requirements for States’ reporting of progress toward 

reducing the number of uninsured children to include changes in SCHIP 

enrollment as an acceptable measure of this progress. 

While CMS acknowledged that enrollment data does not measure 

changes in insurance coverage, CMS provided justifications for 

accepting enrollment data as a proxy measure of change in the number 

of uninsured children. These justifications focused on challenges that 

some States face estimating changes in the number of uninsured 

children. However, this practice is not consistent with current 

requirements for States to report on changes in the number of 

uninsured children. If CMS decides that the challenges for some States 

to obtain information on changes in the number of uninsured children 

outweigh the benefits of the information, then CMS should change the 

regulations to reflect this decision. 

CMS should ensure the integrity, validity, and usefulness of the SCHIP 

Annual Report and SCHIP enrollment data. 

Toward this end, regardless of the option chosen above, CMS should 

work with States in three specific areas (1) encourage States to obtain 

the best estimates available of children’s insurance coverage, (2) ensure 

States provide relevant and timely information on SCHIP progress as 

required, and (3) ensure the accuracy of SCHIP enrollment data. 

CMS should encourage States to obtain the most precise and reliable State-level 

estimates of number of children with health insurance. 

Many States face difficulties obtaining precise and reliable health 

insurance estimates, including concerns about the imprecision and 

unreliability of State-level estimates derived from CPS and the 

costliness of State-initiated surveys. 

Some States use CPS data in some way but do not directly address 

CMS’s requirement to estimate change in the number of uninsured, 

e.g., States that compare enrollment data to a CPS estimate of 

uninsured children at one point in time. For these States, CMS should 

provide technical assistance to SCHIP staff on how to best use CPS data 

to measure progress in reducing the number of uninsured children. 

On behalf of States that have concerns about the reliability of CPS data, 

CMS should continue departmental efforts to collaborate with the 

Bureau of the Census on increasing CPS sample sizes to make this data 

more useful for each State. CMS should also encourage these States to 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 19 
RECOMMENDATIONS 

revisit CPS data to determine whether ongoing CPS sample size 

improvements have alleviated their concerns. 

Further, CMS should work with States that continue to distrust CPS 

estimates to identify or develop alternative data sources. CMS could 

compile and share strategies that other States have used to measure 

their progress in reducing the number of uninsured children. 

Recognizing the costs of a new State survey, CMS can encourage States 

to take advantage of existing surveys and data sources. For example, 

States can minimize costs by adding questions on children’s health 

insurance status to the existing CDC’s Behavioral Risk Factor 

Surveillance Survey. The Urban Institute’s National Survey of 

America’s Families also provides State-level estimates of children’s 

health coverage for 13 States every few years. CMS could encourage 

these 13 States to use this information to assess their progress toward 

reducing the number of uninsured children. 

CMS should improve its oversight to ensure that States submit relevant and timely 

information on progress toward reducing the number of uninsured children. 

While CMS reported having a process for overseeing the quality and 

timeliness of States’ reporting, this process should be evaluated and 

improved because it has not fully resolved deficiencies in States’ Annual 

Reports. Similar to the methods we used for this report, CMS should 

carefully review States’ Annual Reports to ensure that States provide 

clear and timely information on their progress toward reducing the 

number of uninsured children that meets CMS’s requirements. If a 

State’s Annual Report describes performance goals, measures, or 

outcomes that do not describe progress toward reducing the number of 

uninsured children, CMS should follow up with that State for further 

clarification and provide technical assistance as needed. Likewise, CMS 

should provide guidance to States on updating outdated goals to reflect 

current program conditions. CMS should also reinforce the importance 

of submitting the SCHIP Annual Report by the January 1st deadline. 

CMS should ensure the accuracy of SCHIP enrollment data. 

Even beyond its use as a proxy measure for insurance progress, 

enrollment data provide its own important measure of program success. 

CMS should rectify inconsistencies between States’ 2002 enrollment 

data reported in CMS’s enrollment report and data reported in States’ 

Annual Reports by reviewing the two numbers to identify and reconcile 

any discrepancies. CMS has recently implemented a new reporting 

system for States’ enrollment data and anticipates that this system will 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 20 
RECOMMENDATIONS 

alleviate the data concerns cited in our report. CMS should review 2003 

data to verify that this new system ensures data accuracy. 

In addition, CMS should reconsider its methodology which double 

counts children that transfer from one SCHIP program to another 

during the same FY in States with both Medicaid expansion and 

separate SCHIP programs. CMS should devise criteria for categorizing 

such children in a way that produces an unduplicated count of “ever 

enrolled” children, such as counting them in the program in which they 

were enrolled for the greatest duration. CMS should also publish 

additional types of data it collects, such as quarterly point-in-time 

enrollment numbers, to provide a more robust picture of SCHIP 

enrollment. 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 21 
 AGENCY COMMENTS 

CMS provided comments on the draft report. CMS agreed with several 

of our recommendations and described steps the agency has 

implemented to improve the integrity of the States’ SCHIP Annual 

Reports. The complete text of CMS’s comments can be found in 

Appendix B. CMS also provided technical comments on the report. 

CMS provided responses to each of our recommendations. 

Recommendation 1. CMS should resolve the inconsistency between the 

requirement that States report on changes in the number of uninsured 

children and CMS’s practice of accepting enrollment data as a proxy. 

CMS Response. In response to our recommendation to resolve 

inconsistency, CMS stated that they will work with Congress to clarify 

how progress should be measured and reported. CMS also noted that 

their regulations require States to “describe the State’s progress in 

reducing the number of low-income, uncovered children” but do not 

specify a particular methodology for meeting this requirement. 

Further, CMS agreed that they need to work with States to improve 

State measurement efforts. Consistent with our recommendations, 

CMS stated that they are taking steps to enhance technical assistance 

to improve State measurement capabilities, including scheduling a 

technical assistance session at an upcoming national conference. CMS 

also highlighted some of the same difficulties States face in measuring 

changes in the number of uninsured children that we discussed in our 

report. 

Recommendation 2. CMS should ensure the integrity, validity, and 

usefulness of the SCHIP Annual Report and enrollment data. 

CMS Response. CMS stated that, in addition to providing technical 

assistance to States, they are reviewing all State reports on progress 

toward covering the uninsured. Also, CMS has been encouraging States 

to revisit CPS data in light of ongoing improvements to this survey, as 

we recommended. 

Further, CMS has implemented a new, web-based State Annual Report 

Template System, and CMS highlighted this system’s role in 

implementing this recommendation. CMS expects that this system will 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 22 
AGENCY COMMENTS 

assist the agency to more effectively analyze State reports on progress 

toward covering the uninsured. Consistent with our recommendations, 

CMS stated that they are using this new system to more closely review 

the information States provide and to request that States provide more 

clear and accurate responses in their SCHIP Annual Reports. CMS also 

expects the system to help improve CMS’s oversight of the SCHIP 

Annual Report submission process. Additionally, this system will be 

linked to the SCHIP enrollment data that CMS collects and reports, and 

CMS stated that this will eliminate the discrepancies we found between 

enrollment data in the SCHIP Annual Reports and in CMS’s enrollment 

summaries. 

In response to our recommendation that CMS reconsider its 

methodology for “ever enrolled” data which double counts certain 

children who transfer between SCHIP programs, CMS stated that the 

number of double-counted children is limited, and that they do not 

believe this impacts the total enrollment significantly. We suggested 

that CMS also publish the “point-in-time” enrollment data they collect, 

which is not duplicative, but CMS noted that this data is not as reliable 

as “ever enrolled” data. CMS is working with States to improve data 

reliability. 

Throughout their comments, CMS also highlighted national data that 

show significant reductions in the number of uninsured low-income 

children since the implementation of SCHIP and indicate CMS’s and 

States’ success in achieving SCHIP program goals. As we noted in our 

report, national data does provide evidence of success in reducing the 

number of uninsured low-income children, the primary goal of SCHIP. 

We hope that our recommendations and the steps that CMS has 

implemented will contribute to continued national progress toward this 

goal and ensure that each State shows similar success. 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 23 
 APPENDIX ~ A 

States’ Responses to CMS’s Requirement in their 2002 Annual Reports 

States whose responses measured progress in reducing the number of uninsured children 

1 Alabama 12 Montana 

2 Arizona 13 New Hampshire 

3 Delaware 14 New Jersey 

4 Florida 15 New York 

5 Georgia 16 Ohio 

6 Idaho 17 Rhode Island 

7 Kansas 18 South Carolina 

8 Kentucky 19 Utah 

9 Louisiana 20 Washington 

10 Massachusetts 21 West Virginia 

11 Maryland 22 Wisconsin 

Source: States’ SCHIP Fiscal Year 2002 Annual Reports 

States whose responses measured SCHIP enrollment 

1 Alaska 11 North Carolina 

2 Arkansas 12 North Dakota 

3 California 13 Oklahoma 

4 Colorado 14 Pennsylvania 

5 Illinois 15 South Dakota 

6 Iowa 16 Texas 

7 Maine 17 Vermont 

8 Michigan 18 Virginia 

9 Missouri 19 Wyoming 

10 Nebraska 

Source: States’ SCHIP Fiscal Year 2002 Annual Reports 

States whose responses measured something other than children’s insurance or enrollment 

1 Indiana 3 Oregon 

2 Mississippi 

Source: States’ SCHIP Fiscal Year 2002 Annual Reports 

States whose Annual Reports did not include any response to CMS’s requirement 

1 District of Columbia 2 New Mexico 

Source: States’ SCHIP Fiscal Year 2002 Annual Reports 

States that did not submit an Annual Report by June 1, 2003 

1 Connecticut 3 Minnesota 

2 Hawaii 4 Nevada 

Note: Tennessee was not required to submit a 2002 Annual Report because they had no enrolled children in SCHIP. 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 24 
 A P P E N D I X ~ B 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 25 
 OEI-05-03-00280 A S SSESSING TATES ’ P ROGRESS 26
 OEI-05-03-00280 A S SSESSING TATES ’ P ROGRESS 27
 OEI-05-03-00280 A S SSESSING TATES ’ P ROGRESS 28
 OEI-05-03-00280 A S SSESSING TATES ’ P ROGRESS 29
 A C K N O W L E D G M E N T S 

This report was prepared under the direction of William Moran, Regional 

Inspector General for Evaluation and Inspections in the Chicago regional 

office. Other principal Office of Evaluation and Inspections staff who 

contributed include: 

Emily Melnick, Team Leader 

Erin Lemire, Lead Analyst 

Amy Hennessy, Presidential Manag ment Fellow e

Annamarie Lopata, Intern 

Linda Hall, Program Specialist 

Elise Stein, Director, Public Health and Human Services 

 O E I - 0 5 - 0 3 - 0 0 2 8 0 ASSESSING S TATES ’ P ROGRESS 30 
 END NOTES 

1 Social Security Act, Section 2108(a) 

2 Social Security Act, Section 2108(a) and 42 C.F.R. § 457.750(b)(1) 

3 Title XXI of the Social Security Act (State Children’s Health Insurance 

Program (SCHIP) enabling legislation) was created by the Balanced 

Budget Act of 1997 (Public Law 105-33). 

4 Social Security Act, Sections 2101(a) and 2110(b) 

5 Social Security Act, Sections 2102 and 2103 

6 SCHIP Preliminary Annual Enrollment Report for Fiscal Year (FY) 

2002. Posted January 31, 2003. Available online at 

http://www.cms.hhs.gov/schip/enrollment/schip02.pdf. 

7 State Health Access Data Assistance Center. “Issue Brief: The Current 

Population Survey (CPS) and State Health Insurance Coverage 

Estimates.” March 2001. Available online at 

http://www.shadac.org/publications/issuebriefs/IssueBrief1.pdf 

8 Bureau of the Census and the Bureau of Labor Statistics. “Current 

Population Survey - Design and Methodology.” Technical Paper 

63RV, Appendix J. March 2002. Available online at 

http://www.census.gov/prod/2002pubs/tp63rv.pdf 

9 National Governor’s Association and National Association of State 

Budget Officers. “The Fiscal Survey of the States.” December 2003. 

Available online at www.nga.org/cda/files/FSS1203.PDF 

10 Ibid. 

11 Ibid. 

12 Kaiser Commission on Medicaid and the Uninsured. “Out in the Cold: 

Enrollment Freezes in Six State Children’s Health Insurance 

Programs Withhold Coverage from Eligible Children.” 

December 2003. Available online at 

http://www.kff.org/medicaid/4159.cfm 

OEI-05-03-0280 ASSESSING S TATES ’ P ROGRESS 31 
13 Kaiser Commission on Medicaid and the Uninsured. “Out in the Cold: 

Enrollment Freezes in Six State Children’s Health Insurance 

Programs Withhold Coverage from Eligible Children.” 

December 2003. Available online at 

http://www.kff.org/medicaid/4159.cfm. As of November 2003, 

Alabama, Colorado, Florida, Maryland, Montana, and Utah had frozen 

their SCHIP enrollment. 

14 Current Population Survey, March 2002. Health Insurance Table 

H108: “Health Insurance Coverage Status and Type of Coverage by 

Selected Characteristics for Children Under 18: 2001.” 

Available online at 

http://ferret.bls.census.gov/macro/032002/health/h08_000.htm 

15 Current Population Survey, March 1988 - March 2003. Historical 

Health Insurance Tables. Table HI-3: “Health Insurance Coverage 

Status and Type of Coverage--Children Under 18 by Age: 1987 to 

2002.” Available online at 

http://www.census.gov/hhes/hlthins/historic/hihistt3.html 

16 National Center for Health Statistics, Centers for Disease Control and 

Prevention. “Health Insurance Coverage: Estimates from the National 

Health Interview Survey, January – June 2003.” Posted January 

2004. Available online at 

http://www.cdc.gov/nchs/data/nhis/earlyrelease/insur200401.pdf 

17 Urban Institute, “Children’s Insurance Coverage and Service Use 

Improve.” July 31, 2003. Available online at 

http://www.urban.org/url.cfm?ID=310816 

18 Urban Institute, “Children’s Insurance Coverage and Service Use 

Improve.” July 31, 2003. Available online at 

http://www.urban.org/url.cfm?ID=310816 

19 State Health Access Data Assistance Center. “Issue Brief 1: The 

Current Population Survey (CPS) and State Health Insurance 

Coverage Estimates.” March 2001. Available online at 

http://www.shadac.org/publications/issuebriefs/IssueBrief1.pdf 

20 Ibid. 

OEI-05-03-0280 ASSESSING S TATES ’ P ROGRESS 32 
21 State Health Access Data Assistance Center. “Issue Brief 3: State 

Health Insurance Coverage Estimates: Why State-Survey Estimates 

Differ from CPS.” July 2001. Available online at 

http://www.shadac.org/publications/issuebriefs/IssueBrief3.pdf 

22 Office of the Assistant Secretary for Planning and Evaluation (ASPE). 

“Understanding Estimates of the Uninsured: Putting the Differences 

Into Context.” ASPE Research Note. Update June 2000. Available 

online at http://aspe.hhs.gov/health/kids.htm 

23 State Health Access Data Assistance Center. “Issue Brief 3: State 

Health Insurance Coverage Estimates: Why State-Survey Estimates 

Differ from CPS.” July 2001. Available online at 

http://www.shadac.org/publications/issuebriefs/IssueBrief3.pdf 

24 Ibid. 

25 Mathematica Policy Research, Inc. Chapter IX: “State Progress 

Toward Reducing the Number of Uninsured Low-Income Children.” 

Implementation of the State Children’s Health Insurance Program: 

Synthesis of State Evaluations. March 2003. Available online at 

http://www.mathematica-mpr.com/PDFs/impchildhlth.pdf 

26 Social Security Act, Section 2101(a) 

27 42 C.F.R. § 457.750(c)(2) 

28 Office of Management and Budget, Program Assessment Rating Tool. 

“Assessing Program Performance for the FY 2004 Budget.” 2003. 

Available online at 

http://www.whitehouse.gov/omb/budget/fy2004/pma.html 

29 Social Security Act, Section 2108(a)(1) and 42 C.F.R. § 457.750 

30 42 C.F.R. § 457.710 

31 Social Security Act, Section 2108(a)(1) 

32 42 C.F.R. § 457.750(c)(2) 

OEI-05-03-0280 ASSESSING S TATES ’ P ROGRESS 33