Department of Health and Human Services 

OFFICE OF 

INSPECTOR GENERAL 

SCHIP:  S  P
TATES ROGRESS IN 

R N
EDUCING THE  UMBER OF 

U C
NINSURED  HILDREN

Inspector General 
August 2004 
OEI-05-03-00280  Office of Inspector General 

http://oig.hhs.gov 
The mission of the Office of Inspector General (OIG), as mandated by Public Law 95-452, 
as amended, is to protect the integrity of the Department of Health and Human Services 
(HHS) programs, as well as the health and welfare of beneficiaries served by those 
programs. This statutory mission is carried out through a nationwide network of audits, 
investigations, and inspections conducted by the following operating components: 
Office of Audit Services 
The OIG's Office of Audit Services (OAS) provides all auditing services for HHS, either by 
conducting audits with its own audit resources or by overseeing audit work done by others. 
Audits examine the performance of HHS programs and/or its grantees and contractors 
in carrying out their respective responsibilities and are intended to provide independent 
assessments of HHS programs and operations in order to reduce waste, abuse, and 
mismanagement and to promote economy and efficiency throughout the department. 
Office of Evaluation and Inspections 
The OIG's Office of Evaluation and Inspections (OEI) conducts short-term management 
and program evaluations (called inspections) that focus on issues of concern to the 
department, the Congress, and the public. The findings and recommendations contained 
in the inspections reports generate rapid, accurate, and up-to-date information on the 
efficiency, vulnerability, and effectiveness of departmental programs. The OEI also 
oversees State Medicaid fraud control units, which investigate and prosecute fraud and 
patient abuse in the Medicaid program. 
Office of Investigations
The OIG's Office of Investigations (OI) conducts criminal, civil, and administrative 
investigations of allegations of wrongdoing in HHS programs or to HHS beneficiaries 
and of unjust enrichment by providers. The investigative efforts of OI lead to criminal 
convictions, administrative sanctions, or civil monetary penalties. 
Office of Counsel to the Inspector General 
The Office of Counsel to the Inspector General (OCIG) provides general legal services to 
OIG, rendering advice and opinions on HHS programs and operations and providing all 
legal support in OIG's internal operations. The OCIG imposes program exclusions and 
civil monetary penalties on health care providers and litigates those actions within the 
department. The OCIG also represents OIG in the global settlement of cases arising 
under the Civil False Claims Act, develops and monitors corporate integrity agreements, 
develops model compliance plans, renders advisory opinions on OIG sanctions to the 
health care community, and issues fraud alerts and other industry guidance.    A B S T R A C T  

The Balanced Budget Refinement Act of 1999 requires that every 
3 years the Office of Inspector General assess the progress made by 
States in reducing the number of uninsured low-income children.  This 
is our second inspection to fulfill this mandate.  
As of June 1, 2003, 46 States submitted State Childrens Health 
Insurance Program (SCHIP) 2002 Annual Reports.  Of these, 44 
provided some response to the Centers for Medicare & Medicaid 
Servicess (CMS) requirement that all States describe their progress in 
reducing the number of uninsured low-income children in their Annual 
Reports.  However, only 22 of these States directly addressed the CMS 
regulation to report a change in the number of uninsured children in the 
State. Of these 22 States, 17 reported a reduction in the number of 
uninsured children, 3 reported an increase, and 2 reported no change. 
National data indicate that the rate of uninsured children nationally 
has declined. 
Instead of measuring changes in insurance among children, 19 other 
States responded to this requirement by reporting on SCHIP 
enrollment.  Three States reported on something other than the number 
of uninsured children or SCHIP enrollment. Two States submitted 
Annual Reports that did not address their progress in reducing the 
number of uninsured low-income children. 
We recommend that CMS resolve the inconsistency between the 
requirement that States report on changes in the number of uninsured 
children and the practice of accepting enrollment data as a proxy. We 
also recommend that CMS ensure the integrity, validity, and usefulness 
of the SCHIP Annual Report and SCHIP enrollment data.  
  OEI-05-03-00280  ASSESSING STATES PROGRESS  i    E X E C U T I V E   S U M M A R Y  

OBJECTIVE 
(cid:190)  To fulfill the congressional mandate under the State Childrens 
Health Insurance Program (SCHIP) to assess States progress in 
reducing the number of uninsured, low-income children  
(cid:190)  To describe States self-assessment methods 
(cid:190)  To review the Centers for Medicare & Medicaid Servicess (CMS) 
oversight of States assessments 
BACKGROUND 
Enacted in 1997, SCHIP made approximately $40 billion in Federal 
matching funds available to States over 10 years to provide coverage to 
uninsured children below 200 percent of the Federal poverty level.   
The Balanced Budget Refinement Act of 1999 requires that every 
3 years the Office of Inspector General (OIG) (1) evaluate whether 
States are enrolling Medicaid eligible children in SCHIP, and (2) assess 
the progress made by States in reducing the number of uninsured low-
income children, including their progress in meeting the strategic 
objectives and performance goals included in the State child health 
plan. This report addresses the second mandate.  The first mandate 
will be addressed in a separate report.  This is the second time OIG has 
conducted these inspections to fulfill the mandate. 
Congress also mandated and CMS regulated that States annually 
assess and report on their progress in reducing the number of uninsured 
low-income children. 
To assess States progress in reducing the number of uninsured low-
income children, we reviewed the fiscal year (FY) 2002 Annual Reports 
submitted by 46 States.  We determined that a State directly addressed 
CMSs regulations if the State reported on a change, or lack of change, 
in the number and/or rate of uninsured children measured at two or 
more points in time.  In addition, we conducted case studies of six States 
and consulted national data sources to further inform Congress on 
national progress in reducing the rate of uninsured children. 
FINDINGS 
In FY 2002, 22 States described changes in the number of uninsured 
children, and 17 of these States reported a decline.  Forty-six States 
submitted FY 2002 SCHIP Annual Reports to CMS.  These reports are 
  OEI-05-03-00280  ASSESSING STATES PROGRESS  ii  E X E C U T I V E   S U M M A R Y  
required to provide assessments of States progress in reducing the 
number of uninsured low-income children.  However, only 22 of these 
States directly addressed CMSs requirement to provide an estimate of 
changes in the number of uninsured low-income children.  The other   
24 States did not report sufficient information to determine whether the 
number of uninsured children had changed. 
Of the 22 States that provided data on change in the number of 
uninsured children, 17 States reported a decrease, 3 reported an 
increase, and 2 reported no change in the number of uninsured children. 
Several national data sources show a reduction in the national number 
of uninsured children. 
In response to CMSs requirement, States provided Current 
Population Survey, State survey, and enrollment data. The 22 States 
that measured changes in the number of uninsured children relied on 
the Census Bureaus Current Population Survey (CPS) or State survey 
data. 
Instead of providing data on change in the number of uninsured 
children, 19 States provided enrollment data in response to the 
requirement to describe their progress in reducing the number of low-
income uninsured children.  There are several reasons why an increase 
in enrollment does not directly correspond to a decrease in the number 
of uninsured. Some children who enroll in SCHIP had a  previous 
source of insurance, and so their SCHIP enrollment does not change the 
number of uninsured children.  For example, some children transition to 
SCHIP from Medicaid. Likewise, a small percentage of SCHIP enrollees 
may have had prior private health coverage, despite SCHIP provisions 
to minimize shifts from private to public insurance.  Further, external 
factors that result in loss of private coverage, such as a rise in 
unemployment, could cause uninsurance rates for children to increase 
despite increases in SCHIP enrollment.  
Three additional States provided responses that measured something 
other than insurance or enrollment, and two States submitted Annual 
Reports that did not provide any response to CMSs requirement. 
States challenges to addressing their progress to reduce the number of 
low-income uninsured children include limitations associated with CPS 
estimates and costs of conducting a State survey.  CPS limitations 
include small sample sizes for some States and underestimates of 
enrollment in public programs.  Congress has funded annual 
improvements to the CPS through FY 2006. 
  OEI-05-03-00280  ASSESSING STATES PROGRESS  iii  E EX XE EC CU UT TI IV VE E   S SU UM MM MA AR RY Y  
CMS has not ensured that States describe changes in the number of 
uninsured children. As the Federal administrator of SCHIP, CMSs 
responsibilities include holding States accountable for directly 
addressing SCHIP regulations. CMS has not ensured that States report 
data on changes in the number of uninsured low-income children, as 
evidenced by the 28 of 50 States that did not directly address this 
requirement in 2002. 
Though CMS regulations require States to provide an annual estimate 
of changes in the number of uninsured in the State, in practice, CMS 
accepts enrollment data as a proxy measure of States progress in 
reducing the number of uninsured children. CMS recognizes that 
enrollment data does not measure changes in insurance coverage but 
reports that it accepts an enrollment proxy because of challenges States 
face in assessing their progress in reducing the number of uninsured 
children. 
RECOMMENDATIONS 
CMS should resolve the inconsistency between the requirement that 
States report on changes in the number of uninsured children and 
CMSs practice of accepting enrollment data as a proxy. We present 
two options for resolving this inconsistency. 
Option 1: Enforce exisiting regulations that require States to report 
changes in the number of uninsured children. 
Option 2: Broaden the requirements for States reporting of progress 
toward reducing the number of uninsured children to include changes in 
SCHIP enrollment as an acceptable measure of this progress. 
CMS should ensure the integrity, validity, and usefulness of the 
SCHIP Annual Report and SCHIP enrollment data. Toward this end, 
regardless of the option chosen above, CMS should work with States in 
the following three specific areas. 
(1) CMS should encourage States to obtain the most precise and 
reliable State-level estimates of number of children with health 
insurance. 
(2) CMS should improve its oversight to ensure that States submit 
relevant and timely information on progress toward reducing the 
number of uninsured children. 
(3) CMS should ensure the accuracy of SCHIP enrollment data. 
OEI-05-03-00280  ASSESSING STATES PROGRESS  iv  E X E C U T I V E   S U M M A R Y

AGENCY COMMENTS 
CMS provided comments on the draft report.  CMS agreed with several 
of our recommendations and described steps the agency has 
implemented to improve the integrity of the States SCHIP Annual 
Reports.  The complete text of CMSs comments can be found in 
Appendix B. 
  OEI-05-03-00280  ASSESSING STATES PROGRESS  v    T A  B  L E   O F   C O N T E N T S  

ABSTRACT ....................................................... i 

EXECUTIVE SUMMARY ........................................ ii 

INTRODUCTION ................................................ 1 

FINDINGS ....................................................... 7 

Insurance Trends .................................................................................7 

  States Data Sources ............................................................................9 

  CMS Oversight...................................................................................14 

RECOMMENDATIONS ......................................... 18

AGENCY COMMENTS ......................................... 22

APPENDICES 
A: SUMMARY OF STATE RESPONSES ..................... 24 

B: CMS COMMENTS ........................................... 25 

ACKNOWLEDGMENTS ......................................... 30 

END NOTES .................................................... 31
   I N T R O D U C T I O N  

OBJECTIVE 
(cid:190)  To fulfill the congressional mandate under the State Childrens 
Health Insurance Program (SCHIP) to assess States progress in 
reducing the number of uninsured, low-income children 
(cid:190)  To describe States self-assessment methods 
(cid:190)  To review the Centers for Medicare & Medicaid Services (CMS) 
oversight of States assessments 
BACKGROUND 
Congressional Mandates to Evaluate SCHIP 
Office of Inspector General (OIG). Section 703 of the Balanced Budget 
Refinement Act (BBRA) of 1999 requires that every 3 years OIG    
(1) evaluate whether States are enrolling Medicaid eligible children in 
SCHIP, and (2) assess the progress made by States in reducing the 
number of uninsured low-income children, including their progress in 
meeting the strategic objectives and performance goals included in the 
State child health plan (State plan).  BBRA requires OIG to conduct the 
assessment in a sample of States that administer SCHIP separately 
from their Medicaid programs. 
Government Accountability Office (GAO).  BBRA also requires GAO to 
monitor OIG evaluations.  In response to OIGs 2001 evaluation, 
Assessment of State Evaluations Reports (OEI-05-00-00240), GAO 
released a March 2002 report entitled Inspector General Reviews 
Should be Expanded to Further Inform Congress. The GAO report 
found that increasing the number of States under review and including 
those that administer Medicaid expansion programs and Medicaid-
SCHIP combination programs would better inform Congress.  Further, 
GAO suggested that OIG identify and evaluate States that use more 
rigorous methods to measure their progress and to review other sources 
of information that assist States in improving their evaluations. 
OIG concurred with GAOs recommendation to expand the scope of 
future reviews of SCHIP.  In addition, OIG agreed to consider other 
data sources to assess progress, while continuing to focus on what 
States currently do to measure their program performance. 
States. Through section 2108(a) of the Social Security Act (the Act), 
Congress also requires States to annually assess their own SCHIP 
progress, including the progress made in reducing the number of 
uninsured low-income children, and report to the Secretary by  
  OEI-05-03-00280  ASSESSING STATES PROGRESS  1  I N T R O D
 U C T I O N  
January 1 on the results of their assessment.1  Section 2107(a) of the 
Act requires that each SCHIP State plan include a description of the 
programs strategic objectives and performance goals, including those 
related to increasing health insurance coverage for low-income children, 
and methods for assessing progress toward these goals.2 
CMS also issued regulations implementing this mandate of Section 
2108(a) of the Act.  Title 42 C.F.R.  457.750(b)(1) requires that, in their 
Annual Reports, States must describe the States progress in reducing 
the number of uncovered, low-income children.  The regulations also 
outline States options for estimating the number of uninsured, low-
income children, using the Bureau of the Censuss Current Population 
Survey (CPS), State surveys, or another appropriate source. 
Whichever source of data is chosen, according to   
42 C.F.R.  457.750(c)(2), the State must provide an annual estimate of 
changes in the number of uninsured in the State. 
The State Childrens Health Insurance Program:  Overview 
Enacted in 1997 as Title XXI of the Act, SCHIP legislation made 
approximately $40 billion in Federal matching funds available to States 
over a 10-year period to provide coverage to uninsured children below 
200 percent of the Federal poverty level.3   The programs goal is to 
expand coverage to uninsured children with incomes that exceed the 
States Medicaid eligibility levels, but remain too low to purchase 
private health insurance coverage.4  Within Federal guidelines, States 
set eligibility standards for the program; determine the type, amount, 
duration, and scope of services offered; and set payment rates for 
providers.5 
SCHIP is a State and Federal partnership in which all States and the 
District of Columbia have approved State plans, and 49 States and the 
District of Columbia have enrolled children (herein referred to as the 
50 States).a  Unlike Medicaid, SCHIP is not an entitlement program, 
and States have the discretion to cap enrollment, create waiting lists of 
eligible children, or not offer a SCHIP at all.  Title XXI requires States 
with separate programs to screen SCHIP applicants for Medicaid 
eligibility before enrolling them in SCHIP.   
States have three options for covering uninsured children under   
Title XXI. They can (1) institute a separate childrens health insurance 
a Tennessee does not have any children enrolled in its Title XXI program. 
  OEI-05-03-00280  ASSESSING STATES PROGRESS  2  I N T R O D U C T I O N  
program, (2) expand Medicaid eligibility, or (3) institute both a separate 
SCHIP and a Medicaid expansion, known as a combination program.   
In fiscal year (FY) 2002, 5,315,229 children were enrolled in SCHIP 
under Title XXI.6 
The Current Population Survey 
The Current Population Survey (CPS), sponsored by the Bureau of the 
Census and the Bureau of Labor Statistics, is a primary source of data 
on the number of uninsured children.  CPSs main purpose is to collect 
labor force statistics, but it also collects data on health insurance 
coverage through the Annual Demographic Supplement survey 
conducted each March. 
CPS provides a consistent source of longitudinal data on the number 
and rate of uninsured children each year.7   CPS is the only source of 
both national and State-level estimates of the number and rate of 
uninsured children for all 50 States.  The State-level CPS estimates are 
used to determine each States annual SCHIP-funding allotment.   
Some States have concerns about the precision and reliability of CPS 
data due to small State sample sizes.  In 1999, Congress responded in 
Public Law 106-113 to these concerns by allocating $10 million annually 
through FY 2006 to the Bureau of the Census to 
make appropriate adjustments to the annual Current Population 
Survey...in order to produce statistically reliable annual State 
data on the number of low-income children who do not have health 
insurance coverage, so that real changes in the uninsured rates of 
children can reasonably be detected.8 
Adjustments include increasing sample sizes to improve estimates of 
the number of uninsured low-income children. 
Other National Data on Insurance Rates for Children 
The National Health Interview Survey conducted annually by the 
Centers for Disease Control and Prevention (CDC) collects national-
level data on a broad range of health topics, including the health 
insurance status for children. 
The Behavioral Risk Factor Surveillance System conducted annually by 
CDC collects State-level health insurance data on adults.  States may 
choose to fund additional State specific questions on the health 
insurance status of children. 
The Urban Institute, a non-profit organization, conducts the National 
Survey on Americas Families to collect information on a variety of 
  OEI-05-03-00280  ASSESSING STATES PROGRESS  3  I N T R O D
 U C T I O N  
issues, including childrens current health insurance status and SCHIP 
enrollment.  This survey provides national data and State-level data for 
13 States. Surveys were conducted in 1997, 1999, and 2002. 
State Fiscal Constraints and Impact on SCHIP   
According to The Fiscal Survey of States, conducted by the National 
Governors Association and the National Association of State Budget 
Officers, States continue to face budgetary pressures.  This report found 
that budget gaps are lingering as spending pressures persist,  
particularly from Medicaid and other health care.9   States have 
curtailed spending, and FY 2004 expenditures are expected to rise only 
by 0.2 percent over FY 2003, representing the smallest nominal increase 
since 1979.10  In FY 2004, 13 States enacted decreased budgets.11 
As States encounter difficulty producing required Federal matching 
funds for Title XXI programs, some States have elected to cap SCHIP 
enrollment, increase premiums, increase the rigor of the enrollment 
process, and lower the upper limit for income eligibility to adjust to the 
increasing budget deficits.12  As of November 2003, six States had 
frozen their SCHIP enrollment.13 
Related Work by the Office of Inspector General 
In February 2001, OIG released two separate reports on SCHIP to fulfill 
the congressional mandate to OIG. In Ensuring Medicaid Eligibles are 
not Enrolled in SCHIP, (OEI-05-00-00241), OIG found that between 97 
and 99 percent of SCHIP participants were correctly enrolled in SCHIP 
in FY 1999. In Assessment of State Evaluations Reports,
(OEI-05-00-00240), OIG found that questionable evaluations by the 
States undermine the reliability of reported success in reducing the 
number of uninsured children.  OIG also found that State evaluations 
have conceptual and technical weaknesses.  
SCOPE 
This evaluation addresses the second directive of the BBRA mandate, to 
assess States progress in reducing the number of uninsured low-income 
children.  As required by Congress, OIG is concurrently conducting a 
separate evaluation to determine if Medicaid-eligible children are being 
incorrectly enrolled in SCHIP.    
To assess State progress, this report evaluates States assessments of 
their progress toward reducing the number of uninsured children. We 
also assessed the methods States use to determine this progress, as 
described in their FY 2002 SCHIP Annual Reports.  In addition, we 
  OEI-05-03-00280  ASSESSING STATES PROGRESS  4  I N T R O D U C T I O N  
provide national level data on changes in the rate of uninsured children. 
We included all States and the District of Columbia (herein referred to 
as a State) that submitted FY 2002 Annual Reports by June 1, 2003.  
Annual Reports were due by January 1, 2003.  Connecticut, Hawaii, 
Minnesota, and Nevada had not submitted reports by June 1, 2003, and 
Tennessee was exempt from the Annual Report requirement because 
the State had no children enrolled in its SCHIP program. Therefore, we 
reviewed Annual Reports from 46 States. 
We analyzed States progress and methods to the extent that sufficient 
data and information was provided in their Annual Reports.  We did not 
conduct an independent evaluation of the reported outcomes of 
individual States performance objectives.  
METHODOLOGY 
To assess States progress in reducing the number of uninsured low-
income children, we reviewed the FY 2002 Annual Reports submitted by 
46 States. From these reports we determined whether States directly 
addressed CMSs regulations to describe the States progress toward 
reducing the number of uninsured low-income children by providing an 
annual estimate of changes in the number of uninsured in the State. 
We determined that a State directly addressed CMSs requirement if 
the State reported on a change, or lack of change, in the number and/or 
rate of uninsured children measured at two or more points in time.  
Though Congress and CMS-specified low-income children, we 
considered States that provided information on progress toward 
reducing the number of all uninsured children as having addressed 
CMSs requirement. Data limitations prevented more detailed analyses, 
including comparisons across States. 
We also undertook in-depth analyses and conducted site visits in six 
case study States:  California, Florida, Massachusetts, North Carolina, 
Rhode Island, and Utah.  These States were identified by CMS and 
SCHIP stakeholders as having notable evaluation expertise or 
experience.  Diversity in size, geography, program administration, and 
measurement methodologies were additional criteria for selection. 
These States encompass all three types of SCHIP program design  
(i.e., separate child health program, SCHIP Medicaid expansion, and 
combination). These States are not representative of the universe of 
States. 
  OEI-05-03-00280  ASSESSING STATES PROGRESS  5  I N T R O D U C T I O N  
In addition to their Annual Reports, we further reviewed the methods 
by which the six case study States evaluated their outcomes through 
interviews with their SCHIP staff, researchers, and advocates. 
We interviewed CMS staff to gain further understanding of how they 
oversee States Annual Reports and their role in monitoring SCHIP 
progress toward reducing the number of uninsured children. 
We also obtained information on the national trends in insurance 
coverage for children from (1) the CPS, (2) the National Health 
Interview Survey, and (3) the Urban Institutes National Survey of 
Americas Families. 
This inspection was conducted in accordance with the Quality 
Standards for Inspections issued by the Presidents Council on Integrity 
and Efficiency. 
  OEI-05-03-00280  ASSESSING STATES PROGRESS  6    F I N D I N G S

In FY 2002, 22 States described changes in the  While 44 of the 46 States provided 
number of uninsured children, and 17 of these  some response to CMS in their 
2002 Annual Reports, only 22 of 
States reported a decline. 
these States directly addressed 
CMSs regulations requiring States to describe progress in reducing the 
number of uninsured low-income children by estimating changes in the 
number of uninsured in the State.b  We determined that a State 
directly addressed CMS regulations if the State reported a change, or 
lack of change, in the number and/or rate of uninsured children 
measured at two or more points in time.  For additional information on 
States responses, see Appendix A. 
Seventeen of these twenty-two States reported a reduction in the number or 
rate of uninsured children. 
Of the 22 States that directly addressed CMSs requirement to measure 
insurance coverage changes, 17 reported a decrease in the number of 
uninsured children. Three States reported an increase in the number of 
uninsured children and two States measured no change. Table 1 below 
summarizes these States results. 
Table 1.  State-Reported Changes in Insurance for Children 
States reporting a decrease in the number of uninsured children  17 
States reporting an increase in the number of uninsured children  3 
States measuring no change in the number of uninsured children  2 
Total State responses  22 
Source: States SCHIP Annual Reports to CMS, FY 2002 
Of the 17 States measuring a decrease in uninsured children, 5 
specifically reported a reduction in the number of uninsured children 
over time; 9 States reported a decrease in the rate of uninsured 
children; and 3 States reported reductions in both the number and rate 
of uninsured children. 
State-reported reductions in the number of uninsured children range 
from a 5 to 37 percent decline.  For example, 1 States estimate of 
uninsured children declined from 311,000 to 197,000 children, a 
decrease of 37 percent. State-reported changes in insurance rates 
ranged from 1.5 percentage points (representing an 18 percent 
b Though Congress and CMS specified low-income children, we include States that 
provided information on progress toward reducing the number of all uninsured children. 
  OEI-05-03-00280  ASSESSING STATES PROGRESS  7  F I N D I N G
 S  
reduction in the rate of uninsured children) to 6.3 percentage points 
(representing a 66 percent reduction). For example, one State reported 
its estimated rate of uninsured children dropped from 9.5 percent in 
1996 to 3.2 percent in 2001.  This represents a 66 percent reduction in 
the rate for uninsured children. 
State-reported progress in reducing the number of uninsured children is 
not comparable across these 17 States.  States measure change over 
different periods of time and for different populations.  For example, 
three States reported change over a 1-year period, while other States 
compared current estimates of the number of uninsured children to 
estimates from up to 5 years earlier.  Some States also provided 
estimates from more than two points in time. 
Further, 12 of the 17 States reported changes in the number of 
uninsured children overall, while 5 reported specifically on children in 
certain income ranges (e.g., children up to 200 percent of poverty).  Only 
2 of the 17 States provided information on the statistical significance of 
their reported decreases in the number of uninsured children.   
Five of the twenty-two States that directly addressed CMS requirements 
reported an increase or no change in the number of uninsured children.   
Specifically, three States reported an increase in the number of 
uninsured children, and two States reported no change.  Two of these 
five States attributed their lack of measurable insurance progress to 
factors related to their measurement of the number of uninsured 
children.  Specifically, one States estimate of the number of uninsured 
children decreased, but this change was not statistically significant, so 
we categorized this State as measuring no change. Another State 
reported that their increase in the number of uninsured children was 
due, in part, to a change in the weighting of its latest CPS estimates. 
National data show a decline in the number of uninsured children. 
To further inform Congress on the national progress toward reducing 
the number of uninsured children, we consulted several data sources on 
national insurance coverage trends for children.  These estimates are 
not directly comparable to one another because they provide data on 
different populations (e.g., all children versus low-income children) or 
different periods of time.  However, all three sources provide evidence 
that the national rate of uninsured children has declined.  
Specifically, CPS data demonstrate a decline in the rate of uninsured 
children nationally.14   In 1998, CPS estimated that 15.4 percent of all 
children under 18 years old lacked health coverage.  By 2002, this 
  OEI-05-03-00280  ASSESSING STATES PROGRESS  8  F I N D I N G S  
estimate dropped to 11.6 percent, which represents a 25 percent 
reductionc in the rate of uninsured children.15 
The National Health Interview Survey, a survey administered by CDC 
which collects health data from a random sample of individuals 
nationwide, shows a statistically significant reduction in the rate of 
uninsured children from 1997 to the first half of 2003.  In 1997, this 
survey estimated that nationally, 13.9 percent of all children lacked 
health insurance at the time of the survey.  By 2003, this rate had 
dropped to 9.4 percent.16 
The Urban Institute found that the rate of uninsured low-income 
children below 200 percent of the poverty level fell from 22.5 percent in 
1999 to 16.8 percent in 2002. 17   This decrease was associated with 
increased Medicaid and SCHIP coverage.18 
To Respond to CMSs requirement, States use 

CPS, State Survey, and Enrollment Data.
 The 22 States that directly 
addressed CMSs requirement 
relied on the CPS or State survey data to measure a change in the 
number of uninsured children over time.  Instead of measuring SCHIP 
progress in reducing the number of uninsured children, 19 additional 
States provided enrollment data in response to CMSs requirement. 
Three additional States provided responses that measured something 
other than childrens insurance or enrollment progress, and two States 
submitted Annual Reports that did not provide any response to this 
requirement. Table 2 summarizes States responses from their 2002 
Annual Reports. 
Table 2.  States Annual Report Assessments of Changes in the Number of Uninsured Children 
States whose responses measured changes in the number of uninsured children  22 
States whose responses measured SCHIP enrollment  19 
States whose responses measured something other than childrens insurance or enrollment  3 
States whose Annual Reports did not provide any response to CMSs requirement  2 
Total  46 
 Source:  States SCHIP Annual Reports to CMS, FY 2002 
c The CPS revised its survey question during this time, which may affect the magnitude of this change. 
  OEI-05-03-00280  ASSESSING STATES PROGRESS  9  F I N D I N G
 S  
Of the 22 States that directly addressed CMSs requirement, 12 relied on 
CPS data. 
Twelve States, including one case-study State detailed below, used CPS 
data to measure the outcome of their efforts to reduce the number or 
rate of uninsured children in their CMS Annual Report. Annually, CPS 
provides a free, easily accessible source of longitudinal data on the 
number and rate of uninsured children. CPS provides State-level 
insurance estimates for all children and specific estimates for children 
below 200 percent of the poverty level. For many States, CPS is the 
only data source available for estimating changes in the number of 
uninsured children.19 
All States, as well as Congress, CMS, OIG, and the public, have access 
to annual, State-level CPS estimates of uninsured low-income children.  
However, some States face limitations by relying solely on CPS data for 
State-level estimates of uninsured children.20   For States with smaller 
populations, CPS sample sizes are relatively small.21   The smaller the 
sample size, the less precise the estimate.  These estimates may not 
detect incremental changes in insurance rates over time.  For example, 
one State noted that its CPS estimates showed a reduction in the 
number of uninsured children, but this reduction was not statistically 
significant because the States CPS sample size was too small.  Another 
State reported that its total CPS-estimated eligible population was less 
than its current Medicaid and SCHIP enrollment. 
Beyond the concerns about sample size, CPS is an employment survey 
that includes health insurance questions at the end of the survey.  Some 
researchers believe that a survey focused on health issues is more likely 
to elicit accurate responses about health insurance status.22   Further, 
CPS asks respondents in March about their health insurance coverage 
at any time during the past calendar year, rather than their current 
insurance status.23   Researchers expressed concern that CPS 
respondents may erroneously report on current insurance status.24 
Finally, CPS data tend to underestimate the number of children 
enrolled in Medicaid, and these children may be incorrectly counted as 
uninsured.25  One State noted that its CPS estimate of SCHIP coverage 
was 83 percent lower than the States actual SCHIP coverage. 
Rhode Island.  Rhode Island finds CPS data valuable to track State 
health insurance trends and progress in reducing the number of 
uninsured children, despite its limitations. Though State SCHIP staff 
acknowledged the imprecision of CPS point estimates, they expressed 
confidence in the overall direction of change indicated by CPS data.  The 
  OEI-05-03-00280  ASSESSING STATES PROGRESS  10  F I N D I N G
 S  
exact number of uninsured children is less important to the State than 
knowing whether the rate of uninsured children is declining.   
To confirm CPS insurance trends for children, State staff consult 
several additional data sources.  Rhode Island added a question on 
childrens insurance status for its State sample in the CDC- 
administered Behavioral Risk Factor Surveillance System. The State 
also collects health insurance status data through a State household 
survey usually conducted every 5 years.  Finally, the State consults  
private coverage data from BlueCross BlueShield, which accounts for 69 
percent of the States health insurance market, and data on the use of 
uncompensated care provided by safety net providers. 
The other 10 States that directly addressed CMSs requirement used State 
surveys. 
Ten States conducted statewide surveys that enabled them to measure 
and report on change in the number or rate of uninsured children over 
time.  By developing their own surveys States are able to tailor their 
questions to overcome some of the limitations associated with CPS 
insurance rate estimates and address State-specific concerns.  Three of 
these ten States were included in our case studies, and their efforts are 
detailed below. 
Florida.  While Florida SCHIP staff expressed confidence in their CPS 
insurance estimates, the State sought more detailed information on its 
uninsured population.  State staff believed that estimates of the rate of 
uninsured children for different regions and demographic groups would 
help Florida better understand and more effectively address uninsured 
subpopulations. Florida fielded telephone surveys in 1998 and 2002 
designed to produce estimates of insurance specific to 17 regions and 
various demographic groups. The surveys also sought to determine if 
uninsured children were likely to be eligible for Medicaid or SCHIP.   
Massachusetts.  According to Massachusettss staff, their State survey 
has several advantages over CPS for estimating the States rate of 
uninsured children, including a larger sample size which allows a more 
precise estimate.  And, unlike CPS, this survey focuses exclusively on 
health insurance.  Further, Massachusetts tested its survey questions 
extensively using cognitive interviews and focus groups to ensure that 
the survey would yield valid and reliable results. 
Massachusettss survey results support its contention that CPS 
overestimates its number of uninsured children.  In 2002 the State 
survey estimated that 3 percent of all children were uninsured, while 
  OEI-05-03-00280  ASSESSING STATES PROGRESS  11  F I N D I N G
 S  
CPS estimated 7.8 percent. Estimates for Massachusetts from the 
National Survey of Americas Families align more closely with the 
results of the State survey than the CPS for this State.  
Utah. From 1986 to 2001 Utah conducted a telephone survey every 
5 years to collect comprehensive information on childrens health, 
including their health insurance status. In its FY 2002 Annual Report, 
Utah compared its estimated rates of uninsured children from 1996 to 
2001 and found that the rate for uninsured children decreased from 8.5 
percent to 7 percent.  However, the State noted a limitation of this 
comparison. In 2001, Utah changed its health insurance questions. The 
State cannot determine how much of the change in its estimated 
insurance rates since 1996 should be attributed to the change in the 
survey questions, rather than an actual change in insurance coverage 
during that time. 
In response to CMSs requirement, 19 additional States reported on 
enrollment instead of progress in reducing uninsured children. 
In their Annual Reports 19 additional States responded to CMS by 
reporting on SCHIP enrollment, without providing information on the 
changes in the number or rate of uninsured children over time.  These 
States reported a variety of enrollment information, including 
enrollment at a point in time; changes in enrollment over time; 
comparisons of SCHIP enrollment to an estimate of uninsured children 
at a point in time, known as a penetration rate; or some combination of 
these.  Fourteen States reported changes in SCHIP enrollment over 
time.  Of these, 13 States reported enrollment increases, and 1 State 
reported enrollment had remained stable.  
While SCHIP enrollment increases provide one measure of program 
success, these numbers do not indicate changes in States rates or 
number of uninsured children.  Some States interpret increases in 
enrollment, an output, as net gains in health insurance coverage, an 
outcome. However, enrollment increases are not analogous to insurance 
increases for several reasons.  Some children transition to SCHIP from 
Medicaid, and so their SCHIP enrollment does not change the number 
of uninsured children. Likewise, a small percentage of SCHIP enrollees 
may have had prior private health coverage, despite SCHIP provisions 
to minimize the shift from private to public sector insurance, which is 
known as crowd out.  
Further, external factors may affect the insurance rate for low-income 
children.  For example, if a large number of children lose their private 
health coverage due to a rise in unemployment or a rise in private 
  OEI-05-03-00280  ASSESSING STATES PROGRESS  12  F I N D I N G
 S  
insurance premiums, the number of uninsured children may increase 
despite concurrent increases in SCHIP enrollment. 
States offered technical and economic reasons for using enrollment data. 
Eleven States, including three States that used enrollment data, cited 
data limitations associated with CPS estimates in their Annual Reports.  
States expressed concerns that CPS produces unreliable and imprecise 
estimates for their States. As mentioned above, this is primarily due to 
small State sample sizes, the fact that CPS is a lengthy labor-related 
survey with health insurance questions at the end, and historical CPS 
undercounts of public health insurance participation.  For example, one 
State explained that the CPS estimate of the total number of children 
under 100 percent of poverty in the State was lower than the actual 
number of children under 100 percent of poverty that were enrolled in 
the States Medicaid program.  Further, the States enrollment in 
SCHIP is 128 percent of the CPS baseline estimate of all uninsured 
children in the State.  Another State also considered the CPS 
measurement of insurance coverage at any time during the prior year, 
rather than at a point in time, a disadvantage.   
While a State survey could overcome CPS precision and reliability 
concerns, a State survey may be cost-prohibitive.  In interviews, SCHIP 
staff from four States pointed to recent economic downturns and fiscal 
constraints to explain why they did not focus on assessing their progress 
in reducing the number of uninsured.  Respondents explained that their 
priorities have shifted from reducing the number of uninsured children 
and increasing program enrollment to maintaining current enrollment 
levels and preserving SCHIP benefits.  One States SCHIP staff 
acknowledged their inability to reliably measure their progress in 
reducing the number of uninsured children but viewed this as a 
necessary tradeoff, stating that they would rather spend their money to 
cover kids than to count them.   
North Carolina.  North Carolina illustrates these explanations for using 
enrollment data.  Their performance objective related to reducing the 
number of uninsured children is to enroll as many children as possible 
that can be covered within available funds.  This State estimates that 
there are 35,000 more SCHIP-eligible children than the States target 
enrollment goal, which is the maximum number of children the budget 
will cover. According to SCHIP staff, they have insufficient funds to 
cover all eligible children, in part, because CPS underestimated their 
number of eligible children, and they have exhausted their Federal 
SCHIP allotment. 
  OEI-05-03-00280  ASSESSING STATES PROGRESS  13  F I N D I N G S  
In their Annual Report North Carolina provided CMS with a baseline 
insurance rate derived from an adjustment of CPS data.  However, 
neither the State staff nor the researchers who calculated this baseline 
have confidence in the accuracy of this estimate.  This lack of confidence 
in the CPS estimate, combined with a lack of funding for a State survey, 
has motivated a continuing reliance on enrollment numbers to assess 
and report success.  
In the Annual Report North Carolina specifies its performance goal as 
its current number of enrollees.  In effect, the States' goal is to maintain 
current enrollment in the program, not to reduce the overall insurance 
rate.  State staff report that State funding availability and the 
assumption that SCHIP is enrolled to capacity are the factors used to 
determine their SCHIP goal.  State SCHIP staff explain that because 
they lack reliable data to demonstrate that there are additional eligible 
but uninsured children, they must set their goals within budgeted 
funds. 
CMS has not ensured that States describe  CMS has not held States
accountable for directly 
changes in the number of uninsured children. 
addressing the CMS requirement 
that States describes their changes in the number of uninsured low-
income children. As detailed above, 24 of the 46 States that submitted 
Annual Reports to CMS did not describe their progress in reducing the 
number of uninsured children.  Four additional States had not 
submitted Annual Reports by June 1, 2003, despite a deadline of 
January 1, 2003. 
CMS accepts enrollment data as a proxy measure of SCHIP progress. 
Six years after SCHIP enactment, less than half of States report 
directly on their progress in reducing the number of uninsured as 
required by CMS.  In lieu of this direct assessment, CMS accepts and 
uses enrollment data from States as an alternative measure of program 
success.  CMSs staff explained that States enrollment numbers provide 
a reliable indicator of States SCHIP progress, as long as States 
continue to take measures to prevent crowd out (i.e., substitution of 
SCHIP for private health insurance).  In addition, CMS consults CPS 
data available from the Census Bureau to track national health 
insurance trends for children as well as State-level CPS data for each 
State. 
There is incongruity between CMSs requirement that States describe 
their progress in reducing the number of uninsured children and CMSs 
acceptance of enrollment data from States as a substitute for a measure 
  OEI-05-03-00280  ASSESSING STATES PROGRESS  14  F I N D I N G
 S  
of change in the number of uninsured children. Title XXI states that 
the main long-term goal of SCHIP is to expand health assistance to 
uninsured low-income children.26  The mandates that States assess and 
report their progress in reducing the number of uninsured low-income 
children, and that OIG assess States progress toward this goal 
demonstrate Congresss interest in this specific outcome.  CMS 
regulations also require States to provide an annual estimate of changes 
in the number of uninsured children.27 
However, CMS sets its SCHIP Government Performance and Results 
Act (GPRA) goal around increasing enrollment.  In its assessment of 
SCHIP, the Office of Management and Budget (OMB) noted that CMSs 
GPRA goals do not measure the impact of SCHIP on the rate of 
uninsured children, and OMB recommended that CMS develop goals to 
measure this impact.28 
CMS reported to us that it sets its GPRA goal around increasing SCHIP 
enrollment because of the difficulties associated with measuring 
changes in insurance rates, particularly in the context of changes in 
economic conditions and private insurance markets.  CMS also 
emphasized the limitations of CPS data for State-level insurance 
estimates, particularly for smaller States, and recognized that many 
States cannot afford to field their own State insurance coverage surveys.  
Further, CMS respondents pointed to studies that have shown minimal 
crowd out to demonstrate the link between enrollment and insurance 
increases. 
Accuracy of SCHIP enrollment data raises additional concerns. 
CMS emphasized limitations of available health insurance data. 
However, we also found limitations with one of the primary enrollment 
data elements for SCHIP -- the annual number of children ever 
enrolled by State for each FY. CMS produces an annual summary of 
the number of children ever enrolled in SCHIP by State, based on 
unverified, State-submitted data.  The Annual Report template also 
asks States for the number of children ever enrolled in SCHIP.  For 10 
States, we found discrepancies between the numbers of ever enrolled 
children reported in the Annual Reports and those reported in the CMS 
enrollment summary. For instance, between FYs 2001 and 2002, one 
States Annual Report showed an increase in enrollment of 7,000 
children, while the CMS summary reported a decrease of 15,000 
children for the same State.  CMS described a new data reporting 
system that it has implemented since our review that it expects will 
improve the accuracy of State-reported enrollment data. 
  OEI-05-03-00280  ASSESSING STATES PROGRESS  15  F I N D I N G
 S  
In addition, CMSs enrollment summary double counts children who 
transfer from one SCHIP program to another during the year.  This 
duplicate count occurs for children in the 20 States with both Medicaid 
expansion and separate programs (combination States) who move from 
one program to the other within the same State. It also occurs for 
children who move from one States to another States SCHIP.  CMS 
recognized this limitation of its data on ever enrolled children. 
However, CMS pointed out that it collects other types of unpublished 
enrollment data, such as quarterly point-in-time enrollment counts, 
that are not subject to this problem. 
CMS has not ensured that States submit relevant, clear, and timely 
information in their Annual Reports. 
The law and CMS regulations require States to submit SCHIP Annual 
Reports to describe their progress toward meeting the strategic 
objectives and performance goals described in their State plans.29  CMS 
regulations for State plans, which reflect Section 2107(a) of the Act, set 
forth that States must specify one or more performance goals for each 
strategic objective and must describe how performance will be measured 
through objective, independently verifiable means and compared 
against performance goals.30   SCHIP law specifically requires States to, 
assess the operation of the State plan, including the progress made in 
reducing the number of uncovered low-income children.31  And CMS 
regulations require States to provide an annual estimate of changes in 
the number of uninsured.32 
CMS reports that its project officers are in contact with the States to 
provide technical assistance that States need to complete their Annual 
Reports and to check on States progress.  CMS also reported that it 
discusses content and deadlines of the Annual Report on monthly 
SCHIP Technical Advisory Group calls.   
Yet, these efforts have not ensured that States establish and report 
strategic objectives, performance goals, and methodologies that are 
relevant to measuring change in the number of uninsured children.  For 
example, one State described its performance goal related to reducing 
the number of uninsured children as expanded capacity of the States 
data systems and additional field staff training.  Another State 
described increased administrative and outreach capacity as its 
performance goals and the hiring of 47 outstationed eligibility workers 
as its progress related to reducing the number of uninsured children.  
While these may be creditable goals for achieving other SCHIP 
objectives, they are not performance goals that lead to measurement of 
  OEI-05-03-00280  ASSESSING STATES PROGRESS  16  FF II NN DD II NN GG
 SS  
changes in the number of uninsured children. In comparison, an 
example of a relevant, objective, and measurable performance goal 
related to reducing the number of uninsured children was one States 
goal that the percent of children with creditable coverage for the entire 
year, whose family income is between 150% and 200% of the Federal 
Poverty Level, will be increased from 89.7% in 1998 to 95% in 2003. 
CMS has not required States to resubmit sections of the Annual Report 
when they provide unclear information or outdated goals. In their 2002 
Annual Reports, seven States submitted outdated goals related to 
reducing the number of uninsured children. For example, one State 
continued to cite a performance goal related to reducing the number of 
uninsured children that the State had already achieved more than 
2 years earlier. 
CMS has not held States responsible for timely State Annual Report 
submissions. Twelve States Annual Reports were submitted after the 
January 1, 2003 deadline.d  Late submissions ranged from 2 days to 
5 months past this deadline. In addition, four States had not submitted 
2002 Annual Reports at the time of our analysis in June 2003. CMS has 
indicated that those States have since submitted 2002 Annual Reports, 
but these reports were more than 5 months late. 
d Eight additional States reports did not include a submission date. 
OEI-05-03-00280  ASSESSING STATES PROGRESS  17    R E C  O M M E N D A T  I O N S  

In response to our congressional mandate, we found that 22 States 
reported a change in the number or rate of uninsured children in their 
FY 2002 Annual Report.  The remaining 28 States did not report 
sufficient information to allow us to assess whether they had reduced 
their numbers of uninsured children.e  CMS has not ensured that 
States comply with CMSs requirement that States describe changes 
in the number of uninsured children.  As such, we offer two options 
for CMS to select in order to increase States compliance. In addition, 
we offer a three-prong approach to assist CMS in working with States to 
ensure the integrity, validity, and usefulness of all data submitted by 
States in their Annual Report. 
CMS should resolve the inconsistency between the requirement that States 
report on changes in the number of uninsured children and CMSs practice 
of accepting enrollment data as a proxy. 
Currently, there is inconsistency between CMSs requirement that 
States report on changes in the number of uninsured and CMSs 
acceptance of enrollment data as a proxy for progress toward reducing 
the number of uninsured children. As CMS recognizes, enrollment data 
does not measure changes in insurance coverage.  We present two 
options for resolving this inconsistency. 
Option 1: Enforce exisiting regulations that require States to report changes in 
the number of uninsured children. 
As the agency responsible for overseeing SCHIP, CMS should hold 
States accountable to directly address its requirement that States 
describe their progress in reducing the number of uninsured low-income 
children by providing an annual estimate of changes in the number of 
uninsured in the State. If a States response does not demonstrate 
change in the number of uninsured children, CMS should follow up with 
that State to provide further clarification and technical assistance as 
needed.  
e Tennessee is excluded from this count because the State has no children enrolled in 
SCHIP. 
  OEI-05-03-00280  ASSESSING STATES PROGRESS  18  R E C O M M E N D A T I O N S  
Option 2: Broaden the requirements for States reporting of progress toward 
reducing the number of uninsured children to include changes in SCHIP 
enrollment as an acceptable measure of this progress. 
While CMS acknowledged that enrollment data does not measure 
changes in insurance coverage, CMS provided justifications for 
accepting enrollment data as a proxy measure of change in the number 
of uninsured children. These justifications focused on challenges that 
some States face estimating changes in the number of uninsured 
children.  However, this practice is not consistent with current 
requirements for States to report on changes in the number of 
uninsured children. If CMS decides that the challenges for some States 
to obtain information on changes in the number of uninsured children 
outweigh the benefits of the information, then CMS should change the 
regulations to reflect this decision. 
CMS should ensure the integrity, validity, and usefulness of the SCHIP 
Annual Report and SCHIP enrollment data.   
Toward this end, regardless of the option chosen above, CMS should 
work with States in three specific areas  (1) encourage States to obtain 
the best estimates available of childrens insurance coverage, (2) ensure 
States provide relevant and timely information on SCHIP progress as 
required, and (3) ensure the accuracy of SCHIP enrollment data.  
CMS should encourage States to obtain the most precise and reliable State-level 
estimates of number of children with health insurance. 
Many States face difficulties obtaining precise and reliable health 
insurance estimates, including concerns about the imprecision and 
unreliability of State-level estimates derived from CPS and the 
costliness of State-initiated surveys. 
Some States use CPS data in some way but do not directly address 
CMSs requirement to estimate change in the number of uninsured, 
e.g., States that compare enrollment data to a CPS estimate of 
uninsured children at one point in time.  For these States, CMS should 
provide technical assistance to SCHIP staff on how to best use CPS data 
to measure progress in reducing the number of uninsured children.   
On behalf of States that have concerns about the reliability of CPS data, 
CMS should continue departmental efforts to collaborate with the 
Bureau of the Census on increasing CPS sample sizes to make this data 
more useful for each State.  CMS should also encourage these States to 
  OEI-05-03-00280  ASSESSING STATES PROGRESS  19  R E C O M M E N D A T I O N S  
revisit CPS data to determine whether ongoing CPS sample size 
improvements have alleviated their concerns. 
Further, CMS should work with States that continue to distrust CPS 
estimates to identify or develop alternative data sources.  CMS could 
compile and share strategies that other States have used to measure 
their progress in reducing the number of uninsured children.  
Recognizing the costs of a new State survey, CMS can encourage States 
to take advantage of existing surveys and data sources. For example, 
States can minimize costs by adding questions on childrens health 
insurance status to the existing CDCs Behavioral Risk Factor 
Surveillance Survey. The Urban Institutes National Survey of 
Americas Families also provides State-level estimates of childrens 
health coverage for 13 States every few years.  CMS could encourage 
these 13 States to use this information to assess their progress toward 
reducing the number of uninsured children. 
CMS should improve its oversight to ensure that States submit relevant and timely 
information on progress toward reducing the number of uninsured children. 
While CMS reported having a process for overseeing the quality and 
timeliness of States reporting, this process should be evaluated and 
improved because it has not fully resolved deficiencies in States Annual 
Reports.  Similar to the methods we used for this report, CMS should 
carefully review States Annual Reports to ensure that States provide 
clear and timely information on their progress toward reducing the 
number of uninsured children that meets CMSs requirements.  If a 
States Annual Report describes performance goals, measures, or 
outcomes that do not describe progress toward reducing the number of 
uninsured children, CMS should follow up with that State for further 
clarification and provide technical assistance as needed.  Likewise, CMS 
should provide guidance to States on updating outdated goals to reflect 
current program conditions.  CMS should also reinforce the importance 
of submitting the SCHIP Annual Report by the January 1st deadline. 
CMS should ensure the accuracy of SCHIP enrollment data. 
Even beyond its use as a proxy measure for insurance progress, 
enrollment data provide its own important measure of program success. 
CMS should rectify inconsistencies between States 2002 enrollment 
data reported in CMSs enrollment report and data reported in States 
Annual Reports by reviewing the two numbers to identify and reconcile 
any discrepancies.  CMS has recently implemented a new reporting 
system for States enrollment data and anticipates that this system will 
  OEI-05-03-00280  ASSESSING STATES PROGRESS  20  R E C O M M E N D A T I O N S  
alleviate the data concerns cited in our report.  CMS should review 2003 
data to verify that this new system ensures data accuracy.   
In addition, CMS should reconsider its methodology which double 
counts children that transfer from one SCHIP program to another 
during the same FY in States with both Medicaid expansion and 
separate SCHIP programs.  CMS should devise criteria for categorizing 
such children in a way that produces an unduplicated count of ever 
enrolled children, such as counting them in the program in which they 
were enrolled for the greatest duration. CMS should also publish 
additional types of data it collects, such as quarterly point-in-time 
enrollment numbers, to provide a more robust picture of SCHIP 
enrollment. 
  OEI-05-03-00280  ASSESSING STATES PROGRESS  21    A G E N C Y   C O M M E N T S  

CMS provided comments on the draft report.  CMS agreed with several 
of our recommendations and described steps the agency has 
implemented to improve the integrity of the States SCHIP Annual 
Reports.  The complete text of CMSs comments can be found in 
Appendix B.  CMS also provided technical comments on the report. 
CMS provided responses to each of our recommendations. 
Recommendation 1.  CMS should resolve the inconsistency between the 
requirement that States report on changes in the number of uninsured 
children and CMSs practice of accepting enrollment data as a proxy. 
CMS Response.  In response to our recommendation to resolve 
inconsistency, CMS stated that they will work with Congress to clarify 
how progress should be measured and reported.  CMS also noted that 
their regulations require States to describe the States progress in 
reducing the number of low-income, uncovered children but do not 
specify a particular methodology for meeting this requirement.  
Further, CMS agreed that they need to work with States to improve 
State measurement efforts.  Consistent with our recommendations, 
CMS stated that they are taking steps to enhance technical assistance 
to improve State measurement capabilities, including scheduling a 
technical assistance session at an upcoming national conference. CMS 
also highlighted some of the same difficulties States face in measuring 
changes in the number of uninsured children that we discussed in our 
report. 
Recommendation 2.  CMS should ensure the integrity, validity, and 
usefulness of the SCHIP Annual Report and enrollment data. 
CMS Response.  CMS stated that, in addition to providing technical 
assistance to States, they are reviewing all State reports on progress 
toward covering the uninsured.  Also, CMS has been encouraging States 
to revisit CPS data in light of ongoing improvements to this survey, as 
we recommended.   
Further, CMS has implemented a new, web-based State Annual Report 
Template System, and CMS highlighted this systems role in 
implementing this recommendation.  CMS expects that this system will 
  OEI-05-03-00280  ASSESSING STATES PROGRESS  22  A G E N C Y   C O M M E N T S  
assist the agency to more effectively analyze State reports on progress 
toward covering the uninsured.  Consistent with our recommendations, 
CMS stated that they are using this new system to more closely review 
the information States provide and to request that States provide more 
clear and accurate responses in their SCHIP Annual Reports.  CMS also 
expects the system to help improve CMSs oversight of the SCHIP 
Annual Report submission process. Additionally, this system will be 
linked to the SCHIP enrollment data that CMS collects and reports, and 
CMS stated that this will eliminate the discrepancies we found between 
enrollment data in the SCHIP Annual Reports and in CMSs enrollment 
summaries. 
In response to our recommendation that CMS reconsider its 
methodology for ever enrolled data which double counts certain 
children who transfer between SCHIP programs, CMS stated that the 
number of double-counted children is limited, and that they do not 
believe this impacts the total enrollment significantly.  We suggested 
that CMS also publish the point-in-time enrollment data they collect, 
which is not duplicative, but CMS noted that this data is not as reliable 
as ever enrolled data. CMS is working with States to improve data 
reliability. 
Throughout their comments, CMS also highlighted national data that 
show significant reductions in the number of uninsured low-income 
children since the implementation of SCHIP and indicate CMSs and 
States success in achieving SCHIP program goals.  As we noted in our 
report, national data does provide evidence of success in reducing the 
number of uninsured low-income children, the primary goal of SCHIP.  
We hope that our recommendations and the steps that CMS has 
implemented will contribute to continued national progress toward this 
goal and ensure that each State shows similar success. 
  OEI-05-03-00280  ASSESSING STATES PROGRESS  23    A P P E N D I X   ~   A  
States Responses to CMSs Requirement in their 2002 Annual Reports 
States whose responses measured progress in reducing the number of uninsured children 
1  Alabama  12  Montana 
2  Arizona  13  New Hampshire  
3  Delaware  14  New Jersey 
4  Florida  15  New York 
5  Georgia  16  Ohio 
6  Idaho  17  Rhode Island 
7  Kansas  18  South Carolina 
8  Kentucky  19  Utah 
9  Louisiana  20  Washington 
10  Massachusetts  21  West Virginia 
11  Maryland  22  Wisconsin 
Source: States SCHIP Fiscal Year 2002 Annual Reports 
States whose responses measured SCHIP enrollment 
1  Alaska  11  North Carolina 
2  Arkansas  12  North Dakota 
3  California  13  Oklahoma 
4  Colorado  14  Pennsylvania 
5  Illinois  15  South Dakota 
6  Iowa  16  Texas 
7  Maine  17  Vermont 
8  Michigan  18  Virginia 
9  Missouri  19  Wyoming 
10  Nebraska 
Source: States SCHIP Fiscal Year 2002 Annual Reports 
States whose responses measured something other than childrens insurance or enrollment 
1  Indiana  3  Oregon 
2  Mississippi 
Source: States SCHIP Fiscal Year 2002 Annual Reports 
States whose Annual Reports did not include any response to CMSs requirement 
1  District of Columbia  2  New Mexico 
Source: States SCHIP Fiscal Year 2002 Annual Reports 
States that did not submit an Annual Report by June 1, 2003 
1  Connecticut  3  Minnesota 
2  Hawaii  4  Nevada 
Note: Tennessee was not required to submit a 2002 Annual Report because they had no enrolled children in SCHIP.  
  OEI-05-03-00280  ASSESSING STATES PROGRESS  24    A P P  E N D I X   ~   B  
  OEI-05-03-00280  ASSESSING STATES PROGRESS  25    OEI-05-03-00280  ASSESSING STATES PROGRESS  26   OEI-05-03-00280  ASSESSING STATES PROGRESS  27   OEI-05-03-00280  ASSESSING STATES PROGRESS  28   OEI-05-03-00280  ASSESSING STATES PROGRESS  29   A C K N O W L E D   G M E N T S  

This report was prepared under the direction of William Moran, Regional 
Inspector General for Evaluation and Inspections in the Chicago regional 
office.  Other principal Office of Evaluation and Inspections staff who 
contributed include: 
Emily Melnick, Team Leader 
Erin Lemire, Lead Analyst 
Amy Hennessy, Presidential Management Fellow
Annamarie Lopata, Intern 
Linda Hall, Program Specialist 
Elise Stein, Director, Public Health and Human Services 
  OEI-05-03-00280  ASSESSING STATES PROGRESS  30    E N D   N O T E S  

1  Social Security Act, Section 2108(a) 
2  Social Security Act, Section 2108(a) and 42 C.F.R.  457.750(b)(1) 
3  Title XXI of the Social Security Act (State Childrens Health Insurance 
Program (SCHIP) enabling legislation) was created by the Balanced 
Budget Act of 1997 (Public Law 105-33). 
4  Social Security Act, Sections 2101(a) and 2110(b) 
5  Social Security Act, Sections 2102 and 2103 
6  SCHIP Preliminary Annual Enrollment Report for Fiscal Year (FY)  
2002. Posted January 31, 2003.  Available online at 
http://www.cms.hhs.gov/schip/enrollment/schip02.pdf. 
7 State Health Access Data Assistance Center.  Issue Brief:  The Current 
Population Survey (CPS) and State Health Insurance Coverage 
Estimates.  March 2001.  Available online at 
http://www.shadac.org/publications/issuebriefs/IssueBrief1.pdf 
8   Bureau of the Census and the Bureau of Labor Statistics.  Current 
Population Survey - Design and Methodology.  Technical Paper 
63RV, Appendix J.  March 2002.  Available online at 
http://www.census.gov/prod/2002pubs/tp63rv.pdf 
9 National Governors Association and National Association of State 
Budget Officers.  The Fiscal Survey of the States.  December 2003.  
Available online at www.nga.org/cda/files/FSS1203.PDF 
10 Ibid. 
11 Ibid. 
12 Kaiser Commission on Medicaid and the Uninsured.  Out in the Cold: 
Enrollment Freezes in Six State Childrens Health Insurance 
Programs Withhold Coverage from Eligible Children.     
December 2003.  Available online at 
http://www.kff.org/medicaid/4159.cfm 
OEI-05-03-0280  ASSESSING STATES PROGRESS  31  13  Kaiser Commission on Medicaid and the Uninsured.  Out in the Cold: 
Enrollment Freezes in Six State Childrens Health Insurance 
Programs Withhold Coverage from Eligible Children.     
December 2003.  Available online at 
http://www.kff.org/medicaid/4159.cfm. As of November 2003, 
Alabama, Colorado, Florida, Maryland, Montana, and Utah had frozen 
their SCHIP enrollment. 
14 Current Population Survey, March 2002.  Health Insurance Table  
H108: Health Insurance Coverage Status and Type of Coverage by 
Selected Characteristics for Children Under 18:  2001. 
Available online at 
http://ferret.bls.census.gov/macro/032002/health/h08_000.htm 
15	 Current Population Survey, March 1988 - March 2003.  Historical 
Health Insurance Tables.  Table HI-3:  Health Insurance Coverage 
Status and Type of Coverage--Children Under 18 by Age: 1987 to 
2002. Available online at 
http://www.census.gov/hhes/hlthins/historic/hihistt3.html 
16  National Center for Health Statistics, Centers for Disease Control and 
Prevention. Health Insurance Coverage: Estimates from the National 
Health Interview Survey, January  June 2003.  Posted January 
2004. Available online at 
http://www.cdc.gov/nchs/data/nhis/earlyrelease/insur200401.pdf 
17 Urban Institute, Childrens Insurance Coverage and Service Use 
Improve.  July 31, 2003.  Available online at 
http://www.urban.org/url.cfm?ID=310816 
18 Urban Institute, Childrens Insurance Coverage and Service Use 
Improve.  July 31, 2003.  Available online at 
http://www.urban.org/url.cfm?ID=310816 
19 State Health Access Data Assistance Center.  Issue Brief 1: The 
Current Population Survey (CPS) and State Health Insurance 
Coverage Estimates.  March 2001.  Available online at 
http://www.shadac.org/publications/issuebriefs/IssueBrief1.pdf 
20 Ibid. 
OEI-05-03-0280  ASSESSING STATES PROGRESS	 32  21 State Health Access Data Assistance Center.  Issue Brief 3: State 
Health Insurance Coverage Estimates:  Why State-Survey Estimates 
Differ from CPS.  July 2001.  Available online at 
http://www.shadac.org/publications/issuebriefs/IssueBrief3.pdf 
22 Office of the Assistant Secretary for Planning and Evaluation (ASPE).  
Understanding Estimates of the Uninsured:  Putting the Differences 
Into Context.  ASPE Research Note.  Update June 2000.  Available 
online at http://aspe.hhs.gov/health/kids.htm 
23 State Health Access Data Assistance Center.  Issue Brief 3: State 
Health Insurance Coverage Estimates:  Why State-Survey Estimates 
Differ from CPS.  July 2001.  Available online at 
http://www.shadac.org/publications/issuebriefs/IssueBrief3.pdf 
24 Ibid. 
25  Mathematica Policy Research, Inc.  Chapter IX:  State Progress 
Toward Reducing the Number of Uninsured Low-Income Children. 
Implementation of the State Childrens Health Insurance Program: 
Synthesis of State Evaluations.  March 2003.  Available online at 
http://www.mathematica-mpr.com/PDFs/impchildhlth.pdf 
26 Social Security Act, Section 2101(a) 
27 42 C.F.R.  457.750(c)(2) 
28 Office of Management and Budget, Program Assessment Rating Tool.  
Assessing Program Performance for the FY 2004 Budget. 2003. 
Available online at 
http://www.whitehouse.gov/omb/budget/fy2004/pma.html 
29 Social Security Act, Section 2108(a)(1) and 42 C.F.R.  457.750 
30 42 C.F.R.  457.710 
31 Social Security Act, Section 2108(a)(1) 
32 42 C.F.R.  457.750(c)(2) 
OEI-05-03-0280  ASSESSING STATES PROGRESS  33