# Presentation: 785673

## OVERVIEW

- Problems With State-Level Estimates in National Surveys of the Uninsured
- Statistically Enhancing the Current Population Survey (CPS) to Obtain Better State-Level Estimates
- State-Specific Surveys
- State surveys
- Urban institute survey
- Community tracking study survey

## MEASURING THE UNINSURED

- Surveys Can Measure the Number of People Without Insurance at a Particular Point, and Their Characteristics (Age, Sex, Income, Employment Status)
- Later Surveys Using the Same Methodology Can Measure Changes Over Time
- But Surveys Cannot Definitively Measure the Impact of CHIP on the Number of Uninsured Children
- Imprecision in surveys
- CHIP may substitute for private insurance

## CURRENT POPULATION SURVEY

- Nationwide Monthly Census Bureau Survey
- 48,000 Households Interviewed Each Month
- 75% phone, 25% in-person
- 90 % response rate
- Rotating sample
- half interviewed again the following year
- Main Purpose Is to Determine Employment Status
- March survey has supplemental questions on health insurance coverage

## ADVANTAGES OF THE CPS

- Done Every Year
- Methodology Is Consistent From State to State and (Generally) From Year to Year
- Large National Sample Makes National-Level Estimates Reasonably Reliable
- Large Percentage of In-Person Interviews Improves Coverage and Reliability
- Rotating Sample Helps in Measuring Year-to-Year Changes

## LIMITATIONS OF THE CPS

- State-Level Estimates of the Uninsured in the Annual March CPS Are Unreliable
- State sample sizes are small
- over 1000 households in only 11 States
- Results in large margins of error, especially for sub-populations such as low-income uninsured children
- Three-year averages smooth out year-to-year variations, but large margins of error remain
- total number of children uninsured, using 3-year 1994-1996 average  (10/97 CPS analysis)
- California:  1,466,000 to 1,886,000
- Delaware:  12,000 to 32,000

## LIMITATIONS OF THE CPS (CONT.)

- Other Problems With the CPS
- May over-estimate number of full-year uninsured
- full-year vs. point-in-time responses
- Does not directly ask people whether they are uninsured
- people may actually have insurance when they say they don’t
- May under-estimate the number of people on Medicaid
- federal administrative data show 20-30% more Medicaid enrollees than CPS does

## STATISTICALLY ENHANCING THE CPS

- Purpose Is to Increase Reliability of State-Level CPS Estimates for Subpopulations
- Reduces margin of error by increasing sample sizes
- Can get more reliable detail on characteristics of  lower-income families with children
- family  income, age of children, parental employment, insurance coverage

## STATISTICALLY ENHANCING THE CPS (Cont.)

- Increases Sample Sizes in One State by “Borrowing” Similar Households From Other States
- Households must be similar with respect to age, race, income, family type, employment status
- Relationships between household characteristics and insurance coverage must be similar in borrowing and lending states
- can be tested by looking at unemployment rates, economic growth, composition of employment by firm size, and other data

## STATISTICALLY ENHANCING THE CPS (CONT.)

- Potential Uses and Limitations
- Provides better information for program planning and design
- permits more informed decisions about cost sharing, outreach, crowd-out, potential fiscal impact
- Can provide better estimates of year-to-year changes in number of low-income uninsured children than unenhanced CPS
- but large margins of error still remain
- methodological limits of CPS also remain

## STATE-SPECIFIC SURVEYS

- State-Specific Surveys May Be More Reliable Than Enhanced CPS
- But will not be consistent from State to State unless part of a national survey
- To measure impact of CHIP on uninsured, baseline survey must be conducted before implementation of CHIP, and another afterward
- Good State-specific surveys can be costly

## STATE-SPECIFIC SURVEYS (CONT.)

- Elements of a Good State Survey
- Cost of roughly $100-200,000 per state, depending on sample size, method of data collection, desired response rate, and length of interview
- sample size of 2,000-3,000 households
- combination of phone and some in-person interviews
- in-person is much more expensive
- response rate of 70% or higher
- reasonably focused questions

## STATE-SPECIFIC SURVEYS (CONT.)

- Example:  1998 Maine Health Insurance Coverage Survey
- Conducted by Mathematica for University of Southern Maine and Maine Medicaid Agency
- cost about $150,000 (mostly public, with some private funds)
- Surveyed 2,500 households with children
- entirely by phone (75% response rate)
- Comparison of survey results to state Medicaid enrollment files
- one-third of 450 households reporting Medicaid coverage could not be linked to state files
- nearly 20 percent of those enrolled in Medicaid told surveyors they were not

## STATE-SPECIFIC SURVEYS (CONT.) 

- Urban Institute Survey of 13 “Assessing the New Federalism” States (AL, CA, CO, FL, MA, MI, MN, MS, NJ, NY, TX, WA, WI)
- Average of 3,300 households per state
- One round in 1997 (pre-CHIP) and additional rounds in 1999 and 2001
- results of 1999 round won’t be available for BBA-required March 2000 state CHIP evaluations
- Survey over-samples lower-income families with children
- 30% of 50,000 households surveyed
- 1999 round will ask several CHIP-specific questions

## Community Tracking Study Survey
Conducted by Center for Studying Health System Change and Mathematica
Focus is on communities rather than states
30,000 households in 60 communities nationwide
12 high-intensity sites are oversampled
5 sites are in states not in Urban Institute survey
Pre- and post-CHIP surveys of insurance coverage
one round in 1996-97, another in 1998-99
second round data not available until mid-2000

- Conducted by Center for Studying Health System Change and Mathematica
- Focus is on communities rather than states
- 30,000 households in 60 communities nationwide
- 12 high-intensity sites are oversampled
- 5 sites are in states not in Urban Institute survey
- Pre- and post-CHIP surveys of insurance coverage
- one round in 1996-97, another in 1998-99
- second round data not available until mid-2000

- STATE-SPECIFIC SURVEYS (CONT.)

## Annual March CPS Is Not Reliable at the State Level
Statistically Enhanced CPS Can Produce More Reliable State Estimates, but Probably Not Reliable Enough to Track Impact of CHIP on Number of Uninsured Children
Some State-Specific Surveys May Be More Reliable Than Enhanced CPS, but Will Not Be Comparable Across All States 

- Statistically Enhanced CPS Can Produce More Reliable State Estimates, but Probably Not Reliable Enough to Track Impact of CHIP on Number of Uninsured Children
- Some State-Specific Surveys May Be More Reliable Than Enhanced CPS, but Will Not Be Comparable Across All States

- CONCLUSIONS

## Precise State-By-State Impact of CHIP on Number of Uninsured Children Will Likely Remain Unknown
Implications for States
Be prepared to respond to and use CPS, urban institute, CTS, and other surveys
Federal government will have difficulty holding states accountable for impact of CHIP on number of uninsured children
CHIP officials should think hard about what accountability requires within their own states

- Implications for States
- Be prepared to respond to and use CPS, urban institute, CTS, and other surveys
- Federal government will have difficulty holding states accountable for impact of CHIP on number of uninsured children
- CHIP officials should think hard about what accountability requires within their own states

- CONCLUSIONS (CONT.)