Department of Health and Human Services 
OFFICE  OF 
INSPECTOR  GENERAL 
Medicaid  Managed  Care: 
The  Use  of  Surveys  as  a 
Beneficiary  Protection  Tool 
JUNE  GIBBS  BROWN 
Inspector  General 
MAY 1997 
OEI-01-95-00280  OFFICE  OF INSPECTOR  GENERAL 
The  mission  of the  Office  of Inspector  General  (OIG),  as mandated  by Public  Law 95-452, is 
to protect  the  integrity  of the  Department  of Health  and Human  Services  programs  as well as 
the  health  and welfare  of beneficiaries  served  by them.  This statutory  mission  is carried  out 
through  a nationwide  program  of audits,  investigations,  evaluations,  sanctions,  and  fraud  alerts. 
The  Inspector  General  informs  the  Secretary  of program  and management  problems  and 
recommends  legislative,  regulatory,  and operational  approaches  to correct  them. 
OFFICE  OF EVALUATION  AND  INSPECTIONS 
The  OIGs  Office  of Evaluation  and Inspections  (OEI)  conducts  short-term  management  and 

program  evaluations  (called  inspections)  that  focus  on issues of concern  to the  Department, 

the  Congress,  and  the  public.  The  inspection  reports  provide  findings  and recommendations 

on the  efficiency,  vulnerability,  and effectiveness  of departmental  programs. 

OEIs  Boston  Regional  Office  prepared  this report  under  the  direction  of Mark.  R. Yessian, 

Ph. D., Regional  Inspector  General.  Principal  OEI  staff included: 

Boston Region  Headquarters 

Joyce  M. Greenleaf,  Project Leader  Jennifer  Antico,  Program Specialist 

Theodore  T. Wall, Progam Analyst 

For  copies of this report, please  call 617/565-1050.  Department of Health and Human Services 
OFFICE  OF 
INSPECTOR  GENERAL 
Medicaid  Managed  Care: 
The  Use  of  Surveys  as  a 
Beneficiary  Protection  Tool 
JUNE  GIBBS  BROWN 
Inspector  General 
MAY  1997 
OEI-01-95-00280  EXECUTIVE  SUMMARY 
PURPOSE: 

To  assess  the  usefulness  of beneficiary  surveys  as  a beneficiary  protection  tool  for 
Medicaid  managed  care. 
BACKGROUND 

State  Medicaid  agencies  are  increasingly  enrolling  their  beneficiaries  in  full-risk 
managed  care  plans.  Beneficiary  surveys  are  among  the  many  tools  available  to  them 
as  a beneficiary  protection.  These  surveys  can  be  of value  in  two  basic  ways: 
To Gain Insights on Plan Performance:  State  Medicaid  agencies  rely  on  health  plans  to 
provide  the  full  range  of  medical  services  for  their  enrolled  populations.  Overseeing 
the  plans  abilities  to  provide  appropriate  services  is an  important  agency  function. 
Surveys  present  an  opportunity  to  give  agencies  information  about  plan  performance 
from  the  perspectives  of  enrolled  beneficiaries. 
To Help Benejkiaties  Make Informed Enrollment Decisions:  Beneficiaries  that  are 
informed  about  their  enrollment  decisions  and  the  choices  they  have  are  likely  to  be 
better  protected  than  those  who  are  less  informed  about  their  choices.  Survey  results 
present  an  opportunity  to  help  beneficiaries  learn  from  the  experiences  of  others  in 
managed  care,  thereby  helping  them  to  make  an  informed  enrollment  choice. 
This  inspection  seeks  a deeper  understanding  of  the  usefulness  of beneficiary  surveys 
in  the  two  ways  described  above.  It  is based  on  interviews  with  the  Medicaid  agency 
leadership  in  11 States,  site  visits  to  4 of  those  States,  and  a survey  of  the  managed 
care  plans  enrolling  Medicaid  beneficiaries  in  all  11 States. 
FINDINGS 

Surveys of Limited  Use as Ben-y  Protection 
To  date,  surveys  provided  little  useful  information  about  plan  performance  to 

Medicaid  agencies.  Agency  leadership  found  their  survey  results  largely  confirming 

what  they  knew  from  other  sources.  They  also  found  them  to  be  of  questionable  value 

with  regard  to  the  technical  quality  of  care  and  were  reluctant  to  base  corrective 

actions  on  survey  data. 

Surveys  have  yet  to  provide  beneficiaries  with  information  to  help  them  choose  a plan. 

In  one  groundbreaking  survey,  Medicaid  beneficiaries  found  the  survey  data  to  be 

unwieldy.  In  general,  little  is known  about  what  information  beneficiaries  would  like 

to  have  when  faced  with  a choice  of plans. 

i  Agencies  and Plans Face Basic Hurdles in Survqbzg Medicaid  Bene&iaries 
The  intermittent  nature  of  Medicaid  eligibility  is such  that  only  about  half  of  the 
Medicaid  beneficiaries  eligible  in  one  month  will  remain  eligible  12 months  later.  Low 
response  rates  also  create  problems  for  agencies  and  plans.  Two-thirds  of  the  plans  in 
our  survey  reported  response  rates  of  20 percent  or  less  in  their  mail  surveys.  These 
hurdles  make  such  surveys  expensive  to  conduct  in  a credible  fashion. 
Strategic Survey Uses Emerging 
Some  agencies  had  some  success  in  using  more  narrowly  focused  surveys  aimed  at 
answering  questions  such  as:  Are  enrolled  beneficiaries  getting  the  orientation 
information  they  need  from  the  health  plans ?  What  information  do  beneficiaries  want 
available  to  choose  a health  plan ?  Others  have  presented  their  survey  data  in  such  a 
way  as  to  allow  comparisons  to  average  plan  scores. 
Notwithstanding  Limitations,  Plans Find Surv~s  Usw 
In  fact,  plans  conduct  surveys  even  where  their  contract  with  the  Medicaid  agency 
lacks  a specific  survey  requirement.  Their  survey  results  have  led  to  a variety  of 
operational  improvements  and  policy  changes. 
7&e HCFA  should either establish a work group or technical advisory group on Medicaid 
benejiciary surveys or add surveys to the agenda of an t&ting  group.  Either group should 
provide policy-level  guidance on how to make cost-flective  use of beneficiary survq.~s. 
This  group  would  provide  a forum  for  more  targeted  policy-level  discussion  about  the 
best  use  of  surveys.  At  a minimum,  the  group  should  address  the  following  questions: 
b Given  that  resources  are  limited,  what  level  of  emphasis  should  Medicaid 
agencies  give  to  using  surveys  for  overseeing  performance  and  providing 
information  for  beneficiary  education  and  choice? 
b How  can  beneficiary  surveys  be  used  more  effectively  as  a means  for 
overseeing  the  performance  of managed  care  plans? 
b How  can  surveys  be  used  more  effectively  to  provide  information  meaningful 
to  beneficiaries  faced  with  a choice  of  health  plans? 
b How  can  the  agencies  contracts  with plans be used to best benefit  the 
agencies  needs  for  survey  information? 
b How  can  Medicaid  agencies  best  deal  with  the  hurdles  inherent  in  surveying  a 
Medicaid  population? 
ii  The HCFA should devote greater attention to how the Medicaid  agencies are using 
ben@ciary  survqs.  It should revise its written guides for reviewing and monitoring 
Medicaid  managed care init&ivtz~ to call attention to the knportance of using ben@cimy 
surveys in more focuse4&strategic ways. 
Toward  that  end,  it should  consider  adding  the  questions  below  to  its  guides: 
b What  does  the  agency  aim  to  achieve  with  its beneficiary  surveys?  To  what 
extent  is it related  to  beneficiary  protection? 
b How  narrowly  has  it defined  the  surveys  purpose?  Could  a more  targeted 
use  of  a survey  enhance  its value  as  a beneficiary  protection  tool? 
b Will  the  survey  fill a gap  in  the  agencys  system  for  overseeing  quality  in  the 
health  plans?  Will  it provide  information  that  the  agency  knows  beneficiaries 
rely  upon  in  choosing  a health  plan? 
b Does  the  agency  have  a cost-effective  plan  for  overcoming  the  hurdles 
associated  with  surveying  Medicaid  beneficiaries? 
b Has  the  agency  weighed  the  relative  costs  of  surveying  beneficiaries  with  the 
relative  usefulness  of  the  survey  results? 
CoMMENTsoNmDm  REPORT 
We  solicited  and  received  comments  on  the  draft  report  from  HCFA  and  the 

American  Association  of  Health  Plans  (AAHP).  We  include  the  complete  text  of  their 

comments  in  appendix  C.  Below  we  summarize  the  major  comments  on  our 

recommendations  and,  in  italics,  summarize  our  responses. 

The  HCFA  partially  concurred  with  our  recommendation  on  establishing  a group  for 

policy-level  guidance  on  making  cost-effective  use  of beneficiary  surveys.  It  noted  its 

existing  Medicaid  Managed  Care  Technical  Advisory  Group  could  fulfill  this  role.  It 

also  noted  its  ongoing  collaboration  with  the  Agency  for  Health  Care  Policy  and 

Research,  which  is leading  the  Consumer  Assessment  of  Health  Plans  Study.  The 

AAHP  supported  our  recommendation  and  suggested  HCFA  include  some  health  plan 

representatives  in  the  group.  We recognize that HCFAs  existing group  could  be 

appropriate to provide  a forum  for policy-level discussion about the use of surveys.  We 

appreciate AAHPs  support of our recommendation  and  urge HCFA  to consider  including 

health plan  representatives in its group for  this discussion. 

Our  second  recommendation  called  for  HCFA  to  devote  greater  attention  to  how 

Medicaid  agencies  are  using  surveys  and  to  revise  its written  guides  toward  that  end. 

The  HCFA  concurred  with  points  made  in  our  text:  that  some  surveys  have  multiple 

objectives  and  need  to  be  more  focused.  The  HCFA  plans  to  include  a special  session 

on  surveys  in  its  annual  Medicaid  Managed  Care  College  and  will  stress  the 

... 
111  importance  of  surveys  in  its  technical  assistance  to  HCFA  regional  office  and  State 
Medicaid  staff.  The  AAHP  supported  our  recommendation.  We welcome  HCFA.s 
plans for  survey sessions at its Managed  Care  College and  through its ongoing  technical 
assistance.  We urge it to use these venues to stress more strategic and focused  uses of 
surveys.  And  we reemphasize  the need for HCFA  to revise its written guides for  reviewing 
Medicaid  managed  care initiatives. In its comments,  HCFA  did not indicate that it would 
revise these guides along the lines we suggested.  We appreciate AAHPs  support of our 
recommendation. 
iv  TABLE  OF  CONTENTS 
EXECUTIVESUMMARY 

INTRODUCJTION  .................................................  1 

FINDINGS  ......................................................  3 

 Surveys  of  Limited  Use  as  a Beneficiary  Protection  ......................  3 

 Basic  Hurdles  Faced  in  Surveying  Medicaid  Population  ...................  6 

 Strategic  Survey  Uses  Emerging  ....................................  8 

 Notwithstanding  Limitations,  Plans  Find  Surveys  Useful  ..................  9 

RECOMMENDATIONS  ...........................................  11 

COMMENTS  ON THE  DRAFT  REPORT  .............................  14 

APPENDICES 

A:  Background  on  Medicaid  Managed  Care  ............................  A-l 
B:  Methodology  ................................................  B-l 
C:  Complete  Comments  on  the  Draft  Report  .........................  C-l 
D:  Endnotes  ...................................................  D-l  INTRODUCTION 
PURPOSE 

To  assess  the  usefulness  of beneficiary  surveys  as  a beneficiary  protection  tool  for 
Medicaid  managed  care. 
Surveys  of  consumers  enrolled  in  managed  care  plans  are  becoming  increasingly 
popular.  They  can  provide  important  insights  to  the  managed  care  experience.  The 
surveys  results  can  be  used  to  market  a specific  health  plan,  promote  managed  care 
as  a delivery  system,  or  simply  answer  research  questions.  In  fact,  the  Health  Care 
Financing  Administration  (HCFA)  often  requires  Medicaid  agencies  implementing   
managed  care  waivers  to  conduct  surveys.  And  for  any  of  the  agencies  that  are 
increasingly  enrolling  their  beneficiaries  into  managed  care  plans  (see  appendix  A  for 
more  background  on  Medicaid  and  managed  care),  surveys  can  be  helpful  in  another 
way:  beneficiary  protection.  In  that  regard  they  can  be  of value  in  two  basic  ways:  to 
gain  insights  on  plan  performance  and  to  help  beneficiaries  make  educated  enrollment 
choices. 
To Gain Insights on Plan Pe$onnance:  State  Medicaid  agencies  rely  on  health 

plans  to  provide  the  full  range  of  medical  services  for  their  enrolled 

populations.  Overseeing  the  plans  abilities  to  provide  appropriate  services  is 

an  important  agency  function.  Surveys  present  an  opportunity  to  give  agencies 

information  about  plan  performance  from  the  perspectives  of  enrolled 

beneficiaries. 

To Help Benejiciaties  Make Informed Enrollment Deckions:  Beneficiaries  that 

are  informed  about  their  enrollment  decisions  and  the  choices  they  have  are 

likely  to  be  better  protected  than  those  who  are  less  informed  about  their 

choices.  Survey  results  present  an  opportunity  to  help  beneficiaries  learn  from 

the  experiences  of  others  in  managed  care,  thereby  helping  them  to  make  an 

informed  enrollment  choice. 

Of  course,  surveys  are  not  the  only  tools  available  to  State  agencies  for 
beneficiary  protection;  they  are  one  of  many.   For  example,  Medicaid  agencies  rely 
on  their  contracts,  plan  site  visits,  complaints,  focus  groups,  and  encounter  data, 
among  others  means,  to  protect  beneficiaries.  But  surveys,  like  complaints,  are  one  of 
the  few  tools  that  actually  require  some  direct  interactions  between  the  beneficiaries 
and  the  agency. 
1 
 THIS INQUIRY 

The  HCFA  recently  issued  a guide  for  States  to  help  them  develop  customer 
satisfaction  surveys.  It  provides  some  tips  on  developing  surveys,  an  inventory  of 
recent  surveys,  and  contact  people  at  the  agencies  for  those  interested  in  further 
information. 
This  inspection  seeks  a deeper  understanding  of the  usefulness  of  surveys  as 
beneficiary  protection  tools  in the  two  ways  described  above:  plan  performance  and 
beneficiary  education.  Such  an  understanding  is becoming  increasingly  important  as 
the  proportion  of  Medicaid  beneficiaries  enrolled  in  managed  care  plans  continues  to 
mount.  Our  focus  in  this  inspection  is on  mail  and  telephone  surveys  rather  than 
focus  group  surveys.  We  know  little  now  about  how  effective  such  mail  and  telephone 
surveys  have  been  as  a beneficiary  protection  tool.  This  inspection  also  offers  some 
assessment,  from  the  perspective  of  the  Medicaid  managed  care  leadership,  of  the 
surveys  included  in  HCFAs  recent  guide. 
In  conducting  this  inspection,  we  interviewed  the  Medicaid  agency  leadership  in 
11 States  that  have  enrolled  significant  numbers  of  Medicaid  beneficiaries  into  full-risk 
managed  care  plans. 3  All  of  the  11 States  in  our  study  have  had  experience  in 
surveying,  directly  or  through  contractors,  their  Medicaid  populations.  Most  often, 
they  conducted  satisfaction  and  disenrollment  surveys  and  used  a mail  or  telephone 
approach.  A  few  agencies  also  conducted  some  focus  groups.  For  this  inspection,  we 
drew  on  the  11 agencies  experiences  with  mail  and  telephone  surveys  rather  than 
focus  groups.  We  also  conducted  our  own  survey  of  full-risk  managed  care  plans 
enrolling  Medicaid  beneficiaries  in those  States.  (See  appendix  B for  more  detailed 
information  on  our  methodology). 
We  conducted  this  inspection  in  accordance  with  the  Quality Standards for 
Inspections  issued  by  the  Presidents  Council  on  Integrity  and  Efficiency. 
2 
 FINDINGS 
Surveys, as current& used by the 11 State Medicaid  agencies in our sample,  are of limited 
value as a beqticiq  protection. 
b Surveys provide  little useful information  about plan performance  to Medicaid 
agencies. 
All  of  the  11 agencies  in  our  study  The 1995 Minnesota HDI  Sumy 
had  experience  with  surveying  beneficiaries 
over  the  past  2 to  3 years.  Eight  of  the  in the spring of lPPS,  the Minnesota 
11 conducted  recent  surveys  that  included  Health Data Pnstituie (HDI)--a 
goals  of  assessing  plans  performance;  3 of  public-private organization created  by 
these  surveys  were  still  underway  at  this  the State in 1993  to improve health 
writing.  The  bulk  of  their  experience  was  care for Minnesotans--surveyed 
with  satisfaction  and  disenrollment  surveys.  17,591  Minnesotans  enrolled  in 46 
plans,  including Medicaid,  Medicare, 
According  to  the  agency  leadership,  arid the private& insured.  HDls 
neither  satisfaction  nor  disenrollment  goals were to provide  standardized 
surveys  provided  much  in  the  way  of  new  measurement  of health plaan 
insights  into  plan  performance.  The  performance  to consumers, 
insights  they  gained  about  plan  purchasers,  and plans, 
performance  from  satisfaction  surveys  were 
largely  limited  to  enrollee  perceptions  After the survey, HDI contracted for 
about  customer  service  issues,  such  as  the proje&  evahation.  One issue 
waiting  times.  While  relevant  to  a plans  the evaluation explored  was whether 
performance,  the  agency  officials  pointed  the survey results helped policymakers 
out  that  they  were  already  aware  of  such  monitor and  evaluate health system 
concerns  through  other  means,  such  as  perJonnance+  In the evaluation 
complaints  and  contract  compliance  visits.  report, policymakers  did not identify 
Disenrollment  surveys  were  similarly  any ways the survey helped  them 
disappointing.  Although  thousands  of  monitor pefownance. 
beneficiaries  disenrolled  from  plans  every 
(Source:  Mimesotu  Health Data &.&A&  1995 
month,  only  a small  percent--often 
Corz.wner Sufvty Evaluation Report, Ju+  1996) 
1 percent  or  less--disenrolled  voluntarily; 
most  disenrolled  due  to  a loss  of  eligibility. 
So  the  pool  of  potential  survey  - -
respondents  from  which  to  gain  insight  into  the  performance  of  any  one  plan  was 
limited.  One  agency  used  over  100 different  codes  to  reflect  reasons  for 
disenrollment.  With  so  many  codes  for  so  few  disenrollees,  it is unlikely  a pattern 
would  emerge  to  provide  insight  into  plan  performance. 
Agency  leadership  also  indicated  that  surveys  had  little  value  with  regard  to 
technical  quality  of  care  (meaning  examinations,  follow-through  and  treatment).4  For 
assessing  technical  quality  of  care,  they  relied  more  heavily  on  other  indicators  of 
performance,  such  as  encounter  data,  medical  record  reviews,  contract  compliance, 
3 
 and  complaints.    Surveys  are  a limited  tool  to  gain  insight  into  so  complex  a topic  as 
the  quality  of  care.  Agency  leadership  questioned  the  meaning  of  satisfaction  data: 
could  care  that  is truly  managed  lead  to  lower  satisfaction  due  to  restricted  access  to 
specialists?  They  questioned  the  link  between  satisfaction  and  quality.  They  noted 
that  surveys  were  not  good  sources  for  identifying  poor  quality-of-care.  Indeed,  high 
satisfaction  and  poor  quality  can  occur  simultaneously,  as the  Massachusetts  agency 
learned  when  its  plans  with  the  highest  scores  on  quality  indicators  scored  lower  in 
satisfaction.6  Two  officials  noted  that  while  the  plans  performance  matters  to  the 
agency,  it is the  physicians  that  matter  most  to  the  beneficiaries. 
What  surveys  revealed  about  the  plans  versus  what  they  revealed  about  the 
physicians  or  group  practices  complicated  their  usefulness  in  certain  markets.  For 
example,  in  the  Twin  Cities  area  of  Minnesota  and  certain  markets  in  Missouri,  plan 
networks  overlap  considerably.  This  means  many  of  the  same  physicians  and  group 
practices  participate  in  the  same  plans,  so  plan-specific  interpretations  of  survey  data 
become  questionable. 
To  date,  the  agencies  have  been  unable  to  rely  on  survey  data  to  make  many 
useful  distinctions  over  time  or  among  plans.  For  example,  the  11 States  in  our  study 
have  yet  to  conduct  the  same  survey  twice,  which  would  allow  them  to  compare  results 
over  time.7  Also,  survey  results,  such  as  satisfaction  scores,  tended  to  cluster  at  the 
high  end.  Officials  from  two  agencies  leadership  noted  that  their  surveys  would 
establish  a baseline  for  future  comparisons--in  one  case  a previous  survey  aimed  to  do 
the  same  but  low  response  rates  rendered  it unusable. 
In  most  cases,  agency  officials  reported  they  would  be  reluctant  to  direct 
corrective  actions  to  plans  based  on  survey  data  alone.  One  agency  official  referred  to 
the  survey  data  as  too  anecdotal  to  base  any  formal  corrective  actions.  An  official  in 
another  State  intended  to  establish  minimum  performance  levels,  or  benchmarks,  for 
future  surveys.  The  agency  would  penalize  or  direct  some  action  against  plans  failing 
to  score  above  the  benchmarks.  For  satisfaction--on  which  all the  plans  in  the  State 
had  scored  in  the  90s  on  the  most  recent  survey--the  official  was  considering  a 
benchmark  of just  70 percent,  thus  it is unlikely  the  benchmarks  would  be  unmet. 
Most  agencies  did,  however,  share  their  survey  results  with  their  plans  and  expected 
the  data  to  prompt  improvements. 
4 
 b The  surveys have yet to provide beneficiaries with information  to help them 
choose  a plan. 
Of  the  11 Agencies  in  our  study, 
6 had  completed  or  were  in the  process  7he  1995  A4hwmta  HDI Survey 
of  completing  surveys  that  included 
getting  information  back  to  beneficiaries  In  October of 1995j  60 newspapers 
as  a goal.  At  the  time  of  this  writing,  all  across the State published  the results of 
but  one  of  these  surveys  were  still  the survey as an insert to the Sunday 
underway.  edition.  The results were also available 
in libran-es. 
In  Minnesota,  county  staff 
educate  the  Medicaid  beneficiaries  about 
their  health  plan  choices.  An  evaluation  of  the  HDI  survey  found  that  while  widely 
disseminated,  consumers  in  general  did  not  find  it especially  helpful  in  choosing  a 
health  plan.*  It  also  found,  through  a focus  group,  that  Medicaid  beneficiaries  found 
the  information  to  be  unwieldy,  a sentiment  reflected  in  the  two  counties  we  visited. 
The  staff  there  responsible  for  counseling  beneficiaries  about  their  plan  choices  did 
not  use  the  HDI  survey  results  to  help  beneficiaries  make  a choice,  saying  it contained 
too  much  information  to  be  useful. 
Though  widely  heralded  as  a groundbreaking  survey,  the  HDI  experience  raises 
some  important  questions  about  what  is really  known  about  what  consumers,  and 
particularly  Medicaid  beneficiaries,  want  to  know  when  faced  with  a choice  of  plans. 
How  important  is survey  data,  such  as satisfaction  scores  or  waiting  times?  The 
Massachusetts  agency  has  a project  underway  that  aims  to  measure  how  useful  its 
report  cards  are  to  beneficiaries  faced  with  a choice  of  plans.  The  report  card 
includes  data  from  satisfaction  surveys,  as well  as some  quality  indicators  and  other 
information.  That  agency  plans  to  survey  its  newly  enrolled  beneficiaries  and  assess 
the  report  cards  impact  on  their  choice  of  plans.  In  Oregon,  where  a consumer 
scorecard  project  funded  by  the  Federal  government  has  been  underway,  officials 
learned  that  consumers  are  primarily  interested  in the  quality  of  services  offered  by 
their  doctors  and  hospitals,  and  less  interested  in  data  on  their  health  plan  itself. 
Other  researchers  have  come  to  different  conclusions:  ...research  by  the  Picker 
Institute  and  others  seems  to  show  that  patients  and  consumers  dont  yet  know  what 
measures  are  actually  meaningful  to  them.  Some  research  has  suggested  that 
consumers  tend  to  find  satisfaction  data  suspect  and  prefer  relying  on  their  own  family 
members  and  friends  when  choosing  a plan.ll 
5 
 Both agencies and plans face  basic hurdles in survqirzg the Medicaid population. 
w Intermittent  Eligibility 
Historically,  Medicaid  eligibility 
has  been  tied  to  programs  that  provide 
temporary  assistance  such  as Aid  to  b 
Families  with  Dependent  Children. 
Thus,  beneficiaries  tend  to  come  in  and 
out  of  eligibility  as their  income  and  Percent 
family  composition  changes.  The  Month  Continuously 
Enrolled 
HEDIS  Work  Group  compiled  data  on 
the  extent  to  which  beneficiaries  were  1  100% 
continuously  enrolled  over  a 36-month 
6  14% 
period  (see  table  l).l  That  data  shows 
12  55% 
a steady  decline  in  continuous 
enrollment,  with  just  55 percent  enrolled  24  29% 
for  1 year,  29 percent  for  2 years,  and 
36  17% 
17 percent  for  3 years. 
Note:  Based on 8 States experiences 
from  1O/91 through 12193 
The  relatively  short  lengths  of  Source:  Medicaid  HEDIS  DraJ?, July 
time  that  beneficiaries  are  enrolled  limits  199.5 
the  extent  of  their  opportunity  to  have 
much  contact  with  or  receive  care  from 
the  plans.  Thus  the  universe  from  which 
an  agency  or  plan  would  draw  its  survey  sample  is also  limited.  In  fact,  78 percent  of 
plans  responding  to  our  survey  identified  intermittent  eligibility  as  an  obstacle  to  using 
surveys.  Thirty-eight  percent  identified  it as  a major  obstacle.  The  plans  identified 
intermittent  eligibility  as  an  obstacle  more  often  than  anything  else.  In  the  words  of 
some  plan  officials,  members  are  simply  not  enrolled  long  enough  to  use  services  and 
on  and  off  again  eligibility...do  not  lend  themselves  to  a meaningful  managed  care 
experience. 
b Low Response  Rates 
Both  plans  and  agencies  reported  low  response  rates  with  their  surveys.  In  fact, 
according  to  our  survey  of  plans,  66 percent  of  the  plans  that  had  conducted  a mail 
survey  of  their  Medicaid  enrollees  had  response  rates  of  20 percent  or  less.  Low 
response  rates  were  common  in  the  agencies,  too.  Indeed,  in  Massachusetts  the 
agency  was  able  to  contact  just  7 percent  of  its  sample  in  one  survey.  The  Missouri 
agency  expanded  its  sample  after  the  response  rate  to  its  first  mailing  was  just 
8 percent.  And  early  experiences  with  surveys  in Florida,  Oregon,  and  Utah  had 
response  rates  too  low  for  the  survey  results  to  be  usable. 
Many  factors  contribute  to  low  response  rates  within  the  Medicaid  population. 
Medicaid  populations  tend  to  be  more  transient  than  other  populations,  meaning 
6 
 agencies  and  plans  often  contend  with  outdated  addresses  and  telephone  numbers.  In 

Ohio,  the  agency  found  about  70 percent  of  its  sample  population  to  be  either  invalid 

or  with  incorrect  phone  numbers.  Differences  in  language  and  literacy  skills--either  on 

the  part  of  the  agency  or  plan  administering  the  survey  or  on  the  part  of  the 

beneficiary  responding  to  the  survey--also  lead  to  low  response  rates.  Two-thirds  of 

the  plans  responding  to  our  survey  identified  literacy  problems  and  half  identified 

language  problems  as  obstacles  to  using  surveys. 

b  costs 
Conducting  credible  surveys  is costly.  It  requires  well-thought  out  methodology, 
and  sampling  designs,  carefully  constructed  questions,  adequate  response  rates,  and 
careful  analysis.  Each  of  those  stages  can  tax  a plan  or  agencys  staff,  time,  and  funds. 
For  example,  agencies  and  plans  often  lack  the  experience  or  skills  needed  to  develop 
a credible  questionnaire.13  One  agency  official  pointed  out  that  its  earlier  experience 
with  surveying  enrollees  failed  in  large  part  due  to  poor  survey  design.  Thirty-two 
percent  of  the  plans  in  our  survey  identified  the  lack  of  a suitable  off-the-shelf 
questionnaire  as  an  obstacle  to  using  surveys;  23 percent  identified  their  limited 
capacity  to  develop  their  own  questionnaires  as  an  obstacle. 
Even  if an  agency  or  plan  has  a suitable  questionnaire,  actually  carrying  out  the 
survey  can  be  labor-intensive.  It  can  involve  making  calls,  stuffing  and  labeling 
envelopes,  tracking  responses,  targeting  follow-up,  and  analyzing  responses,  among 
other  tasks--or  at  a minimum:  overseeing  the  contractors  work.  One  agency  shared 
the  costs  of  its  mail  survey  by  requiring  the  plans  to  mail  the  questionnaire  themselves. 
Forty  percent  of  the  responding  plans  in  our  survey  identified  limited  capacity  to 
administer  surveys  and  33 percent  identified  limited  capacity  to  assess  results  as 
obstacles  to  using  them. 
Among  agencies,  costs  are  also  high.  For  example,  to  achieve  respectable 
response  rates,  the  Oregon  agency  used  an  intensive  5-step  process  that  involved,  in 
multiple  languages:  an  advance  letter  mailing,  an  initial  survey  mailing,  a reminder 
postcard,  a second  survey  mailing,  and  a second  reminder  postcard  or  phone  call.  The 
Oregon  agencys  intensive  effort  paid  off  with  a 63 percent  response  rate.  Officials 
there  note,  however,  that  it is too  much  to  do  annually. 
Other  agencies  have  undertaken  similarly  intensive  survey  projects.  For 
example,  the  Utah  agency  also  undertook  a survey  that  involved  agency  and  field  staff
-as  opposed  to  a contractor--making  calls  to  a sample  of  1,000  enrolled  beneficiaries. 
They  completed  579  surveys  over  a 2-month  period,  resulting  in  an  adjusted  response 
rate  of  73 percent.r4  The  Florida  agency  is also  in  the  midst  of  its intensive  survey 
effort  that  involves  multiple  mailings,  similar  to  the  Oregon  effort.  As  of  this  writing, 
Florida  has  a response  rate  in  the  low  30s  and  is extending  its  surveying  by  5 weeks. 
7 
 Some agencies are beginning to use surveys in strategic ways, with potentially  promising 
results. 
b More  Focused  Surveys 
Much  of  agencies  experiences  to 
date  has  been  with  large-scale 
satisfaction  surveys  that  aimed  to  meet 
multiple  objectives.  But  some  agencies 
have  conducted  surveys  with  a narrower 
focus  and  found  the  results  to  be  quite 
useful.  In  the  words  of  one  agency 
leader,  The  usefulness  of  surveys  is 
likely  to  be  closely  correlated  to  how 
focused  they  are. 
The  survey  described  at  the  right 
is an  example  of  a focused  survey 
designed  to  meet  specific  information 
needs  of  the  agency.  We  also  learned  of 
another  focused  survey  in  Utah,  where 
agency  staff  spent  some  late  nights  in 
hospital  emergency  rooms  interviewing  Medicaid  beneficiaries.  That  survey  gave  the 
agency  new  insights  into  inappropriate  emergency  room  usage  and  led  to  changes  in 
its  policies  on  emergency  room  copayments  and  reimbursement. 
Other  agencies  have  had  similar  success  with  narrowly  focused  surveys.  For 
example,  the  Rhode  Island  agency  conducted  two  surveys:  one  to  determine  what 
factors  beneficiaries  relied  upon  to  select  their  health  plans  and  one  to  determine 
what  factors  they  relied  upon  to  choose  a primary  care  physician.  The  Massachusetts 
enrollment  broker  reduced  its field  staff  from  45 to  10 based  in  part  on  information 
learned  through  a focused  survey. 
b Better  Presentation  of Survey Data 
As we  have  seen,  to  date  the  agencies  in  our  study  have  not  repeated  the  same 
survey,  thereby  precluding  them  from  presenting  survey  results  over  time. 
Nonetheless,  some  agencies  have  presented  their  survey  data  in  such  a way  as  to 
maximize  its  utility.  For  example,  the  Oregon  agency  divided  its  presentation  of 
survey  results  between  plans  in  urban  and  rural  areas.  It  included  the  average  scores 
for  plans  in  each  area,  and  used  colors  to  indicate  which  plans  scores  were  above  or 
below  average.  Of  course,  whether  the  average  score  was  good  or  bad  is unknown. 
Nevertheless,  presenting  the  data  with  an  average  provided  some  context  in 
understanding  one  plans  performance  relative  to  the  average. 
8  Notwithstanding  the limitations of benejkiiuy  surveys, health plans  still jind  them to be of 
some use in identijjhg  and responding to enrollee concerns. 
b Plans Conduct  Surveys Even  Without  Contractual  Requirements 
Eight  of  the  11 agencies  in  our  sample  required  that  the  plans  they  contract 
with  conduct  surveys.  According  to  our  survey,  the  majority  of  plans  responding  from 
all  11 States  had  surveyed  Medicaid  beneficiaries  in  the  past  2 years.r5  Yet  even 
plans  without  the  contractual  requirement  conducted  surveys.  In  fact,  the  majority  of 
responding  plans  (85  percent)  from  those  3 States  conducted  surveys  anyway.  On 
average,  the  plans  responding  from  those  States  conducted  about  4 surveys  over  the 
past  2 years.  Overall,  83 percent  of  the  responding  plans  surveyed  their  memberships, 
including  Medicaid,  an  average  of  about  5 times  over  the  past  2 years. 
The  actual  contract  requirements 
regarding  surveys  varied  from  agency  to  Institutionalizing the Sw~ty  &3txxs 
agency,  but  overall,  the  contract 
language  regarding  surveys  was  quite  One Medicaid-only plan  we visited had 
limited.  Seven  of  the  eight  contracts  institutionalized its surveys so that each 
required  the  plans  to  conduct  the  quarteq  it sent out satisfaction surveys 
surveys  annually  and  to  report  results  to  to a 25 percmt  random  sampEe of 
the  agencies,  but  none  specified  members  who had  been  continuously 
methodologies  for  the  plans  to  use.  Two  enroiled for  tj months.  Each  month  it 
contracts  specified  that  the  survey  must  sent out disenrolbnent  surveys to 
cover  perceived  problems  in  quality,  fOUpercent  of voluntary disenrollees, 
availability,  and  accessibility  of  health  Both surveys are in English  and 
care  services,  but  the  others  lacked  Spanish,  Response  rates hovered 
specific  requirements  for  the  surveys  around  10 percent for  these surveys, 
contents.  and  the plan  considered  the results a 
roiling mme  of feedback  to share  with 
The  lack  of  many  specific  itsproviders. 
contractual  requirements  allows  plans 
great  latitude  in  surveying  beneficiaries, 
and  we  found  great  variation  among  the  plans  approaches  to  surveying.16  For 
example,  for  their  most  recent  survey,  64 percent  of  plans  conducted  a mail  survey, 
24 percent  telephone,  and  about  4 percent  each  self-administered,  in-person,  and  focus 
group.  Plans  also  often  surveyed  their  whole  membership,  rather  than  just  Medicaid 
beneficiaries.  In  fact,  44 percent  of  responding  plans  reported  that  their  most  recent 
survey  included  non-Medicaid  enrollees.  Targeting  certain  Medicaid  enrollees  for 
recent  surveys  was  also  common  among  responding  plans.  Of  the  80 percent  that 
targeted  in  some  way,  51 percent  targeted  Medicaid  disenrollees;  45 percent,  Medicaid 
enrollees  in  certain  geographic  areas;  44 percent,  users  of  certain  Medicaid  services; 
33 percent,  users  of  certain  physicians;  and  26 percent,  Medicaid  enrollees  with  certain 
diagnoses. 
9 
 w Plan Surveys Lead  to Improvements 
Even  though  plans  face  the  same 
hurdles--such  as  low  response  rates--as  Percent of Plans Rating Surveys as 
agencies  in  surveying  their  Medicaid  Us&i  for IdentifLig  Medicaid  Emlke 
enrollees,  the  plans  in  our  own  survey  Con~ems About.-
routinely  surveyed  their  memberships. 
Their  surveys  gave  them  some  cues  Quality of Care  75% 
about  what  mattered  to  their  (interpersonul communication,  listening 
memberships,  thereby  guiding  their  responsiveness, manne;r, courtesy) 
improvements  and  helping  them 
compete.r7  Quality of Customer  Service 
73% 
The  operational  focus  of  plans  (responsiveness to questiovts, concerns, 
surveys  is evident  by  the  variety  of  and  complaints) 
changes  they  prompted  within  the  plans. 
Indeed,  56 percent  of  the  responding  Waiting Times  6.5% 
plans  explained  their  surveys  usefulness  (for appointments,  and  in waiting and 
in  terms  of  the  improvements  to  which  exam rooms) 
they  led.  This  reflects  national  trends  in 
health  plans  using  surveys:  in  1995,  Access  55% 
virtually  all  plans--99  percent--reported  (to specialty care such  as mental 
using  of  surveys  for  quality  improvement,  health, prenatal,  substance  abuse,  and 
provider  feedback,  and  marketing.*  to emergency rooms) 
of
According  to  our  survey,  plans  Technical  Quality  Care  365% 
made  both  operational  and  policy  (examinations>follow-through,  and 
changes  based  on  insights  they  gained  treatment) 
from  surveying  their  Medicaid  enrollees. 
For  example,  they  expanded  their  Note:  Categories a&pted$om  the oraft  Medicaid  HEDIS 

document,  .iu& 1995.  N=69-79. 

provider  networks,  translated  written 
Source:  1996 DIG Sutvq. 

materials,  revised  prescription  drug 
policies,  added  toll-free  telephone  lines, 
changed  referral  policies,  and  increased  hours  of  operation,  among  others.  A  few 
plans  also  noted  that  they  used  surveys  to  give  feedback  to  their  physicians,  and  in 
some  cases,  had  established  incentives  for  physicians  who  scored  high  on  surveys. 
10 
 RECOMMENDATIONS 
This  report  raises  some  questions  about  the  usefulness  and  cost-effectiveness  of 
surveys  in  protecting  beneficiaries  from  the  vulnerabilities  associated  with  full-risk 
managed  care  plans.  There  are  two  basic  ways  in which  surveys  can  serve  as  a 
beneficiary  protection  tool:  (1)  to  provide  Medicaid  agencies  with  information  on  the 
performance  of  the  plans  enrolling  Medicaid  beneficiaries,  and  (2)  to  provide  the 
beneficiaries  facing  a choice  of  health  plans  information  to  help  them  choose.  Given 
the  partnership  nature  of  the  Medicaid  program,  the  Federal  and  State  governments 
should  work  cooperatively  to  identify  ways  in which  surveys  can  best  be  used  as  a cost-
effective  beneficiary  protection  tool.  Toward  that  end,  we  direct  the  following 
recommendations  to  the  Health  Care  Financing  Administration: 
The HCFA  should either establkh  a work group or technical advisory group on Medicaid 
benejkiary  survqs  or add surveys to the agenda of an existing group.  Either group should 
provide policy-level  guidance on how to make cost-effective use of benejkia~  surveys. 
Such  a group  would  provide  a forum  for  more  targeted  policy-level  discussion 
amongst  State  Medicaid  leadership  on  how  to  best  use  surveys  as  a beneficiary 
protection.  Medicaid  policymakers  need  to  figure  out  how  surveys  can  best  fit  in with 
the  array  of  beneficiary  protection  tools,  such  as  encounter  data  and  HEDIS  and 
contracts,  at  their  disposal.  The  involvement  of  officials  at  the  policy  level  is key, 
because  without  direction  from  that  level,  the  danger  exists  that  the  group  will  become 
consumed  with  the  methodological  minutiae  of  conducting  surveys  rather  than  the 
information  needs  they  can  fill. 
The  survey  group  should  aim  to  build  on  the  existing  Customer  Satisfaction 
Surveys  for  Medicaid  Beneficiaries:  A State  Guide  recently  issued  by  HCFA.  At  a 
minimum,  the  group  should  address  the  following  questions  with  an  eye  toward 
maximizing  surveys  potential  as  a beneficiary  protection  tool: 
b Given  that  resources  are  limited,  what  level  of  emphasis  should  Medicaid 
agencies  give  to  using  surveys  for  overseeing  performance  and  providing 
information  for  beneficiary  education  and  choice? 
Medicaid  agencies  need  to  maximize  the  potential  of  surveys  as  a 
beneficiary  protection  tool,  and  this  will  involve  tough  choices.  Using  a 
single  survey  to  serve  multiple  objectives  can  limit  its  effectiveness  in 
meeting  any  one  objective.  Conducting  a survey  and  having  credible 
data  takes  an  intensive  effort.  Medicaid  agencies  need  to  consider  the 
strengths  and  weaknesses  of  their  existing  beneficiary  protection  systems 
and  weigh  the  relative  merit  of using  a survey  for  oversight  or 
educational  purposes. 
11 
 b How  can  beneficiary  surveys  be  used  more  effectively  as  a means  for 
overseeing  the  performance  of  health  plans? 
Medicaid  agencies  rely  on  a variety  of  data  and  information  to  assess  the 
performance  of  managed  care  plans.  Surveys  could  serve  to  provide 
information  about  topics  which  other  oversight  tools  fail  to  address  or 
address  insufficiently.  For  example,  surveys  could  be  used  to  gain  a 
better  understanding  of  the  experiences  of  the  chronically  ill or  of  those 
enrolled  in  managed  care  but  not  receiving  services  (do  they  consider 
themselves  healthy?  do  they  know  they  are  enrolled?  do  they  know  how 
to  access  services?). 
b How  can  surveys  be  used  more  effectively  to  provide  information  meaningful 
to  beneficiaries  faced  with  a choice  of  health  plans? 
The  Agency  for  Health  Care  Policy  and  Research  (AHCPR)  is engaged 
in  a 5-year  Consumer  Assessment  of  Health  Plan  Survey  project  aimed 
at  using  surveys  to  develop  report  cards  for  consumers  facing  a choice  of 
plans.  l9  What  can  Medicaid  agencies  learn  from  this  project?  What  is 
known  about  what  types  of  information  Medicaid  beneficiaries  need 
when  making  a choice? 
b How  can  the  agencies  contracts  with  managed  care  plans  best  benefit  the 
agencies  needs  for  survey  information? 
Medicaid  agencies  have  an  opportunity  with  their  contracts  to  call  for 
the  plans  to  conduct  specific  surveys  and  even  to  stipulate  the  sampling 
design,  the  question  phrasing,  and  the  survey  methodology.  How  shoulc 
agencies  best  take  advantage  of  that  opportunity?  Should  they  aim  to 
have  all their  plans  conduct  uniform  surveys  according  to  contract 
specifications? 
b How  can  Medicaid  agencies  best  deal  with  the  hurdles  inherent  in  surveying  a 
Medicaid  population? 
Conducting  a credible  survey  always  requires  careful  consideration  to  a 
variety  of  methodological  issues,  such  as sampling  and  approach 
(telephone,  mail,  or  in-person,  for  example).  But  surveying  a Medicaid 
population  presents  unique  hurdles,  such  as short  and  discontinuous 
eligibility. 
12 
 The HCFA  should devote greater attention  to how the Medicaid  agencim are using 
bene&imy  survqs.  It should revise its written guides for reviewing and nwnitoring 
Medicaid  managed care initiativ~  to call attention to the importance of using benejicky 
surveys in more focusesdtr,ategic ways. 
The  HCFA  already  identifies  surveys  and  their  uses  as a point  of  review  in  its 
guides  for  reviewing  and  monitoring  Medicaid  managed  care  initiatives.20  It  could, 
however,  enhance  the  attention  it gives  to  surveys  by  incorporating  greater  discussion, 
inquiry,  and  assessment  of  agencies  use  of  surveys  at  each  point  for  formal  review  or 
monitoring.  Toward  that  end,  it should  consider  adding  the  questions  below  to  its 
guides: 
b What  does  the  agency  aim  to  achieve  with  its beneficiary  surveys?  To  what 
extent  is it related  to  beneficiary  protection? 
b How  narrowly  has  it defined  the  surveys  purpose?  Could  a more  targeted 
use  of  a survey  enhance  its value  as a beneficiary  protection  tool? 
b Will  the  survey  fill a gap  in  the  agencys  system  for  overseeing  quality  in  the 
health  plans?  Will  it provide  information  that  the  agency  knows  beneficiaries 
rely  upon  in  choosing  a health  plan? 
b Does  the  agency  have  a cost-effective  plan  for  overcoming  the  hurdles 
associated  with  surveying  Medicaid  beneficiaries? 
b Has  the  agency  weighed  the  relative  costs  of  surveying  beneficiaries  with  the 
relative  usefulness  of  the  survey  results? 
13 
 COMMENTS  ON  THE  DRAFT  REPORT 
We  solicited  and  received  comments  on  the  draft  report  from  the  Health  Care 
Financing  Administration  (HCFA)  and  the  American  Association  of  Health  Plans 
(AAHP).  We  include  the  complete  text  of  the  detailed  comments  in  appendix  C. 
Below,  we  summarize  HCFAs  and  AAHPs  major  comments  on  the  recommendations 
and  then,  in  italics,  offer  our  responses. 
RECOMMENDATIONSFROMTHE  DRAFT REPORT 

T%eHCFA  should either establish a work group or technical adtiory  group on Medicaid 
benej?ciary surveys or add surveys to the agenda of an existing group.  Either group should 
provide policy-level  guidance on how to make cost-flective  uses of benejZiary  surveys. 
The  HCFA  partially  concurred  with  this  recommendation.  It  pointed  out  that 
its  existing  Medicaid  Managed  Care  Technical  Advisory  Group  has  a work  group 
currently  working  on  consumer  information  and  surveys.  The  HCFA  agreed  with  our 
assessment  that  agencies  often  conduct  surveys  for  multiple  purposes  but  disagreed 
with  our  assessment  that  these  were  often  of limited  value.  The  HCFA  pointed  out 
that  it is not  feasible  for  States  to  develop  single-purpose  surveys.  It  further  noted 
that  it believes  States  have  obtained  some  meaningful  information  from  surveys,  even 
where  that  information  was  not  what  the  State  intended  to  acquire.  And  it noted  its 
ongoing  collaboration  with  the  Agency  for  Health  Care  Policy  and  Research,  which  is 
leading  the  Consumer  Assessment  of  Health  Plans  Study. 
The  AAHP  supported  this  recommendation  and  suggested  that  HCFA  include 
some  health  plans  in  this  work  group.  It  reinforced  the  importance  of weighing  the 
relative  costs  of  surveys  with  the  relative  usefulness  of  the  survey  results  for  not  only 
State  agencies  but  also  health  plans. 
We recognize that HCFAs  Technical Advisory Group  could  be an appropriate 
group  to provide policy-level guidance  on the cost-effective use of beneficiary surveys and 
we urge it to continue  collaborating on the Consumer  Assessment project.  We maintain, 
however, that based on this inquiry, narrowly focused  surveys have been  more  useful to 
State agencies  as a beneficiary protection  tool. 
We appreciate AAHPs  support for  our recommendation  and  urge HCFA  to 
consider  including health plan  representatives in its Technical Advisory Group  to discuss 
the cost-effective use of surveys. 
14 
 The HCFA should devote greater attention to how the Medicaid  agencies are using 
benejkiay  surveys.  It should revire its written guides for reviewing and monitoring 
Medicaid  managed care intitivm  to call attention to the importance of using benejitiry 
surveys in more focus4  strategic ways. 
The  HCFA  concurred  with  points  raised  in  our  findings  and  text:  that  some 
surveys  have  multiple  objectives  and  need  to  be  more  focused.  It  pointed  out  that 
surveys  are  a very  important  part  of  State  agencies  requirements  under  the  Quality 
Assurance  Reform  Initiative,  which  is being  replaced  with  a system  of  standards  for 
both  Medicare  and  Medicaid.  It  anticipates  that  surveys  will  be  an  important 
component  of  that  new  system.  The  HCFA  plans  to  include  a special  session  on 
survey  development  and  use  of  survey  data  in  its  annual  Managed  Care  College  and 
will  stress  the  importance  of  surveys  in  its  technical  assistance  to  HCFA  regional  office 
and  State  Medicaid  staff. 
The  AAHP  supported  this  recommendation. 
We welcome HCFA  s plans for survey sessions at its Managed  Care  College and 
through  its ongoing  technical assistance and  urge it to use these venues to stress more 
focused  and strategic uses of surveys.  And  we reemphasize  the need for HCFA  to revise 
its written guides for  reviewing and monitoring Medicaid  managed  care initiatives. In  its 
comments,  HCFA  did not indicate that it would revise these guides  along the lines we 
suggested. 
We appreciate AAHPs  support of our recommendation. 
15 
 APPENDIX  A 
BACKGROUNDONMEDICAIDMANAGEDCARE 

Medicaid  Expansion  into Managed  Care Programs 
Over  the  past  15 years,  States  have  increasingly  used  managed  care  to  provide 
medical  services  for  Medicaid  beneficiaries.  This  trend  has  accelerated  in  the  past  few 
years:  in  1991,  2.7 million  Medicaid  beneficiaries  were  enrolled  in  managed  care,  by 
1993,  that  number  grew  to  4.8 million,  and  in  1996,  13 million.  As  of  June  1996, 
39 percent  of  all  Medicaid  beneficiaries  were  enrolled  in  some  kind  of  managed  care 
arrangement. 
To  date,  States  have  primarily  enrolled  adults  and  children  in  low-income 
families  into  managed  care,  whereas  aged  or  disabled  beneficiaries  remain  under  fee-
for-service  systems.  By  1996,  over  500  managed  care  organizations  were  providing 
services  to  13 million  Medicaid  beneficiaries. 
The  movement  to  enroll  Medicaid  beneficiaries  in  managed  care  began  in 
earnest  in  the  early  198Os, as States  experienced  significant  fiscal  pressures  due  to 
rising  Medicaid  costs.  While  States  viewed  managed  care  as  a way  to  contain 
Medicaid  costs,  they  were  constrained  by  Federal  standards  required  for  Medicaid 
enrollment  in  managed  care. 
In  response  to  mounting  concerns,  Congress  allowed  States  greater  flexibility  to 
deviate  from  those  standards  through  amendments  to  the  1981  Omnibus  Budget 
Reconciliation  Act.21  For  example,  the  amendments  allow  States  to  pursue  freedom-
of-choice  waivers  (under  section  1915  of  the  Social  Security  Act)  that  release  them 
from  certain  Federal  provisions,  such  as the  free-choice-of-provider  provision.  To 
date,  42 States  have  freedom-of-choice  waivers. 
States  also  can  receive  research  and  demonstration  waivers  under  section 
1115  of  the  Social  Security  Act.  Since  1992,  many  States  have  aggressively  pursued 
such  waivers.  States  implementing  or  pursuing  1115 waivers  often  extend,  as  a part  of 
their  demonstration,  insurance  benefits  to  those  not  otherwise  eligible  for  Medicaid, 
such  as  the  working  poor  and  their  families.  As  of  this  writing,  HCFA  has  approved 
18 research  and  demonstration  waivers.  Of  those,  12 States  have  implemented  their 
programs,  5 are  pending  implementation,  and  one  has  no  plans  to  implement.22  The 
HCFA  is currently  reviewing  nine  States  applications.23 
A  -1 
 Defining Managed  Care 
Although  managed  care  organizations  vary,  they  generally  feature  a focus  on 
primary,  preventive  health  care  and  care  coordination.  That  focus  is believed  to 
improve  care  and  access  for  enrollees.  It is also  thought  to  promote  cost  containment, 
thus  slowing  the  rate  of  increase  in  health  care  spending. 
The  managed  care  organizations  enrolling  Medicaid  beneficiaries  can  generally 
be  defined  as fitting  into  one  of  two  basic  types:  health  maintenance  organizations 
and  fee-for-service  primary  care  case  management  (PCCM)  programs.  Both  types 
feature  coordinated  care.  But  each  carries  a different  level  of  financial  risk.  Health 
maintenance  organizations  (hereafter  referred  to  as health  plans  or  plans)  are  full-risk 
plans  that  contract  with  Medicaid  for  a fixed  fee  per  person  and  provide 
comprehensive  services.  24  PCCM  programs  comprise  providers,  usually  primary  care 
physicians,  willing  to  serve  as gatekeepers  and  take  responsibility  for  approving  and 
coordinating  enrollees  care.  Medicaid  pays  PCCM  providers  on  a fee-for-service 
basis,  but  they  receive  a case  management  fee  to  cover  their  added  responsibilities. 
Thus  PCCM  providers  are  at  no  financial  risk. 
Some  agencies  also  contract  with  plans  separately  for  certain  aspects  of  care, 
such  as  mental  health,  substance  abuse,  and  dental  care.  The  arrangements  for  these 
carve-outs  vary  in  terms  of  the  services  included  and  the  level  of financial  risk  the 
plan  assumes. 
A -2 
 APPENDIX  B 

We  drew  on  three  major  sources  of  data  for  this  study:  (1)  interviews  with  the 
Medicaid  managed  care  leadership  in  11 States,  (2)  a survey  of  the  full-risk  managed 
care  plans  enrolling  beneficiaries  in those  11 States,  and  (3)  the  model  contracts  used 
by  the  Medicaid  agencies  in  contracting  with  the  managed  care  plans. 
II  Participating State Medicaid Agencies 
At  the  outset  of  our  study,  we  chose  the  13 State  Medicaid  programs  that  had 
made  a significant  commitment  to  enrolling  their  beneficiaries  in  full-risk  managed 
care  plans,  represented  different  geographic  areas  of  the  country,  and  with  waivers  in 
various  stages  of  approval  or  implementation.  Medicaid  agencies  in  11 of  these  States 
participated:  Arizona,  California,  Florida,  Massachusetts,  Minnesota,  Missouri,  New 
York,  Ohio,  Oregon,  Rhode  Island,  and  Utah.  Two  State  agencies  declined  to 
participate.  Of  the  participating  11 States,  6 had  approved  and  implemented 
1115  waivers  (Arizona,  California,  Minnesota,  Ohio,  Oregon,  and  Rhode  Island),  2 had 
approved  waivers  pending  implementation  (Florida  and  Massachusetts),  and  3 had 
pending  waivers  (Missouri,  New  York,  and  Utah).  Together,  the  11 States  represent 
about  37 percent  of  all Medicaid  beneficiaries  in  the  country.  The  results  of  our  study 
are  not  generalizable  to  other  State  Medicaid  agencies. 
Interviews and Site Viits 
We  conducted  structured  telephone  interviews  with  the  Medicaid  managed  care 
leadership  in  eight  of  the  State  Medicaid  agencies.  We  visited  the  Medicaid  agencies 
in  five  of  the  States  on  one  or  more  occasions  to  conduct  the  interviews  in  person  and 
meet  with  additional  agency  staff,  advocates,  and  representatives  of  the  States 
legislature  and  managed  care  plans.  We  conducted  site  visits  in  California, 
Massachusetts,  Minnesota,  Rhode  Island,  and  Utah.  During  both  the  telephone 
interviews  and  site  visits  we  collected  any  relevant  documents  such  as  survey  reports, 
contracts,  and  evaluations. 
Contract Review 
We  reviewed  the  model  contracts  in use  at  the  time  of  our  interviews  for 
requirements  related  to  surveys. 
of
Survey  Full-Risk Managed  Care Plans in 11 States 
We  received  lists  of  full-risk  managed  care  plans  enrolling  Medicaid 
beneficiaries  from  each  of  the  11 agencies  in  our  sample.  We  then  sent,  via  facsimile, 
a questionnaire  asking  about  the  plans  recent  survey  activity.  We  sent  a second 
survey  to  nonresponding  plans  after  about  3 weeks,  and  then  followed  up  by  telephone 
B-l  with  remaining  nonresponding  plans  after  about  6-8 weeks.  The  universe  for  this 
survey  included  162 plans  in  11 States.  Eight  of  the  162 plans  were  invalid  because 
they  no  longer  enrolled  Medicaid  beneficiaries,  therefore  our  adjusted  universe 
included  154 plans.  We  received  105 completed  questionnaires  in  time  to  be  included 
in  our  analysis.  Our  adjusted  response  rate  was  68 percent. 
We  attempted  to  conduct  a non-respondent  analysis  to  determine  how 
nonresponding  plans  differ  from  responding  plans,  if they  do  at  all.  Because  we 
lacked  certain  data  elements,  such  as tax  status  and  size  of  enrolled  population  for 
some  plans,  we  were  unable  to  complete  the  analysis.  The  results  of  our  health  plan 
survey  are  not  generalizable  to  all plans. 
B-2 
 APPENDIX  C 
0N
Com4~m-s  m  DRAFT  REPORT 
In  this  appendix,  we  present  in  full  the  comments  from  the  Health  Care 
Financing  Administration  and  the  American  Association  of  Health  Plans. 
C-l  DEPARTMENT  OF HEALTH  &HUMAN  SERVICES  Health Care Financing Administration 
The Administrator 
Washington, D.C.  20201 
DATE: 
APR 23 1997 
TO: 	 June Gibbs Brown 
Inspector General 
FROM: 	 Bruce C. Vlade 
Administrator 
SUBJECT: 	 Office of Inspector General (OIG) Draft Report: Medicaid Managed  Care: 
The Use of Surveys as a Beneficiary  Protection Tool, (OEI-01-95-00280) 
. 
We reviewed the above-referenced  report that finds surveys have been of limited use as a 
beneficiary  protection  tool. 
Our detailed comments on the report recimmendations  are attached for your 
consideration.  Thank you for the opportunity  to review and comment on this report. 
Attachment  Comments of the Health Care Financing Administration  (HCFAI 

on Offrce of Insnector General (OIG) Draft Report: 

Medicaid Managed Care: The Use of Surveys as a Beneficiarv  Protection  Tool. 

(OEI-01-95-00280) 

OIG Recommendation 
HCFA should either establish  a work group or technical  advisory group on Medicaid 
beneficiary  surveys or add surveys to the agenda of an existing group.  Either group 
should provide policy-level  guidance on how to make cost-effective  use of beneficiary 
surveys. 
HCFA Resuonse 
We partially  concur with the intent of the recommendation  to provide policy-level 
guidance on how to make cost-effective  use of beneficiary  surveys. 
HCFA already has a Medicaid Managed Care Technical Advisory  Group (MMCTAG) 
that works in conjunction  with the Medicaid Managed Care Team (MMCT)  in the Office 
of Managed Care (OMC).  The MMCTAG  is comprised of state Medicaid  agency staff, 
the American Public Welfare Association,  and the MMCT staff.  The MMCTAG 
provides valuable input essential for the accuracy  and completion of many-HCFA 
projects.  As part of its fiscal year  1996 agenda, the MMCTAG formed six work groups 
to address issues they consider to be the most urgent in the rapid transition  from fee-for-
service to managed care.  One of these groups is working on consumer information, 
which has as one of its goals the development  of a document on survey instruments  tested 
and validated. 
The report states beneficiary  sufveys developed by state Medicaid agencies are developed 
for different purposes and to meet various objectives.  We agree with that assessment. 
However, due to limited state resources,  it is not feasible for states to develop surveys for 
a single purpose.  When they are developed and used for a single purpose,  they are ody 
one part of the overall assessment.  The purposes of Medicaid surveys range from internal 
assessment of a plans performance,  to assessing how satisfied beneficiaries  are with a 
plan and its services, to identifying  areas for potential improvement,  etc.  The purpose  of 
the survey dictates how it should be constructed,  what should be included,  to whom and 
when it should be administered,  and most importantly,  how the results can and should be 
compiled and utilized.  Because of the variation  among states administration  and 
coverage, it is very difficult to come up with one method or instrument  that will satisfy 
everyones needs.  Although  states are currently  experimenting  with different  instruments 
and formats, we believe they have obtained some meaningful information  about 
beneficiaries  (maybe not what they started out to acquire, but nonetheless,  useful).  Page 2 
We concur with the need for additional policy-level  guidance on survey use.  HCFA is 
already collaborating with the Agency for Health Care Policy and Research on the 
development  of its Consumer Assessment  of Health Plans Survey (CHAPS).  This survey 
includes questions for the Medicaid population,  as well as commercial and Medicare 
populations.  In addition, HCFA recently  prepared and distributed copies of Customer 
Satisfaction Surveys for Medicaid Beneficiaries:  A State Guide to Medicaid Associate 
Regional Administrators  for release to their respective  states for input.  This document, 
containing tiormation  on specific survey instruments  developed by states and national 
organizations, will assist states in developing or improving existing customer satisfaction 
surveys for the Medicaid population.  Although we are making progress, there still needs 
to be more work done in this area, particularly  on the use of beneficiary  survey results. 
OIG Recommendation 
HCFA should devote greater attention to how the Medicaid agencies are using 
beneficiary  surveys.  It should revise its written guides for reviewing and monitoring  . 
Medicaid managed care initiatives  to call attention to the importance  of using beneficiary 
surveys in more focused, strategic ways. 
HCFA Response 
We concur that some state surveys are very general, attempt to address too many issues at 
once, and need to be more focused.  Surveys are a very important part of states 
requirements  under the Quality Assurance Reform Initiative (QARI).  As states become 
more experienced in survey design, analysis,  and use, we expect the surveys to be more 
focused in what questions they are trying to answer and the purposes they are used for. 
We have, however, made some progress in this area and are working to improve it even 
further.  The existing MMCTAG  established  a work group on consumer information, 
whose goal is to collect information  on what Medicaid recipients want to  about 
know 
health plans and providersand how beneficiaries  think information should be presented. 
The MhKTAG  also plans to work with Harvard Medical School and/or Rand in the 
piloting of the CHAPS Medicaid  survey.  In addition, the QARI is being replaced  with a 
system that will contain quality  standards for both Medicaid and Medicare.  We 
anticipate additional focus on the importance of surveys and baseline data to measure 
demonstrable  improvement  in beneficiary  care and outcomes.  OMC also sponsors a 
Managed Care College each year.  We plan to include a special session on survey 
development  and use of survey data.  Finally,  as part of our continuing technical 
assistance to regional office and state Medicaid staffl we will emphasize the importance 
of surveys and how they can be used to monitor managed care initiatives.  American  Association  of 
HEALTH  PLAN3 
April 11, 1997 
June Gibbs Brown 

Inspector General 

Offrce of Inspector  General 

Department  of Health and Human Services 

330 Independence  Avenue,  SW 

Washington,  DC 20201 

Dear Ms. Brown: 

On behalf of the American  Association  of Health Plans (AAHP), thank you for the opportunity  to 

review and provide comments on the Office of Inspector Generals (OIGs) draft inspection 

report, Medicaid  Managed Care: Use of Surveys as a Beneficiary  Protection  Tool.  AAHP 

represents approximately  1000 health plans serving over 100 million  Americans  nationwide. 

Many of AAHPs members provide services to Medicaid beneficiaries. 

AAHP agrees with the basic findings  and recommendations  of the report.  /&HP  fully supports 

initiation  of new efforts designed to improve information  to beneficiaries  about their enrollment 

options to help them make better informed  c&ices.  We also recognize that, for a number of 

reasons, State mandated  surveys may be of limited value as a beneficiary  protection  tool and may 

not provide useful information  to beneficiaries. 

AAHP specifically  supports the recommendation  that HCFA form a work group or technical 

advisory  group to provide policy-level  guidance on how to make cost-effective  use of beneficiary 

surveys.  We also appreciate and agree with the recommendation  that HCFA should consider the 

relative costs of surveying  beneficiaries  with the relative usefulness of the survey results. 

Consideration  of this issue is equally relevant whether the survey costs are incurred by the State 

Medicaid Agency directly or by the health plan. 

Our suggestions  and comments  are relatively  ininor.  We suggest several refinements  to the 

Executive  Summary to clarify  a few issues and to give greater prominence  to some important 

points made in the body of the report.  The following  are suggested revisions  to the Executive 

summary: 

0 	 At the end of the second paragraph of the Background section of the Executive  Summary, 
we suggest that the OIG add a sentence or two noting that there are also many other 
methods to measure plan performance,  including HEDIS and the evaluation  of other 
utilization  data.  Also, we suggest that the text note that to obtain a beneficiary 
perspective  on health plans, surveys are rarely used alone and complement  other types of 
performance  measurement  data. 
. 
0  In the next paragraph of the Executive  Summary, we suggest that the narrative  note that 
---.--- -e-(...p-.- - - - -- . ._  _  ___  __  ._ 
1129  Twentieth  Street.  N\V   Suite  600    \Vnshineton.  DfJ 2W36   202-778-3200    FAX:  202-331-7487  June Gibbs Brown 
April  11, 1997 
Page 2 
other tools, such as descriptions  of benefits,  access requirements,  and the delivery  system, 
may assist the consumer in making informed  enrollment  decisions. 
0 	 We suggest that the Executive  Summary note that there are two types of surveys.  The 
first type, which is the focus of the report, is State mandated surveys (either conducted  by 
the State Medicaid Agency or the health plan on its behalf).  The second type of survey is 
one conducted voluntarily  by health plans.  We suggest that the Executive  Summary  note 
briefly that health plans frequently  conduct their own surveys and that these surveys may 
lead to improvements  by the plans.  (This point is noted on page 10 of the draft report). 
AAHP also recommends  that the OIG suggest that HCFA obtain the participation  of health plans 
in its Workgroup  activities. 
Further, we suggest that the OIG identify  as an issue for consideration  the possibility  of giving 
health plans the discretion to decide, subject to State approval, the surveys focus.  The draft 
report suggests that survey needs may vary among health plans.  If so, it would be desirable  to 
give health plans the flexibility  to decide-how best to meet those needs. This suggestion  could be 
incorporated  in the fourth bullet on page 12 of the draft report. 
Thank you again for the opportunity  to comment  on this draft report.  If your staff would like to 
discuss further our comments, please call me at (202) 778-3209 or Michelle Fried, Counsel  for 
Federal Programs, at (202) 778-8484. 
Sincerely, 
k&W  . 
Executive  Director, Regulatory  Affairs  APPENDIX  D 
ENDNOTEB 

1.  The  Federal  government  requires  Medicaid  agencies  enrolling  their  beneficiaries 
into  managed  care  to  meet  certain  quality  assurance  requirements.  Among  these 
requirements  are  those  that  call  for  each  managed  care  plan  to  have  an  internal 
quality  assurance  plan,  grievance  procedure,  and  total  enrollment  comprised  of  not 
more  than  75 percent  Medicaid  and/or  Medicare  beneficiaries.  Federal  regulations 
also  require  the  plans  to  allow  disenrollment  on  demand  within  1 month  of  requesting 
it,  although  this  can  be  waived  for  a 6-month  lock-in  period.  Statutory  requirements 
limit  the  manner  in which  plans  can  pay  their  physicians,  generally  referred  to  as  the 
physician  incentive  program.  Agencies  must  conduct  periodic  medical  audits  to  ensure 
quality  and  access  of  services  and  provide  data  on  reasons  for  beneficiary 
disenrollment.  Agencies  must  also  ensure  that  an  independent,  external  review  of  the 
quality  of  services  be  conducted  annually  for  each  managed  care  plan.  The  agencies, 
of  course,  can  build  upon  these  Federal  requirements,  and  the  Health  Care  Financing 
Administration  can  waive  certain  requirements  through  the  different  waiver  programs. 
2.  C. Baldwin,  Customer  Satisfaction  Surveys  for  Medicaid  Beneficiaries:  A  State 
Guide,  (Baltimore,  MD:  Office  of  State  Health  Reform  Demonstrations,  HCFA) 
November  1996. 
3. The  11 States  are:  Arizona,  California,  Florida,  Massachusetts,  Minnesota, 
Missouri,  New  York,  Ohio,  Oregon,  Rhode  Island,  and  Utah. 
4.  By technical  quality  of  care  we  mean  treatment,  examinations,  follow-through,  and 
preventive  care  as  opposed  to  interpersonal  communications  and  courtesy.  We  base 
this  distinction  on  categories  presented  in  the  Draft  Medicaid  HEDIS  document 
published  in  July  of  1995. 
5.  The  agencies  in  our  sample  were  all in various  stages  of  the  process  of  getting  and 
using  encounter  data  to  help  them  assess  plan  performance.  For  the  most  part,  as  of 
this  writing,  agencies  lack  validated  encounter  data.  They  also  were  in various  stages 
of  requiring  their  plans  to  report  HEDIS  indicators. 
6.  R.  H.  Brook,  C. J. Kamberg,  and  E.  A.  McGlynn,  Health  System  Reform  and 
Quality,  Journal  of the American  Medical Association  276(6)  (August  14,  1996):  478 
and  Massachusetts  Healthcare  Purchaser  Group,  Cost/Quality  Challenge,  March 
1996. 
7. 	 In  Oregon,  the  State  agency  conducted  a baseline  survey  in  1994,  then  conducted 
another  survey  in  1996.  The  agency  revised  the  second  survey,  thereby  affecting  the 
direct  comparability  of  results  between  the  two  surveys.  (The  Oregon Health  Plan 
D-l  Medicaid  Demonstration  Project:  1994  and  1996  Comparative  Client Satisfaction Data, 
Preliminary Aggregate Report.) 
8. Minnesota  Health  Data  Institute,  Evaluation  Report  of  the  Minnesota  Health  Data 
Institute  1995  Consumer  Survey,  (July  1996) 
9. Joseph  Mangano,  Ed.,  Oregon  Consumers  Favor  Doctor  Data  Quality 
Management  Update 6(18)  (September  18,  1996). 
10. Robert  Cunningham,  Ed.,  Consumers  Gaining  Leverage  in  Quality  Assurance 
Arena,  Medicine  & Health  (June  17, 1996).  See  also  Harold  S. Luft,  Modifying 
managed  Competition  to  Address  Cost  and  Quality,  Health Affairs  (Spring  1996):  23-
28  and  Frederick  Schneider  Research,  Medicaid  and  Managed  Care:  Focus  Group 
Studies  of  Low  Income  Medicaid  Beneficiaries  in  Five  States,  (Washington  DC:  May 
1996). 
11. Henry  J. Kaiser  Family  Foundation,  Americans  as Health  Care  Consumers:  The 
Role  of  Quality  Information,  (October  1996)  and  Robert  Cunningham,  Ed., 
Consumers  Gaining  Leverage  in  Quality  Assessment  Arena. 
12.  The  Medicaid  HEDIS  (Health  Plan  Employer  Data  and  Information  Set)  Work 
Group  comprises  individuals  and  organizations  that  worked  together  to  develop 
measures  for  monitoring  the  performance  of  managed  care  plans  serving  the  Medicaid 
population.  The  National  Committee  for  Quality  Assurance  sponsored  the  Work 
Group  in  association  with  HCFA  and  the  American  Public  Welfare  Association. 
13.  The  Agency  for  Health  Care  Policy  and  Research  (AHCPR)  is currently  funding  a 
5-year  project  aimed,  in  part,  at  developing  and  testing  questionnaires  to  assess  plans. 
This  project,  called  the  Consumer  Assessment  of  Health  Plans  Study  (CAHPS),  is at 
year  3 and  beginning  to  field  test  its  questionnaires.  The  CAHPS  encompasses  all 
types  of  health  plans  as well  as the  privately  and  publicly  (Medicare  and  Medicaid) 
insured.  The  Health  Care  Financing  Administration  has  called  for  managed  care 
plans  enrolling  Medicare  beneficiaries  to  use  the  CAHPS  questionnaire.  Eventually 
the  AHCPR  expects  that  its  CAHPS  questionnaires  will  be  available  to  interested 
parties  without  charge. 
14.  The  response  rate  was  calculated  after  deducting  204 beneficiaries  identified  as 
ineligible  for  reasons  such  as  disconnected  phones,  moved  out  of  the  area,  and 
deceased. 
15.  Of  the  17 respondents  that  indicated  they  had  not  conducted  any  surveys  that 
included  Medicaid  beneficiaries  in  the  past  2 years,  14 had  plans  to  conduct  such 
surveys  within  the  next  year. 
16.  In  another  recent  survey  of  health  plans  (with  Medicare  risk  and  cost  contracts), 
the  Office  of  Inspector  General  also  found  great  variety  in  plan  approaches  to  surveys 
(Office  of  Inspector  General,  HMO  Customer  Satisfaction Suweys,  March  1996). 
D-2 
 17.  In  another  recent  survey  of  health  plans  (with  Medicare  risk  and  cost  contracts), 
the  Office  of  Inspector  General  found  that  plans  used  their  survey  results  as  much  for 
marketing  as  for  improvements.  (Office  of  Inspector  General,  HMO  Customer 
Satisfaction Suweys,  March  1996) 
18.  American  Association  of  Health  Plans,  1995 AAHP  HMO  PPO  Trends  Report. 
19.  The  CAHPS  aims  to  help  consumers  choose  the  best-suited  health  plan  for  their 
needs  by  developing  and  testing  questionnaires  to  assess  plans,  reporting  the  survey 
information  to  consumers,  and  evaluating  their  usefulness.  This  ambitious  project 
encompasses  the  privately  insured  as well  as the  Medicaid  and  Medicare  populations. 
As  of  this  writing,  the  CAHPS  project  is at year  3 and  beginning  to  field  test  its 
surveys. 
20.  HCFA,  Monitoring  Guide  for  Medicaid  Managed  Care  Initiatives,  Working 
Draft  (September  1996)  and  Medicaid  Pre-Implementation  Review  Guide  for  Section 
1115  Demonstration  Waivers,  Working  Draft,  (August  1995). 
21. The  1981  amendments  also  allow  States  to  enroll  Medicaid  beneficiaries  in  limited-
risk  managed  care  organizations  (i.e.,  no  risk  for  inpatient  care)  that  fail  to  meet 
Federal  qualifications. 
22. 	The  following  States  have  implemented  their  1115 waivers:  Alabama,  Arizona, 
California,  Delaware,  Hawaii,  Minnesota,  Ohio,  Oklahoma,  Oregon,  Rhode  Island, 
Tennessee,  and  Vermont.  The  following  States  have  approved  1115 waivers  that  are 
pending  implementation:  Florida,  Illinois,  Kentucky,  Maryland,  and  Massachusetts. 
South  Carolina  has  an  approved  waiver  but  no  plans  for  implementation. 
23.  States  with  1115  waivers  under  review  are:  Georgia,  Kansas,  Louisiana,  Missouri, 
New  Hampshire,  New  York,  Texas,  Utah,  and  Washington. 
24. Federal  regulations  define  comprehensive  services  as either  inpatient  hospital 
services  and  one  other  mandatory  service  or  three  or  more  mandatory  services 
(42  CFR  434.21).  Mandatory  services  are  defined  in  statute  as inpatient  and 
outpatient  care,  physicians  services,  and  laboratory  and  diagnostic  services,  among 
others  (42  USC  1396d(a)). 
D-3