United States General Accounting Office

GAO Report to Ranking Minority Member,

Committee on Commerce, House of

Representatives

May 1997 MEDICAID MANAGED

CARE

Challenge of Holding

Plans Accountable

Requires Greater State

Effort

GAO/HEHS-97-86

GAO United States

General Accounting Office

Washington, D.C. 20548

Health, Education, and

Human Services Division

B-270335 

May 16, 1997

The Honorable John D. Dingell

Ranking Minority Member

Committee on Commerce

House of Representatives

Dear Mr. Dingell:

Over the past decade, Medicaid expenditures have soared. By fiscal year

1996, they reached $160 billion—nearly quadrupling fiscal year 1986

expenditures. Although the annual growth rate abated significantly in

1996, Medicaid expenditures continue to exert strong pressure on federal

and state budgets. To help bring these costs under control, states

increasingly are mandating significant numbers of their Medicaid

population to enroll in managed care programs. By emphasizing primary

and preventive care and treatment, it is hoped that managed care will

improve beneficiary health care while curbing health care costs.

As of June 1996, about 11 million Medicaid beneficiaries were enrolled in

“capitated” managed care programs.1

 Under a capitated managed care

model, states contract with managed care plans, such as health

maintenance organizations (HMO), and pay them a monthly, or capitated,

fee per Medicaid enrollee to provide most medical services—which are

coordinated through primary care physicians. This model, with its fixed

prospective payment for a package of services, creates an incentive for

plans to provide preventive and primary care and to ensure that only

necessary medical services are provided. However, managed care also can

create an incentive to underserve or even deny beneficiaries access to

needed care since plans and, in some cases, providers can profit from not

delivering services. Moreover, Medicaid beneficiaries required to enroll in

managed care may find it difficult to seek alternative care if they find that

plan providers fail to meet their needs.

Because of your concern about these issues, we reviewed state efforts to

hold managed care plans accountable for meeting Medicaid program goals

and for providing beneficiaries enrolled in capitated managed care plans

the care they need. As agreed with your office, we focused our study on

the difficulties that purchasers, including states, have in monitoring

1

Based on the most current data available from the Health Care Financing Administration (HCFA) on

managed care enrollment. Another 4 million individuals were enrolled in noncapitated managed care

programs. Of the total 15 million managed care enrollees, about 2 million were enrolled in more than

one plan, according to HCFA.

Page 1 GAO/HEHS-97-86 Medicaid Managed Care Accountability
B-270335 

managed care programs and on state efforts to (1) ensure Medicaid

beneficiaries have access to appropriate providers, (2) assess the

adequacy of medical care provided through contracted plans, and

(3) determine beneficiary satisfaction with plan performance.

To understand the types of issues states face in ensuring accountability

and quality in their capitated Medicaid managed care programs and the

steps taken to address these issues, we visited four states—Arizona,

Pennsylvania, Tennessee, and Wisconsin. At the time of our review, these

four states collectively had almost 1.9 million Medicaid beneficiaries

enrolled in their managed care programs. To analyze and illustrate state

actions, we focused the scope of our work on 10 core accountability

measures or processes deemed essential by HCFA and experts we

contacted.2

 We reviewed these states’ contracts with managed care plans

and other plan requirements, as well as their efforts to monitor plan

performance. Appendix I provides more detailed information on our scope

and methodology.

Results in Brief Ensuring that managed care plans provide enrollees the care that they

need is a formidable task for private and public purchasers alike. In

establishing their managed care programs, purchasers can require

contracted plans to meet certain conditions—such as maintaining

adequate provider networks and complying with data collection

requirements—that help to hold them accountable for providing enrollees

with appropriate care. However, establishing criteria for these conditions

and monitoring plan compliance are often difficult because of a lack of

population-based standards or benchmarks for what constitutes

appropriate care or expected outcomes. In addition, individual-level data

on patient care, such as those that are generated in a claims-based

fee-for-service system, are not readily available. For states, establishing

standards of care and tapping into alternative information sources on

service utilization to assess the care that Medicaid beneficiaries receive

can be a challenge. The four states that we visited—Arizona, Pennsylvania,

Tennessee, and Wisconsin—have built access and data collection

requirements into their contracts with managed care plans. We found,

however, that plan compliance with the requirements we reviewed does

not necessarily ensure that beneficiaries are receiving the care that they

need.

2

Our work did not include a complete assessment of each state’s entire quality assurance process.

Page 2 GAO/HEHS-97-86 Medicaid Managed Care Accountability
B-270335 

A number of these states’ requirements aim to ensure managed care plans

develop and maintain provider networks that are sufficient to meet the

needs of Medicaid beneficiaries. Some are criterion-based, such as

patient-to-primary-care-physician ratios. For example, two states required

that plans not exceed a maximum patient-to-primary-care-physician ratio

of 2,500 to 1. Compliance with such a requirement, however, does not

necessarily demonstrate that a network is sufficient to meet the needs of

Medicaid beneficiaries. Patient-to-primary-care-physician ratios generally

do not consider the number of networks a primary care physician

participates in or a physician’s capacity or willingness to see Medicaid

patients. Of the states that we visited, only Arizona required physicians to

report their work load in full-time-equivalent terms and identified primary

care physicians who participate in more than one plan and could be

counted more than once. The four states also require plans to provide a

full range of specialty services, even if this means beneficiaries must be

referred to providers outside the plan’s network. However, because there

are no established standards for specialists, these states have not specified

the types and numbers of specialists to include in plan networks, making it

difficult for these states to measure the adequacy of plan specialist

networks before awarding a contract. Once plans have a contract, states

can monitor the numbers and types of specialists participating in the

network, but this does not necessarily indicate whether beneficiaries

actually gain access to specialty care when they need it.

Given the difficulties associated with gauging the adequacy of a provider

network, the four states that we visited have taken additional steps to

assess the adequacy of the medical care that beneficiaries enrolled in

managed care receive. For example, each state has looked at aggregated

statistics on the use of specific services. Some have found that, compared

with fee-for-service, Medicaid managed care recipients were more likely to

receive certain preventive and diagnostic services, such as childhood

immunizations and cancer screenings. Arizona, Tennessee, and

Pennsylvania also have invested in developing encounter data—the

individual-level data on all services provided to all patients. Encounter

data can enable states to conduct their own analyses on a wider array of

services than is possible using aggregated statistics. These analyses allow

states to examine patterns of care across plans, such as differences in

service delivery by selected types of services, beneficiary groups, and

providers. To date, Arizona has made the most use of its encounter data,

including using them as the state begins to develop quality indicators.

Tennessee’s early efforts primarily focused on developing and validating

its encounter data; more recently, the state has begun to use these data to

Page 3 GAO/HEHS-97-86 Medicaid Managed Care Accountability
B-270335 

assess service utilization patterns. Pennsylvania’s use of encounter data

was even more limited. All four states also use data from plan-conducted

clinical studies and state-conducted medical record audits to help assess

patient care. Improved plan and state methodologies, however, could

increase the usefulness of the data collected from these reviews.

The four states that we visited also have sought to assess the adequacy of

patient care by tapping into information provided directly by Medicaid

beneficiaries enrolled in managed care, such as patient satisfaction

surveys and data gathered from grievance processes. While it is important

to gauge patients’ satisfaction with the care they receive, satisfaction data

generally are not reliable measures of quality; most people lack the

knowledge needed to adequately evaluate the appropriateness of the care

they receive—or do not receive. In addition, newcomers to managed care

may not fully understand how the system operates to effectively access

services, advocate on their own behalf, or register dissatisfaction with

their plan or provider. This is especially true for individuals with diverse

language and cultural needs. Regardless, we found that if the states we

visited improved certain methodologies for designing satisfaction surveys

and stratified their survey and grievance data, they would have a better

understanding of the needs and concerns of their Medicaid beneficiaries

enrolled in managed care—especially those with special needs or chronic

illnesses, who may experience problems in accessing services but whose

numbers are too small to show up in analyses of broad-based data.

Background Medicaid, a joint federal-state health financing program for the poor,

provides health care for about 37 million low-income people.3

 In fiscal year

1996, Medicaid expenditures accounted for more than 20 percent of state

budgets. To help control expenditures and expand access to health care,

36 states have mandated enrollment for some portion of their Medicaid

population in managed care programs. As of June 1995, nearly 14 percent

of Medicaid eligibles were enrolled in capitated programs.

Under a capitated managed care system, states pay contracted plans a

monthly per-enrollee amount before services are delivered—a distinct

departure from the traditional claims-based fee-for-service system in

which providers are paid for each service as bills are submitted. In turn,

the plans employ or subcontract with primary care physicians, who

3

Medicaid was established in 1965 as title XIX of the Social Security Act (42 U.S.C. 1396 et seq.).

Medicaid is administered at the state level, with federal oversight by HCFA within the Department of

Health and Human Services (HHS).

Page 4 GAO/HEHS-97-86 Medicaid Managed Care Accountability
B-270335 

coordinate the delivery of health services.4 Some plans pay their

subcontracted providers on a fee-for-service basis for care provided, while

others pass certain financial risks on to providers by linking the providers’

revenues or profits to the total number of services provided to plan

enrollees. While capitated managed care has strong cost-containment

incentives, it also provides incentives for plans and providers to limit

services—not only must plans and providers absorb all costs that exceed

the capitation rate, they profit if the capitation rate exceeds their costs.

Nationwide, most states initially implemented Medicaid capitated

managed care programs by allowing beneficiaries to enroll on a voluntary

basis in limited geographic areas. These programs were largely targeted to

low-income families who received financial assistance under Aid to

Families With Dependent Children (AFDC) and pregnant women and

children who qualified for Medicaid. Increasingly, states are mandating

beneficiary enrollment and expanding their programs to more geographic

areas. In addition, they are beginning to include more populations with

specialized needs, such as blind or disabled individuals who qualify for

Medicaid under the Supplemental Security Income (SSI) program. As we

reported in July 1996, 17 states had extended their Medicaid managed care

programs to these more vulnerable populations.5

States must comply with certain federal statutory requirements for the

development and oversight of their managed care programs. HCFA can

waive some of these requirements—such as a beneficiary’s freedom to

choose any provider—to enable states to restrict beneficiaries to the

providers participating in a managed care network. Waivers also allow

states to expand the scope of their programs to populations not otherwise

eligible for Medicaid. These waivers are of two types: program or

demonstration.6

 Program waivers allow states to require beneficiaries to

join a managed care plan, but beneficiaries are generally allowed to switch

plans every 30 days. Demonstration waivers provide states with greater

flexibility, and while they are more difficult to obtain than program

waivers, they have been granted more frequently in recent years. States

request demonstration waivers to establish mandatory programs that lock

beneficiaries into one plan for periods of up to 12 months or to expand

4

Primary care physicians may be general internal medicine practitioners, family and general

practitioners, pediatricians, or obstetricians and gynecologists.

5

See Medicaid Managed Care: Serving the Disabled Challenges State Programs (GAO/HEHS-96-136,

July 31, 1996).

6

Program waivers are authorized under section 1915 of the Social Security Act. Demonstration waivers

are also known as section 1115 waivers, after the section of the Social Security Act that authorizes

them.

Page 5 GAO/HEHS-97-86 Medicaid Managed Care Accountability
B-270335 

eligibility to uninsured populations. Table 1 compares various

characteristics of the two waiver types.

Table 1: Characteristics of Managed

Care Programs Under Program and

Demonstration Waivers

Program waivers Demonstration waivers

General characteristics

Allow for waiver of certain Medicaid

requirements.

Allow for waiver of a broader range of

Medicaid requirements.

Waivers renewed for 2- to 5-year periods. Generally not renewed.a

Generally used to establish primary care

case management programs and home-and

community-based service programs.

More recently used to establish broad

changes in Medicaid programs.

Characteristics pertaining to capitated managed care

Plans must comply with 25% requirement for

private enrollment.

Plans may enroll Medicaid patients

exclusively.

Full range of mandatory services must be

offered.

Benefit package may be modified.b

Beneficiaries may be mandated to enroll in

plan and can be locked in for no longer than

1 month.c

Beneficiaries may be locked in for up to 12

months.

aThe Congress has authorized renewal of some demonstration waivers.

bTo date, only Oregon has been permitted to modify the benefit package for traditional Medicaid

beneficiaries. Other states have been permitted to offer a modified package only to those newly

eligible for Medicaid coverage under the demonstration.

cLock-in is up to 6 months for capitated plans meeting certain federal requirements.

At the time of our review, Arizona and Tennessee had demonstration

waivers for their mandatory statewide programs, which served both AFDC

and SSI populations. Wisconsin had a program waiver for its mandatory

program, which served only the AFDC populations in 5 of its 72 counties. In

contrast, Pennsylvania had voluntary and mandatory managed care

programs. The voluntary program—the larger program at the time of our

review—served both AFDC and SSI populations in 13 of its 90 counties.7

 This

program required no federal waiver.

To ensure that states comply with statutory and HCFA requirements, HCFA

reviews state contracts with managed care plans. It also monitors state

programs through independent evaluations and periodic reviews of

state-submitted information on expenditures, medical services, and

7

At the time of our review, 475,000 beneficiaries were enrolled in Pennsylvania’s voluntary managed

care program. With the February 1997 expansion of its mandatory program, 177,000 beneficiaries were

participating in the voluntary program as of April 1, 1997; 66,000 are projected to be in the voluntary

program by 1998.

Page 6 GAO/HEHS-97-86 Medicaid Managed Care Accountability
B-270335 

enrollment data, which HCFA requires all states to report. The nature of

HCFA’s requirements and oversight role depends on the waiver type. For

example, under a demonstration waiver, HCFA develops terms and

conditions that vary by state, depending on the provisions being waived.

(For a more detailed discussion of federal regulations and HCFA

requirements for waiver programs, see app. II.)

Difficulties in

Monitoring Managed

Care Stem From

Limited Standards of

Care and Data on

Service Utilization

Purchasers of managed care face a number of difficulties in ensuring

enrollees receive the care that they need. In contrast to fee-for-service

care—where the incentive is to oversupply services to increase

revenues—capitated managed care, with its fixed payment system,

contains incentives to provide fewer services to maximize short-term

profits. Assessing how well the care delivered matches beneficiary needs

is difficult because few aggregate or population-based utilization

standards or benchmarks on delivery of care patterns have been

established for managed care. Benchmarks derived from providers and

patients in the fee-for-service sector may not be appropriate since service

utilization patterns are expected to change under managed care.

Even where standards of care do exist for selected conditions or

procedures, such as for prenatal care or childhood immunizations,

monitoring the actual delivery of such services in capitated managed care

is difficult because data on service utilization do not flow as readily as in a

fee-for-service environment. In fee-for-service care, data on service

delivery are captured in individual claims, which are submitted for

payment as services are provided. In managed care, however, purchasers

prospectively pay plans a monthly fee for services not yet provided. Unless

plans capture service data in another form, it is difficult to identify the

services actually provided. Without specific data collection requirements,

providers may lack the incentive to accurately report individual-level data

on all services provided since payment is not linked to documentation of

the care provided.

To learn more about what transpires in the delivery of managed care

services—and to help ensure that the health care services they contract

for are appropriately provided—private and public purchasers also rely on

other measurement tools and data sources. Among these are reviews of

patient medical records and surveys of patients on the care they receive

from plans. Reviews of medical records can help purchasers assess the

care provided to individual patients. These assessments, however, are

time-consuming and costly and are generally undertaken for small

Page 7 GAO/HEHS-97-86 Medicaid Managed Care Accountability
B-270335 

numbers of patients. Information from patients, while more accessible

than medical records, can be problematic as well. Patients typically are

not in a position to know what specific care or services they need for a

given condition and often cannot assess the appropriateness of the care

they receive—or do not receive. In addition, patients new to managed care

may confuse differences in the way the system is meant to operate with

deficiencies in the care provided. Problems associated with obtaining

meaningful patient survey information may be even more pronounced for

those in the surveyed population with unique language or cultural needs or

who are unaccustomed to receiving routine health care in a structured

system. Educating and informing prospective and newly enrolled

beneficiaries about managed care and helping them learn how to use the

system—as some states have done in their Medicaid managed care

programs—can mitigate these problems.8

Most states are grappling with these and other issues associated with

adapting and developing systems and processes for managed care—a

relatively new health care environment. According to some experts, many

states are struggling to maintain the staff needed to establish and oversee

their programs, since frequent turnover of staff with managed care

expertise is common. It is not surprising, then, that states are at various

stages in their program development and monitoring efforts. The four

states that we visited have taken a number of steps to overcome these

various challenges and improve the odds that their money is well spent in

their managed care programs. These states have established contract

requirements that aim to ensure that participating managed care plans

have the capacity to provide adequate care to enrolled Medicaid

beneficiaries. Prominent among these requirements are standards for

plans’ provider networks. In monitoring participating plans, these states

obtain information to assess actual services delivered from various

sources, including plan-collected and -submitted data, state reviews of

patient medical records, and beneficiaries’ reports on their experiences.

Measures to Assess

Beneficiaries’ Access

to Care Are Still

Evolving

Before a contract is awarded, managed care plans must demonstrate that

their provider networks are sufficient to meet the anticipated needs of

enrolled Medicaid beneficiaries. In an attempt to measure provider

network sufficiency, the states we visited—Arizona, Pennsylvania,

Tennessee, and Wisconsin—have focused on quantitative or other

measures related to primary care physicians and specialist care. But just

8

See Medicaid: States’ Efforts to Educate and Enroll Beneficiaries in Managed Care

(GAO/HEHS-96-184, Sept. 17, 1996).

Page 8 GAO/HEHS-97-86 Medicaid Managed Care Accountability
B-270335 

as there are few standards for health service utilization, there are few

standards for what constitutes a sufficient provider network. Three of the

four states have established a specific number of primary care physicians

that a plan must have, and all require plans to provide a full range of

specialty services. The states also have relied on criteria that measure

beneficiaries’ ability to reach their primary care physician within a

reasonable time, in terms of maximum travel distances and waiting times.

After contract award, the states use various monitoring techniques to

determine the extent to which provider practices are in fact open to

Medicaid beneficiaries. These measures, however, do not necessarily

ensure that beneficiaries have access to the care that they need. Whether

these measures provide meaningful information on beneficiary access is

largely dependent on whether state monitoring efforts are independent

and systematic and go beyond plan-reported, paper-based indications of

compliance.

Compliance With Primary

Care Physician

Requirements Provides

Incomplete Information

About Network Adequacy

One criterion that states have established in an effort to ensure a sufficient

provider network relates to the availability of primary care physicians,

expressed as a ratio of enrolled beneficiaries per primary care physician.9

At the time of our review, Arizona and Tennessee used a maximum

patient-to-primary-care-physician ratio of 2,500 to 1, as required by the

conditions of their demonstration waivers, and Pennsylvania required

plans to meet a ratio of 1,600 to 1.10 Wisconsin did not have specific

contractual requirements for plans but looked for a ratio of approximately

1,200 to 1. To monitor plan compliance with these ratios, the states require

plans to submit updated provider listings either annually, to coincide with

contract renewal, or as frequently as monthly. The states also require

plans to report all changes to the network as they occur and to note in

their provider directories given to beneficiaries those providers who

currently do not accept new patients.11

The states that we visited have found that plans in their managed care

programs have complied with their patient-to-primary-care-physician

ratios. But compliance with these ratios may not indicate actual physician

capacity or Medicaid beneficiaries’ access to care. We believe that the

9

In 1994, there was one primary care physician for every 1,173 United States citizens (based on GAO

analysis of the HHS Area Resource File).

10Beginning in October 1997, Arizona will require plans to have maximum

patient-to-primary-care-physician ratios of 1,800 to 1 for adults and 1,200 to 1 for children under age 13.

11Typically, plan directories are updated annually; consequently, beneficiaries must directly contact the

managed care plan for more current information on physician availability.

Page 9 GAO/HEHS-97-86 Medicaid Managed Care Accountability
B-270335 

number of primary care physicians and their availability to treat patients

may be overstated for two reasons. First, if the state reviews a plan’s

network capacity by looking at only that plan rather than looking at all

plans collectively, providers who participate in more than one plan may be

counted more than once. Second, if the state does not use full-timeequivalency data to determine network capacity, network physicians’

other lines of business, such as treating Medicare or privately insured

patients, are not taken into account. Of the states that we visited, only

Arizona reviews provider participation across plans and assesses provider

capacity in full-time-equivalency terms.

For patient-to-primary-care-physician ratios to be an effective measure of

patient access to care, states must also ensure that plan physicians are

actually available to treat Medicaid beneficiaries. Ratios simply indicate

the number of physicians that have contracted with a plan; they do not

indicate the number of patients physicians are willing to treat and the

extent to which physicians actually provide services. Arizona and

Pennsylvania independently assess the extent to which physician practices

are open to Medicaid beneficiaries by periodically or randomly calling

physician offices to determine whether they are accepting new Medicaid

patients. When done on a systematic basis, these checks can better ensure

that provider practices are open to Medicaid beneficiaries.

To improve oversight of its provider networks, Arizona increased plans’

quarterly provider reporting requirements in September 1996. Previously,

plans were required to report provider names and the number of

beneficiaries seen by providers. Plans must now submit additional data on

provider access, such as the maximum number of Medicaid beneficiaries

that a provider will accept, the total number of beneficiaries currently

assigned to a plan, the providers who are accepting new members,

specialty services that are available, and foreign languages spoken by

providers.

Specialist Mix Is Not

Specified in Plan Contracts

Patients often require more specialized care than their primary care

physician can provide—such as oncological, urological, or pediatric

subspecialty care. Yet, assessing beneficiary access to such care is even

more difficult than assessing access to primary care physicians. And as

states move more beneficiaries with special needs or chronic conditions

into managed care, ensuring beneficiary access to appropriate specialty

services will become even more critical.

Page 10 GAO/HEHS-97-86 Medicaid Managed Care Accountability
B-270335 

Assessing the availability of specialty providers within a network is

problematic for at least two reasons. First, there are no criteria or

standards—in fee-for-service or managed care—for the number and mix of

specialists needed to serve a population or for when and how often

beneficiaries should be referred to specialists. Second, some

specialists—especially those that are used infrequently—often are not

included in the network and can only be accessed outside the network.

Since these specialists are not in the network, states cannot readily assess

their availability to beneficiaries when needed.

The four states that we visited require that contracted plans provide a full

range of specialty services, even if this requires referring beneficiaries to

providers outside the plan’s network. Without recognized standards,

requirements for specialists are often vague and expressed in terms of an

objective—such as, “provide access to necessary specialty care”—that

cannot be measured before the contract award. While the four states do

not specify in the contracts the types and numbers of specialists that plans

must include in their networks, they do count the number and type of

specialists available in any one plan. This is done by reviewing the listing

of providers in a plan network during the contracting and contract renewal

processes. Each of these states also reviews plan listings periodically and

requires plans to report all changes in their specialty networks. For

example, Tennessee officials told us that the state uses a zip-code-based

computer program to check the location of specialists each quarter—or

more frequently if inquiries or questions come up on a particular provider

type. States then rely on the judgment of their experienced contract

staff—which may include health care professionals—to determine

whether the plans’ specialist networks are adequate.

This type of review, however, does not inform states of the specialist

services that Medicaid beneficiaries may use or request, especially if the

services involve out-of-network referrals. In the absence of accepted

standards for specialty care, states have relied on other information

sources—such as data on service utilization and beneficiary

satisfaction—to help them monitor the appropriateness of care provided.

Maximum Travel and

Waiting Requirements Are

Additional Criteria for

Assessing Beneficiary

Access to Care

As additional criteria for assessing the adequacy of provider networks,

states commonly stipulate the maximum time and distance beneficiaries

must travel to their primary care physician’s office, the number of days

they must wait for an appointment to see their physician, and the time they

wait in the physician’s office to be examined. These measures were

Page 11 GAO/HEHS-97-86 Medicaid Managed Care Accountability
B-270335 

developed on the premise that certain factors—such as lengthy travel

times and distances, a physician’s failure to schedule appointments in a

timely manner, and excessive wait times in the physician’s office—can

discourage patients from seeking care.

In general, the states we visited established maximum travel requirements

of 20 miles or 30 minutes for urban areas and 30 miles or 30 minutes for

rural areas. For their specialist networks, the four states do not have or

grant some exceptions to travel requirements, since significant portions of

their Medicaid population can reside in rural areas where certain types of

specialty care may not be available. These states also require providers to

meet state-established standards for scheduling appointments and

attending to beneficiaries in a timely manner.12

To determine whether plans are complying with the time and distance

travel requirements, each state we visited reviews documentation on the

location of provider sites prior to contracting with a plan and on a periodic

schedule after contracting. Arizona and Tennessee, for example, use a

computer-based zip-code program. This allows them to determine whether

the locations of the primary care physicians and other providers in the

network, such as dentists and hospitals, meet contractual time and

distance requirements. In addition, each state requires plans to provide

transportation for beneficiaries who require medical attention and who

cannot get to their provider’s location on their own, such as the elderly.

These four states use various approaches—involving plan-provided data or

independent checks—to monitor provider compliance with wait-time

requirements. For example, each state requires plans to include

appointment standards in its subcontracts with providers and to review

logs to ensure that providers comply. During their periodic compliance

reviews, the states review the plans’ procedures to ensure that providers

have complied with the standards; the states also review selected sign-in

and appointment books. For example, Tennessee found during one

quarterly review of plan networks that some plans were not complying

with the contractual travel requirements for their dental networks.

According to state officials, the plans were notified of the deficiencies, and

most took actions to resolve them. In at least one case, a plan did not take

action and the state withheld 10 percent of the plan’s capitation payment

12Generally, the four states’ requirements to schedule appointments with primary care physicians are

same-day appointments for emergency care, within 24 to 48 hours for urgent or sick care, and 2 to 3

weeks for routine preventive care. Requirements for in-office waiting times are generally 30 to 45

minutes. Some states also specify appointment standards for specialty referrals and dental services, as

well as maternity, mental health, and substance abuse services.

Page 12 GAO/HEHS-97-86 Medicaid Managed Care Accountability
B-270335 

until it was in compliance. States also have independently assessed

provider compliance by periodically or randomly calling provider offices

to schedule appointments. However, such periodic calls may not be

systematic enough to provide the information required to identify

problems. As one means of gaining additional insight into beneficiary

experience with providers, Arizona and Tennessee have included

questions in their beneficiary satisfaction surveys that are specifically

related to wait times.

Systems for Assessing

Adequacy of Medical

Care Provided Can Be

Further Improved

Recognizing the challenges associated with measuring the capacity of plan

provider networks and their ability to ensure adequate care, states can use

other accountability measures and processes to assess the actual care

Medicaid beneficiaries receive. To identify average levels or recent

changes in the use of services by beneficiaries enrolled in managed care,

states can use plan reports of utilization statistics, which summarize

selected services provided to specific populations. For more extensive

analyses, states can use encounter data, which are individual-level data for

each service provided to each enrollee. Encounter data allow states to

identify the care received by any individual and the provision of any

procedure. In addition, states can conduct, or require plans to conduct,

other analyses—such as clinical studies and medical record audits—that

review the full medical records to assess the appropriateness of the care

received by a sample of beneficiaries.13

The four states that we visited have taken different approaches in

implementing their data collection methods. For example, Pennsylvania

and Wisconsin have relied primarily on collecting and using aggregated

utilization statistics to measure the adequacy of care. Arizona and

Tennessee also have required plans to collect and submit encounter data.

However, both states have found that developing reliable and useful

encounter data has required lengthy and continuing investment. As a

result, these two states have primarily used their encounter data to

identify services that may be over- or underutilized and health care areas

on which to focus their studies and audits. Arizona has begun to use its

encounter data to develop standards for measuring the quality of the care

provided.14 For all four states, we found that certain improvements in their

13For specific federal regulations and guidance related to these accountability measures and processes,

see table II.1.

14Arizona further uses its encounter data to estimate the cost of serving beneficiaries in each county

and to set capitation rates by county. See Arizona Medicaid: Competition Among Managed Care Plans

Lowers Program Costs (GAO/HEHS-96-2, Oct. 4, 1995).

Page 13 GAO/HEHS-97-86 Medicaid Managed Care Accountability
B-270335 

methodologies for conducting clinical studies and medical record audits

could increase the usefulness of these reviews.

States Challenged to

Develop Effective

Utilization Statistics and

Encounter Data for

Monitoring Managed Care

Although utilization statistics and encounter data both capture patient use

of services, utilization statistics are summary data that are generally relied

on to show the frequency with which a service is accessed by a specific

population. As such, there are several difficulties associated with using

these summary statistics in assessing services provided under managed

care. For example, utilization statistics are often compiled for only specific

types of services, such as mammograms or childhood immunizations.

Consequently, utilization statistics cannot be used to determine the full

range of services that beneficiaries may receive. For example, in a prior

study, we found that analyses of utilization statistics on early and periodic

screening, diagnosis, and treatment (EPSDT) could not determine the actual

number of EPSDT-eligible children who received required screenings or

whether children with post-screen referrals actually received follow-up

diagnosis and treatment.15 Utilization statistics also have generally not

been used to assess the performance of individual providers. To obtain

utilization statistics for other sets of services or services supplied by

individual providers, states would need to modify their reporting

requirements—which could be problematic for plans if they have to alter

their data systems or provider reporting requirements to meet new state

requirements.

Unlike utilization statistics, which aggregate service use by a population,

encounter data document all services that individual patients receive.

While encounter data have certain limitations, these data provide states

more flexibility to detect problems in beneficiary care by identifying

patterns of service use by individual beneficiaries and services provided

by individual providers. For example, these data can be used to assess the

participation of any provider or group of providers and analyze patterns of

care for specific diagnoses or procedures. With encounter data, states also

can explore service delivery beyond what is captured by utilization

statistics. These statistics allow plans and providers to “teach to the test,”

that is, focus on service delivery areas that they know will be measured,

perhaps to the exclusion of other services. Since encounter data

encompass all services for all beneficiary populations, all plan-provided

services are subject to state review, and the services being reviewed can

15See Medicare Managed Care: More Competition and Oversight Would Improve California’s Expansion

Plan (GAO/HEHS-95-87, Apr. 28, 1995).

Page 14 GAO/HEHS-97-86 Medicaid Managed Care Accountability
B-270335 

be changed periodically without adding to the administrative burden of

plans.

Certain analyses of encounter data, however, are currently constrained by

the limited number of benchmarks or standards against which states can

measure the care provided for certain conditions. For example, while

recognized standards exist for prenatal care and childhood immunizations,

many diagnoses have multiple alternative treatments or therapies. Vocal

chord stress, for instance, might be treated with medication, voice therapy,

or surgery. Individual-level encounter data cannot show the

appropriateness of many treatments provided to beneficiaries, nor can

they provide information about beneficiaries who do not seek treatment.

Regardless of these constraints, encounter data are valuable in that they

support a wider array of analyses than do utilization statistics. Moreover,

they provide the potential for supporting even more analyses as additional

benchmarks are developed.

To ensure that the utilization statistics and encounter data are usable, data

collection standards must be established and plans must be monitored for

compliance with these standards. Validating the accuracy and

completeness of encounter data requires additional measures—especially

in a fully capitated system where provider payment is not directly linked

to the documentation of each service provided. Although the data

collection efforts in the four states that we visited varied

considerably—due, in part, to the federal guidelines and requirements

associated with their waiver type—overall, their use of encounter data to

identify problems in beneficiary care has been minimal.

Wisconsin requires contracted plans to collect and submit on a quarterly

basis utilization statistics on 59 types of health care services, including

maternal and child health, mental health, and emergency room visits.16

Using fee-for-service experience as the benchmark, the state

analyzes—and publicly discloses in periodic reports—plan-submitted

statistics on certain services, such as childhood immunizations; lead

testing; mammograms; and dental, vision, and hearing examinations. For

example, the state found that certain preventive services, such as Pap

smears and childhood immunizations, increased in managed care, whereas

emergency room visits decreased—a redistribution of service settings that

is consistent with the goals of managed care. In addition, the state found

16For each indicator for which summary statistics are reported, plans also report a complete

individual-level patient history file, which documents all services that the patient has received. This

system allows Wisconsin to determine the actual number of beneficiaries who receive a service and to

conduct more extensive analyses on areas of concern.

Page 15 GAO/HEHS-97-86 Medicaid Managed Care Accountability
B-270335 

that beneficiary use of dental services was less in managed care than in

fee-for-service care and took action to improve beneficiary access to these

services. In relying on utilization statistics rather than on encounter data,

however, Wisconsin cannot easily or independently assess the care that

Medicaid beneficiaries receive beyond the 59 services. Such analyses

would require encounter data for 100 percent of the services provided,

which the state has opted not to collect in an effort to limit the

administrative burden that collecting encounter data can place on the

state and the plans.

Pennsylvania requires each plan participating in its voluntary managed

care program to collect both utilization statistics and encounter data. But

the state does not prescribe a method for collecting and validating the

encounter data and does not require plans to routinely submit these data

to the state for review and analysis.17 In April 1996, the state required all

plans to submit EPSDT data for a specific time period to respond to

concerns regarding Medicaid beneficiaries’ access to this service.

As part of their demonstration waiver requirements,18 Tennessee and

Arizona require plans to collect and submit encounter data on 100 percent

of services, potentially giving these states the greatest flexibility in their

analyses. Tennessee’s efforts to date have focused on providing technical

assistance to help plans configure their computer systems and data

elements to ensure accuracy. HCFA is also providing the state with

technical assistance in developing methodologies to analyze its encounter

data. Despite these start-up issues, the state has begun to analyze its

encounter data and to review provider practice patterns to identify

potential over- or underutilization of care.

Of the states we visited, Arizona has the most comprehensive system for

validating and analyzing plan-generated encounter data and has made the

most use of these analyses to guide state actions. To ensure that its

encounter data are accurate, complete, and timely, the state conducts two

validation studies of plan-submitted encounter data each contract year.

The first study takes a random sample of medical records and manually

compares these records with the encounter data submitted by the plans.

The second study compares the number of inpatient maternity

17Pennsylvania requires plans to submit encounter data for Medicaid beneficiaries enrolled in the

state’s mandatory managed care program. At the time of our review, 75,000 beneficiaries were enrolled

in mandatory managed care. Nearly 500,000 are now enrolled as a result of the state’s February 1997

expansion.

18HCFA requires all states with a demonstration waiver to collect encounter data for 100 percent of

services.

Page 16 GAO/HEHS-97-86 Medicaid Managed Care Accountability
B-270335 

hospitalizations with newborn reports from hospitals. The state also uses

its encounter data to evaluate individual plan performance. Furthermore,

beginning October 1997, Arizona will require plans to compile encounter

data on specific measures, including mammography screening, cervical

cancer screening, children’s dental services, and well-child care.19 In

essence, Arizona will require plans to use their encounter data to compute

certain utilization statistics. This should make the information available to

the state sooner. Arizona also is creating a quality management system

that uses outcome-based standards and, over the past several years, has

been creating a baseline for these standards. The system will seek to

produce data that could indicate whether preventive care—such as dental

visits for children, mammograms, and Pap smears—prevent more serious

health problems.

In the three states with mandatory managed care programs, plans may be

sanctioned if they do not comply with utilization statistics or encounter

data requirements. Tennessee assesses a 10-percent withhold on

capitation payments for each month that a plan does not comply with data

submission requirements. If the problem is not corrected within 6 months,

the state keeps the withheld funds. Arizona imposes a financial penalty

based on the number of data errors identified in its plans’ encounter data.

Wisconsin may impose financial penalties if plans do not meet accuracy

and timeliness requirements. Over the last 2 years, the amount collected in

penalties by Tennessee and Arizona has been minimal, and Wisconsin has

not assessed any penalties.

More Targeted Clinical

Studies and Medical

Record Audits Are Needed

to Assess Impact of

Medicaid Managed Care

Recognizing the limitations of utilization statistics and encounter data,

each of the four states that we visited uses reviews of samples of

individual patient’s medical records to determine whether appropriate and

adequate care has been provided. Each state requires contracted plans to

conduct at least one clinical study each year. A clinical study focuses on

certain aspects of health care services, such as maternal health, to answer

questions about the quality and appropriateness of care that has been

provided. Each state also conducts its own medical record audits—as

required by federal regulation—either internally or through a contracted

external review organization, such as the State Peer Review Organization.

Medical record audits also have the potential to assess the appropriateness

19The required measures are based on Medicaid Health Plan Employer Data Information Set (HEDIS).

Medicaid HEDIS provides guidance that states may use to measure, improve, and report on health plan

performance. Medicaid HEDIS was the collaborative effort of representatives from state Medicaid

agencies, managed care plans, the National Committee on Quality Assurance (NCQA), professional

health and welfare organizations, beneficiary advocacy groups, HCFA, the U.S. Public Health Service,

and others.

Page 17 GAO/HEHS-97-86 Medicaid Managed Care Accountability
B-270335 

of the care provided as well as determine whether patients’ medical

records properly document the health care and services that they received.

The states’ current approaches to conducting these studies and audits

could be improved to yield more useful findings. For example, we found

that clinical studies often focus on the Medicaid population as a whole and

less frequently target populations that may not fare as well under managed

care, such as people with disabilities or chronic illnesses.20 In addition, the

states’ sample sizes for its medical record audits appear to be insufficient

to enable states to draw conclusions about the adequacy of the

documentation of beneficiary care, particularly for certain populations or

conditions too small to show up in pure random samples.

Managed Care Plans’ Clinical

Studies

Pennsylvania, Tennessee, and Wisconsin allow plans to select a topic to

study from one or more health care areas that the state identifies.21 In

selecting study areas, the three states rely on the professional judgment

and experience of staff in the state Medicaid agency and on independent

expert opinion the states may seek. While studies of this nature can

provide states with valuable information on plan performance and a

baseline for evaluating subsequent plan performance, we found that the

plan-conducted studies had several limitations. For example, only

Wisconsin specified study areas that included conditions for which

beneficiaries might require more care than the general Medicaid

population, such as children with special care needs. Problems in the care

for such beneficiaries may not be detected in studying samples of the

overall Medicaid population because, relative to the larger population,

their numbers are too small. In addition, allowing plans to select their

clinical study topics gives them the latitude to select a topic where

improvement may be needed, but it also allows them to select a topic that

would yield positive results about plan performance. Finally, allowing

plans to select topics to study does not enable states to compare results

across plans for certain conditions or topics.

We also found that the states we visited conducted limited reviews of plan

methodologies for clinical studies. Pennsylvania, for example, reviews

plan methodologies during periodic quality assurance audits—after the

plans’ clinical studies have been completed. Wisconsin recently revised its

process for reviewing plan methodologies. Previously, the state required

20See “Differences in 4-Year Health Outcomes for Elderly and Poor, Chronically Ill Patients Treated in

HMO and Fee-for-Service Systems: Results From the Medical Outcomes Study,” The Journal of the

American Medical Association, Vol. 276, No. 13 (Oct. 2, 1996).

21Arizona requires plans to submit a topic for state approval.

Page 18 GAO/HEHS-97-86 Medicaid Managed Care Accountability
B-270335 

plans to submit a one-paragraph description of their methodologies, on

which the state would base its approval. For the contract year beginning

October 1996, Wisconsin began to require plans to submit detailed

descriptions of the study topic and the methodology for conducting the

study. To validate the results of the plans’ clinical studies, Pennsylvania

and Tennessee review a sample of patient records during compliance

reviews for their annual contracts. However, these states generally pull

only a handful of records to verify the clinical study results.

States’ Medical Record Audits Medical record audits document problems with patient medical records,

such as incomplete patient histories, lack of indication of follow-up care,

and illegibility and unavailability of records. These audits also can help

identify underlying causes of service delivery or access problems. If a

plan’s medical records are inadequate or indicate a service delivery

problem, a state can require the plan to take corrective action. All four

states that we visited use utilization statistics, encounter data, or both to

focus their audits of Medicaid beneficiary medical records. For example,

Wisconsin found through its analyses of utilization statistics that use of

dental services was infrequent and, therefore, conducted a medical record

audit to determine why Medicaid beneficiaries were not getting dental

care.

While these states’ periodic audits of beneficiary records have revealed

weaknesses in the documentation of beneficiary care, we found that their

audit methodologies often yielded results that were not statistically valid

or may not have been sufficient to identify problems experienced by

different groups of Medicaid beneficiaries. Specifically, the samples of

records that the states used to conduct their audits generally were not

stratified by specific conditions or populations—which could result in an

underrepresentation of Medicaid beneficiaries with special needs—and

the sample sizes may not have been adequate to identify areas that warrant

further investigation. For example, Wisconsin uses a random sample of

about 2 percent of cases for medical record audits. Arizona bases its

sample sizes on the number of beneficiaries enrolled in a plan, with the

sample sizes ranging from 30 to 100 patient records. Although the state

agrees these sample sizes are not statistically valid, it believes that these

audits, when combined with other periodic on-site reviews, are sufficient

to identify best practices as well as problem areas to target for corrective

action.

Page 19 GAO/HEHS-97-86 Medicaid Managed Care Accountability
B-270335 

States Could Learn

More From Improved

Design and Analysis

of Data on Beneficiary

Experience With

Managed Care

In assessing the performance of managed care plans, indicators of

beneficiary satisfaction can complement other analyses of provider

network capacity and the services provided. To gauge the extent to which

beneficiaries are satisfied with Medicaid managed care and their managed

care plan, the states that we visited review the results of beneficiary

satisfaction surveys, grievance data, and the rates at which beneficiaries

choose to switch plans or, if enrolled in a voluntary program, leave

managed care altogether. While analyses of such data have helped these

states identify problems that Medicaid beneficiaries have with managed

care, such as difficulty in scheduling appointments or accessing

specialists, certain improvements in the design of their surveys and data

collection methods could enhance the usefulness of their analyses.

Improved Beneficiary

Survey Designs Could Yield

More Useful Data

Although the results of patient satisfaction surveys may not be the best

indicator of quality care, periodic satisfaction surveys—administered

statewide or at the plan level—can help measure the degree to which

Medicaid beneficiaries are happy with the providers and services offered

in their managed care plan. To ensure that survey findings are reliable and

useful in identifying areas that need systemwide improvement, the survey

design and process must be methodologically sound, however.22 In

addition, the survey must be designed to address several difficulties

inherent in surveying the Medicaid population, such as effectively

administering a survey instrument in multiple languages and overcoming

historically low response rates.

Plan-Conducted Beneficiary

Surveys

Pennsylvania, Tennessee, and Wisconsin require plans to conduct periodic

surveys to assess beneficiary satisfaction with network providers and

services. Pennsylvania and Tennessee review the plans’ survey

methodologies during their annual contract compliance reviews, but they

do not routinely examine the survey methods before the surveys are

conducted. Wisconsin requires plans to describe in detail the methodology

they intend to use before the contract is awarded. Once the contract is

awarded, the state reviews and approves each plan’s survey methodology

and instrument before it is administered. However, none of these states

prescribes a methodology for conducting satisfaction surveys.

Consequently, these states cannot compare survey results across plans.

22Medicaid HEDIS provides states with technical guidance on designing a satisfaction survey for

Medicaid beneficiaries. This guidance includes how to identify topics for surveys and the trade-offs

between phone and mail surveys, such as data collection costs, time required for data collection,

response rates, sample sizes, and overall quality of data. In addition, HHS is developing beneficiary

satisfaction surveys for both general and special needs populations, which should be of further help to

the states once completed.

Page 20 GAO/HEHS-97-86 Medicaid Managed Care Accountability
B-270335 

Statewide Surveys Statewide surveys allow states to compare results across plans for various

access and quality measures, such as use of specialty services, average

waiting time for physician office visits, and beneficiary perception of the

quality of care provided. As a condition of its demonstration waiver,

Tennessee is required to conduct statewide satisfaction surveys annually.

Arizona—on its own initiative—will soon complete a statewide beneficiary

satisfaction survey. Pennsylvania and Wisconsin plan to conduct surveys

as part of their planned program expansions.

Since implementing its managed care program in 1994, Tennessee has

conducted two annual statewide surveys. HCFA required that the state’s

beneficiary satisfaction survey include questions on referrals to

specialists, average waiting time for physician office visits, and reasons for

disenrollment. Tennessee has used these surveys to identify trends in

service use. For example, the state found that between 1993—the last year

of fee-for-service care—and 1995, hospital use decreased 6 percent, and

visits to doctors’ offices increased 8 percent—a redistribution of service

settings that is hoped for in managed care. In addition, its 1995 survey

showed that 75 percent of enrollees were satisfied with the care they

received through their managed care plan compared with a 61-percent

satisfaction rate in 1994. Although this showed improvement in overall

beneficiary satisfaction, the state also could use these survey results to

further explore remaining causes of dissatisfaction for one fourth of the

beneficiaries.

Arizona is issuing the results of its third statewide beneficiary survey in

spring 1997.23 The comprehensive survey included questions on the use of

health services, time elapsed in getting an appointment with a physician

and in waiting in the physician’s office, problems with access to specialty

care, and an overall rating of the plan and quality of care. The state

conducted telephone interviews, which allowed interviewers to verify that

they were speaking with the appropriate beneficiary and to ask

appropriate follow-up questions. Of the current 450,000 beneficiaries

enrolled in the program, over 14,000 were interviewed. The state intends

to use the survey results to provide feedback to plans.

23Arizona’s first survey, administered in 1989 by an advocacy group, was a comprehensive look at all

health services used. The second, administered in 1995 by the Arizona State University Survey

Research Laboratory, was a focused survey on prenatal and maternity care.

Page 21 GAO/HEHS-97-86 Medicaid Managed Care Accountability
B-270335 

Targeted Analyses of

Grievance Data Help

Identify Areas That Need

Improvement

To satisfy a federal requirement for operating a Medicaid managed care

program, states must ensure that participating plans have an internal

grievance process through which beneficiaries can report their

dissatisfaction with plan providers, services, and benefits. Through these

grievance processes, the states that we visited have been able to identify

and address a number of beneficiary concerns. Some states also look at

individual beneficiary grievances to identify specific and localized

problems. Other opportunities for analyzing grievance data, however,

exist. For example, monitoring the volume of grievances

filed—particularly across plans—could reveal previously unidentified

problems. Even a low number of grievances could indicate that

beneficiaries do not understand the grievance process.

Arizona requires beneficiaries to submit grievances directly to the plan.

Pennsylvania and Wisconsin have no such requirement but encourage this

practice; they also allow beneficiaries to submit grievances directly to the

state. Tennessee requires beneficiaries to submit grievances directly to the

state. After receiving a grievance, the plans must provide beneficiaries

with resolution and action in a reasonable time frame, ranging from 30 to

90 days. If a beneficiary is not satisfied with a plan’s decision, the

beneficiary can appeal to the state. Most grievances are resolved at the

plan level, however, according to officials in the states we visited. At a

minimum, the plans that directly receive grievances are required to

periodically report to the state the number and type of grievances they

received—such as denial of requests for out-of-plan services or difficulty

in locating a provider or in scheduling an appointment—and the status of

these cases. To probe beyond such aggregated information, which may

mask specific or localized problems, Arizona and Wisconsin informed us

that they review each grievance that plans receive.

In addition to the grievance process, each state has developed other means

for beneficiaries to voice their concerns. For example, Tennessee has a

toll-free information hotline to respond to beneficiary questions and

concerns. Tennessee also sponsors hotlines run by advocacy groups to

answer questions posed by beneficiaries with special needs, such as

persons with acquired immune deficiency syndrome (AIDS) or human

immunodeficiency virus (HIV), hemophiliacs, and persons with disabilities,

as well as the general Medicaid population. In addition to state-run

hotlines, Wisconsin requires each plan to have a beneficiary advocate who

serves as a liaison between the state, the plan, and the beneficiary. The

plan advocate identifies major areas of concern, such as lack of access to

Page 22 GAO/HEHS-97-86 Medicaid Managed Care Accountability
B-270335 

mental health care, and works with the plan and the state to correct the

problem. This can obviate the need for beneficiaries to register grievances.

States’ Analyses of

Disenrollment Data Could

Help Identify Problem

Areas

Beneficiaries who disenroll from a managed care plan may do so because

of dissatisfaction with the care they receive through the plan. Therefore,

collecting and analyzing data on disenrollments can provide important

insights into plan performance. In a voluntary program, such as the one we

visited in Pennsylvania, beneficiaries can switch plans or return to

fee-for-service care. In mandatory programs—such as those in Arizona,

Tennessee, and Wisconsin—beneficiaries can switch plans during open

seasons, which occur every 6 or 12 months.24 The states we visited,

however, generally do not conduct routine disenrollment studies.

According to officials in these states, they would conduct a disenrollment

study if a significant number of disenrollments were detected.25 They

believe that disenrollments—especially in low numbers—could signify a

number of occurrences other than beneficiary dissatisfaction or problems

with the plan. For example, in 1992, Arizona conducted a disenrollment

study and found that most of the beneficiaries who changed plans during

open enrollment—which was less than 5 percent of all beneficiaries in

managed care—did so for reasons other than plan dissatisfaction.

Specifically, the state found that some beneficiaries disenrolled because

they wanted to continue to see a provider who was no longer in their

plan’s network. Others switched to have all family members in one plan,

and still others wanted to enroll in a plan where provider location was

more convenient. Unless it sees a substantial change in enrollment rates

during an annual open season, Arizona has no plans to conduct another

study.

More analyses of these disenrollment data—even if the rate at which

beneficiaries leave or switch plans is low—could reveal significant

problems. Disenrollments concentrated in an area or among people having

similar needs, such as people with AIDS, may indicate a potential problem

in a plan. Also, any plan having higher disenrollment rates than other plans

may merit scrutiny to determine the reason.

24Under a demonstration waiver, states can obtain federal authorization to require beneficiaries to

remain enrolled in a specific health plan for 12 months. Under certain circumstances, such as

relocation, beneficiaries can change plans at other times.

25In 1995, disenrollment rates in Arizona and Tennessee—the two demonstration waiver states where

enrollment only changes annually—were 4 and 6 percent, respectively.

Page 23 GAO/HEHS-97-86 Medicaid Managed Care Accountability
B-270335 

Observations In view of the billions of dollars that are being paid prospectively to

managed care plans and the questions about the degree to which managed

care is meeting the health care needs of Medicaid beneficiaries, there is a

new demand for public accountability. The continuing trend toward

expansion of mandatory, capitated Medicaid managed care programs

requires that states have the ability to adequately oversee their contracts

with health plans and ensure that states get what they are paying for.

However, developing systems to hold plans accountable for ensuring that

Medicaid beneficiaries receive the care that they need has been a

challenge for states—especially since there are few benchmarks and

standards against which states can measure beneficiary access to network

providers and the appropriateness of the care provided.

The four states that we visited have made progress toward developing

accountability measures to ensure that beneficiaries have access to quality

care. As they expand or refine their Medicaid managed care programs,

these states continue to scale the steep learning curve to becoming an

effective purchaser of managed care. Yet, to instill greater public

confidence that managed care can effectively and efficiently meet the

health care needs of Medicaid beneficiaries, more effort is needed. For

example, to varying extents, these states could improve their

methodologies for collecting and analyzing data—especially encounter

data—on beneficiary care. They could better target their clinical studies,

medical record audits, beneficiary satisfaction surveys, and reviews of

grievance data on specific services and beneficiary groups—particularly

those with special needs or conditions whose numbers may be too small

to show up in broad-scale surveys or studies. The need for these

improvements takes on even greater importance for those states planning

to expand their managed care programs to other geographic areas or

populations, such as people with disabilities or other special needs.

Agency Comments

and Our Evaluation

We provided a draft of this report to the Administrator, HCFA. We also

provided a draft to Medicaid officials in each of the four states we visited

and to independent experts and researchers from the Center for Health

Care Strategies, Medical College of Virginia, and National Academy for

State Health Policy. Each provided technical or clarifying comments,

which we incorporated as appropriate.

In addition to technical comments, each state informed us of recent or

planned initiatives for ensuring plan and provider accountability in their

programs. For example, in February 1997, Pennsylvania implemented

Page 24 GAO/HEHS-97-86 Medicaid Managed Care Accountability
B-270335 

HealthChoices—a mandatory managed care program projected to serve

more than a half million Medicaid beneficiaries in the state by 1997. With

this new program, Pennsylvania plans to improve those accountability

measures used under its voluntary program or adopt new accountability

measures, such as disenrollment studies. HCFA, Arizona, and Wisconsin

also noted that the draft did not discuss all accountability or quality

assurance measures that were in use at the time of our review and were

concerned that the account of state efforts was incomplete. In the final

report, we state that the focus of our work was on 10 key measures or

processes that states commonly use to assess plan accountability—not on

each state’s entire quality assurance process.

Arizona and Wisconsin also commented that the draft report did not

reflect in all respects their experience with managed care. For example,

they disagreed with our premise that since managed care plans receive

prospective capitated payments, there is a financial incentive to limit or

not provide needed services. HCFA echoed this comment. The two states

pointed out, for example, that the use of certain preventive services

increased when they moved to managed care. In response to their

comments, we revised the report to include examples of their experience

in the changing patterns of care under managed care. Nevertheless, in a

prepaid capitated system, the incentive remains to provide fewer services

in order to maximize short-term profits, as the HCFA Administrator recently

testified before the Senate Committee on Finance. Concern about plans

and providers having a short-term focus is exacerbated by the fact that

significant numbers of beneficiaries frequently gain and lose Medicaid

eligibility within a short period of time.

Arizona also made a number of comparisons between managed care and

fee-for-service and suggested that our report include such comparisons.

The purpose of our report was not to weigh the merits of one system

against those of another. Rather, we set out to identify potential problems

Medicaid beneficiaries may have in accessing services through managed

care and state efforts to address these access issues.

Several reviewers, including HCFA, agreed with our conclusion that certain

measures of physician capacity do not adequately ensure beneficiary

access to care. It was suggested that we report on other important criteria

states use to assess the adequacy of provider networks—specifically,

beneficiary travel and waiting times. The final report reflects additional

information on this issue.

Page 25 GAO/HEHS-97-86 Medicaid Managed Care Accountability
B-270335 

Finally, the experts we consulted generally agreed with the accuracy and

comprehensiveness of our presentation of the issues. They also

emphasized that the transformation of existing state systems and

processes to an effective managed care program—especially one with

meaningful oversight mechanisms—requires great change accompanied by

continuous refinements and adaptations. Each state—with varying levels

of experience with managed care, resources, and in-house

expertise—understandably approaches this evolutionary process with

varying strategies and time frames. Even as states confront their many

challenges in implementing managed care, strong and consistent

accountability systems remain integral to their success in meeting the

needs of Medicaid beneficiaries. This perspective is more fully reflected in

the final report.

As arranged with your office, unless you announce its contents earlier, we

plan no further distribution of this report until 30 days after the date of

this letter. At that time, we will send copies of this report to the Secretary

of Health and Human Services, the Administrator of HCFA, state officials in

the four states we visited, appropriate congressional committees, and

other interested parties. We will also make copies available to others upon

request.

Please contact me on (202) 512-7114 or Kathryn G. Allen on (202) 512-7059

if you or your staff have any questions. Major contributors to this report

are listed in appendix III.

Sincerely yours,

William J. Scanlon

Director, Health Financing and

 Systems Issues

Page 26 GAO/HEHS-97-86 Medicaid Managed Care Accountability
Page 27 GAO/HEHS-97-86 Medicaid Managed Care Accountability
Contents

Letter 1

Appendix I 

Scope and

Methodology

30

Appendix II 

Federal and State

Oversight Roles of

Medicaid Managed

Care

33

Appendix III 

Major Contributors to

This Report

36

Tables Table 1: Characteristics of Managed Care Programs Under

Program and Demonstration Waivers

6

Table I.1: Selected Accountability Measures and Processes 30

Table I.2: Description of the Four States’ Medicaid Managed Care

Programs That GAO Reviewed

31

Table II.1: Federal Regulations, Requirements, and Guidance for

Selected Accountability Measures and Processes Within States’

Medicaid Managed Care Programs

35

Abbreviations

AFDC Aid to Families With Dependent Children

AIDS acquired immune deficiency syndrome

EPSDT early and periodic screening, diagnosis, and treatment

HCFA Health Care Financing Administration

HEDIS Health Plan Employer Data Information Set

HHS Department of Health and Human Services

HIV human immunodeficiency virus

HMO health maintenance organization

NCQA National Committee on Quality Assurance

SSI Supplemental Security Income

Page 28 GAO/HEHS-97-86 Medicaid Managed Care Accountability
Page 29 GAO/HEHS-97-86 Medicaid Managed Care Accountability
Appendix I 

Scope and Methodology

To ensure quality in their Medicaid managed care programs, states use a

number of quality assurance and oversight mechanisms. For example,

some states require accreditation of plans by independent entities, such as

the National Committee on Quality Assurance. Most states build into their

contracts with managed care plans a variety of accountability measures

and processes. Once states have contracted with plans, they monitor the

plans to ensure that they comply with these requirements.

Based on our review of federal requirements and guidelines and state

contracts and supporting documents, we identified a number of

accountability measures and processes that states commonly include in

their contracts with managed care plans. States can rely on various

controls to assess compliance with some of these measures—including

plan licensing, physician credentialing, and fiscal solvency requirements.

Some accountability measures are more difficult to develop controls for

and, therefore, are more difficult to monitor.

To assess state efforts to hold managed care plans accountable for

meeting Medicaid program goals and providing beneficiaries enrolled in

capitated managed care plans the care they need, we narrowed the focus

of our study to three areas: ensuring an adequate provider network,

tracking the medical care provided to beneficiaries, and assessing

beneficiary satisfaction. From among numerous quality assurance

measures and processes, we identified 10 that states commonly use to

monitor plan compliance in these accountability areas. Health Care

Financing Administration (HCFA) officials and experts we contacted agreed

that these measures and processes are essential to ensuring that plans

meet the terms of their contracts. (See table I.1.)

Table I.1: Selected Accountability

Measures and Processes Accountability area Measure or process

Adequacy of provider network —Patient-to-primary-care-physician ratio

—Access to specialists

—Travel distances and waiting times

Adequacy of medical care —Utilization statistics

—Encounter data

—Clinical studies

—Medical record audits

Beneficiary satisfaction —Satisfaction surveys

—Grievance procedures

—Disenrollment data

Page 30 GAO/HEHS-97-86 Medicaid Managed Care Accountability
Appendix I 

Scope and Methodology

To examine how these accountability measures and processes were

implemented, we visited four states—Arizona, Pennsylvania, Tennessee,

and Wisconsin—and reviewed their systems for ensuring access to quality

care in their managed care programs. To select these states, we first

identified a universe of 14 states that, as of June 1994, had more than

100,000 beneficiaries enrolled in capitated Medicaid managed care

programs. We chose that date because we believed that, even for states

just entering the managed care market, they would have had at least 2

years at the time we began our review to develop and implement their

accountability systems. We then judgmentally selected four states that

would provide a mix of experiences for a variety of factors. These factors

included type of program (Medicaid demonstration waiver, program

waiver, or voluntary nonwaiver), years of managed care experience, size

of program, and geographic diversity. (See table I.2 for a brief description

of each state’s Medicaid managed care program.)

Table I.2: Description of the Four

States’ Medicaid Managed Care

Programs That GAO Reviewed

State Program name and description

Arizona Arizona Health Care Cost Containment System is a mandatory

statewide demonstration program, operational since 1982, with

Aid to Families With Dependent Children (AFDC) and

Supplemental Security Income (SSI) enrollment of over 450,000

in 14 participating health plans. Arizona did not have a

Medicaid program before the waiver.

Pennsylvania The state’s voluntary nonwaiver program has been operational

since 1972, with enrollment of about 435,000 AFDC and SSI

beneficiaries in 18 counties in 11 health maintenance

organizations (HMO) as of June 1996. (The scope of our work

focused on the state’s voluntary program.) The state also had a

mandatory program, Health Pass, in certain areas of

Philadelphia from 1986 to January 31, 1997. On February 1,

1997, a new program waiver, HealthChoices, began in 5

Philadelphia-area counties; the voluntary program will continue

in 13 counties. Within the next 2 years, the state plans to apply

for a statewide demonstration waiver.

Tennessee TennCare is a mandatory, statewide demonstration waiver

program, operational since 1994, with enrollment of 849,000

AFDC and SSI Medicaid beneficiaries and over 334,000

uninsured persons in 12 participating HMOs at the time of our

review.a

Wisconsin The state’s mandatory program waiver for its AFDC population

has been operational since 1984 in 5 counties, with enrollment

of 138,000 in 11 participating HMOs. A modified waiver was

approved October 1, 1996, to expand mandatory enrollment to

68 of the state’s 72 counties.

aAs of January 1997, Tennessee began to contract solely with HMOs. Before this, Tennessee also

contracted with other health plans, such as preferred provider organizations.

Page 31 GAO/HEHS-97-86 Medicaid Managed Care Accountability
Appendix I 

Scope and Methodology

For each state, we reviewed the contractual and other requirements the

states have established with plans for these selected accountability

indicators. We also interviewed officials from the state Medicaid, health,

and insurance agencies to discuss specific contract requirements with

managed care plans, state oversight activities and state actions available

or taken as a result of monitoring, and state plans for changes in and

expansions to their managed care programs.

To identify federal requirements and guidance available to states to

oversee their Medicaid managed care programs, we interviewed officials

from HCFA’s central office and Chicago and Philadelphia regional offices,

the Department of Health and Human Services’ (HHS) Office of Inspector

General, and state-level advocacy groups. We also interviewed experts

with The George Washington University Center for Health Policy

Research, the National Association of Managed Care Regulators, the

National Committee on Quality Assurance, and the National Association of

Insurance Commissioners. Finally, we consulted with the following

experts and researchers in the course of our work: Jane Horvath, National

Academy for State Health Policy, Washington, D.C.; Robert Hurley,

Medical College of Virginia, Richmond, Virginia; and Stephen Somers and

Karen Brodsky, Center for Health Strategies, Princeton, New Jersey.

We performed our work between October 1995 and March 1997 in

accordance with generally accepted government auditing standards.

Page 32 GAO/HEHS-97-86 Medicaid Managed Care Accountability
Appendix II 

Federal and State Oversight Roles of

Medicaid Managed Care

By virtue of the mandated federal-state Medicaid partnership, states must

meet certain federal requirements when developing their managed care

programs. States may implement managed care programs under one of

three options. The first option is a nonwaiver program that allows states to

contract with managed care plans to deliver health care services to

Medicaid beneficiaries who voluntarily participate. Certain conditions

must be met, such as allowing beneficiaries the freedom to stay in a

traditional fee-for-service system or enroll with a managed care plan from

which they can disenroll at any time. Plans also must adhere to a “75-25

rule,” which prohibits participating managed care plans from enrolling

75 percent or more Medicaid and Medicare beneficiaries. The managed

care program in Pennsylvania that we reviewed is a program of this type.

The other two options for managed care—program and demonstration

waivers—allow HCFA to waive certain provisions of the Medicaid statute,

including beneficiaries’ freedom to choose from among participating

providers. Under a program waiver, enrollment can be mandatory, but

states are still required to ensure that plan enrollment of Medicaid and

Medicare beneficiaries does not reach 75 percent and, in most cases, plans

cannot lock in enrollment for more than 1 month. Wisconsin operates its

mandatory managed care program under a program waiver. Under a

demonstration waiver, states may be given permission to contract with

plans that do not comply with the 75-25 rule and to exclusively enroll

Medicaid beneficiaries. They also have been permitted to lock in

beneficiary enrollment for up to 12 months. The managed care programs

in Arizona and Tennessee are statewide mandatory programs operated

under demonstration waivers. In addition, some states, such as Tennessee,

have used demonstration waivers to expand eligibility to include

non-Medicaid-eligible people who were formerly uninsured.

Certain federal regulations, requirements, and guidance influence the

development of state managed care programs and state monitoring of

managed care plan performance. The extent of these requirements often

depends on waiver type and can vary by state. In general, HCFA monitors

the planning for and implementation of demonstration waivers more than

for program waivers. The initial terms and conditions of approval for

demonstration waivers are more detailed than for program waivers and

are more specific in the content and timing of reporting requirements. For

example, HCFA’s terms and conditions for a demonstration waiver have

required that states specify in their contracts with plans a specific

patient-to-primary-care-physician ratio that plans must meet. HCFA also

requires that most states establish travel-related requirements for plan

Page 33 GAO/HEHS-97-86 Medicaid Managed Care Accountability
Appendix II 

Federal and State Oversight Roles of

Medicaid Managed Care

networks, such as maximum times and distances beneficiaries must travel

to reach their primary care physician. In contrast, under a program waiver,

HCFA suggests that states establish a patient-to-primary-care-physician

ratio or that providers be located near beneficiaries. Under demonstration

waivers, HCFA also requires states to provide an overall quality assurance

monitoring plan and, as part of that plan, requires states to specify a

minimum data set of encounter data. This minimum data set must receive

prior HCFA approval. Program waivers, in contrast, have guidelines on

quality assurance programs but not as many specific requirements. For

example, HCFA does not require states to develop encounter data under

program waivers.

Table II.1 summarizes federal requirements and guidance by selected

accountability measures and processes.

Page 34 GAO/HEHS-97-86 Medicaid Managed Care Accountability
Appendix II 

Federal and State Oversight Roles of

Medicaid Managed Care

Table II.1: Federal Regulations,

Requirements, and Guidance for

Selected Accountability Measures and

Processes Within States’ Medicaid

Managed Care Programs

Accountability measure or

process Federal regulation

Additional HCFA

requirements and

guidancea

Patient-to-primary-carephysician ratioPlans must ensure that

beneficiaries in managed

care have the same access

to providers and services

as beneficiaries in

fee-for-service plans.

(42 C.F.R. 434.20(c)(2))

For demonstration waivers,

states must meet maximum

2,500 to 1 ratio. For

program waivers, HCFA

suggests ratios be

evaluated.

Availability of specialists Services are to be the same

as those provided under

fee-for-service plans. (42

C.F.R. 434.20(c)(2))

HCFA suggests that states

have a system for

authorizing and

coordinating specialty

services.

Utilization statistics and

encounter data

Requires plans to maintain

appropriate record systems

for services provided to

enrollees. (42 C.F.R.

436.6(a)(7))

HCFA requires all states

with demonstration waivers

to collect 100% encounter

data and requires all states

to quarterly report

aggregated statistics on

selected services.

Clinical studies Plans must have an internal

quality assurance system.

(42 C.F.R. 434.34)

HCFA suggests states

conduct quality-of-care

studies.

Medical record audits States must annually

conduct an audit of medical

records. (42 C.F.R. 434.53)

HCFA requires states to

comply with federal

regulation for medical

record audits.

Beneficiary satisfaction

surveys

Plans must have an internal

quality assurance system.

(42 C.F.R. 434.34)

For demonstration waivers,

a state may be required to

conduct a survey as HCFA

prescribes. For program

waivers, HCFA suggests

plans conduct periodic

surveys.

Grievance procedures Plans must have an internal

grievance procedure

approved by the state that

provides for prompt

resolution. (42 C.F.R.

434.32)

HCFA requires states to

report grievance data

quarterly.

Disenrollment studies States must monitor

enrollment and termination

practices. (42 C.F.R. 434.63)

HCFA suggests states

analyze enrollment statistics.

aFor demonstration waivers, additional requirements and guidance are in the terms and

conditions that HCFA develops when it approves a state’s waiver request. These terms and

conditions can vary by state. For risk-based managed care plans, such as program waivers,

HCFA’s guidance is included in Monitoring Risk-Based Managed Care Plans: A Guide for State

Medicaid Agencies, report prepared under contract for the Medicaid Bureau/HCFA by the

Medicaid Management Institute of the American Public Welfare Association (Washington, D.C.:

HHS, July 1993).

Page 35 GAO/HEHS-97-86 Medicaid Managed Care Accountability
Appendix III 

Major Contributors to This Report

Kathryn G. Allen, Acting Associate Director, (202) 512-7059

Karyn L. Papineau, Evaluator-in-Charge, (202) 512-7155

Karin A. Lennon, Senior Evaluator

Karen M. Sloan, Communications Analyst

Betty J. Kirksey, Evaluator

Daniel S. Meyer, Senior Evaluator

(101385) Page 36 GAO/HEHS-97-86 Medicaid Managed Care Accountability
Ordering Information

The first copy of each GAO report and testimony is free.

Additional copies are $2 each. Orders should be sent to the

following address, accompanied by a check or money order

made out to the Superintendent of Documents, when

necessary. VISA and MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address

are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 6015

Gaithersburg, MD 20884-6015

or visit:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders may also be placed by calling (202) 512-6000 

or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any

list from the past 30 days, please call (202) 512-6000 using a

touchtone phone. A recorded menu will provide information on

how to obtain these lists.

For information on how to access GAO reports on the INTERNET,

send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov

PRINTED ON RECYCLED PAPER
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. G100