United States General Accounting Office

GAO Report to the Ranking Minority Member,

Committee on Labor and Human

Resources, U. S. Senate

September 1995 HEALTH CARE

Employers and

Individual Consumers

Want Additional

Information on Quality

GAO/HEHS-95-201

GAO United States

General Accounting Office

Washington, D.C. 20548

Health, Education, and

Human Services Division

B-257441 

September 29, 1995

The Honorable Edward M. Kennedy

Ranking Minority Member

Committee on Labor and Human Resources

United States Senate

Dear Senator Kennedy:

Employers and individual consumers are no longer concerned only about

the escalating cost of health care; they are increasingly concerned that

efforts to reduce health care costs may now also be reducing its quality.

Some cost-control efforts might unduly encourage providers to withhold

care. Other cost-control efforts restrict or eliminate individuals’ choice of

provider. As employers negotiate for lower premiums or limit employees’

access to providers, they want to ensure that their employees still receive

quality care. Individual consumers want to be assured that they have

access to quality providers and that they make the right health care

decisions. As a result, both employers who purchase health care and

individual consumers have demanded more information about quality.

In response to these demands, some states, large employers, and health

plans have been publishing performance reports describing the quality of

health care providers. These “report cards” include information such as

the frequency with which preventive services are provided and the degree

of success in treating certain diseases. The federal government, as the

nation’s largest health care purchaser, has also become increasingly

involved in the movement to develop and publish performance reports,

especially for Medicare and Medicaid beneficiaries.

Little has been known, however, about how useful published reports have

been or how they could be made more helpful. As a result, you asked us to

study (1) how consumers use available published comparative data and

(2) what information consumers want.1

To obtain the views of individual consumers, we attempted to contact over

1,000 persons who had requested a report card published by a state agency

or health plan. We were not able to obtain usable information from most

of these persons—some did not respond to our attempts, some were

unavailable, some did not remember receiving the report card, and some

1

Employers are consumers of health plan services to the extent that they use administrative services,

including data. Recognizing that, some plans include both employers and individual consumers in their

customer satisfaction surveys.

Page 1 GAO/HEHS-95-201 Health Care Report Cards
B-257441 

had used the information for work-related research rather than to select a

provider or plan. We conducted telephone interviews with the remaining

153 people who had requested the report card for use in making health

care purchasing decisions. We also conducted 7 group interviews with a

total of 64 employees at 7 locations.2

 These consumers may not have had

previous experience with report cards.

To obtain the views of employers, we interviewed representatives of 65

businesses around the country with health coverage for fewer than 5 to

over 100,000 employees. Because the employers and individual consumers

are not nationally representative, their experiences and opinions cannot

be generalized to all employers and consumers.

We conducted our review from November 1994 to June 1995 in accordance

with generally accepted government auditing standards. (See app. I for a

more detailed discussion of our scope and methodology.)

Results in Brief Many employers and individual consumers we interviewed are using

information that measures and compares the quality of health care

furnished by providers and health plans when making their purchasing

decisions. For example, employers are using report cards to select and

monitor the performance of providers and plans furnishing services to

their employees, negotiate with insurance carriers, and market managed

care plans to employees. As one employer remarked, “We’d like to get

some kind of value-based decision for purchasing health care.” Individuals

are using report cards to choose providers or plans, to enhance their

knowledge of providers or plans, and to reassure themselves of their own

or their employers’ provider choices.

Employers and individual consumers we interviewed wanted performance

reporting efforts to continue. In fact, they are requesting more data than

are publicly available. However, they believed the information would be

more useful if their concerns about the reliability and validity of the data

were addressed. For example, some individual consumers used terms such

as “self-serving,” “one-sided,” and “nontrustworthy” to describe the reports

they received from health plans.

Employers and individual consumers we interviewed also reported that

the most useful information would measure health care outcomes. One

2

Four of these worksites were federal agencies. Many private sector employers we contacted refused

our request to discuss health plan purchasing decisions with their employees.

Page 2 GAO/HEHS-95-201 Health Care Report Cards
B-257441 

comment was, “The number one thing people ask . . . is not . . . . ‘Am I

going to get that mammogram?’ it’s . . . . ‘Am I going to die?’” However,

they acknowledged that it is very difficult to attribute outcomes to quality

of care rather than to factors such as the patient’s health or lifestyle

choices. They also said they want standardized and comparable health

care information to assess health care providers’ or health plans’

performance equally.

Many of the employers we interviewed are getting some of the data they

want through business coalitions, consultants, and their own data

collection efforts. But these sources are not available to individual

consumers, and few employers were sharing these data with their

employees. One employer said, “I don’t know if the data we’d be giving

them would be the complete picture.” Although some employers stated

that their employees did not want or would not understand data

comparing quality, their employees told us such information would be

helpful.

Background Employers have been the driving force behind the growing move to

compare health care providers and plans on the basis of their

performance. These employers have worked both individually and

collaboratively with providers, health plans, and government to produce

information that will allow them to assess the quality of the care they

purchase. Health plans have been publishing reports comparing their

performance to their peers or to a national standard. State governments

have published comparative information, often focused on specific

procedures performed in hospitals. Although the federal government was

responsible for the first widespread public disclosure of hospital

performance data in 1987, it discontinued this practice in 1993. As a payer

of health care services on behalf of Medicare and Medicaid beneficiaries,

the Health Care Financing Administration (HCFA) lags behind others in

making performance data public.

Report cards can include a variety of performance indicators, either

structural, process, or outcome based. Structural indicators measure the

resources and organizational arrangements in place to deliver care, such

as the ratio of nurses to inpatient beds. Process indicators measure the

physician and other provider activities carried out to deliver the care, such

as the rates of childhood immunization. Outcome indicators measure the

results of the physician and other provider activities, such as mortality,

morbidity, and customer satisfaction.

Page 3 GAO/HEHS-95-201 Health Care Report Cards
B-257441 

Employers Took the

Initiative in Report Card

Development

In 1989, a group of employers initiated one of the most significant efforts

to identify uniform and standardized performance indicators. This effort

resulted in the creation of a performance measurement system known as

the Health Plan Employer Data and Information Set (HEDIS). Several

business coalitions and health care organizations used the first HEDIS

measures in 1991. The nonprofit National Committee for Quality

Assurance (NCQA) has led the effort to revise the measures, issuing HEDIS

2.0 in 1993 and HEDIS 2.5 in 1995. Current HEDIS measures focus on process

indicators. (See table 1 for a list of some key HEDIS measures.)

Table 1: Selected HEDIS Performance

Measures

Quality/access Utilization Physician network

Membership/

finance

Childhood

immunization

Cholesterol screening

Mammography

Pap smear

Prenatal care visit

Diabetic retinal exam

Major affective

disorder follow-up

Members visiting

provider

Asthma admission

rate

Low birthweight rate

Coronary bypass

rate

Angioplasty rate

Cardiac

catheterization rate

Hysterectomy rate

Prostatectomy rate

Laminectomy rate

Cesarean section

rate

Obstetrical hospital

stay

Readmission for

chemical

dependency

Hospital days/ 1,000

enrollees

Physician turnover

Board certification

Membership

disenrollment

Medical loss ratio

Administrative loss

ratio

Revenue

requirements per

member per month 

Tier rates

Source: NCQA Report Card Pilot Project Technical Report (Washington, D.C.: NCQA, Feb. 1995).

Using HEDIS as a base, some employers have begun to distribute to their

employees educational materials that include outcome measures. For

example, the California Public Employees’ Retirement System (CalPERS)

recently distributed to its employees a performance report about the

health plans it offers. Although it had furnished some comparative

information to its employees in previous years, the information generally

Page 4 GAO/HEHS-95-201 Health Care Report Cards
B-257441 

featured cost and benefits. CalPERS’ May 1995 Health Plan

Quality/Performance Report is its first effort at distributing comprehensive

information that includes both specific quality performance indicators and

member satisfaction survey results. The quality performance data are

based on HEDIS indicators measuring health maintenance organizations’

(HMO) success with providing childhood immunizations, cholesterol

screening, prenatal care, cervical and breast cancer screening results, and

diabetic eye exams. Employee survey results include employee

satisfaction with physician care, hospital care, and the overall plan, and

the results of a question asking whether members would recommend the

plan to a fellow employee or friend.

Some employers are using third-party health care accrediting

organizations to measure health plan performance using structural

indicators. These employers are requiring the health plans they contract

with to be accredited by organizations such as NCQA and the Joint

Commission on Accreditation of Healthcare Organizations. Furthermore,

some accrediting agencies publicize their accreditation decisions, which

allows employers and individual consumers to consider accreditation

status in their health care purchasing decisions. For example, a

consortium of employers has elected to exclude a Florida HMO from new

business with its employer-sponsored health plans because of the HMO’s

failure to obtain accreditation.

Health Plans Make

Performance Data Public

Health plans have published comparative information intended to assist

individual consumers in their health care choices and health care

providers in their quality improvements. For example, in 1993, Kaiser

Permanente Northern California Region released a report on 102

performance measures divided into the following categories: childhood

health, maternal care, cardiovascular disease, cancer, common surgical

procedures, other adult health, and mental health/substance abuse. (See

fig. 1.) Although Kaiser was one of the first health plans to publish this

kind of information, an increasing number of health plans are now

providing similar information.

Page 5 GAO/HEHS-95-201 Health Care Report Cards
B-257441 

Figure 1: Example of Health Plan

Presentation of Comparative Health

Care Data

Source: Kaiser Permanente, Northern California Region, 1993 Quality Report Card (Oakland,

Calif.: Kaiser Permanente, 1993).

Health plans have been exploring new ways to make information readily

available and understandable to individual consumers. For example, on

September 15, 1995, HealthPartners, Inc., will initiate a consumer-oriented

Page 6 GAO/HEHS-95-201 Health Care Report Cards
B-257441 

program using touch-screen computers.3 Initially, at least 50 computers

will be installed permanently at 50 employer sites, and at least 100

computers will be rotated among other employers. This will allow

employees to obtain details about any one of the plans’ primary care sites,

such as its physicians’ credentials, on-site services offered, and specialists

to which its physicians refer. Because health plan members are expected

to enroll in a specific care delivery system—a set of primary care sites

with affiliated specialists—HealthPartners will furnish data about each

care system to help plan members make a decision about which one to

join. Currently these data include preventive screening rates and patient

satisfaction measures. HealthPartners anticipates expanding the

availability of touch-screen computers to more public spaces, such as

shopping malls, after physician concerns about data confidentiality and

other matters are resolved.

State Legislatures Mandate

Public Dissemination of

Data

The states have also been active in providing information about provider

performance to the public. Forty states have mandated the collection,

analysis, and public distribution of health care data, such as hospital use,

charges or cost of care, effectiveness of health care, and performance of

hospitals.4,5 For example, Pennsylvania has released four report cards on

the hospitals and physicians in the state performing coronary artery

bypass graft surgery (CABG) since 1992. Providing both costs and mortality

rates, the reports are publicized through the local media and are available

free to consumers. (See fig. 2.)

3

HealthPartners, Inc., is the parent company of health care organizations that include group and staff

model HMOs located in Minnesota’s Twin Cities. They provide health care services and coverage to

more than 650,000 members.

4

The 10 states that have not mandated these activities are Alabama, Alaska, Hawaii, Idaho, Louisiana,

Michigan, Mississippi, Montana, Nebraska, and Wyoming. Colorado’s legislature eliminated funding for

its state data commission as of July 1995, and North Carolina’s program will cease in October 1995.

5

A self-insured company that administers its own health plan may not be under any obligation to report

its performance. In Employer-based Health Plans: Issues, Trends, and Challenges Posed by ERISA

(GAO/HEHS-95-167, July 25, 1995), we reported that the National Governors’ Association believed that

the Employee Retirement Income Security Act of 1974 (ERISA) prohibited their states from developing

standard data collection systems applicable to all health plans.

Page 7 GAO/HEHS-95-201 Health Care Report Cards
B-257441 

Figure 2: Example of Pennsylvania

State Comparison of Health Care

Providers

Source: Pennsylvania Health Care Cost Containment Council, A Consumer Guide to Coronary

Artery Bypass Graft Surgery, Vol. IV (Harrisburg, PA: June 1995). Statistics were based on 1993

data.

Federal Government Is

Moving Slowly

In 1987, HCFA initially publically released hospital mortality information,

but did so only in response to a request under the Freedom of Information

Page 8 GAO/HEHS-95-201 Health Care Report Cards
B-257441 

Act (5 U.S.C. 552). The published information, collected as part of HCFA’s

oversight efforts, included the observed and expected mortality rates for

Medicare beneficiaries in each hospital that performed CABG surgery. HCFA

published the information annually until 1993, when the HCFA

Administrator discontinued the reports. He cited problems with the

reliability of HCFA’s methods to adjust the data to account for the influence

of patient characteristics on the outcomes. HCFA has not published any

other information about the performance of Medicare providers.

HCFA’s responsibility to Medicare beneficiaries in the selection and

oversight of Medicare contract HMOs is similar to that of employers to their

employees in selecting health plans. However, HCFA does not routinely

provide beneficiaries the results of its monitoring reviews or other

performance-related information such as HMO disenrollment rates. In

August 1995, we recommended that HCFA publish (1) comparative

performance data it collects on HMOs such as complaint rates,

disenrollment rates, and rates and outcomes of appeals and (2) the results

of its investigations or any findings of noncompliance by HMOs.6

Our recommendation that HCFA publish performance data was consistent

with the views of experts we interviewed about the federal government’s

role in ensuring that Medicare beneficiaries receive quality care. These

experts cited the need for gathering health plan information such as

(1) performance measures, (2) patient satisfaction, and (3) assurances that

basic organizational standards have been met. Furthermore, they believed

that when the information is obtained, it should be shared with

beneficiaries to assist them in their health care purchasing decisions.7

Although HCFA has not been publishing data on Medicare providers, it is

collaborating with others to publish performance information about

Medicaid providers. HCFA has been participating with NCQA and the

American Public Welfare Association on behalf of the State Medicaid

Agencies Directors Group to tailor HEDIS to the particular needs of state

Medicaid agencies, health plans that serve Medicaid recipients, and the

recipients themselves. In July 1995 the work group released the first draft

of Medicaid HEDIS8

 and is expected to release a final version of the

document in Fall 1995 after considering comments received.

6

Medicare: Increased HMO Oversight Could Improve Quality and Access to Care (GAO/HEHS-95-155,

Aug. 3, 1995).

7

Medicare: Enhancing Health Care Quality Assurance (GAO/T-HEHS-95-224, July 27, 1995).

8

Draft Medicaid HEDIS: An Adaptation of the Health Plan Employer Data and Information Set 2.0/2.4,

NCQA (Washington, D.C.: NCQA, July 1995).

Page 9 GAO/HEHS-95-201 Health Care Report Cards
B-257441 

Public/Private Partnerships

Form to Produce and

Disseminate Outcome Data

Like HEDIS, many of the most recent initiatives to provide data involve a

partnership between private and public players. For example, a more

recent public/private initiative that includes some of the major employers

involved in developing HEDIS is the Foundation for Accountability (FAcct),

created in June 1995. At a meeting of the Jackson Hole Group, some of the

nation’s largest employers and HCFA, together representing more than

80 million people, or almost a third of the U.S. population, agreed to

combine their expertise and purchasing power. This action grew out of

employer frustration with current performance data that focus on plan and

provider structure and process rather than outcomes of care. FAcct intends

to recommend measures of health care quality that can be easily

understood by the general public so that people can make informed

decisions when choosing a health plan. FAcct also hopes to encourage the

common adoption of these standards to establish uniformity and minimize

health plan reporting burdens as well as develop a means of educating

diverse audiences about the significance and applications of health plan

accountability.

Little Is Known About

What Information Is

Needed or Wanted

Experts have noted that studies performed to determine how consumers

make decisions when no comparative information on quality has been

available may not be helpful in determining what information consumers

would actually use. Adding to the conclusions of numerous researchers

that individual consumers give more weight to information from

acquaintances than to expert opinion, researchers at Brandeis University

reported in 1994 that Massachusetts state employees they surveyed valued

information about quality but did not value report card information. From

this apparent contradiction, the researchers concluded that survey

respondents view quality as something other than what is described in

report cards.9

In 1995, NCQA reported that almost all consumers participating in focus

groups NCQA sponsored stated that they would use better evaluative

information if it were available to them.10 In addition, when NCQA provided

participants with sample report cards, NCQA noted that in every group,

participants were able to critically evaluate the information, raising the

same questions about the validity of the data that experts debate.

9

Brandeis University: The Heller School, Consumer Information: Decisive Factors in Health Plan

Choice (Waltham, Mass.: Aug. 15, 1994).

10NCQA Consumer Information Project Focus Group Report: Executive Summary (Washington, D.C.:

NCQA, Spring 1995).

Page 10 GAO/HEHS-95-201 Health Care Report Cards
B-257441 

In 1994, we reported that while performance measures or report cards

could be a useful tool to educate consumers about the health care that

plans provide, the report cards being developed may not reflect the needs

of some users.11 Employers have been the primary users of information

comparing quality of care; little is known about the extent to which this

information is meeting individual consumers’ needs.

The sections that follow discuss in more detail the results of our efforts to

determine, from the consumers’ perspectives, the extent to which they use

quality of care information in making health care choices and the types of

information consumers find useful in arriving at decisions.

Consumers We

Interviewed Use

Information

Comparing Quality in

Health Care

Decision-Making

Many of the employers and individual consumers of health care we talked

with are increasingly using information that compares the quality of care

furnished by health care providers or health plans to make purchasing

decisions and to encourage providers and plans to improve the quality of

their care. However, some of those we interviewed told us they are not

using the information because they are unaware that it exists, they have

not been able to find it in some markets, they believe the available

information does not meet their needs, or they lack the resources or time

to find and use the information. Further, they stated that the information

would be more useful if their concerns about the reliability and validity of

the information were addressed.

Use of Quality Information

Varies

Most of the employers we spoke with were either actively seeking or using

information on quality or stated that they would use it if it were available

to help choose health plans or individual providers for their employees.

Many employers told us that because they limited the employee’s choice of

provider by using an approach that restricted or encouraged the use of

specific providers, they felt a greater need to ensure that they provided

access to quality providers. For example, the human resource manager of

a midwestern manufacturing firm told us that

“we’d like to get some kind of value-based decision for purchasing health care. The pure

pricing arrangements, the deals . . . have not really been a complete answer for us. Those

arrangements don’t address quality, and we’re coming to believe that that’s got to be the

cornerstone of your health care plan.”

11Health Care Reform: “Report Cards” Are Useful But Significant Issues Need to Be Addressed

(GAO/HEHS-94-219, Sept. 29, 1994).

Page 11 GAO/HEHS-95-201 Health Care Report Cards
B-257441 

Employers’ use of the data varied considerably. Some of the larger

self-insured employers were using data to select individual providers to

include in their own network. Other self-insured employers preferred to

leave the selection of providers and quality assurance functions up to

either the HMO or the plan administrator with whom they contracted. In the

words of the benefits manager for a major Northeast financial services

employer,

“I think that they [HMOs] should be in the business of comparing hospitals, picking out the

high-quality, cost-effective providers; that’s what I’m paying them to do. I just want to make

sure they’re doing it, and feel comfortable that they’re doing it.”

These employers used the comparative data as a “red flag,” signaling a

possible decline in quality. For example, one large southeastern

self-insured employer stated that he watched for trends in performance

measures that might serve as a warning that a problem was developing.

Some smaller employers reported that they had neither the resources nor

the time to find or use report cards but wanted the information to be

available to the insurance agents or purchasing alliance staff they relied on

to make health insurance recommendations.

Employers told us that they are also using the data as a tool to market a

specific plan to their employees or to negotiate contract terms with the

insurance carriers. Numerous employers told us that providing employees

with data comparing quality of care was particularly helpful in convincing

their workers that managed care plans do not compromise the quality of

care provided. Employers stated that they use the data to influence

providers and plans to improve quality. For example, one employer told us

that during contract negotiations, data were used comparing hospitals on

specific procedures, such as hysterectomies, to encourage hospitals to

reduce unnecessary surgeries.

The individual consumers we talked with in Pennsylvania, California, and

Minnesota who had requested and received specific report cards generally

used the information and found it to be very helpful in making health care

purchasing decisions. These consumers received either (1) information

about patient outcomes for physicians and hospitals performing specific

procedures or (2) information on a specific plan.

More specifically, individual consumers in Pennsylvania and California

reported using the procedure-specific reports to

Page 12 GAO/HEHS-95-201 Health Care Report Cards
B-257441 

• select the best surgeon or hospital because they or someone in their family

anticipated having the surgery described in the report,

• select the best surgeon or hospital for procedures other than those

described in the report,

• review the ranking of the surgeon who had performed their surgery before

they had obtained the report,

• ask more informed questions of their doctors,

• increase their general knowledge,

• provide advice to others, or

• satisfy their curiosity.

Individual consumers using a plan-specific report card told us that they

used the information to select a health plan or to increase their knowledge

about the health plan chosen by their employer, such as the services

provided or the financial health of the plan. Consumers using either a planor procedure-specific report card who had no choice of provider reported

that the information gave them reassurance.

Although most individual consumers we interviewed found the report card

helpful, some did not. Some consumers reported that they did not use the

information because it focused on one procedure or health plan, or

because it was limited to a specific state or area. Other consumers told us

that they were unaware the information existed until after they or a family

member no longer needed it. For example, a Pennsylvania woman stated

that she wished she had known about this information before her mother

died after heart surgery, because it might have helped her select a

provider.

Consumers’ Concerns

About Comparative Data

Limit Data’s Usefulness

Both employers and individual consumers echoed many of the same

concerns expressed by health care experts and previously reported by us

that comparative information may not be measuring what it is intended to

measure. Experts have varying beliefs about what information should be

included in a report card because of acknowledged difficulties with the

reliability and validity of data sources and systems designed to measure

quality. Areas of concern for purchasers we interviewed focused on risk

adjustment, age of data, subjectivity, and bias.

More specifically, consumers, both corporate and individual, questioned

whether procedure-specific data were properly adjusted to account for

differences in patient characteristics that might contribute to adverse

outcomes. They were skeptical about whether factors such as age, severity

Page 13 GAO/HEHS-95-201 Health Care Report Cards
B-257441 

of condition, and functional status12 could be accounted for to ensure that

outcomes were an accurate reflection of provider quality. We have

reported previously that severity-adjusted performance measurement

systems are in a relatively early stage of development and may not provide

information for accurately comparing hospitals’ performance. We

concluded that additional information and methodological improvements

are needed to provide more useful data on which to base purchasing

decisions.13

Numerous individual consumers commented that the report card data they

had received were too old to accurately reflect current provider or plan

performance. For example, a consumer using plan-specific information

stated that the information was not helpful because it was already 2 years

old when published. Another commented that even in a short time, cost

data can become outdated. A consumer using health-plan specific

information told us that

“they [the report cards] are to reassure the public, but they can’t be used to make health

care decisions because they are too general and outdated from the time the data was

gathered until the decision is made.”

Some consumers stated that selecting a health care provider is a

subjective decision that is difficult to quantify. In the words of a

Pennsylvania consumer, while the report card was a good publication, “it

is limited by trying to objectify something that will always be subjective.”

For example, consumers differ in what they want from a provider. Some

consumers mentioned that it is more important for some patients to feel at

ease with their doctor than it is for others. Although many consumers

stated that they wanted information on customer satisfaction, others felt it

was of limited value because “just because you’re happy with your doctor

doesn’t mean I would be happy with him or her.” Another individual

consumer questioned a patient’s ability to assess a doctor’s medical

knowledge, technical skills, and ability.

Some employers explained that the subjective nature of the health care

purchasing decision results in their reluctance to use quantitative data to

select providers for their employees. As expressed by a representative of a

major East Coast manufacturer:

12“Functional status” is the extent to which people are able to perform activities of daily living and

their basic social roles.

13Health Care: Employers Urge Hospitals to Battle Costs Using Performance Data Systems

(GAO/HEHS-95-1, Oct. 3, 1994).

Page 14 GAO/HEHS-95-201 Health Care Report Cards
B-257441 

“Quality is in the eyes of the beholder . . . . It is not appropriate for the employer to place a

value on one outcome over another. It is up to the patient to place that value. Is it more

important that I be alive but it’s okay if I’m hurt or I’d rather die than be [incapacitated]?”

A representative of a medium-sized manufacturing firm stated that

“you know I think that a big part of the problem, and we’re guilty of it too, is imposing our

own tastes or beliefs on other people . . . . In health care we do a lot of deciding of what’s

good for people on the basis of our own beliefs, and the issues that [concern] a $9 an hour

person are not the same ones that I’m contending with . . . . The highly paid person may not

have any problem in going out of network—may be able to afford to go to Mayo’s [the

Mayo clinic] and decide, ‘hey, that’s where I’m going with this problem. I’m not going to

stick around [city deleted].’ Whereas somebody on the shop floor has got to stay in [city

deleted].”

Individual consumers questioned the objectivity of the health care data

produced and distributed by the provider or plan. Many consumers stated

they would be less likely to believe the information if it is gathered and

reported by the provider or plan rather than an independent third party.

For example, one individual stated that “an unscrupulous provider could

make sure they hit home runs on all of these particular items [the quality

measurements] . . . .” Individual consumers who requested and received

report card information from health plans used terms such as

“self-serving,” “one-sided,” and “nontrustworthy” to describe the report.

These respondents saw the purpose of the reports as a provider’s public

relations effort to “blow its own horn” or use the report as a “marketing

tool” rather than to provide information to the consumer.

Consumers Want

More Information

Consumers we interviewed want more information than they currently

have. Both employers and individual consumers want information that

emphasizes outcomes rather than process or structure measures of

quality. They want standardized information that allows them to compare

providers and plans. Few employers we interviewed are sharing

unpublished data with employees, and they differ from one another on

whether or not they believe their employees would use it to make

decisions. Individual consumers generally stated that they wanted reports

on quality to make decisions, but many emphasized that such reports

would never be the sole source of information; they would only augment

the advice of others.

Page 15 GAO/HEHS-95-201 Health Care Report Cards
B-257441 

Consumers Want

Outcomes More Than

Other Kinds of Measures

When emphasizing that they want information on the outcome of health

care provided, consumers are asking for a measure that allows them to

select providers who will improve their health status or that of their

employees. For example, in describing the need for outcome data, one

employer stated that rather than just knowing how many women received

mammography screening for breast cancer, he wanted to know if the

number of women who died or were incapacitated from breast cancer was

being reduced.

A major northeastern food manufacturer used outcomes to relate quality

assurance in health care to its manufacturing quality assurance program to

explain that “outcome data . . . is the only way to measure quality . . . .

Once you have the outcome, you can go back and look at the processes

themselves.”

A large West Coast employer stated that what the company really wants is

information on health status.

“What we’d like as a measure is we’d like to know that the plan has improved the health

status of the population served . . . . That might be different for some subpopulations. So,

[we would like to see reports] moving much more to population-based approaches.”

A medium-sized manufacturer stated that

“in general, you’re looking for quality and you’re looking for value, so maybe [we need]

more of a functional analysis. There is some subjective information that needs to be

obtained along with the length of stay and cost of stay and some of these other factors that

we’re just not getting yet . . . . You need to do a kind of functional analysis as well, to say 30

days after that angioplasty was that patient back at work, and were they working 40 hours

per week, and were they doing their job . . . . How’s your quality of life after you’ve had

this?”

A large northeastern financial services firm said

“the number one thing people ask . . . when they’re considering an HMO . . . is not like gee,

‘Am I going to get that mammogram,’ it’s ‘What if I get sick, am I going to die, are they going

to take care of me?’”

Both employers and individual consumers stated that although data

reflecting the outcomes may be the best measure of provider quality, it is

Page 16 GAO/HEHS-95-201 Health Care Report Cards
B-257441 

very difficult to know whether outcomes result from quality of care or

factors such as the patient’s condition or lifestyle choices.14

Data Must Be Standardized

and Easily Comparable

Both employers and individual consumers told us that they want

standardized data that could be used to compare health care providers’

and health plans’ performances. Though noting that efforts such as HEDIS

exist, employers told us that these measurement systems are still in the

developmental phase. They also said that without standardization, such as

in definitions of disease or methodology for analyzing data, this

information is not comparable regionally or nationally. Numerous

employers were participating in standardization efforts such as the

Midwest Business Group on Health’s efforts to standardize customer

satisfaction surveys. Many of the larger employers we interviewed hire

consulting firms that provide them with some level of standardization and

comparability. An official at a medium-sized manufacturing firm stated

that

“the government should prescribe some standards and force providers to adhere to these

standards in the publishing of information. The government should say, ‘You’re going to

code this disease this way, and you do it consistently and uniformly . . . .’”

Individual consumers stated that the way the information was presented

was very important to them. For example, some wanted to have providers

or plans compared side-by-side on one or two pages. Consumers using the

procedure-specific reports uniformly praised the table format that

provided this kind of direct comparison. (See fig. 2 for an example of a

table providing a comparison of providers from a procedure-specific

report.)

Some individual consumers wanted the information to cover a wider

geographic area, and others emphasized the need for community-specific

data. For example, some Pennsylvania consumers stated that the report

card for that state pertained only to providers in Pennsylvania. Consumers

living in Philadelphia would like to have had this type of information for

surrounding states because their providers, while close to their homes,

were located in other states. The same concern came up in a midwestern

city that bordered two states.

14Some experts estimate that it will take 10 to 15 years and millions of dollars to develop a technically

sophisticated measurement system that is able to accurately attribute outcomes to care.

Page 17 GAO/HEHS-95-201 Health Care Report Cards
B-257441 

Employers Have Sources

of Data Not Available to

Employees

Many employers are getting some of the quality of care data they want

through business coalitions, consultants, and their own data collection

efforts. This information is generally unavailable to the individual

consumer, and few employers we interviewed were sharing these data

with their employees. Employers differed in their opinion of whether or

not their employees would use these data in making health care choices.

Some employers stated that they do, or would, share information on

quality with their employees because such information would help their

employees make informed decisions. For example, a midwestern service

employer stated that it would be important to

“hav[e] some report card concepts that the employees could understand the information,

user friendly . . . consistent . . . . I want to have a tool for the employees to make that

decision. If the employees are making that decision, they are going to change the

marketplace. They are going to improve the quality of the system because the doctors and

the hospitals are going to have to alter their practices because of the information that has

been gathered and is presented and understood by employees. They are then making

intelligent decisions as far as where to get their health care . . . . Empower the people to

make the decision.”

A major employer located in the Northeast stated that

“if we are going to have value-based purchasing which would drive a competitive

marketplace in health care, we have to involve consumers who make the ultimate choice.

Therefore the information has to be relevant for them.”

Other employers believed that their employees did not want or would not

understand data comparing quality. A large East Coast manufacturer

stated that

“I think it’s not speaking to how they make decisions. I think we’d overwhelm them . . . .

Also, I don’t know if the data we’d be giving them would be the complete picture.”

Another large East Coast manufacturer stated that

“I’ve been in health care benefits for 15 years. I don’t know how to make the choice. What

happens to poor Harry the Huffer working on the shop floor when you give him . . . the

morbidity in this hospital is here, and you know the readmission rate is this, and the

reinfection rate is this, and the guy says, ‘I don’t know what I should do.’ Because what

they do to our counselors is say, ‘I don’t want to make choices.’”

Page 18 GAO/HEHS-95-201 Health Care Report Cards
B-257441 

Nevertheless, employees we interviewed disagreed with those employers

who said that employees would not use the information. Employees’

concerns included issues of validity and reliability such as risk assessment

and accuracy rather than their ability to understand the data. Most of the

individual consumers who had requested the published reports found

them to be easy to understand, using terms such as “clear,” “concise,” and

“well organized.” They found the charts and tables particularly useful. For

those who had some problems understanding the reports, additional

assistance was useful. For example, a Pennsylvania consumer who had

been unable to fully understand the published report on her own had no

trouble after it was explained by the state agency that had produced the

report. Another Pennsylvania consumer stated that the first report card

she received was difficult to understand but that by the third report she

received, she found it very useful.

Many individual consumers emphasized that published information would

never be the sole source of data for their health care decisions but would

be used in addition to other information such as personal consultation

with their physician, friends, family members, or coworkers.

Conclusion Data comparing health care plans and providers helped the consumers we

interviewed make their health care purchasing decisions. However,

performance reports have not yet achieved their fullest potential.

Consumers said they needed more reliable and valid data, more readily

available and standardized information, and a greater emphasis on

outcome measures.

Meeting the information needs of individual consumers continues to lag

behind meeting the employer needs. Attention must be paid to ensuring

that individual consumers have access to health care data. While

employers themselves have initiated efforts to cooperate with one another,

few we interviewed are making complete health care data available to

assist individual consumers in making purchasing decisions. Relevant

stakeholders have not yet addressed the issues of disseminating

performance data to individual consumers so that they can make

responsive, informed decisions about their health care coverage.

We are sending copies of this report to interested congressional

committees and other interested parties. We will make copies available to

others on request.

Page 19 GAO/HEHS-95-201 Health Care Report Cards
B-257441 

This report was prepared under the direction of Carlotta C. Joyner,

Associate Director. Other major contributors to this report include Sandra

K. Isaacson, Assistant Director; Susan Lawes; Lise Levie; Lesia Mandzia;

and Janice Raynor. Please call me on (202) 512-6806, or Dr. Joyner on

(202) 512-7002, if you have any questions.

Sincerely yours,

Janet L. Shikles

Assistant Comptroller General

Page 20 GAO/HEHS-95-201 Health Care Report Cards
Page 21 GAO/HEHS-95-201 Health Care Report Cards
Appendix I 

Scope and Methodology

To obtain information on how consumers use data comparing the quality

of health care providers or health plans and what information they want

when making health care purchasing decisions, we interviewed both

employers and individual consumers. To obtain the view of employers, we

interviewed officials at over 60 businesses. The size of these businesses

ranged from under 5 employees to over 100,000 employees. These

employers were selected on the basis of the following criteria: (1) size of

workforce (small, medium, and large); (2) geographic variability; and

(3) variation in whether or not they used published report cards. “Small”

employers were defined as those with fewer than 50 employees, “medium”

as having 50 to 499, and “large” as 500 or more employees. Because the

businesses were not randomly selected, their experiences and opinions

cannot be generalized to all employers. We also interviewed a major

private sector management consulting firm that supplies comparative

health care data to employers.

To obtain the views of individual consumers who had received a report

card, we conducted telephone interviews during January, February, and

March 1995 with 153 consumers who had requested and received

published report cards to determine how they used the information (see

table I.1). The report cards they received were published by either

California or Pennsylvania state agencies or by health maintenance

organizations in California and Minnesota. These report cards were

selected because they were the most recently available in which the

issuing entity had a record of requesters and the state or HMO was willing

to assist in the study. The consumers we talked with had received this

information sometime during 1993 and 1994. The reports published by the

state agencies contained only procedure-specific indicators, while the

health plan reports focused on various plan and procedure quality

indicators related to the individual health plans.

Table I.1: Summary of Telephone

Survey With Individual Consumers

State or health plan

Number of individuals

who requested report

card information

Number of individual

consumers we

intervieweda

Pennsylvania 633 120

California 43 6

Medica 338 22

Kaiser Permanente (HMO) 73 5

Total 1,087 153

aThese do not include those consumers who requested the report solely for their work, those who

did not receive or could not recall the details of the report, or those who chose not to or were

unable to participate.

Page 22 GAO/HEHS-95-201 Health Care Report Cards
Appendix I 

Scope and Methodology

Although we attempted to contact all 1,087 individuals who had requested

the report cards issued by those states or health plans, many of these

individuals did not choose to participate, could not recall receiving the

information, or had requested the information for reasons other than

making health care purchasing decisions, such as for school or work.

Because we spoke with only a small number of individuals who had

requested information for consumer-related purposes and they were not

chosen at random, their experiences and opinions cannot be generalized

to the entire consumer population that requested report card information.

We also conducted interviews with seven groups of employees around the

country who may not have had previous experience with such reports. We

conducted these interviews with employees from four federal government

agencies and three private corporations—manufacturing, sales, and

service. These employees were selected because their employers offered

them more than one health insurance plan to choose from when making

their health care insurance purchasing decisions. The number of

participants in each group ranged from 8 to 10 and included employees

with varying marital, family, and age status as well as employees enrolled

in both indemnity and managed care plans.

Page 23 GAO/HEHS-95-201 Health Care Report Cards
Related GAO Products

Medicare: Increased HMO Oversight Could Improve Quality and Access to

Care (GAO/HEHS-95-155, Aug. 3, 1995).

Employer-Based Health Plans: Issues, Trends, and Challenges Posed by

ERISA (GAO/HEHS-95-167, July 25, 1995).

Medicare: Enhancing Health Care Quality Assurance (GAO/T-HEHS-95-224,

July 27, 1995).

Health Care: Employers Urge Hospitals to Battle Costs Using Performance

Data Systems (GAO/HEHS-95-1, Oct. 3, 1994).

Health Care Reform: “Report Cards” Are Useful but Significant Issues

Need to Be Addressed (GAO/HEHS-94-219, Sept. 29, 1994).

Access to Health Insurance: Public and Private Employers’ Experience

With Purchasing Cooperatives (GAO/HEHS-94-142, May 31, 1994).

Managed Health Care: Effect on Employers’ Cost Difficult to Measure

(GAO/HRD-94-3, Oct. 19, 1993).

(101456) Page 24 GAO/HEHS-95-201 Health Care Report Cards
Ordering Information

The first copy of each GAO report and testimony is free.

Additional copies are $2 each. Orders should be sent to the

following address, accompanied by a check or money order

made out to the Superintendent of Documents, when

necessary. Orders for 100 or more copies to be mailed to a

single address are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 6015

Gaithersburg, MD 20884-6015

or visit:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders may also be placed by calling (202) 512-6000 

or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any

list from the past 30 days, please call (301) 258-4097 using a

touchtone phone. A recorded menu will provide information on

how to obtain these lists.

For information on how to access GAO reports on the INTERNET,

send an e-mail message with "info" in the body to:

info@www.gao.gov

PRINTED ON RECYCLED PAPER
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. G100