HEALTH CARE 

TRANSPARENCY

Actions Needed to 

Improve Cost and 

Quality Information for 

Consumers

Report to Congressional Requesters

October 2014

GAO-15-11

United States Government Accountability Office
United States Government Accountability Office

Highlights of GAO-15-11, a report to 

congressional requesters

October 2014

HEALTH CARE TRANSPARENCY

Actions Needed to Improve Cost and Quality 

Information for Consumers

Why GAO Did This Study

The cost and quality of health care 

services can vary significantly, with 

high cost not necessarily indicating 

high quality. As consumers pay for a 

growing proportion of their care, they 

have an increased need for cost and 

quality information before they receive 

care, so they can plan and make 

informed decisions. Transparency tools 

can provide such information to 

consumers and others.

GAO was asked to study cost and 

quality information for consumers. This 

report examines (1) information on cost 

and quality available to consumers 

from selected transparency tools, 

(2) characteristics of effective 

transparency tools, (3) limitations, if 

any, in the effectiveness of CMS 

transparency tools, and (4) CMS 

efforts to expand cost and quality 

information available through 

transparency tools. GAO analyzed 

information from two private tools—

selected because they had both cost 

and quality information—and CMS’s 

five transparency tools, reviewed 

research to identify best practices for 

conveying information to consumers, 

interviewed CMS and HHS officials 

and subject matter expects, and 

reviewed CMS and HHS planning 

documents and relevant criteria for 

effective planning in the federal 

government.

What GAO Recommends

GAO recommends that HHS’s CMS 

take steps to improve the information 

in its transparency tools and develop 

procedures and metrics to ensure that 

tools address consumers’ needs. HHS 

concurred with the recommendations 

and provided technical comments that 

were incorporated as appropriate.

What GAO Found

Results obtained from two selected private consumer transparency tools GAO 

reviewed—websites with health cost or quality information comparing different 

health care providers—show that some providers are paid thousands of dollars 

more than others for the same service in the same geographic area, regardless 

of the quality of such services. For example, the cost for maternity care at 

selected acute care hospitals in Boston, all of which rated highly on several 

quality indicators, ranged between $6,834 and $21,554 in July 2014.

Transparency tools are most effective if they provide information relevant to 

consumers and convey information in a way that consumers can readily 

understand. GAO identified key characteristics of effective transparency tools 

through a literature review and interviews with experts. The information that is 

most relevant to consumers relates directly to their personal circumstances, such 

as information on specific procedures they are considering, and allows them to 

make meaningful distinctions among providers based on their performance. 

Characteristics of such relevant information include describing key differences in 

quality of care and costs, particularly for what consumers are likely to pay out of 

pocket based on their specific circumstances. In addition, effective transparency 

tools must take specific steps to make the information they present 

understandable by consumers. For example, tools must enable consumers to 

discern patterns by summarizing related information and allowing consumers to 

customize information to focus on what is most relevant to them.

The Centers for Medicare & Medicaid Services (CMS) operates five transparency 

tools—Nursing Home Compare, Dialysis Facility Compare, Home Health 

Compare, Hospital Compare and Physician Compare—that are limited in their 

provision of relevant and understandable cost and quality information for 

consumers. In particular, GAO found that the tools lack relevant information on 

cost and provide limited information on key differences in quality of care, which 

hinders consumers’ ability to make meaningful distinctions among providers 

based on their performance. Because none of the tools contain information on 

patients’ out-of-pocket costs, they do not allow consumers to combine cost and 

quality information to assess the value of health care services or anticipate the 

cost of such services in advance. Additionally, GAO found substantial limitations 

in how the CMS tools present information, such as, in general, not using clear 

language and symbols, not summarizing and organizing information to highlight 

patterns, and not enabling consumers to customize how information is presented.

CMS, part of the Department of Health and Human Services (HHS), has taken 

some steps to expand access to cost and quality information for consumers, but 

has not established procedures or metrics to ensure the information it collects 

and reports meets consumer needs. Both HHS and CMS have set goals to report 

on measures that meet consumer needs. However, CMS’s process for 

developing and selecting cost and quality measures for its tools has been heavily 

influenced by the concerns of providers rather than consumers. Without 

procedures or metrics focusing on consumer needs, CMS cannot ensure that 

these efforts will produce cost and quality information that is relevant and 

understandable to consumers seeking to make meaningful distinctions.

View GAO-15-11. For more information, 

contact Linda T. Kohn at (202) 512-7114 or 

kohnl@gao.gov.
Page i GAO-15-11 Health Cost and Quality Transparency

Letter 1 

Background 6

Information from Two Selected Consumer Transparency Tools 

Consistently Demonstrated Substantial Cost Variation 

Regardless of Quality for Certain Services 11

Effective Transparency Tools Provide Cost and Quality 

Information That Is Relevant and Understandable to

Consumers 14

CMS Transparency Tools Are Limited in Their Provision of 

Relevant and Understandable Cost and Quality Information for 

Consumers 19

CMS Has Taken Limited Steps to Expand Access to Cost and 

Quality Information, but Has Not Established Procedures to 

Ensure This Information Meets Consumer Needs 24

Conclusions 30

Recommendations for Executive Action 31

Agency Comments and Our Evaluation 32

Appendix I Examples of Variation in Cost and Quality Information Available 

to Consumers for Selected Procedures 35

Appendix II Assessment of CMS Transparency Tools 41

Appendix III Comments from the Department of Health and Human Services 42

Appendix IV GAO Contact and Staff Acknowledgments 46

Tables

Table 1: Types of Health Care Quality Measures 7

Table 2: Existing and Expected Future Transparency Tools of the 

Centers for Medicare & Medicaid Services (CMS) 9

Table 3: Information Obtained through Telephoning Selected 

Ambulatory Surgical Centers and Hospital Outpatient 

Departments 13

Contents
Page ii GAO-15-11 Health Cost and Quality Transparency

Table 4: Cost and Quality of a Laparoscopic Gallbladder Surgery 

at Ambulatory Surgical Centers (ASCs) and Hospital 

Outpatient Departments in the Denver, Colorado-Area 36

Table 5: Cost and Quality of MRI of the Lower Back at StandAlone Imaging Centers and Acute Care Hospitals in the 

Indianapolis, Indiana Area 37

Table 6: Cost and Quality of Maternity Care at Acute Care 

Hospitals in the Boston, Massachusetts Area 38

Table 7: Cost and Quality Information Obtained for an Inguinal 

Hernia Repair or Diagnostic Colonoscopy at Ambulatory 

Surgical Centers (ASC) and Hospital Outpatient 

Departments in Two Health Care Markets Based on 

Telephoning Selected Providers 39

Table 8: Assessment of CMS Transparency Tools on Relevance 

and Understandability of Information for Consumers 41
Page iii GAO-15-11 Health Cost and Quality Transparency

Abbreviations

AHRQ Agency for Healthcare Research and Quality

APCD All-payer-claims database

ASC ambulatory surgical center

CMS Centers for Medicare & Medicaid Services

GPRAMA Government Performance and Results Act Modernization 

 Act of 2010

HCAHPS Hospital Consumer Assessment of Healthcare Providers 

 and Systems

HHS Department of Health and Human Services

IRF inpatient rehabilitation facility

IT information technology

MAP Measure Applications Partnership

MRI magnetic resonance imaging

PPACA Patient Protection and Affordable Care Act

ResDAC Research Data Assistance Center

QE Qualified Entity

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-15-11 Health Cost and Quality Transparency

441 G St. N.W.

Washington, DC 20548

October 20, 2014

Congressional Requesters

The cost and quality of health care services can vary significantly, with 

evidence suggesting that more expensive health care services are not 

necessarily higher in quality. Providers, such as hospitals or physicians, 

may be paid two or three times more than similar providers for the same 

health care services, without delivering higher quality of care.1 At the 

same time, a growing proportion of health care costs are being paid by 

consumers. Consumers without health insurance are generally 

responsible for paying what the provider charges, rather than discounted 

rates negotiated by insurance companies, while those with insurance also 

face increased costs, as use of high-deductible health plans and other 

forms of cost sharing have increased. In particular, consumers whose 

out-of-pocket costs are based on a percentage of the total cost of care 

can be especially affected by cost variations.2

As consumers are asked to pay more for their care, they run the risk of 

obtaining care from high-cost providers without receiving the highest 

quality care. Therefore, consumers have an increased need for 

transparent information on cost and quality before they receive care, so 

they can make informed decisions when choosing treatments and 

providers. Even when not faced with these decisions, consumers can use 

such information to better anticipate and plan for expenses. Together, 

information on both cost and quality enables consumers to assess the 

Consumers may also face 

fees associated with care received from physicians, laboratories, or 

hospitals that are outside of an insurance network and bill for their 

services separately.

 

1

See Massachusetts Attorney General, Examination of Health Care Cost Trends and Cost 

Drivers (Boston, Mass.: 2011); Institute of Medicine, Variation in Health Care Spending: 

Target Decision Making, Not Geography (Washington, D.C.: 2013); and P.S. Hussey, S. 

Wertheimer, and A. Mehrotra, “The Association between Health Care Quality and Cost: A 

Systematic Review,” Annals of Internal Medicine, vol. 158, no. 1 (2013).

2

For example, Medicare patients are typically responsible for 20 percent of the costs of 

the physician services that they receive, after they meet their annual deductible. On 

average, out-of-pocket costs for Medicare beneficiaries on medical and long-term care 

services amounted to $2,744 per year in 2010. See J. Cubanski, et al., How Much Is 

Enough? Out-of-Pocket Spending Among Medicare Beneficiaries: A Chartbook, Kaiser 

Family Foundation (Menlo Park, Calif.: July 2014).
 

Page 2 GAO-15-11 Health Cost and Quality Transparency

value of health care services, for example by comparing costs within a 

group of providers determined to be of high quality. Research suggests 

that this transparent information is most relevant for services that can be 

planned in advance, so consumers have time to consider it.3 We have 

previously found that this transparent information on cost and quality is 

difficult for consumers to obtain, and that the information that is available 

is not always meaningful to consumers.4

Transparency tools—websites with health cost or quality information 

comparing different providers of health care services—are one way that 

the Department of Health and Human Services (HHS) and other 

organizations provide this information to consumers and other users. 

HHS both publishes its own transparency tools and shares data on cost 

and quality that other public or private entities can use in their 

transparency tools. Specifically, HHS publishes transparency tools 

through its Centers for Medicare & Medicaid Services (CMS), such as its 

series of Compare websites for hospitals, nursing homes, and certain 

other providers that participate in the Medicare program.

In light of variations in cost and quality and consumers’ increased need 

for transparent information prior to receiving health care, you asked us to 

study the cost and quality information available to consumers. This report 

examines

1. information on cost and quality available to consumers for selected 

health care services from selected private transparency tools,

2. characteristics of effective transparency tools,

3. limitations, if any, in the effectiveness of CMS transparency tools in 

providing cost and quality information about providers to consumers, 

and

 

3

For example, to assist decision making, research suggests that health care price 

transparency is most relevant for consumers who are having services that are nonurgent, 

such as a knee replacement, or not complex, such as a colonoscopy. See, for example, 

Paul Ginsburg. “Shopping for Price in Medical Care,” Health Affairs, vol. 26, no. 2 (2007).

4

GAO, Health Care Price Transparency: Meaningful Price Information is Difficult for 

Consumers to Obtain Prior to Receiving Care, GAO-11-791 (Washington, D.C.: 2011); 

GAO, Nursing Homes: CMS Needs Milestones and Timelines to Ensure Goals for the 

Five-Star Quality Rating System Are Met, GAO-12-390 (Washington, D.C.: 2012).
 

Page 3 GAO-15-11 Health Cost and Quality Transparency

4. CMS efforts to expand cost and quality information that could be 

made available to consumers through transparency tools.

To examine information on cost and quality available to consumers for 

selected health care services from selected private transparency tools, 

we obtained examples of information from the transparency tools of a 

health insurer and Castlight Health, a third-party vendor.5 We selected 

these tools based on their capacity to provide consumer-relevant cost 

information on services that could be planned in advance, as well as 

some related information on quality of care. Within these tools, we 

selected services, provider types, and geographic locations based on 

each tool’s ability to provide cost and quality information for those 

services, provider types, and locations. The selected services were: 

laparoscopic gallbladder surgery, magnetic resonance imaging (MRI) of 

the lower back, and maternity care. The selected providers were: 

ambulatory surgical centers (ASC), hospital outpatient departments, and 

acute care hospitals. The selected locations were: Boston 

(Massachusetts), Indianapolis (Indiana), and Denver (Colorado). Both the 

health insurer and Castlight provided cost information based on an 

insurance plan common for their customers.6 We reviewed related 

documentation and interviewed knowledgeable company officials, and 

determined that the data were sufficiently reliable for our purposes. The 

findings from these transparency tools cannot be generalized to other 

tools. To determine the availability to consumers of cost and quality data 

in the absence of transparency tools, we directly contacted 24 randomly 

selected ASCs and hospital outpatient departments in Minneapolis 

(Minnesota) and Portland (Oregon) health care markets by telephone.7

 

5

Castlight serves employers who provide cost and quality health information to their 

employees.

We selected these locations because each of these states had cost and 

quality initiatives to provide information to consumers, and each market 

6

The health insurer provided estimates based on a point-of-service, high-deductible plan 

offered by employers. Castlight provided estimates based on a standard health plan with a 

$2,000 deductible, $5,000 out-of-pocket maximum, 100 percent coverage for certain 

preventive services, and 80 percent in-network coverage after application of the 

deductible. A point-of-service plan is a hybrid of a preferred provider organization plan and 

a health maintenance organization plan. It resembles a health maintenance organization 

plan for in-network services, and services received outside of the network are usually 

reimbursed based on a fee schedule or usual and customary charges.

7

We contacted each provider up to three times in an attempt to get a response between 

May 28 and July 7, 2014.
 

Page 4 GAO-15-11 Health Cost and Quality Transparency

had the largest number of hospitals in those states.8

To examine characteristics of effective transparency tools, we reviewed 

the literature on best practices for conveying cost and quality information 

to consumers and interviewed subject matter experts. We searched 

multiple online databases to identify relevant articles and reports 

published from 2008 to 2013,

During the calls we 

requested the cost of two services for an uninsured patient—an inguinal 

hernia repair and a diagnostic colonoscopy, as well as requested any 

quality information available for the service or facility. These two services 

were selected because each could be planned in advance and is 

commonly performed in an ambulatory setting. We did not assess the 

accuracy of the cost and quality information provided by these selected 

providers.

9 the most recent available at the time of our 

work. We also identified nine researchers as experts on the basis of their 

publications and other activities in the area of communicating cost and 

quality information to consumers. Six of these experts provided their 

perspectives in interviews. We synthesized our literature review and 

interviews into a draft description of key characteristics of effective 

transparency tools, and then obtained comments from the remaining 

three consumer transparency experts and revised the description of the 

characteristics as appropriate.

 

8

We used hospital referral regions, as defined by the Dartmouth Atlas of Health Care, to 

identify these regional health care markets. We randomly selected ASCs and hospitals to 

contact from the National Provider Identifier Registry.

Minnesota’s transparency tools include: Minnesota Hospital Price Check, Minnesota 

HealthScores, Minnesota Nursing Home Report Card, and Minnesota Community 

Measurement Health Care Quality Report. Oregon’s transparency tools include: Oregon 

PricePoint, Compare Hospital Costs, and Partner for Quality Care. We also selected 

Minnesota because it requires providers to make estimated costs of treatment and 

estimated costs that must be paid by the patient available upon request. See Minn. Stat. 

§§ 62J.81-62J.823.

9

Databases we searched included the ABI/Inform Professional Advanced; Allied & 

Complementary Medicine; BIOSIS Previews; British Library Inside Conferences; British 

Nursing Index; Embase; Emcare; Gale Group Health Periodicals Database; HSELINE: 

Health and Safety; Incidence & Prevalence; International Pharmaceutical Abstracts; King’s 

Fund; MEDLINE; National Technical Information Service; Public Affairs Information 

Service International; PsycINFO; SciSearch; Social SciSearch; ProQuest; and Web of 

Science.
 

Page 5 GAO-15-11 Health Cost and Quality Transparency

To examine the limitations, if any, in the effectiveness of CMS 

transparency tools in providing cost and quality information about 

providers to consumers, we identified five CMS transparency tools that 

provide comparative cost or quality information about providers: Hospital 

Compare, Nursing Home Compare, Dialysis Facility Compare, Home 

Health Compare, and Physician Compare. We then reviewed each tool to 

determine the extent to which it had characteristics of effective 

transparency tools, using the list we identified in our second objective. We 

interviewed CMS officials responsible for planning, developing and testing 

the HHS transparency tools, and reviewed HHS documentation on the 

process for developing the tools and HHS’s future plans for improving the 

tools.

To examine CMS efforts to expand cost and quality information available 

to consumers through transparency tools, we interviewed CMS and HHS 

officials with oversight and involvement in public reporting and CMS 

transparency initiatives, including officials who develop and maintain 

CMS’s Compare websites. We also reviewed HHS strategic planning 

documents related to health care quality and public reporting, and 

reviewed relevant criteria from the Government Performance and Results 

Act Modernization Act of 2010 (GPRAMA) as incorporated in our 

guidance on assessing performance.10 We then interviewed consumer 

transparency experts, as noted above, and officials representing privateand state-sponsored organizations that use CMS data in their 

transparency tools to identify CMS and other HHS agency actions that 

could help to expand information available to consumers. Because our 

study focused on CMS’s efforts to promote transparency for consumers, 

we did not examine other factors affecting access to cost and quality 

information, such as barriers to obtaining information on payment rates 

negotiated between individual providers and private sector payers.11

We conducted this performance audit from December 2013 to October 

2014 in accordance with generally accepted government auditing 

standards. Those standards require that we plan and perform the audit to 

 

10See Pub. L. No. 111-352, 124 Stat. 3866 (2011). GPRAMA amends the Government 

Performance and Results Act of 1993, Pub. L. No. 103-62, 107 Stat. 285 (1993); GAO, 

Executive Guide: Effectively Implementing the Government Performance and Results Act,

GAO/GGD-96-118 (Washington, D.C.: June 1996).

11See GAO-11-791 for more information on these other factors affecting access to cost 

and quality information.
 

Page 6 GAO-15-11 Health Cost and Quality Transparency

obtain sufficient, appropriate evidence to provide a reasonable basis for 

our findings and conclusions based on our audit objectives. We believe 

that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives.

Transparency tools are a way to make information on health care cost

and quality transparent to consumers and others, and are a key part of 

HHS’s strategy to improve the quality and affordability of health care.

There are multiple ways to assess the cost of health care. For example, 

cost can be measured based on the amount of money providers set as a 

“charge” for various individual services, but these charges typically do not 

represent the actual amounts paid by insurers or consumers. The cost 

that an insured consumer is responsible for paying to receive services is 

called an out-of-pocket cost, which depends on the consumer’s individual 

provider choices and insurance benefit design.12

There also are a variety of different types of clinical quality measures—

standard, evidence-based metrics used to assess the performance of 

health care providers—that address different aspects of quality. 

According to HHS’s Agency for Healthcare Research and Quality’s 

(AHRQ) National Quality Measures Clearinghouse, types of quality 

measures include, (1) structure, (2) process, (3) outcome, and (4) patient 

experience of care.

In addition, any given 

episode of care usually involves payments to multiple providers (e.g., 

surgeons, anesthesiologists, pathologists, etc.), facility fees, and other 

ancillary fees, and any given cost figure may or may not represent the 

total costs for an episode of care by including all of these expenses.

13

 

12For insured consumers, out-of-pocket payment typically includes copayments, 

coinsurance, and deductibles, which are affected by a number of variables such as the 

use of out-of-network providers.

(See table 1 for a description of types of quality 

measures.)

13For a description of the quality measure categories, see Agency for Healthcare 

Research and Quality, National Quality Measures Clearinghouse, 

http://www.qualitymeasures.ahrq.gov/tutorial/varieties.aspx, accessed August 5, 2014. 

The National Quality Measures Clearinghouse also includes other types of clinical quality 

measures, such as access measures, which measure a patient’s ability to attain timely 

and appropriate care.

Background

Information on Health 

Care Cost and Quality
 

Page 7 GAO-15-11 Health Cost and Quality Transparency

Table 1: Types of Health Care Quality Measures

Measure type Description Examples

Structural Reflect the conditions in which providers care for 

patients.

Measures about staffing levels or the volume of 

procedures performed by a provider.

Process Show whether steps, or processes of care, that have 

been proven to benefit patients are followed correctly.

Measures that indicate whether an action was 

completed—such as writing a prescription, 

administering a drug, or having a conversation.

Outcome Report the actual results of care. Measures that indicate a change in patient health 

status, such as lower blood pressure for a patient who 

is hypertensive.

Patient experience Record patients’ perspectives on their care. Measures based on patient reports on their care, often 

obtained through surveys, such as patient responses to 

a question about whether their pain was always well 

controlled during a hospital stay.

Source: National Quality Measures Clearinghouse and National Quality Forum. | GAO-15-11

Quality measures of each type vary greatly in their breadth. Some focus 

on overarching assessments of quality, such as asking patients if they 

would recommend a hospital where they had recently been treated, while 

other quality measures have a much more narrow focus, for example, 

addressing a single step in a process of care. Researchers have 

developed techniques for consolidating information from many separate 

but related narrowly focused quality measures into broader assessments. 

These are called summary measures or composite measures.

The data used to apply these different types of cost and quality measures 

can come from a number of sources. Clinical data extracted from medical 

records can provide important clinical details needed to more fully adjust 

provider performance assessments for differences in the severity of 

illness of the patients that they treat—a process known as risk 

adjustment—but such data are typically costly and time-consuming to 

collect. By contrast, claims data—also known as billing or administrative 

data—are already collected to process provider payments, and therefore 

are more readily available for a large number of patients. However, 

because claims data are collected for payment purposes, they contain 

only a limited set of information relevant for quality measures or for 

making risk adjustments. Finally, patients can be asked directly to report 

on their experiences of the care they received and the outcomes of their 

care that relate to their functional abilities, such as their ability to descend 

stairs. Outcome measures based on these reports are known as patientreported outcomes.
 

Page 8 GAO-15-11 Health Cost and Quality Transparency

In March 2011, HHS first published its National Strategy for Quality 

Improvement in Health Care (the National Quality Strategy), as required 

by the Patient Protection and Affordable Care Act (PPACA).14 The 

National Quality Strategy builds on priorities HHS previously identified in 

its strategic plan for fiscal years 2010-2015, which emphasize the need 

for transparent information to give consumers the means to make more 

informed choices about their health care. Two of the National Quality 

Strategy’s overarching goals—better care and affordable care—relate to 

health care cost and quality transparency. According to the strategy, to 

achieve better care, patients must be given access to understandable 

information and decision support tools that help them manage their health 

and navigate the health care delivery system. To achieve affordable care, 

systems must be created to make health care cost and quality more 

transparent to consumers and providers, so they can make better choices 

and decisions. The strategy also focuses on coordinating and aligning 

efforts across the public and private sectors, for example by establishing 

an aligned set of common cost and quality measures by which to assess 

how well providers and programs support effective care.

As part of its efforts to foster greater transparency of information, HHS 

developed transparency tools for consumers focused on quality, some of 

which predate PPACA and the development of the National Quality 

Strategy. Between 1998 and 2010, HHS—through CMS—launched five 

Compare websites to publicly report certain information, including quality 

information, based on data submitted by different types of providers 

participating in the Medicare program (see table 2). For some, but not all, 

 

14Section 3011 of PPACA required the Secretary of the Department of Health and Human 

Services to establish a national quality strategy, including a comprehensive strategic plan 

and the identification of priorities to improve the delivery of health care services, patient 

health outcomes, and population health. PPACA requires that the strategy be developed 

using a transparent and collaborative process and must include provisions for: (1) agencyspecific plans and benchmarks; (2) coordination among agencies; (3) strategies to align 

public and private payers; and (4) alignment with meaningful use of health information 

technology (IT).

HHS’s National Quality 

Strategy

HHS’s Role in Establishing 

Transparency Tools
 

Page 9 GAO-15-11 Health Cost and Quality Transparency

of these provider types, HHS is required to publicly report certain 

information about provider performance.15

Table 2: Existing and Expected Future Transparency Tools of the Centers for 

Medicare & Medicaid Services (CMS)

Tool name Date launched

Nursing Home Compare 1998

Dialysis Facility Compare 2001

Home Health Compare 2005

Hospital Compare 2005

Physician Compare 2010a

Hospice Quality Reporting Under development

Inpatient Rehabilitation Facility (IRF) Quality Reporting Under development

Long-term Acute Care Hospital Quality Reporting Under development

Source: HHS. | GAO-15-11

a

Physician Compare currently provides quality information related to management of diabetes and 

heart disease for 66 physician groups and 141 Accountable Care Organizations. CMS has plans to 

add additional information to that tool by 2015.

HHS—primarily through AHRQ—also plays a role in research and 

dissemination of information on consumer-focused public reporting. 

AHRQ has supported the research and publication of numerous papers 

that lay out best practices for public reporting of cost and quality 

information to consumers and has widely shared the expertise contained 

 

15HHS’s reporting programs and responsibilities vary, including whether specific 

information is required to be posted on the Compare websites. In some cases, PPACA 

either created the reporting program or required the establishment of the website, but in 

other cases, the reporting program or website was preexisting. For example, section 3001 

of PPACA established the Hospital Value Based Purchasing Program under which valuebased incentive payments are made to hospitals that meet performance standards. 

Section 3001 requires that HHS post information regarding the performance of individual 

hospitals under the program on the preexisting Hospital Compare website. The Medicare 

Plan Finder is another CMS transparency tool that allows consumers to compare 

Medicare health, prescription drug, and supplemental health insurance policies based on 

various characteristics, including some information about the costs and quality of the 

plans. Similarly, CMS launched the Health Insurance Marketplace tool in 2013, which 

compares costs for commercial insurance plans with plans to add quality ratings starting in 

2016. These tools are not included in our report, because they compare insurance 

products rather than comparing the cost and quality of health care providers.
 

Page 10 GAO-15-11 Health Cost and Quality Transparency

in that research through various forums, such as conferences and 

webinars that support wider transparency efforts.16

A variety of private sector entities have developed transparency tools that 

provide cost and quality information to consumers, including health 

insurance carriers, third-party vendors, and regional collaboratives. These 

private-sector tools allow consumers to obtain personalized cost 

estimates, compare different providers, or estimate their out-of-pocket 

costs before receiving a service. For example, many health plans offer 

tools that provide cost estimates to enrolled members.17 Researchers 

calculated that approximately 70 percent of the privately insured 

population had access to cost transparency tools in 2013.18 In addition to 

tools provided directly by health plans, employers may contract with thirdparty vendors to provide transparency tools for their employees.19 Health 

plans and third-party vendors frequently offer some quality information 

along with cost estimates to their enrolled members. Often this 

information is derived from data reported by CMS on its Compare sites 

and may be combined with additional quality information obtained from a 

variety of sources that collect and report data on provider quality.20

 

16See for example, Agency for Healthcare Research and Quality, Best Practices in Public 

Reporting No. 1: How To Effectively Present Health Care Performance Data To 

Consumers, AHRQ Pub. No. 10-0082-EF, (Rockville, Md.: June 2010). AHRQ also has 

several programs designed to support states, private entities, and others that provide cost 

and quality information to consumers, and funds research on public reporting.

17For example, Aetna, Cigna, Humana, and UnitedHealthcare, as well as the Blue Cross 

plans Anthem and Health Care Services Corp., among others, all have cost transparency 

tools either developed internally or contracted through a third-party vendor.

18See Westhealth Policy Analysis, Healthcare Price Transparency: Policy Approaches and 

Estimated Impacts on Spending (Washington, D.C.: Westhealth Policy Center, 2014), 6.

19Castlight, Change Healthcare, and Health Care Blue Book are examples of third-party 

vendors.

20For example, Castlight reported that some of its quality information comes from 

LeapFrog, another private entity that collects quality information primarily on hospitals.

Other Entities’ 

Transparency Tools
 

Page 11 GAO-15-11 Health Cost and Quality Transparency

Several states also have developed transparency tools.21 Unlike health 

plans and third-party vendors, state tools can be accessed by the general 

public and are not restricted to members who pay or enroll for services, 

giving the uninsured access to cost and quality information. Like the 

private-sector tools, state tools generally draw on quality information from 

CMS in combination with information from other sources.

The two selected consumer transparency tools we reviewed show that 

some providers are paid thousands of dollars more than others for the 

same service in the same geographic area, regardless of the quality of 

such services. Specifically, we found this variation in cost to be present 

across multiple services, settings, and geographic areas in the 

information provided by the transparency tools of two different entities (a 

health insurer and Castlight) that we reviewed. For example, the health 

insurer reported that in the Denver area, the estimated total cost of a 

laparoscopic gallbladder surgery in selected ASCs ranged between 

$3,281 and $18,770 (consumers would pay between $3,281 and $6,954 

in estimated out-of-pocket costs) in July 2014. Meanwhile, for the same 

time period, the health insurer reported that the estimated total cost for 

the same service in selected hospital outpatient departments in the 

Denver area ranged from $17,791 to $40,626 (consumers would pay 

between $6,758 and $11,325 in estimated out-of-pocket costs).22

 

21For example, 11 states have developed tools using data collected through all-payerclaims databases (APCD), and 6 additional states have plans to launch similar efforts. 

APCDs are large-scale databases that systematically collect medical, pharmacy, and in 

some cases, dental, claims, and eligibility and provider files from private and public payers 

to be used for multiple purposes, such as to assess the impact of health care reforms or to 

report cost and quality information to consumers.

Similarly, Castlight reported, in May 2014, that for an MRI of the lower 

back at selected acute care hospitals in the Indianapolis, Indiana area, 

the estimated total cost ranged between $277 and $5,184 depending on 

the provider (with consumers paying between $277 and $2,637 in 

22The estimated total costs provided by the health insurer include facility, physician, and 

anesthesia fees. The estimated out-of-pocket costs provided by the health insurer are 

what a consumer would pay if enrolled in one of its self-insured point-of-service, high 

deductible health plan—a common benefit plan for the insurer.

Information from Two 

Selected Consumer 

Transparency Tools 

Consistently 

Demonstrated 

Substantial Cost 

Variation Regardless 

of Quality for Certain 

Services
 

Page 12 GAO-15-11 Health Cost and Quality Transparency

estimated out-of-pocket costs).23

Information from the two selected transparency tools indicates that the 

observed cost variation is not tied to variations in quality, regardless of 

the treatment and geographic area. For example, according to information 

obtained from Castlight, the estimated total cost of maternity care at 

selected acute care hospitals in the Boston area that rated more highly on 

several quality indicators ranged between $6,834 and $21,554 

(consumers would pay between $2,967 and $5,000 in estimated out-ofpocket costs).

(See app. I for more examples of the 

variation in information we obtained from selected consumer 

transparency tools.)

24 Likewise, information from the health insurer for a 

laparoscopic gallbladder surgery showed that a number of hospital 

outpatient departments in the Denver area that rated more highly on 

several quality indicators had lower total costs than other hospital 

outpatient departments that rated less highly on quality.25

Without these transparency tools, which we selected based on their ability 

to provide consumer-relevant cost and quality information, it would not be 

easy for consumers to obtain this information directly from providers. 

Specifically, our effort to gather cost and quality information by calling 

selected providers directly—as an average consumer might—

demonstrated the difficulty of obtaining this type of information without the 

These 

examples are particularly relevant as researchers have found that many 

consumers assume that all providers offer good quality care, while others 

have the misconception that higher-cost providers will provide higher 

quality of care than lower-cost providers.

 

23The estimated total costs provided by Castlight include a standard MRI, radiologist’s fee, 

facility fee, reading and interpretation of the MRI. The estimated out-of-pocket costs 

provided by Castlight are what a consumer would pay if enrolled in a health plan with a 

$2,000 deductible, $5,000 out-of-pocket maximum, 100 percent coverage for preventive 

services, and 80 percent in-network coverage after application of the deductible—a 

common benefit plan for employers that use this third-party vendor.

24The quality indicators provided by Castlight for maternity care include the percent of 

pregnancies for which the baby is delivered at the appropriate developmental age, using 

an appropriate delivery method, and recommended processes of care.

25The quality measures provided by the health insurer for a gall bladder surgery include 

hospitals overall performance in keeping patients safe from preventable harm and medical 

errors and patients experiences with the hospital overall. These measures apply to 

inpatient care overall, and none are based on information relating to outpatient care or 

specifically to gall bladder surgery.
 

Page 13 GAO-15-11 Health Cost and Quality Transparency

assistance of consumer transparency tools.26

Table 3: Information Obtained through Telephoning Selected Ambulatory Surgical

Centers and Hospital Outpatient Departments

Of the 24 ASCs and 

hospitals we contacted to inquire about the cost of an inguinal hernia 

repair and diagnostic colonoscopy for an uninsured patient in the 

Portland, Oregon, and Minneapolis, Minnesota health care markets—

locations selected because they have initiatives to promote 

transparency—we received limited cost information from 54 percent (13) 

and quality information from 29 percent (7) (see table 3).

Information obtained from telephoning selected providers Number of providers

Only cost information 9 

Only quality information 3 

Both cost and quality information 4 

No cost and quality information 8 

Total 24

Source: GAO. | GAO-15-11

Note: We contacted 24 providers between May and July 2014 to obtain cost and quality information 

for an inguinal hernia repair and diagnostic colonoscopy for an uninsured patient at their facility.

Only 5 of the 13 ASCs and hospitals that provided cost estimates were 

able to estimate the total cost, including all of the facility, physician, 

anesthesia and other costs involved. The quality information we received 

was highly disparate across the providers, which precluded making valid 

comparisons among providers on the basis of quality. Therefore, despite 

receiving both cost and quality information from some of the ASCs and 

hospitals we contacted, it was not possible to assess these providers in 

terms of both cost and quality. Our experiences receiving limited cost and 

quality information in two locations that have adopted specific initiatives to 

promote cost and quality transparency—as well as researchers’ 

experiences in different locations—suggest that consumers in other 

locations would face similar difficulties when calling providers for such 

 

26The results of anonymously calling selected providers in Portland, Oregon and 

Minneapolis, Minnesota, yielded similar results to our efforts to obtain cost information 

from selected hospitals and primary care physicians for selected services in Denver, 

Colorado, which we reported in our 2011 report. See GAO-11-791. 
 

Page 14 GAO-15-11 Health Cost and Quality Transparency

information.27 (See app. 1 for more examples of the variation we obtained 

from calling selected providers.) 

Transparency tools are most effective if they both provide information 

relevant to consumers and convey that information in a way that 

consumers can readily understand.

The information on cost and quality that consumers find relevant makes 

up just a portion of the information available for inclusion in transparency 

tools. For example, research shows that most consumers do not find the 

information derived from many specific process-of-care measures that 

clinicians have developed and used to guide quality improvement efforts 

useful. Rather, consumers are most likely to respond to information that 

applies to their personal circumstances, including, for example, 

information on the specific procedures consumers are considering, on 

providers who would be available to perform those procedures, and on 

cost estimates that take account of their particular insurance coverage. 

Consumers also seek information that helps them to make meaningful 

distinctions among providers in terms of cost and quality. The research 

we reviewed has found that consumers value quality measures that show 

differences in clinical outcomes and patient experiences, and cost 

measures that show differences in out-of-pocket expenses. In addition, 

consumers want to know the source of cost and quality information 

 

27See GAO-11-791, 31-37. For other researchers’ attempts to obtain cost information by 

calling selected providers, see J. A. Rosenthal, X. Lu, P. Cram, “Availability of Consumer 

Prices from US Hospitals for a Common Surgical Procedure,” JAMA Internal Medicine, 

vol. 173, no. 6 (March 25, 2013): 427-432.

Effective 

Transparency Tools 

Provide Cost and 

Quality Information 

That Is Relevant and 

Understandable to 

Consumers

Effective Transparency 

Tools Provide Information 

Relevant to Consumers
 

Page 15 GAO-15-11 Health Cost and Quality Transparency

because this source information helps them determine their level of 

confidence in that information.28

Eight of the 15 characteristics of effective transparency tools we identified 

address the extent to which a tool provides substantive quality and cost 

information of relevance to consumers. Specifically, the research we 

reviewed shows that more effective transparency tools:

1. Review a broad range of services so that more consumers’ 

particular needs are included. The more services that are covered 

by a transparency tool (or set of tools), the more likely it is that the tool 

will have information relevant to the particular services of interest to 

any given consumer. It is especially important to include services that 

are predictable and non-urgent in a transparency tool, because these 

services are most likely to afford consumers the opportunity to 

evaluate cost and quality information before receiving the service.

2. Cover a broad range of providers. Transparency tools that provide 

information for all or most of the available providers in a given 

geographic area, regardless of network status or practice setting, give 

consumers more information about their full range of options. For 

example, for procedures that can be conducted in either a hospital 

outpatient department or ASC, it helps consumers to provide 

comparable information for both settings, so that consumers can 

choose from a larger number of providers that offer those procedures.

3. Describe key differences in clinical quality of care, particularly 

patient-reported outcomes. Assessments of the clinical quality of 

care that have been shown to have particular relevance to consumers 

are those that relate to long-term outcomes of the care experienced 

by other patients. Often this is best addressed by patient-reported 

outcomes, which tell consumers the eventual outcome of treatments, 

as reported by previous patients of a particular provider. For example, 

patients receiving hip replacements can be asked, through such 

patient-reported outcomes, to rate their ability to climb stairs both 

before and after their procedures, which enables assessments of the 

procedures’ effects on patients’ mobility.

 

28For example, the research we reviewed shows that consumers have greater confidence 

in information that comes from a source that they perceive to be independent and 

objective. They treat more skeptically information coming from organizations, such as 

health plans or the providers being assessed, that might have a vested interest in how 

different providers are rated.
 

Page 16 GAO-15-11 Health Cost and Quality Transparency

4. Describe key differences in patient experiences with providers. 

Another outcome that matters is patients’ assessment of their 

interactions with providers. Effective transparency tools include 

information on how past patients evaluated providers in terms of 

dimensions such as how well nurses communicate with patients, or 

the responsiveness of clinicians to patients’ needs.

5. Describe key differences in costs, particularly patient out-ofpocket costs. The cost information of greatest relevance to 

consumers is finding out what they will have to pay for a given service 

before it occurs. For example, a tool that provides information on the 

average cost of a selected procedure in a given geographic location 

may be less relevant to consumers than one that takes into account 

the specific costs for that procedure and location given that 

consumer’s specific health insurance coverage.

6. Describe other information related to quality, where appropriate. 

There may be other quality indicators that could have major 

significance to consumers for certain types of services. For example, 

facility inspection results and staffing levels are of particular relevance 

to nursing home care.

7. Provide timely Information. More recent data are intrinsically more 

relevant than data that are several years old. Because consumer 

transparency tools necessarily rely on past data to assess likely cost 

and quality performance in the future, some lag in collecting, 

analyzing, and providing data is inevitable. Data that are no more than 

two years old are generally considered timely.

8. Describe key strengths and limitations of the data. Although the 

research we reviewed shows that few consumers are inclined to delve 

into the many methodological issues that concern appropriate 

techniques for collecting, checking, and analyzing cost and quality 

data, transparency tools can provide both summary assessments of 

strengths and limitations for most consumers, and links to more 

complete explanations for those wanting to pursue these issues in 

greater detail. Such information, along with identification of the 

organization responsible for the tool, provides consumers a basis to 

judge the credibility of the cost and quality information provided.
 

Page 17 GAO-15-11 Health Cost and Quality Transparency

Regardless of its potential relevance to them, consumers only can 

respond to information about cost and quality if they understand it. 

Although cost and quality information can be inherently difficult to 

understand, research suggests ways that transparency tools can make it 

easier for consumers to do so. One factor that increases the difficulty of 

understanding such information is the prevalence of misleading 

preconceptions about the cost and quality of care. For example, 

researchers find that many consumers incorrectly assume that all 

hospitals and physicians provide good quality care, while others assume 

that higher-cost providers will provide higher quality of care than lowercost providers. As a result, transparency tools that address these 

misleading preconceptions can better help consumers to understand the 

information they present.

Other challenges identified by researchers that consumers face include 

absorbing and evaluating large amounts of information about multiple 

providers across different measures of cost and quality, assessing a 

provider who does relatively well on some measures of quality and less 

well on others, and interpreting complex numerical information.29

Therefore, it is important that transparency tools limit the amount of 

information that consumers need to pay attention to and make it easy for 

them to discern overall patterns. One experiment demonstrated that 

improving the organization and presentation of quality information in 

transparency tools led to an increased proportion of consumers who 

could identify the best-performing providers for a given dimension 

of quality. Specifically, with the improvements, the proportion of 

consumers who could identify the best-performing providers increased 

from 18 percent to 76 percent.30

Seven of the 15 characteristics of effective consumer transparency tools 

we identified focus on the extent to which a tool presents its information in 

a way that enables the consumer to grasp and interpret it. Specifically, 

the research we reviewed shows that more effective transparency tools:

 

29E. Peters, et al., “Less Is More in Presenting Quality Information to Consumers,” Medical 

Care Research and Review, vol. 64, no. 2 (April 2007).

30K. Carman, “Improving Quality Information in a Consumer-driven Era: Showing 

Differences Is Crucial to Informed Consumer Choice,” presentation at the 10th National 

Consumer Assessment of Healthcare Providers and Systems User Group Meeting, 

(Baltimore, Md.: Mar. 31, 2006).

Effective Transparency 

Tools Take Steps to Make 

Cost and Quality 

Information 

Understandable to 

Consumers
 

Page 18 GAO-15-11 Health Cost and Quality Transparency

1. Use plain language with clear graphics. Effective consumer 

transparency tools use labels and descriptions that make sense to 

consumers who typically are unfamiliar with clinical terminology and 

who often have difficulty interpreting numerical information. Graphics, 

including symbols, can help to readily convey information on relative 

provider performance, especially when they are designed to display a 

summary assessment of that performance as part of the symbol itself, 

for example one that incorporates the words “superior” or “poor”.

2. Explain purpose and value of quality performance ratings to 

consumers. Effective consumer transparency tools address prevalent 

misleading preconceptions by providing consumers coherent 

explanations of how different quality measures relate to the aspects of 

quality that consumers find relevant. These explanations work best 

when they link individual measures to overarching categories 

indicating what is being achieved, such as effectiveness of care, 

safety, or patient-focused care.

3. Summarize related information and organize data to highlight 

patterns and facilitate consumer interpretation. Two techniques 

that consumer tools can use to help consumers make sense of large 

amounts of information are (a) combining information from multiple 

related measures into summary or composite scores, and 

(b) structuring presentation of the data in ways that make patterns 

evident. For example, listing providers in rank order on selected cost 

and quality dimensions greatly simplifies identification of high and low 

performers.

4. Enable consumers to customize information selected for 

presentation to focus on what is most relevant to them. 

Consumers differ in the priority they assign to different aspects of 

quality. Tools that enable consumers to customize which quality 

information is presented help consumers filter out information of 

lesser consequence to them, and hone in on the information that they 

find most compelling. For example, one consumer may choose to 

focus on providers’ capacity to communicate well with patients, while 

another may focus on providers’ rates of complications and infections.

5. Enable consumers to compare quality performance of multiple 

providers in one view. Transparency tools are most effective when 

they present side-by-side assessments of providers’ performance on 

a given aspect of cost or quality, so consumers can most easily 

compare providers.
 

Page 19 GAO-15-11 Health Cost and Quality Transparency

6. Enable consumers to assess cost and quality information 

together. Consumers cannot make judgments about the value of the 

care offered by providers unless they can consider both cost and 

quality in relation to each other. For example, transparency tools can 

enable consumers to rank order available providers first on selected 

measures of quality, and then, within the high-quality group, show 

those with lower costs.

7. Enable easy use and navigation of the tool. Unless consumers can 

quickly find information of interest to them, they are likely to quickly 

dismiss the potential utility of a consumer transparency tool and move 

on. Extensive testing with consumers can help public and private 

entities providing transparency tools to develop intuitive, user-friendly 

approaches to website navigation and for manipulating how the data 

are presented.

The CMS transparency tools we evaluated are limited in the relevance 

and understandability of cost and quality information they provide to 

consumers.

CMS Transparency 

Tools Are Limited in 

Their Provision of 

Relevant and 

Understandable Cost 

and Quality 

Information for 

Consumers
 

Page 20 GAO-15-11 Health Cost and Quality Transparency

Based on the characteristics we identified concerning information 

provided to consumers by effective transparency tools, we found that 

CMS’s tools demonstrate a number of the characteristics of effective 

transparency tools, such as timeliness of data; however, the tools lack 

relevant information on cost and key differences in quality of care.31

With respect to cost information, none of the tools contain information on 

the specific costs that patients would incur under Medicare—such as the 

out-of-pocket costs to a consumer for a full episode of care. Therefore, 

they do not allow consumers to combine cost and quality information to 

assess the value of health care services, or anticipate and plan for 

expenses related to non-emergency procedures. For example, a 

consumer may wish to compare the costs of similar high-quality 

providers. Depending on the service a consumer is planning to receive, a 

consumer may be able to find some indications of provider quality in CMS 

transparency tools, but would not find relevant information on cost. One 

CMS official stated that CMS does not provide expected patient out-ofpocket costs because the agency does not have information on what 

beneficiaries would pay when they have coverage other than, or in 

addition to, traditional fee-for-service Medicare, as many beneficiaries 

have.

These limitations hinder their relevance and usefulness by consumers, 

particularly consumers’ ability to make meaningful distinctions among 

providers. (See Appendix 2 for our assessment of how CMS tools fare on 

all 15 characteristics of effective tools identified through our literature 

review and interviews with experts.)

32

 

31One of the five tools—Physician Compare— offers very limited information on cost and 

quality to consumers. Specifically the tool offers information about physicians including 

name, practice specialty, gender, and educational qualifications, as well as quality 

measures related to management of diabetes and heart disease for 66 physician groups

and 141 Accountable Care Organizations. Therefore we focused our review on the 

remaining four tools.

However, CMS has the information necessary to create estimates 

of what Medicare beneficiaries likely would pay for different treatments 

and procedures, based on payment levels the program has set for each 

provider and the cost-sharing provisions that apply under the traditional 

fee-for-service Medicare program. For some Medicare beneficiaries, this 

information would provide an estimate of their out-of-pocket costs. For 

those with supplemental insurance coverage, it would not provide full out32This includes beneficiaries who purchase Medigap policies or are covered under 

Medicare Advantage plans.

CMS Transparency Tools 

Lack Certain Relevant 

Information Needed to 

Make Meaningful 

Distinctions about the Cost 

and Quality of Providers
 

Page 21 GAO-15-11 Health Cost and Quality Transparency

of-pocket costs, but could be useful as an indicator of the maximum 

amounts they would have had to pay without supplemental coverage.

We found that all of the CMS transparency tools provide some clinical 

quality information relevant to consumers, but they often lack conditionspecific information for the type of non-urgent procedures that consumers 

can most readily plan for in advance.33 For example, some information on 

hip and knee replacements is included in Hospital Compare, but limited 

information is available in Hospital Compare for many other common 

medical procedures such as a colonoscopy. In addition, with the 

exception of Hospital Compare, none of CMS’s transparency tools 

currently provide information on patient-reported outcomes, which have 

been shown to be particularly relevant to consumers considering common 

elective medical procedures, including hip and knee replacements.34

CMS officials stated that the department plans to expand over time the 

cost and quality information reported in the CMS transparency tools. For 

example, CMS officials described their efforts to develop patient-reported 

outcome measures to include in Hospital Compare, beginning with hip 

and knee replacements. They also plan to expand the information in 

Hospital Compare on costs to the Medicare program to cover a number of 

specific conditions, such as heart attacks, pneumonia, and stroke. 

However, CMS officials stated that the department has no plans to add 

estimated consumer out-of-pocket costs related to specific medical 

procedures to any of its transparency tools.

Based on the characteristics we identified regarding how effective 

transparency tools make cost and quality information understandable to 

consumers, we found limitations in how CMS transparency tools present 

information about comparative provider performance. Specifically, the 

tools have substantial limitations in their use of clear language and 

symbols, in summarizing and organizing information to highlight patterns 

for consumers, and in enabling consumers to customize how information 

 

33The Dialysis Facility Compare tool represents a notable exception because it focuses on 

providing specific information for a single chronic condition.

34Two of the CMS transparency tools, Hospital Compare and Home Health Compare, 

provide information on patient experience of care, such as their perspectives on their 

interactions with physicians and nurses.

Cost and Quality 

Information in CMS’s 

Transparency Tools May 

Not Be Understandable to 

Consumers
 

Page 22 GAO-15-11 Health Cost and Quality Transparency

is presented. Although the CMS tools demonstrate other characteristics of 

effective transparency tools, such as explaining the purpose and value of 

the performance information reported in the tools, and providing the 

capacity to compare up to three providers side by side, the limitations we 

identified may hinder consumers’ ability to understand and use the 

information provided. (See Appendix 2 for our assessment of how CMS 

tools compared to the 15 characteristics of effective tools identified 

through our literature review and interviews with experts.)

The tools generally do not use clear language and symbols. Although 

consumers can follow links to a separate page to obtain plain-language 

explanations of quality measures, the labels for the actual results of the 

tools often use fairly technical terms. For example, one quality measure 

for heart attack patients is labeled “heart attack patients given fibrinolytic 

medication within 30 minutes of arrival.” In addition, many of the quality 

measures are reported in numerical form, most often as percentages. 

With the exception of the “five-star” ratings currently limited to Nursing 

Home Compare, none of the other CMS tools use symbols to help 

consumers interpret the meaning of the information provided.

Additionally, the CMS tools also generally do not summarize results for 

consumers—with the exception of Nursing Home Compare—or organize 

data to highlight patterns. The Nursing Home Compare tool’s five-star 

rating system provides overall assessments of performance in three 

major categories—health inspections, quality measures, and staffing—

each of which summarizes a set of individual measures in those 

categories that the tool also reports. CMS officials told us that the agency 

plans to expand this five-star rating system to its other tools by 2015.35

 

35In 

In 

addition, CMS generally has not structured the presentation of the 

information in its tools in a way that helps consumers detect patterns in 

provider performance. For example, consumers may choose up to three 

providers to compare, but must do so from lists in which the providers are 

sequenced in alphabetical order or by distance from a geographic 

GAO-12-390, we recommended that CMS use strategic planning to establish how 

four planned efforts to improve the five-star system would help achieve its goals to inform 

consumers and improve provider quality, and that CMS develop milestones and timelines 

for the efforts. In August 2013, CMS released a strategic plan that outlines how one of its 

four efforts—evaluating the usability of Nursing Home Compare—aligns with the goals.
 

Page 23 GAO-15-11 Health Cost and Quality Transparency

location, not in rank order according to provider performance.36

In addition, the CMS tools provide consumers very limited ability to 

customize what information is presented, which may not be sufficient to 

allow consumers to focus on information most relevant to them.

Therefore, 

consumers have to sort through the information themselves if they want 

to identify the top performing providers.

37

CMS, in conjunction with other HHS agencies, has made efforts to 

address limitations in the understandability of information on cost and 

quality, such as by supporting research on the topic, and conducting 

consumer testing of the CMS tools over multiple years.

Instead, 

the CMS tools typically allow consumers to filter providers for 

consideration based on their geographic location and whether the 

provider offers one or more of a particular set of services. Expanding the 

options available to consumers to customize the information presented on 

cost and quality according to their individual situations and priorities 

would allow them to reduce the amount of information that they must 

review to identify providers that meet their needs.

38

 

36Unlike the other Compare sites, according to CMS officials, Nursing Home Compare 

does provide the ability to sort providers by performance according to the star ratings on 

several summary measures covering inspections, staffing, quality measures, and the 

facility’s overall rating.

CMS is also 

developing a single web page to serve as a point of entry for all the CMS 

transparency tools and revising the tools to achieve a “unified look and 

feel”. CMS officials told us that, while they expect to continually make 

updates to the tools over time, they do not have specific plans to change 

the way that the tools display information beyond implementation of the 

five-star rating system. In particular, they do not plan to organize 

information to facilitate identification of high-performing providers or 

expand customization options for consumers. CMS officials told us that 

making changes to the tools is complex and resource-intensive, and that 

they have received feedback from providers that are strongly opposed to 

37For the purposes of our review, customizing information refers to the ability of 

consumers to select from the quality measures offered in the tool those that are relevant 

to their personal situation.

38See for example, L&M Policy Research, Quality Reporting on Medicare’s Compare 

Sites: Lessons Learned from Consumer Research, 2001-2013, a report prepared at the 

request of the Centers for Medicare & Medicaid Services, Washington, D.C.
 

Page 24 GAO-15-11 Health Cost and Quality Transparency

organizing the tools’ information in rank order according to provider 

performance.

CMS has taken limited steps in three key areas to expand information on 

cost and quality and increase transparency to consumers and others. 

Although both HHS, through its department-wide strategy, as well as 

CMS, through agency-specific plans, have clearly articulated the goal of 

enhancing cost and quality transparency, these plans lack the procedures 

and performance metrics needed to ensure that the particular needs of 

consumers will be met.

CMS has taken steps to expand cost and quality information in three key 

areas, although we identified some limitations in each step based on the 

research we reviewed and experts we interviewed.

Developing and selecting measures. CMS funds and directs, primarily 

through contracts, the development of new cost and quality measures 

that can then be reported in both CMS and state and private sector 

transparency tools. It also selects among available measures for reporting 

information in its own tools. Through these two activities CMS influences 

the extent to which relevant cost and quality information are made 

available to consumers.

Although CMS’s transparency tools are intended to provide consumers 

with the cost and quality information that they need to assess their health 

care options, the tools also serve a different role for providers. According 

to research conducted to support CMS public reporting efforts, CMS uses 

the tools to create an incentive for providers to improve the quality of care 

they deliver. By publically reporting how providers compare to their peers, 

CMS intends providers to be able to identify shortcomings in their 

performance that can be improved. However, research we reviewed 

demonstrates that information used to motivate providers can be different 

from information needed by consumers. Consequently, researchers who 

CMS Has Taken 

Limited Steps to 

Expand Access to 

Cost and Quality 

Information, but Has 

Not Established 

Procedures to Ensure 

This Information 

Meets Consumer 

Needs

CMS Has Taken Limited 

Steps in Three Key Areas 

to Expand Cost and 

Quality Information 

Available to Consumers 

and Others
 

Page 25 GAO-15-11 Health Cost and Quality Transparency

have assessed CMS’s tools have raised the question of whether CMS 

can ensure that tools intended to influence provider quality can also 

provide consumers with the cost and quality information they need.

According to consumer transparency experts we interviewed and 

research we reviewed, CMS’s processes for measure development and 

selection do not adequately address consumer needs. For example, 

according to a summary of CMS-sponsored consumer testing conducted 

between 2001 and 2013 for the development and support of the CMS 

Compare websites, consumer involvement generally comes late in the 

measure development process. Moreover, CMS’s consumer testing has 

focused on assessing the ability of consumers to interpret measures 

developed for use by clinicians, rather than to develop or select measures 

that specifically address consumer needs.39

CMS officials report several ways that they have recently begun to 

incorporate consumer input into the agency’s measure development and 

selection process, although it is not yet clear if these efforts will ensure 

that measures are focused on consumers. For example:

• In 2013, CMS updated its requirements for measure development 

contracts to address the needs of consumers. Specifically, contractors 

are required to include patient or caregiver participation on technical 

expert panels, which are a part of the measure development 

process.40

 

39See L&M Policy Research, Quality Reporting on Medicare Compare Sites: Lessons 

Learned from Consumer Research, 2001-2013, a report prepared at the request of the 

Centers for Medicare & Medicaid Services, Washington, D.C.

According to one CMS official, the requirements do not 

specifically dictate how contractors must involve these consumers, but 

rather asks contractors to consider meaningful ways to include them. 

They also do not specify how contractors will be evaluated on their 

ability to meet consumer needs or specifically define expected results. 

According to the CMS official, only two measure contracts have taken 

effect since the requirements were updated, so it will take time to 

40The requirements also state that when developing measures, contractors should 

consider their ability to be useful to intended audiences, including consumers, for the 

purpose of public reporting and transparency. See Department of Health & Human 

Services, Centers for Medicare & Medicaid Services, Measure and Instrument 

Development and Support Indefinite Delivery, Indefinite Quantity, Umbrella Statement of 

Work, Washington, D.C., 2013.
 

Page 26 GAO-15-11 Health Cost and Quality Transparency

determine the extent to which they may improve the development of 

cost and quality measures of particular relevance to consumers.

• CMS also has begun to solicit consumer input in the selection of 

measures included in its transparency tools. In 2011, HHS convened 

the Measure Applications Partnership (MAP), a consensus-based 

group that includes representatives of consumer as well as provider 

and other stakeholder organizations, to furnish CMS with input on the 

selection of measures to include in its transparency tools and 

payment programs. Although the inclusion of consumer organizations 

in the MAP gives consumers an opportunity to influence measure 

selection, both a CMS official and a MAP member said that the 

concerns of provider organizations, which command greater technical 

expertise on measurement methodology and therefore are better 

prepared to advocate for their own selection preferences, tend to take 

priority.

Aligning cost and quality measurement. CMS, along with several other 

HHS agencies, collaborates with multiple consumer organizations, state 

and private sector entities that also collect and report cost and quality 

data, to work toward agreement on an aligned core set of cost and quality 

measures.41 According to HHS’s National Quality Strategy, providers are 

often asked to submit quality and other information to multiple payers, 

including Medicare, and having a consistent, or aligned, set of measures 

can help facilitate the collection of this information. Transparency experts 

have indicated that such alignment could not only reduce the providers’ 

reporting burden, but it could also lead to more consistent information on 

provider performance.42

 

41In October 2012, CMS and AHRQ began meeting with a group of large employers who 

purchase health care services and other health care purchaser representatives through 

the Buying Value initiative, to reach agreement on a common set of measures to be 

reported and used by health plans and others. In addition, according to a CMS official, 

CMS is participating in discussions with America’s Health Insurance Plans, a national 

trade association representing the health insurance industry, to similarly identify a set of 

common measures.

However, according to a CMS official and an 

42According to a summary report based on discussions among 125 participants in the 

2011 AHRQ Summit on Public Reporting for Consumers, experts who support greater 

measure alignment cite the benefits of reduced provider burdens, efficiencies of scale, 

comparability of measures, more uniform deployment of evidence-based report designs 

and practices which lead to a more credible reporting enterprise that produces ratings 

useful to consumers. See P. Hussey, H. Luft, and P. McNamara, “Public Reporting of 

Provider Performance at a Crossroads in the United States: Summary of Current Barriers 

and Recommendations on How to Move Forward,” Medical Care Research and Review, 

vol. 71, no. 5 suppl., October 2014, 5S-16S, e-published May 27, 2014.
 

Page 27 GAO-15-11 Health Cost and Quality Transparency

expert involved in the alignment efforts, the extent to which this promise is 

fulfilled remains to be seen, both in terms of the number and breadth of 

the measures for which an agreed alignment is reached, as well as how 

effectively the agreement ensures uniform application of the aligned 

measures. In addition, according to experts we interviewed, the extent to 

which this movement toward measure alignment benefits consumers will 

depend on the extent to which the agreed-upon set of measures are 

relevant to consumer concerns.

Releases of claims data. In addition to the quality data published 

through the Compare sites, CMS has begun to publicly report information 

on provider costs based on Medicare fee-for-service claims data.43

CMS has released more detailed patient-level claims data to CMSapproved qualified entities (QE). Established under PPACA, QEs are 

organizations that are given access to Medicare claims data to enable 

evaluation of the performance of providers and suppliers.

Specifically, beginning in 2013, CMS began releasing large data sets 

containing partially aggregated payment data on individual physicians and 

hospitals. According to experts we interviewed, the release of these data 

help to promote the concept of transparency. However, these experts 

noted that the lack of patient-level information in the files prevents even 

rudimentary risk adjustment—adjustments based on differences in 

patients—which precludes using these data to supply cost and claimsdata-based quality information for state and private sector transparency 

tools.

44

 

43CMS has shared Medicare fee-for-service provider claims data through other programs 

in the past, such as through the Research Data Assistance Center (ResDAC), a free 

service provided by CMS to assist academic and non-profit researchers interested in 

using Medicare and Medicaid data for research. Data obtained through ResDAC is limited 

to use for pre-determined research applications, and according to CMS officials, cannot be 

used more generally for public reporting.

According to 

CMS officials, by having access to more detailed patient-level data, 

QEs—selected based in part on their expertise in analyzing complex 

claims data—could help make available to consumers information on 

quality measures that use claims data, such as mortality and 

readmissions for specific conditions or procedures. HHS has begun to 

44Use of the Medicare data provided to QEs is limited to the purposes specified in section 

10332 of PPACA and to uses in accordance with the Privacy Act of 1974. QEs also must 

have an Information Exchange Agreement or contract with CMS.
 

Page 28 GAO-15-11 Health Cost and Quality Transparency

share these data with 13 organizations that have been approved to 

become QEs, according to CMS officials, but the organizations have only 

just begun to publicly share new cost or quality information based on 

these data.45 Therefore, it is too early to tell to what extent the QE 

program will expand the cost and quality information available to 

consumers.

Although HHS has included transparency as part of its strategic plan, and 

set a specific goal to report on measures that meet consumer needs, it 

has not established specific procedures or performance metrics to ensure 

that the cost and quality information that it collects and publicly reports 

accomplish this goal. A key part of HHS’s strategic plan to ensure optimal 

health care is to give the public the means to make more informed health 

care choices by improving transparency and providing public access to 

HHS’s data on provider performance, among other things.46 Furthermore, 

HHS has set a specific goal to report on measures that are important to 

consumers and other key audiences in ways that can be easily 

understood and readily acted upon.47

However, despite this goal, neither HHS, through its department-wide 

strategy, nor CMS, through its agency-specific strategic plans, have 

established specific procedures or performance metrics to clarify how 

they will ensure that the particular needs of consumers are met, in the 

midst of competing demands from providers. In particular, they have not 

specified how the particular needs of consumers, as distinct from those of 

providers, should be determined, and how to assess progress in 

addressing those needs:

 

45On August 26, 2014, the Oregon Health Care Quality Corporation released the first 

public reports on provider quality of care that incorporated Medicare claims data obtained 

through the QE program.

46See Department of Health and Human Services, “Strategic Goal 4, Objective C: Use 

HHS Data for the health and well being of the American people,” HHS Strategic Plan 

2010-2015, accessed February 24, 2014, 

http://www.hhs.gov/secretary/about/priorities/strategicplan2010-2015.pdf.

47Department of Health and Human Services, Prioritizing the Public in HHS Public 

Reporting: Using and Improving Information on Quality, Cost, and Coverage to Support 

Health Care Improvement (Washington, D.C.: December 2013).

CMS Has Not Established 

Procedures to Ensure the 

Cost and Quality 

Information It Collects and 

Reports Meets Consumer 

Needs
 

Page 29 GAO-15-11 Health Cost and Quality Transparency

• In an analysis conducted for HHS of its public reporting efforts, 

researchers proposed that HHS develop and adopt processes and 

criteria to select measures for reporting to consumers as well as for 

reporting to more specialized audiences.48 HHS subsequently 

released a framework to guide its public reporting activities in 

December 2013. Although the framework acknowledged gaps in 

available measures relevant to consumers and made the task of 

addressing these gaps a priority, it did not outline a process for 

integrating consumer needs into developing or selecting measures, 

nor did it define criteria HHS planned to use in selecting measures for 

reporting to consumers.49

• In November 2012, CMS also produced a draft strategic plan that 

outlines the agency’s public reporting strategy. One of three goals 

contained in the plan is to better meet consumer and other audience 

needs, but the plan does not contain any procedures to address how 

consumer needs should be determined or metrics to assess whether 

identified needs are met. According to CMS officials, the draft plan is 

under ongoing review by CMS officials and has not been finalized.

• Similarly, CMS’s policies with respect to collaborating with other 

entities to align cost and quality measures and its programs to publicly 

report Medicare cost information, through its own public releases and 

via QEs, have not included procedures or performance metrics that 

address the particular needs of consumers.

Establishing procedures to implement goals and performance metrics to 

monitor progress is important for accomplishing agency goals, according 

to our guidance to federal agencies on effectively implementing 

GPRAMA, which describes leading practices for how federal agencies 

should assess their performance.50

 

48Department of Health and Human Services, Office of the Assistant Secretary for 

Planning and Evaluation, Public Reporting of Health Quality and Efficiency Data under the 

U.S. Department of Health and Human Services (Washington, D.C.: Jan. 31, 2013), 70.

 According to the guidance, 

procedures and metrics are particularly important when a government 

agency must respond to multiple priorities and competing demands, such 

49Department of Health and Human Services, Prioritizing the Public in HHS Public 

Reporting: Using and Improving Information on Quality, Cost, and Coverage to Support 

Health Care Improvement (Washington, D.C.: December 2013).

50GAO, Executive Guide: Effectively Implementing the Government Performance and 

Results Act, GAO/GGD-96-118 (Washington, D.C.: June 1996).
 

Page 30 GAO-15-11 Health Cost and Quality Transparency

as CMS faces in meeting the demands of both consumers and providers 

in the development and selection of cost and quality measures and in the 

reporting of information through its transparency tools. Without such 

specific procedures and performance metrics, CMS cannot ensure that its 

efforts to make cost and quality information publicly available will meet 

consumer needs or help them to make meaningful distinctions among 

providers.

Health care costs vary widely without being related to quality. For some 

services, the differences in consumers’ out-of-pocket costs between 

providers can be thousands of dollars. However, as transparent 

information on both cost and quality is difficult for consumers to obtain—

either through transparency tools or calling providers directly—consumers 

often do not realize they could obtain better value in their health care 

decisions, for example by utilizing high-quality, low-cost providers. With 

consumers paying greater shares of their health care costs, it is 

particularly important that they have access to relevant and 

understandable cost and quality information to make value-based 

decisions about services that can be planned in advance, as well as to 

better anticipate and plan for their expenses.

Through HHS’s National Quality Strategy, which builds on HHS’s 

strategic plan, the department has made health care cost and quality 

transparency a priority for itself and its component agencies, including 

CMS. CMS’s transparency tools—five Compare websites—are one way 

the agency has provided cost and quality information to consumers. 

However, CMS addresses multiple strategic aims with these tools, 

including both informing consumers and incentivizing providers to 

improve the quality of their care. Partly as a result, the tools exhibit critical 

weaknesses when assessed in terms of the characteristics that would 

make them most effective for consumers. For example, the tools lack 

information on topics of considerable relevance to consumers, such as 

patient-reported outcome measures and patient out-of-pocket costs. 

Additionally, the tools do not organize cost and quality information in a 

way that may enable consumers to readily understand and compare 

provider performance, or customize how the information is presented to 

enable consumers to identify the best providers for aspects of care that 

they may find most relevant. While providing relevant and understandable 

information on cost and the ability to compare it with provider quality is 

inherently complex, consumers need this information to determine if 

different providers or settings provide a greater overall value for care they 

Conclusions
 

Page 31 GAO-15-11 Health Cost and Quality Transparency

expect to receive and to determine what their health care expenses will 

be in advance.

Although HHS has set goals to promote transparency for consumers, 

and, largely through CMS, has taken steps to expand available cost and 

quality information, CMS has not established procedures or performance 

metrics to ensure that this information is relevant and understandable to 

consumers. Furthermore, each of the steps CMS has taken has 

limitations that may prevent consumer needs from being met. In 

particular, CMS’s process for developing and selecting cost and quality 

measures for its tools has been heavily influenced by the concerns of 

providers rather than consumers, which helps to account for the relative 

lack of cost and quality information in CMS’s tools that consumers would 

find relevant. Especially in such situations where agencies are responding 

to multiple priorities, leading practices for strategic planning call for 

establishing specific procedures and performance metrics to ensure that 

each of those priorities receives due attention. Until CMS establishes 

such procedures and performance metrics with respect to implementing 

their goals for promoting health care transparency, the agency is likely to 

continue to have limited effectiveness in conveying relevant and

understandable information on cost and quality to consumers.

To improve consumers’ access to relevant and understandable 

information on the cost and quality of health care services, we 

recommend that the Secretary of HHS direct the Administrator of CMS to 

take four actions:

1. Include in the CMS Compare websites, to the extent feasible, 

estimated out-of-pocket costs for Medicare beneficiaries for common 

treatments that can be planned in advance;

2. Organize cost and quality information in the CMS Compare websites 

to facilitate consumer identification of the highest-performing 

providers, such as by listing providers in order based on their 

performance;

3. Include in the CMS Compare websites the capability for consumers to 

customize the information presented, to better focus on information 

relevant to them; and

4. Develop specific procedures and performance metrics to ensure that 

CMS’s efforts to promote the development and use of its own and 

others’ transparency tools adequately address the needs of 

consumers.

Recommendations for 

Executive Action
 

Page 32 GAO-15-11 Health Cost and Quality Transparency

We provided a draft of this report to HHS for review, and HHS provided 

written comments, which are reprinted in appendix III. In its comments, 

HHS concurred with each of our recommendations and noted a number 

of activities being done to improve transparency of cost and quality 

information for consumers. HHS stated its intention to ensure that its 

transparency tools, the Compare websites, fully address consumer 

priorities. For example, HHS stated that it was committed, to the extent 

feasible, to providing estimated out-of-pocket costs to Medicare 

beneficiaries for common procedures that can be planned in advance. 

However, HHS noted that, as mentioned in our draft report, there are 

challenges to collecting all of the relevant cost information. Similarly, HHS 

agreed to organize the cost and quality information provided in its 

Compare websites to facilitate consumer identification of the highest 

performing providers, such as by listing providers in order based on

ratings of the quality of their performance. HHS noted its plans to expand 

the use of star ratings, including the ability to sort and filter provider 

listings based on those ratings. HHS also agreed with our 

recommendation to enable consumers to customize the information 

presented to them on the Compare websites. Finally, HHS agreed to 

develop specific procedures and performance metrics to ensure that its 

transparency tools adequately address the needs of consumers. Although

HHS stated that it had already developed many internal procedures and 

performance metrics, it did not identify or describe them in its comments, 

nor did we find any such procedures or performance metrics in the 

planning documents provided to us by HHS during the course of our 

work. HHS also provided us with technical comments, which we 

incorporated as appropriate. 

As agreed with your office, unless you publicly announce the contents of 

this report earlier, we plan no further distribution until 30 days from the 

report date. At that time, we will send copies of this report to the 

Secretary of the Department of Health and Human Services, the 

Administrator of the Centers for Medicare & Medicaid Services, and other 

interested parties. In addition, the report will be available at no charge on 

GAO’s website at http://www.gao.gov. 

Agency Comments 

and Our Evaluation
 

Page 33 GAO-15-11 Health Cost and Quality Transparency

If you or your staff have any questions about this report, please contact 

me at (202) 512-7114 or at kohnl@gao.gov. Contact points for our Office 

of Congressional Relations and Office of Public Affairs can be found on 

the last page of this report. Other major contributors to this report are 

listed in appendix IV. 

Linda T. Kohn

Director, Health Care
Page 34 GAO-15-11 Health Cost and Quality Transparency

List of Requesters

The Honorable Tom Coburn, M.D. 

Ranking Member

Committee on Homeland Security and Governmental Affairs

United States Senate

The Honorable Amy Klobuchar

United States Senate

The Honorable Jeanne Shaheen

United States Senate

The Honorable Pat Toomey

United States Senate
Appendix I: Examples of Variation in Cost and 

Quality Information Available to Consumers for 

Selected Procedures

Page 35 GAO-15-11 Health Cost and Quality Transparency

We obtained examples of cost and quality information that were available 

to certain consumers in specific circumstances by obtaining information 

from selected transparency tools and by contacting providers, acting in 

the role of a consumer, to ask for cost and quality information. The 

examples we obtained from two selected transparency tools (a health 

insurer and Castlight) and from contacting selected providers in two 

health care markets by telephone illustrate the variation in cost and 

quality information that are available to consumers in specific situations, 

such as those who are members of a particular health plan or uninsured. 

Specifically, the examples showed variation in cost, but often did not 

include enough information on quality for it to be considered alongside

cost (see tables 4-7 below).

Appendix I: Examples of Variation in Cost 

and Quality Information Available to 

Consumers for Selected Procedures
Appendix I: Examples of Variation in Cost and 

Quality Information Available to Consumers for 

Selected Procedures

Page 36 GAO-15-11 Health Cost and Quality Transparency

Table 4: Cost and Quality of a Laparoscopic Gallbladder Surgery at Ambulatory Surgical Centers (ASCs) and Hospital 

Outpatient Departments in the Denver, Colorado-Area

Information related to cost Information related to quality

Provider Settinga

Estimated total 

actual costa

(in dollars)

Estimated 

out-of-pocket costb

(in dollars)

Leapfrog Hospital 

Safety Scorec

HCAHPS 

overall ratingd

(percent high)

Complications 

indexe

(100=average)

Provider 1 ASC $3,281 $3,281 N/A N/A N/A

Provider 2 ASC 4,205 4,041 N/A N/A N/A

Provider 3 ASC 4,575 4,115 N/A N/A N/A

Provider 4 ASC 4,716 4,143 N/A N/A N/A

Provider 5 ASC 18,770 6,954 N/A N/A N/A

Provider 6 Hospital outpatient 18,185 6,837 A 68 55

Provider 7 Hospital outpatient 20,260 7,252 A 81 N/A

Provider 8 Hospital outpatient 29,420 9,084 B 69 123

Provider 9 Hospital outpatient 30,079 9,216 B 63 104

Provider 10 Hospital outpatient 32,788 9,758 C 69 153

Provider 11 Hospital outpatient 34,515 10,103 A 72 69

Provider 12 Hospital outpatient 40,626 11,325 A 73 N/A

Source: Health Insurer. | GAO-15-11

a

The estimated total actual cost is the health insurer’s negotiated rate with the provider for an 

uncomplicated laparoscopic gallbladder surgery performed on an outpatient basis as of July 2014. It 

includes typical associated imaging procedures done in conjunction with the procedure, anesthesia 

for 135 minutes, as well as typical associated facility, surgeon, and other professional charges.

b

The estimated out-of-pocket costs are what a member of the health insurer would pay of the 

estimated total actual cost with a health plan of a $4,000 deductible, and 20 percent coinsurance. The 

estimated out-of-pocket costs assume that the member has not yet paid any money towards the 

deductible.

c

The Leapfrog Hospital Safety Score is an A, B, C, D, or F letter grade reflecting how safe hospitals 

are for patients. The grades used in this score are derived from expert analysis of publically available 

data using national evidence-based measures of patient safety. The Leapfrog Hospital Safety Score 

program grades hospitals on their overall performance in keeping patients safe from preventable 

harm and medical errors.

d

The Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey is 

published by the Centers for Medicare & Medicaid Services on Hospital Compare. Patients are 

surveyed to provide information about their hospital experiences. The percentages reflected in this 

column represent patients who gave a “high” score, a rating of 9 or 10 (out of a 10 point scale) to the 

question, “How do patients rate the hospital overall?”

e

The Complications Index is a point score based on the frequency and severity of each complication 

at each hospital. The area average is 100. Therefore, a Complications Index of less than 100 

indicates better performance on quality than the average of the hospitals in the area. A Complications 

Index of more than 100 indicates worse performance on quality than the average of the hospitals in 

the area.
Appendix I: Examples of Variation in Cost and 

Quality Information Available to Consumers for 

Selected Procedures

Page 37 GAO-15-11 Health Cost and Quality Transparency

Table 5: Cost and Quality of MRI of the Lower Back at Stand-Alone Imaging Centers and Acute Care Hospitals in the 

Indianapolis, Indiana Area

Information related to cost Information related to quality

Provider Setting 

Estimated total 

actual costa

(in dollars)

Estimated 

out-of-pocket costb

(in dollars)

MRI of the 

lower backc

HCAHPS Hospital 

Patient Experience 

Measured

Provider 1 Stand-alone imaging center $450 $450 N/A N/A

Provider 2 Stand-alone imaging center 541 541 N/A N/A

Provider 3 Stand-alone imaging center 575 575 N/A N/A

Provider 4 Stand-alone imaging center 1,033 – 3,162 1,033 – 2,233 N/A N/A

Provider 5 Acute care hospital 277 – 5,184 277 – 2,637 Average 3 star

Provider 6 Acute care hospital 910 – 3,905 910 – 2,381 Above average 5 star

Provider 7 Acute care hospital 1,579 – 2,571 1,579 – 2,115 Below average 5 star

Provider 8 Acute care hospital 2,030 2,006 Average 5 star

Provider 9 Acute care hospital 2,523 2,105 Average 4 star

Provider 10 Acute care hospital 3,756 2,351 Below average 1 star

Source: Castlight Health. | GAO-15-11

a

The estimated total actual cost is what a plan that utilizes Castlight negotiated with the provider for a 

standard magnetic resonance imaging (MRI) of the lower back performed at stand-alone imaging 

centers and acute care hospitals as of May 2014. Services include the MRI, radiologist’s fee, facility 

fee, and reading and interpretation of the MRI.

b

The estimated out-of-pocket costs are what a member of the plan that utilizes Castlight would pay of 

the estimated total actual cost. The estimated out-of-pocket cost is based on a health plan with a 

$2,000 deductible, $5,000 out-of-pocket maximum, 100 percent coverage for preventive services, and 

80 percent coverage in network after application of the deductible.

c

This quality measure is from Hospital Compare and evaluates the appropriateness of doing an MRI.

d

The Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey is 

published by the Centers for Medicare & Medicaid Services on Hospital Compare. Patients are 

surveyed to provide information about their hospital experiences. This score represents the patient 

experience rating for a hospital based on the percentage of patients who said that they would 

“definitely” recommend that hospital to family and friends.
Appendix I: Examples of Variation in Cost and 

Quality Information Available to Consumers for 

Selected Procedures

Page 38 GAO-15-11 Health Cost and Quality Transparency

Table 6: Cost and Quality of Maternity Care at Acute Care Hospitals in the Boston, Massachusetts Area

Information related to cost Information related to quality

Provider Setting 

Estimated total 

actual costa

(in dollars)

Estimated 

out of pocket costb

(in dollars)

Patient experience 

measure for 

physicianc

 

(number of ratings)

Leapfrog delivery 

care rating for 

facilityd HCAHPS 

for facilitye

Provider 1 Acute care hospital $3,969 2,394 5 star (1) Average 1 star

Provider 2 Acute care hospital 5,412 2,682 5 star (29) Average 3 star

Provider 3 Acute care hospital 5,900 2,780 5 star (8) Below average 5 star

Provider 4 Acute care hospital 6,834 2,967 5 star (27) Above average 3 star

Provider 5 Acute care hospital 8,744 3,349 5 star (20) Average 5 star

Provider 6 Acute care hospital 9,026 3,405 5 star (2) Above average 3 star

Provider 7 Acute care hospital 20,776 5,000 5 star (17) Above average 5 star

Provider 8 Acute care hospital 21,554 5,000 Not rated Above average 5 star

Source: Castlight Health. | GAO-15-11

a

The estimated total actual cost is what a plan that utilizes Castlight negotiated with the provider for 

maternity care performed at acute care hospitals as of May 2014. The cost includes prenatal office 

visits, required ultrasounds, delivery, and one visit after delivery. It does not include tests, elective 

ultrasounds, imaging, medicine, and anesthesia.

b

The estimated out-of-pocket costs are what a member of the plan that utilizes Castlight would pay of 

the estimated total actual cost. The estimated out-of-pocket cost is based on a health plan with a 

$2,000 deductible, $5,000 out-of-pocket maximum, 100 percent coverage for preventive services, and 

80 percent coverage in network after application of the deductible.

c

The “patient experience measure for physician” rating is a summary measure of how patients view 

their doctor. The data is collected from Castlight users and aggregated with patient experience 

information from external websites, such as Angie’s List.

d

The “Leapfrog delivery care rating” for the facility represents the percent of pregnancies for which the 

baby is delivered at the appropriate developmental age, using an appropriate delivery method, and 

recommended processes of care.

e

The Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey is 

published by the Centers for Medicare & Medicaid Services on Hospital Compare. Patients are 

surveyed to provide information about their hospital experiences. This score represents a patient 

experience rating for each hospital based on the percentage of patients who said that they would 

“definitely” recommend that hospital to family and friends.
Appendix I: Examples of Variation in Cost and 

Quality Information Available to Consumers for 

Selected Procedures

Page 39 GAO-15-11 Health Cost and Quality Transparency

Table 7: Cost and Quality Information Obtained for an Inguinal Hernia Repair or Diagnostic Colonoscopy at Ambulatory 

Surgical Centers (ASC) and Hospital Outpatient Departments in Two Health Care Markets Based on Telephoning Selected 

Providers

Procedure Setting 

Information provided on costa

(in dollars) Information provided on quality

Minneapolis, Minnesota

Inguinal Hernia Repair ASC Estimated total cost – $2,970

(includes facility feeb and meshc

). 

None

Inguinal Hernia Repair ASC Estimated total cost – $3,100

(includes facility, nurse anesthetist, 

surgeon, and anesthesiologist fees). 

Representative said that infection 

rates of facility are less than 

1 percent. Representative anecdotally 

recalled one surgery complication 

related to blood thinners, but no other 

complications.

Inguinal Hernia Repair Hospital outpatient 

department

Estimated total cost – $6,750 – $9,625

(includes facility, physician, and 

anesthesia fees). 

None

Colonoscopy ASC Estimated total cost – $1,733

(includes facility and professional feesd

). 

Representative referred to provider’s 

website which has patient satisfaction 

scores and quality measures, such as 

the completion rate, adenoma 

detection rates, and withdrawal times, 

with benchmarks to understand the 

scores provided.

Colonoscopy ASC Estimated total cost – $3,700 – $5,500

(includes facility, physician, anesthesia, 

and professional fees). 

None

Colonoscopy Hospital outpatient 

department

Estimated total cost – $1,825

(includes facility fee). 

Representative referred to provider’s 

website which reported that the 

hospital scored in the top 3 percent 

nationwide in surgical care 

improvement process of care 

measures in 2009.

Portland, Oregon

Inguinal Hernia Repair ASC Estimated total cost – $3,139 – 3,739

(includes facility and anesthesiologist fees 

and mesh).

None

Inguinal Hernia Repair Hospital outpatient Estimated total cost – $6,000 – $12,000

(includes facility fee). 

None

Inguinal Hernia Repair Hospital outpatient Estimated total cost – $6,400

(includes facility fee). 

None

Colonoscopy ASC Estimated total cost – $800 (adjusted 

pricing for uninsured)

(includes facility, anesthesia, and 

physician fees). 

None
Appendix I: Examples of Variation in Cost and 

Quality Information Available to Consumers for 

Selected Procedures

Page 40 GAO-15-11 Health Cost and Quality Transparency

Procedure Setting 

Information provided on costa

(in dollars) Information provided on quality

Colonoscopy ASC Estimated total cost – $2,074.74

(includes facility fee). 

None

Colonoscopy Hospital outpatient Estimated total cost – $3,500 – $4,000

(includes facility and anesthesia fees). 

None

Colonoscopy Hospital outpatient Estimated total cost – $5,000

(includes facility and physician fees). 

Representative referred to provider’s 

website which has patient experience 

metrics.

Source: GAO. | GAO-15-11

Notes: These results represent the 13 ASCs or hospital outpatient departments that were able to 

provide us with any information about the costs of getting either an inguinal hernia repair or diagnostic 

colonoscopy for an uninsured patient at their facility. Overall we contacted 24 providers to obtain such 

information. 11 providers did not provide cost information. Representatives from these providers did 

not provide cost estimates because they needed additional information, such as the specific 

procedure code or personal information about the patient (date of birth and contact information) which 

we did not provide, or were unavailable.

a

The cost information provided by selected providers represents what the providers would charge an 

uninsured patient, less any discounts for charity care for uninsured patients, if applicable.

b

Facility fees include all fees that would be charged by the facility, such as a hospital, for the 

procedure.

c

Mesh is a medical device that may be used in hernia repair surgery to provide additional support to 

weakened or damaged tissue.

d

Physician and professional fees include the fees that would be charged by the physician or other 

health care professionals involved in the procedure.
Appendix II: Assessment of CMS Transparency 

Tools

Page 41 GAO-15-11 Health Cost and Quality Transparency

We assessed the extent to which the Centers for Medicare & Medicaid 

Services’ transparency tools include the 15 characteristics of effective 

tools we identified through our literature review and interviews with 

experts.1

Table 8: Assessment of CMS Transparency Tools on Relevance and Understandability of Information for Consumers

(See Table 8.) 

Hospital 

Compare

Nursing Home 

Compare

Home Health 

Compare

Dialysis facility 

Compare

Relevance criteria

Review a broad range of services Yes Yes Yes Yes

Cover a broad range of providers Yes Yes Yes Yes

Describe key differences in clinical quality of care Limited Limited Limited Yes

Describe key differences in patient experiences Yes No Yes No

Describe key differences in costs No No No No

Describe other information related to quality, if appropriate Yes Yes Yes Yes

Provide timely Information Yes Yes Yes Yes

Describe key strengths and limitations of the data Yes Yes Yes Yes

Understandability criteria

Written in plain language with clear graphics No No Yes No

Explains purpose and value of quality performance ratings Yes Yes Yes Yes

Summarizes related information and organizes data to 

highlight patterns

No Yes No No

Enables consumers to customize information Very limited Very limited Very limited Very limited

Enables comparison of multiple providers in one view. Yes Yes Yes Yes

Enables consumer to assess cost and quality information 

together.

No No No No

Enables easy use and navigation of the tool. Yes Yes Yes Yes

Source: GAO (analysis). | GAO-15-11

 

1

Based on our initial review of the compare tools we determined that one of the five 

tools—Physician Compare—offers very limited information to consumers. Specifically the 

tool offers information such as the physician’s name, practice specialty, gender, and 

educational qualifications, as well as quality measures related to management of diabetes 

and heart disease for 66 physician groups and 141 Accountable Care Organizations. 

Therefore we focused our review on the remaining four tools.

Appendix II: Assessment of CMS 

Transparency Tools
Appendix III: Comments from the Department 

of Health and Human Services

Page 42 GAO-15-11 Health Cost and Quality Transparency

Appendix III: Comments from the 

Department of Health and Human Services 
Appendix III: Comments from the Department 

of Health and Human Services

Page 43 GAO-15-11 Health Cost and Quality Transparency
Appendix III: Comments from the Department 

of Health and Human Services

Page 44 GAO-15-11 Health Cost and Quality Transparency
Appendix III: Comments from the Department 

of Health and Human Services

Page 45 GAO-15-11 Health Cost and Quality Transparency
Appendix IV: GAO Contact and Staff 

Acknowledgments

Page 46 GAO-15-11 Health Cost and Quality Transparency

Linda T. Kohn, (202) 512-7114 or kohnl@gao.gov

In addition to the contact named above, Will Simerl, Assistant Director; 

N. Rotimi Adebonojo; Jennie Apter; Rebecca Hendrickson; 

Monica Perez-Nelson; Eric Peterson; and Elise Pressma made key 

contributions to this report.

Appendix IV: GAO Contact and Staff 

Acknowledgments

GAO Contacts

Acknowledgments

(291169)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (http://www.gao.gov). Each weekday 

afternoon, GAO posts on its website newly released reports, testimony, 

and correspondence. To have GAO e-mail you a list of newly posted 

products, go to http://www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. 

Visit GAO on the web at www.gao.gov. 

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Please Print on Recycled Paper.