Health Care Transparency: Actions Needed to Improve Cost and Quality Information for Consumers
Published: 2014-10-20T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The cost and quality of health care services can vary significantly, with high cost not necessarily indicating high quality. As consumers pay for a growing proportion of their care, they have an increased need for cost and quality information before they receive care, so they can plan and make informed decisions. Transparency tools can provide such information to consumers and others.
GAO was asked to study cost and quality information for consumers. This report examines (1) information on cost and quality available to consumers from selected transparency tools, (2) characteristics of effective transparency tools, (3) limitations, if any, in the effectiveness of CMS transparency tools, and (4) CMS efforts to expand cost and quality information available through transparency tools. GAO analyzed information from two private tools—selected because they had both cost and quality information—and CMS's five transparency tools, reviewed research to identify best practices for conveying information to consumers, interviewed CMS and HHS officials and subject matter expects, and reviewed CMS and HHS planning documents and relevant criteria for effective planning in the federal government.
section_title: What GAO Found
paragraphs: Results obtained from two selected private consumer transparency tools GAO reviewed—websites with health cost or quality information comparing different health care providers—show that some providers are paid thousands of dollars more than others for the same service in the same geographic area, regardless of the quality of such services. For example, the cost for maternity care at selected acute care hospitals in Boston, all of which rated highly on several quality indicators, ranged between $6,834 and $21,554 in July 2014.
Transparency tools are most effective if they provide information relevant to consumers and convey information in a way that consumers can readily understand. GAO identified key characteristics of effective transparency tools through a literature review and interviews with experts. The information that is most relevant to consumers relates directly to their personal circumstances, such as information on specific procedures they are considering, and allows them to make meaningful distinctions among providers based on their performance. Characteristics of such relevant information include describing key differences in quality of care and costs, particularly for what consumers are likely to pay out of pocket based on their specific circumstances. In addition, effective transparency tools must take specific steps to make the information they present understandable by consumers. For example, tools must enable consumers to discern patterns by summarizing related information and allowing consumers to customize information to focus on what is most relevant to them.
The Centers for Medicare & Medicaid Services (CMS) operates five transparency tools—Nursing Home Compare, Dialysis Facility Compare, Home Health Compare, Hospital Compare and Physician Compare—that are limited in their provision of relevant and understandable cost and quality information for consumers. In particular, GAO found that the tools lack relevant information on cost and provide limited information on key differences in quality of care, which hinders consumers' ability to make meaningful distinctions among providers based on their performance. Because none of the tools contain information on patients' out-of-pocket costs, they do not allow consumers to combine cost and quality information to assess the value of health care services or anticipate the cost of such services in advance. Additionally, GAO found substantial limitations in how the CMS tools present information, such as, in general, not using clear language and symbols, not summarizing and organizing information to highlight patterns, and not enabling consumers to customize how information is presented.
CMS, part of the Department of Health and Human Services (HHS), has taken some steps to expand access to cost and quality information for consumers, but has not established procedures or metrics to ensure the information it collects and reports meets consumer needs. Both HHS and CMS have set goals to report on measures that meet consumer needs. However, CMS's process for developing and selecting cost and quality measures for its tools has been heavily influenced by the concerns of providers rather than consumers. Without procedures or metrics focusing on consumer needs, CMS cannot ensure that these efforts will produce cost and quality information that is relevant and understandable to consumers seeking to make meaningful distinctions.
section_title: What GAO Recommends
paragraphs: GAO recommends that HHS's CMS take steps to improve the information in its transparency tools and develop procedures and metrics to ensure that tools address consumers' needs. HHS concurred with the recommendations and provided technical comments that were incorporated as appropriate.

section_title: Letter
paragraphs: The cost and quality of health care services can vary significantly, with  evidence suggesting that more expensive health care services are not  necessarily higher in quality. Providers, such as hospitals or physicians,  may be paid two or three times more than similar providers for the same  health care services, without delivering higher quality of care. At the  same time, a growing proportion of health care costs are being paid by  consumers. Consumers without health insurance are generally  responsible for paying what the provider charges, rather than discounted  rates negotiated by insurance companies, while those with insurance also  face increased costs, as use of high-deductible health plans and other  forms of cost sharing have increased. In particular, consumers whose  out-of-pocket costs are based on a percentage of the total cost of care  can be especially affected by cost variations. Consumers may also face  fees associated with care received from physicians, laboratories, or  hospitals that are outside of an insurance network and bill for their  services separately.
As consumers are asked to pay more for their care, they run the risk of  obtaining care from high-cost providers without receiving the highest  quality care. Therefore, consumers have an increased need for  transparent information on cost and quality before they receive care, so  they can make informed decisions when choosing treatments and  providers. Even when not faced with these decisions, consumers can use  such information to better anticipate and plan for expenses. Together,  information on both cost and quality enables consumers to assess the  value of health care services, for example by comparing costs within a  group of providers determined to be of high quality. Research suggests  that this transparent information is most relevant for services that can be  planned in advance, so consumers have time to consider it. We have  previously found that this transparent information on cost and quality is  difficult for consumers to obtain, and that the information that is available  is not always meaningful to consumers.
Transparency tools—websites with health cost or quality information  comparing different providers of health care services—are one way that  the Department of Health and Human Services (HHS) and other  organizations provide this information to consumers and other users.  HHS both publishes its own transparency tools and shares data on cost  and quality that other public or private entities can use in their  transparency tools. Specifically, HHS publishes transparency tools  through its Centers for Medicare & Medicaid Services (CMS), such as its  series of Compare websites for hospitals, nursing homes, and certain  other providers that participate in the Medicare program.
In light of variations in cost and quality and consumers’ increased need  for transparent information prior to receiving health care, you asked us to  study the cost and quality information available to consumers. This report  examines  1.  information on cost and quality available to consumers for selected  health care services from selected private transparency tools,  2.  characteristics of effective transparency tools,  3.  limitations, if any, in the effectiveness of CMS transparency tools in  providing cost and quality information about providers to consumers,  and  4.  CMS efforts to expand cost and quality information that could be  made available to consumers through transparency tools.
To examine information on cost and quality available to consumers for  selected health care services from selected private transparency tools,  we obtained examples of information from the transparency tools of a  health insurer and Castlight Health, a third-party vendor. We selected  these tools based on their capacity to provide consumer-relevant cost  information on services that could be planned in advance, as well as  some related information on quality of care. Within these tools, we  selected services, provider types, and geographic locations based on  each tool’s ability to provide cost and quality information for those  services, provider types, and locations. The selected services were:  laparoscopic gallbladder surgery, magnetic resonance imaging (MRI) of  the lower back, and maternity care. The selected providers were:  ambulatory surgical centers (ASC), hospital outpatient departments, and  acute care hospitals. The selected locations were: Boston  (Massachusetts), Indianapolis (Indiana), and Denver (Colorado). Both the  health insurer and Castlight provided cost information based on an  insurance plan common for their customers. We reviewed related  documentation and interviewed knowledgeable company officials, and  determined that the data were sufficiently reliable for our purposes. The  findings from these transparency tools cannot be generalized to other  tools. To determine the availability to consumers of cost and quality data  in the absence of transparency tools, we directly contacted 24 randomly  selected ASCs and hospital outpatient departments in Minneapolis  (Minnesota) and Portland (Oregon) health care markets by telephone.  We selected these locations because each of these states had cost and  quality initiatives to provide information to consumers, and each market   During the calls we  had the largest number of hospitals in those states.requested the cost of two services for an uninsured patient—an inguinal  hernia repair and a diagnostic colonoscopy, as well as requested any  quality information available for the service or facility. These two services  were selected because each could be planned in advance and is  commonly performed in an ambulatory setting. We did not assess the  accuracy of the cost and quality information provided by these selected  providers.
To examine characteristics of effective transparency tools, we reviewed  the literature on best practices for conveying cost and quality information  to consumers and interviewed subject matter experts. We searched  multiple online databases to identify relevant articles and reports  published from 2008 to 2013,work. We also identified nine researchers as experts on the basis of their  publications and other activities in the area of communicating cost and  quality information to consumers. Six of these experts provided their  perspectives in interviews. We synthesized our literature review and  interviews into a draft description of key characteristics of effective  transparency tools, and then obtained comments from the remaining  three consumer transparency experts and revised the description of the  characteristics as appropriate.   the most recent available at the time of our  To examine the limitations, if any, in the effectiveness of CMS  transparency tools in providing cost and quality information about  providers to consumers, we identified five CMS transparency tools that  provide comparative cost or quality information about providers: Hospital  Compare, Nursing Home Compare, Dialysis Facility Compare, Home  Health Compare, and Physician Compare. We then reviewed each tool to  determine the extent to which it had characteristics of effective  transparency tools, using the list we identified in our second objective. We  interviewed CMS officials responsible for planning, developing and testing  the HHS transparency tools, and reviewed HHS documentation on the  process for developing the tools and HHS’s future plans for improving the  tools.
To examine CMS efforts to expand cost and quality information available  to consumers through transparency tools, we interviewed CMS and HHS  officials with oversight and involvement in public reporting and CMS  transparency initiatives, including officials who develop and maintain  CMS’s Compare websites. We also reviewed HHS strategic planning  documents related to health care quality and public reporting, and  reviewed relevant criteria from the Government Performance and Results  Act Modernization Act of 2010 (GPRAMA) as incorporated in our  guidance on assessing performance. We then interviewed consumer  transparency experts, as noted above, and officials representing private-  and state-sponsored organizations that use CMS data in their  transparency tools to identify CMS and other HHS agency actions that  could help to expand information available to consumers. Because our  study focused on CMS’s efforts to promote transparency for consumers,  we did not examine other factors affecting access to cost and quality  information, such as barriers to obtaining information on payment rates  negotiated between individual providers and private sector payers.
We conducted this performance audit from December 2013 to October  2014 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: Transparency tools are a way to make information on health care cost  and quality transparent to consumers and others, and are a key part of  HHS’s strategy to improve the quality and affordability of health care.
subsections: section_title: Information on Health Care Cost and Quality
paragraphs: There are multiple ways to assess the cost of health care. For example,  cost can be measured based on the amount of money providers set as a  “charge” for various individual services, but these charges typically do not  represent the actual amounts paid by insurers or consumers. The cost  that an insured consumer is responsible for paying to receive services is  called an out-of-pocket cost, which depends on the consumer’s individual  provider choices and insurance benefit design. In addition, any given  episode of care usually involves payments to multiple providers (e.g.,  surgeons, anesthesiologists, pathologists, etc.), facility fees, and other  ancillary fees, and any given cost figure may or may not represent the  total costs for an episode of care by including all of these expenses.
For a description of the quality measure categories, see Agency for Healthcare  Research and Quality, National Quality Measures Clearinghouse,  http://www.qualitymeasures.ahrq.gov/tutorial/varieties.aspx, accessed August 5, 2014.  The National Quality Measures Clearinghouse also includes other types of clinical quality  measures, such as access measures, which measure a patient’s ability to attain timely  and appropriate care.
section_title: HHS’s National Quality Strategy
paragraphs: In March 2011, HHS first published its National Strategy for Quality  Improvement in Health Care (the National Quality Strategy), as required  by the Patient Protection and Affordable Care Act (PPACA). The  National Quality Strategy builds on priorities HHS previously identified in  its strategic plan for fiscal years 2010-2015, which emphasize the need  for transparent information to give consumers the means to make more  informed choices about their health care. Two of the National Quality  Strategy’s overarching goals—better care and affordable care—relate to  health care cost and quality transparency. According to the strategy, to  achieve better care, patients must be given access to understandable  information and decision support tools that help them manage their health  and navigate the health care delivery system. To achieve affordable care,  systems must be created to make health care cost and quality more  transparent to consumers and providers, so they can make better choices  and decisions. The strategy also focuses on coordinating and aligning  efforts across the public and private sectors, for example by establishing  an aligned set of common cost and quality measures by which to assess  how well providers and programs support effective care.
section_title: HHS’s Role in Establishing Transparency Tools
paragraphs: As part of its efforts to foster greater transparency of information, HHS  developed transparency tools for consumers focused on quality, some of  which predate PPACA and the development of the National Quality  Strategy. Between 1998 and 2010, HHS—through CMS—launched five  Compare websites to publicly report certain information, including quality  information, based on data submitted by different types of providers  participating in the Medicare program (see table 2). For some, but not all,  of these provider types, HHS is required to publicly report certain  information about provider performance.
HHS—primarily through AHRQ—also plays a role in research and  dissemination of information on consumer-focused public reporting.  AHRQ has supported the research and publication of numerous papers  that lay out best practices for public reporting of cost and quality  information to consumers and has widely shared the expertise contained  in that research through various forums, such as conferences and  webinars that support wider transparency efforts.
section_title: Other Entities’ Transparency Tools
paragraphs: A variety of private sector entities have developed transparency tools that  provide cost and quality information to consumers, including health  insurance carriers, third-party vendors, and regional collaboratives. These  private-sector tools allow consumers to obtain personalized cost  estimates, compare different providers, or estimate their out-of-pocket  costs before receiving a service. For example, many health plans offer  tools that provide cost estimates to enrolled members. Researchers  calculated that approximately 70 percent of the privately insured  population had access to cost transparency tools in 2013. In addition to  tools provided directly by health plans, employers may contract with third- party vendors to provide transparency tools for their employees. Health  plans and third-party vendors frequently offer some quality information  along with cost estimates to their enrolled members. Often this  information is derived from data reported by CMS on its Compare sites  and may be combined with additional quality information obtained from a  variety of sources that collect and report data on provider quality.
Several states also have developed transparency tools.plans and third-party vendors, state tools can be accessed by the general  public and are not restricted to members who pay or enroll for services,  giving the uninsured access to cost and quality information. Like the  private-sector tools, state tools generally draw on quality information from  CMS in combination with information from other sources.
section_title: Information from Two Selected Consumer Transparency Tools Consistently Demonstrated Substantial Cost Variation Regardless of Quality for Certain Services
paragraphs: The two selected consumer transparency tools we reviewed show that  some providers are paid thousands of dollars more than others for the  same service in the same geographic area, regardless of the quality of  such services. Specifically, we found this variation in cost to be present  across multiple services, settings, and geographic areas in the  information provided by the transparency tools of two different entities (a  health insurer and Castlight) that we reviewed. For example, the health  insurer reported that in the Denver area, the estimated total cost of a  laparoscopic gallbladder surgery in selected ASCs ranged between  $3,281 and $18,770 (consumers would pay between $3,281 and $6,954  in estimated out-of-pocket costs) in July 2014. Meanwhile, for the same  time period, the health insurer reported that the estimated total cost for  the same service in selected hospital outpatient departments in the  Denver area ranged from $17,791 to $40,626 (consumers would pay  between $6,758 and $11,325 in estimated out-of-pocket costs).Similarly, Castlight reported, in May 2014, that for an MRI of the lower  back at selected acute care hospitals in the Indianapolis, Indiana area,  the estimated total cost ranged between $277 and $5,184 depending on  the provider (with consumers paying between $277 and $2,637 in  estimated out-of-pocket costs).variation in information we obtained from selected consumer  transparency tools.) (See app. I for more examples of the  Information from the two selected transparency tools indicates that the  observed cost variation is not tied to variations in quality, regardless of  the treatment and geographic area. For example, according to information  obtained from Castlight, the estimated total cost of maternity care at  selected acute care hospitals in the Boston area that rated more highly on  several quality indicators ranged between $6,834 and $21,554  (consumers would pay between $2,967 and $5,000 in estimated out-of- pocket costs). laparoscopic gallbladder surgery showed that a number of hospital  outpatient departments in the Denver area that rated more highly on  several quality indicators had lower total costs than other hospital  outpatient departments that rated less highly on quality. These  examples are particularly relevant as researchers have found that many  consumers assume that all providers offer good quality care, while others  have the misconception that higher-cost providers will provide higher  quality of care than lower-cost providers.
The quality indicators provided by Castlight for maternity care include the percent of  pregnancies for which the baby is delivered at the appropriate developmental age, using  an appropriate delivery method, and recommended processes of care. assistance of consumer transparency tools.hospitals we contacted to inquire about the cost of an inguinal hernia  repair and diagnostic colonoscopy for an uninsured patient in the  Portland, Oregon, and Minneapolis, Minnesota health care markets— locations selected because they have initiatives to promote  transparency—we received limited cost information from 54 percent (13)  and quality information from 29 percent (7) (see table 3).
section_title: Effective Transparency Tools Provide Cost and Quality Information That Is Relevant and Understandable to Consumers
paragraphs: Transparency tools are most effective if they both provide information  relevant to consumers and convey that information in a way that  consumers can readily understand.
subsections: section_title: Effective Transparency Tools Provide Information Relevant to Consumers
paragraphs: The information on cost and quality that consumers find relevant makes  up just a portion of the information available for inclusion in transparency  tools. For example, research shows that most consumers do not find the  information derived from many specific process-of-care measures that  clinicians have developed and used to guide quality improvement efforts  useful. Rather, consumers are most likely to respond to information that  applies to their personal circumstances, including, for example,  information on the specific procedures consumers are considering, on  providers who would be available to perform those procedures, and on  cost estimates that take account of their particular insurance coverage.  Consumers also seek information that helps them to make meaningful  distinctions among providers in terms of cost and quality. The research  we reviewed has found that consumers value quality measures that show  differences in clinical outcomes and patient experiences, and cost  measures that show differences in out-of-pocket expenses. In addition,  consumers want to know the source of cost and quality information  because this source information helps them determine their level of  confidence in that information.
Eight of the 15 characteristics of effective transparency tools we identified  address the extent to which a tool provides substantive quality and cost  information of relevance to consumers. Specifically, the research we  reviewed shows that more effective transparency tools:  1.  Review a broad range of services so that more consumers’  particular needs are included. The more services that are covered  by a transparency tool (or set of tools), the more likely it is that the tool  will have information relevant to the particular services of interest to  any given consumer. It is especially important to include services that  are predictable and non-urgent in a transparency tool, because these  services are most likely to afford consumers the opportunity to  evaluate cost and quality information before receiving the service.  2.  Cover a broad range of providers. Transparency tools that provide  information for all or most of the available providers in a given  geographic area, regardless of network status or practice setting, give  consumers more information about their full range of options. For  example, for procedures that can be conducted in either a hospital  outpatient department or ASC, it helps consumers to provide  comparable information for both settings, so that consumers can  choose from a larger number of providers that offer those procedures.  3.  Describe key differences in clinical quality of care, particularly  patient-reported outcomes. Assessments of the clinical quality of  care that have been shown to have particular relevance to consumers  are those that relate to long-term outcomes of the care experienced  by other patients. Often this is best addressed by patient-reported  outcomes, which tell consumers the eventual outcome of treatments,  as reported by previous patients of a particular provider. For example,  patients receiving hip replacements can be asked, through such  patient-reported outcomes, to rate their ability to climb stairs both  before and after their procedures, which enables assessments of the  procedures’ effects on patients’ mobility.  4.  Describe key differences in patient experiences with providers.
Another outcome that matters is patients’ assessment of their  interactions with providers. Effective transparency tools include  information on how past patients evaluated providers in terms of  dimensions such as how well nurses communicate with patients, or  the responsiveness of clinicians to patients’ needs.  5.  Describe key differences in costs, particularly patient out-of- pocket costs. The cost information of greatest relevance to  consumers is finding out what they will have to pay for a given service  before it occurs. For example, a tool that provides information on the  average cost of a selected procedure in a given geographic location  may be less relevant to consumers than one that takes into account  the specific costs for that procedure and location given that  consumer’s specific health insurance coverage.  6.  Describe other information related to quality, where appropriate.
There may be other quality indicators that could have major  significance to consumers for certain types of services. For example,  facility inspection results and staffing levels are of particular relevance  to nursing home care.  7.  Provide timely Information. More recent data are intrinsically more  relevant than data that are several years old. Because consumer  transparency tools necessarily rely on past data to assess likely cost  and quality performance in the future, some lag in collecting,  analyzing, and providing data is inevitable. Data that are no more than  two years old are generally considered timely.  8.  Describe key strengths and limitations of the data. Although the  research we reviewed shows that few consumers are inclined to delve  into the many methodological issues that concern appropriate  techniques for collecting, checking, and analyzing cost and quality  data, transparency tools can provide both summary assessments of  strengths and limitations for most consumers, and links to more  complete explanations for those wanting to pursue these issues in  greater detail. Such information, along with identification of the  organization responsible for the tool, provides consumers a basis to  judge the credibility of the cost and quality information provided.
section_title: Effective Transparency Tools Take Steps to Make Cost and Quality Information Understandable to Consumers
paragraphs: Regardless of its potential relevance to them, consumers only can  respond to information about cost and quality if they understand it.  Although cost and quality information can be inherently difficult to  understand, research suggests ways that transparency tools can make it  easier for consumers to do so. One factor that increases the difficulty of  understanding such information is the prevalence of misleading  preconceptions about the cost and quality of care. For example,  researchers find that many consumers incorrectly assume that all  hospitals and physicians provide good quality care, while others assume  that higher-cost providers will provide higher quality of care than lower- cost providers. As a result, transparency tools that address these  misleading preconceptions can better help consumers to understand the  information they present.
Other challenges identified by researchers that consumers face include  absorbing and evaluating large amounts of information about multiple  providers across different measures of cost and quality, assessing a  provider who does relatively well on some measures of quality and less  well on others, and interpreting complex numerical information.  Therefore, it is important that transparency tools limit the amount of  information that consumers need to pay attention to and make it easy for  them to discern overall patterns. One experiment demonstrated that  improving the organization and presentation of quality information in  transparency tools led to an increased proportion of consumers who  could identify the best-performing providers for a given dimension   of quality. Specifically, with the improvements, the proportion of  consumers who could identify the best-performing providers increased  from 18 percent to 76 percent.
Seven of the 15 characteristics of effective consumer transparency tools  we identified focus on the extent to which a tool presents its information in  a way that enables the consumer to grasp and interpret it. Specifically,  the research we reviewed shows that more effective transparency tools:  1.  Use plain language with clear graphics. Effective consumer  transparency tools use labels and descriptions that make sense to  consumers who typically are unfamiliar with clinical terminology and  who often have difficulty interpreting numerical information. Graphics,  including symbols, can help to readily convey information on relative  provider performance, especially when they are designed to display a  summary assessment of that performance as part of the symbol itself,  for example one that incorporates the words “superior” or “poor”.  2.  Explain purpose and value of quality performance ratings to  consumers. Effective consumer transparency tools address prevalent  misleading preconceptions by providing consumers coherent  explanations of how different quality measures relate to the aspects of  quality that consumers find relevant. These explanations work best  when they link individual measures to overarching categories  indicating what is being achieved, such as effectiveness of care,  safety, or patient-focused care.  3.  Summarize related information and organize data to highlight  patterns and facilitate consumer interpretation. Two techniques  that consumer tools can use to help consumers make sense of large  amounts of information are (a) combining information from multiple  related measures into summary or composite scores, and   (b) structuring presentation of the data in ways that make patterns  evident. For example, listing providers in rank order on selected cost  and quality dimensions greatly simplifies identification of high and low  performers.  4.  Enable consumers to customize information selected for  presentation to focus on what is most relevant to them.  Consumers differ in the priority they assign to different aspects of  quality. Tools that enable consumers to customize which quality  information is presented help consumers filter out information of  lesser consequence to them, and hone in on the information that they  find most compelling. For example, one consumer may choose to  focus on providers’ capacity to communicate well with patients, while  another may focus on providers’ rates of complications and infections.  5.  Enable consumers to compare quality performance of multiple  providers in one view. Transparency tools are most effective when  they present side-by-side assessments of providers’ performance on  a given aspect of cost or quality, so consumers can most easily  compare providers.  6.  Enable consumers to assess cost and quality information  together. Consumers cannot make judgments about the value of the  care offered by providers unless they can consider both cost and  quality in relation to each other. For example, transparency tools can  enable consumers to rank order available providers first on selected  measures of quality, and then, within the high-quality group, show  those with lower costs.  7.  Enable easy use and navigation of the tool. Unless consumers can  quickly find information of interest to them, they are likely to quickly  dismiss the potential utility of a consumer transparency tool and move  on. Extensive testing with consumers can help public and private  entities providing transparency tools to develop intuitive, user-friendly  approaches to website navigation and for manipulating how the data  are presented.
section_title: CMS Transparency Tools Are Limited in Their Provision of Relevant and Understandable Cost and Quality Information for Consumers
paragraphs: The CMS transparency tools we evaluated are limited in the relevance  and understandability of cost and quality information they provide to  consumers.
subsections: section_title: CMS Transparency Tools Lack Certain Relevant Information Needed to Make Meaningful Distinctions about the Cost and Quality of Providers
paragraphs: Based on the characteristics we identified concerning information  provided to consumers by effective transparency tools, we found that  CMS’s tools demonstrate a number of the characteristics of effective  transparency tools, such as timeliness of data; however, the tools lack  relevant information on cost and key differences in quality of care.  These limitations hinder their relevance and usefulness by consumers,  particularly consumers’ ability to make meaningful distinctions among  providers. (See Appendix 2 for our assessment of how CMS tools fare on  all 15 characteristics of effective tools identified through our literature  review and interviews with experts.)
With respect to cost information, none of the tools contain information on  the specific costs that patients would incur under Medicare—such as the  out-of-pocket costs to a consumer for a full episode of care. Therefore,  they do not allow consumers to combine cost and quality information to  assess the value of health care services, or anticipate and plan for  expenses related to non-emergency procedures. For example, a  consumer may wish to compare the costs of similar high-quality  providers. Depending on the service a consumer is planning to receive, a  consumer may be able to find some indications of provider quality in CMS  transparency tools, but would not find relevant information on cost. One  CMS official stated that CMS does not provide expected patient out-of- pocket costs because the agency does not have information on what  beneficiaries would pay when they have coverage other than, or in  addition to, traditional fee-for-service Medicare, as many beneficiaries  have.of what Medicare beneficiaries likely would pay for different treatments  and procedures, based on payment levels the program has set for each  provider and the cost-sharing provisions that apply under the traditional  fee-for-service Medicare program. For some Medicare beneficiaries, this  information would provide an estimate of their out-of-pocket costs. For  those with supplemental insurance coverage, it would not provide full out-  However, CMS has the information necessary to create estimates  of-pocket costs, but could be useful as an indicator of the maximum  amounts they would have had to pay without supplemental coverage.
We found that all of the CMS transparency tools provide some clinical  quality information relevant to consumers, but they often lack condition- specific information for the type of non-urgent procedures that consumers  can most readily plan for in advance. For example, some information on  hip and knee replacements is included in Hospital Compare, but limited  information is available in Hospital Compare for many other common  medical procedures such as a colonoscopy. In addition, with the  exception of Hospital Compare, none of CMS’s transparency tools  currently provide information on patient-reported outcomes, which have  been shown to be particularly relevant to consumers considering common  elective medical procedures, including hip and knee replacements.
CMS officials stated that the department plans to expand over time the  cost and quality information reported in the CMS transparency tools. For  example, CMS officials described their efforts to develop patient-reported  outcome measures to include in Hospital Compare, beginning with hip  and knee replacements. They also plan to expand the information in  Hospital Compare on costs to the Medicare program to cover a number of  specific conditions, such as heart attacks, pneumonia, and stroke.  However, CMS officials stated that the department has no plans to add  estimated consumer out-of-pocket costs related to specific medical  procedures to any of its transparency tools.
section_title: Cost and Quality Information in CMS’s Transparency Tools May Not Be Understandable to Consumers
paragraphs: Based on the characteristics we identified regarding how effective  transparency tools make cost and quality information understandable to  consumers, we found limitations in how CMS transparency tools present  information about comparative provider performance. Specifically, the  tools have substantial limitations in their use of clear language and  symbols, in summarizing and organizing information to highlight patterns  for consumers, and in enabling consumers to customize how information  is presented. Although the CMS tools demonstrate other characteristics of  effective transparency tools, such as explaining the purpose and value of  the performance information reported in the tools, and providing the  capacity to compare up to three providers side by side, the limitations we  identified may hinder consumers’ ability to understand and use the  information provided. (See Appendix 2 for our assessment of how CMS  tools compared to the 15 characteristics of effective tools identified  through our literature review and interviews with experts.)
The tools generally do not use clear language and symbols. Although  consumers can follow links to a separate page to obtain plain-language  explanations of quality measures, the labels for the actual results of the  tools often use fairly technical terms. For example, one quality measure  for heart attack patients is labeled “heart attack patients given fibrinolytic  medication within 30 minutes of arrival.” In addition, many of the quality  measures are reported in numerical form, most often as percentages.  With the exception of the “five-star” ratings currently limited to Nursing  Home Compare, none of the other CMS tools use symbols to help  consumers interpret the meaning of the information provided.
Additionally, the CMS tools also generally do not summarize results for  consumers—with the exception of Nursing Home Compare—or organize  data to highlight patterns. The Nursing Home Compare tool’s five-star  rating system provides overall assessments of performance in three  major categories—health inspections, quality measures, and staffing— each of which summarizes a set of individual measures in those  categories that the tool also reports. CMS officials told us that the agency   In  plans to expand this five-star rating system to its other tools by 2015.addition, CMS generally has not structured the presentation of the  information in its tools in a way that helps consumers detect patterns in  provider performance. For example, consumers may choose up to three  providers to compare, but must do so from lists in which the providers are  sequenced in alphabetical order or by distance from a geographic  location, not in rank order according to provider performance.consumers have to sort through the information themselves if they want  to identify the top performing providers.
In addition, the CMS tools provide consumers very limited ability to  customize what information is presented, which may not be sufficient to  allow consumers to focus on information most relevant to them. the CMS tools typically allow consumers to filter providers for  consideration based on their geographic location and whether the  provider offers one or more of a particular set of services. Expanding the  options available to consumers to customize the information presented on  cost and quality according to their individual situations and priorities  would allow them to reduce the amount of information that they must  review to identify providers that meet their needs.
For the purposes of our review, customizing information refers to the ability of  consumers to select from the quality measures offered in the tool those that are relevant  to their personal situation. organizing the tools’ information in rank order according to provider  performance.
section_title: CMS Has Taken Limited Steps to Expand Access to Cost and Quality Information, but Has Not Established Procedures to Ensure This Information Meets Consumer Needs
paragraphs: CMS has taken limited steps in three key areas to expand information on  cost and quality and increase transparency to consumers and others.  Although both HHS, through its department-wide strategy, as well as  CMS, through agency-specific plans, have clearly articulated the goal of  enhancing cost and quality transparency, these plans lack the procedures  and performance metrics needed to ensure that the particular needs of  consumers will be met.
subsections: section_title: CMS Has Taken Limited Steps in Three Key Areas to Expand Cost and Quality Information Available to Consumers and Others
paragraphs: CMS has taken steps to expand cost and quality information in three key  areas, although we identified some limitations in each step based on the  research we reviewed and experts we interviewed.
Developing and selecting measures. CMS funds and directs, primarily  through contracts, the development of new cost and quality measures  that can then be reported in both CMS and state and private sector  transparency tools. It also selects among available measures for reporting  information in its own tools. Through these two activities CMS influences  the extent to which relevant cost and quality information are made  available to consumers.
Although CMS’s transparency tools are intended to provide consumers  with the cost and quality information that they need to assess their health  care options, the tools also serve a different role for providers. According  to research conducted to support CMS public reporting efforts, CMS uses  the tools to create an incentive for providers to improve the quality of care  they deliver. By publically reporting how providers compare to their peers,  CMS intends providers to be able to identify shortcomings in their  performance that can be improved. However, research we reviewed  demonstrates that information used to motivate providers can be different  from information needed by consumers. Consequently, researchers who  have assessed CMS’s tools have raised the question of whether CMS  can ensure that tools intended to influence provider quality can also  provide consumers with the cost and quality information they need.
According to consumer transparency experts we interviewed and  research we reviewed, CMS’s processes for measure development and  selection do not adequately address consumer needs. For example,  according to a summary of CMS-sponsored consumer testing conducted  between 2001 and 2013 for the development and support of the CMS  Compare websites, consumer involvement generally comes late in the  measure development process. Moreover, CMS’s consumer testing has  focused on assessing the ability of consumers to interpret measures  developed for use by clinicians, rather than to develop or select measures  that specifically address consumer needs.
CMS officials report several ways that they have recently begun to  incorporate consumer input into the agency’s measure development and  selection process, although it is not yet clear if these efforts will ensure  that measures are focused on consumers. For example:  In 2013, CMS updated its requirements for measure development  contracts to address the needs of consumers. Specifically, contractors  are required to include patient or caregiver participation on technical  expert panels, which are a part of the measure development  process. According to one CMS official, the requirements do not  specifically dictate how contractors must involve these consumers, but  rather asks contractors to consider meaningful ways to include them.  They also do not specify how contractors will be evaluated on their  ability to meet consumer needs or specifically define expected results.  According to the CMS official, only two measure contracts have taken  effect since the requirements were updated, so it will take time to  determine the extent to which they may improve the development of  cost and quality measures of particular relevance to consumers.
CMS also has begun to solicit consumer input in the selection of  measures included in its transparency tools. In 2011, HHS convened  the Measure Applications Partnership (MAP), a consensus-based  group that includes representatives of consumer as well as provider  and other stakeholder organizations, to furnish CMS with input on the  selection of measures to include in its transparency tools and  payment programs. Although the inclusion of consumer organizations  in the MAP gives consumers an opportunity to influence measure  selection, both a CMS official and a MAP member said that the  concerns of provider organizations, which command greater technical  expertise on measurement methodology and therefore are better  prepared to advocate for their own selection preferences, tend to take  priority.
Aligning cost and quality measurement. CMS, along with several other  HHS agencies, collaborates with multiple consumer organizations, state  and private sector entities that also collect and report cost and quality  data, to work toward agreement on an aligned core set of cost and quality  measures. According to HHS’s National Quality Strategy, providers are  often asked to submit quality and other information to multiple payers,  including Medicare, and having a consistent, or aligned, set of measures  can help facilitate the collection of this information. Transparency experts  have indicated that such alignment could not only reduce the providers’  reporting burden, but it could also lead to more consistent information on  provider performance. However, according to a CMS official and an  expert involved in the alignment efforts, the extent to which this promise is  fulfilled remains to be seen, both in terms of the number and breadth of  the measures for which an agreed alignment is reached, as well as how  effectively the agreement ensures uniform application of the aligned  measures. In addition, according to experts we interviewed, the extent to  which this movement toward measure alignment benefits consumers will  depend on the extent to which the agreed-upon set of measures are  relevant to consumer concerns.
Releases of claims data. In addition to the quality data published  through the Compare sites, CMS has begun to publicly report information  on provider costs based on Medicare fee-for-service claims data.  Specifically, beginning in 2013, CMS began releasing large data sets  containing partially aggregated payment data on individual physicians and  hospitals. According to experts we interviewed, the release of these data  help to promote the concept of transparency. However, these experts  noted that the lack of patient-level information in the files prevents even  rudimentary risk adjustment—adjustments based on differences in  patients—which precludes using these data to supply cost and claims- data-based quality information for state and private sector transparency  tools.
CMS has released more detailed patient-level claims data to CMS- approved qualified entities (QE). Established under PPACA, QEs are  organizations that are given access to Medicare claims data to enable  evaluation of the performance of providers and suppliers.CMS officials, by having access to more detailed patient-level data,  QEs—selected based in part on their expertise in analyzing complex  claims data—could help make available to consumers information on  quality measures that use claims data, such as mortality and  readmissions for specific conditions or procedures. HHS has begun to   According to  share these data with 13 organizations that have been approved to  become QEs, according to CMS officials, but the organizations have only  just begun to publicly share new cost or quality information based on  these data. Therefore, it is too early to tell to what extent the QE  program will expand the cost and quality information available to  consumers.
section_title: CMS Has Not Established Procedures to Ensure the Cost and Quality Information It Collects and Reports Meets Consumer Needs
paragraphs: Although HHS has included transparency as part of its strategic plan, and  set a specific goal to report on measures that meet consumer needs, it  has not established specific procedures or performance metrics to ensure  that the cost and quality information that it collects and publicly reports  accomplish this goal. A key part of HHS’s strategic plan to ensure optimal  health care is to give the public the means to make more informed health  care choices by improving transparency and providing public access to  HHS’s data on provider performance, among other things. Furthermore,  HHS has set a specific goal to report on measures that are important to  consumers and other key audiences in ways that can be easily  understood and readily acted upon.
However, despite this goal, neither HHS, through its department-wide  strategy, nor CMS, through its agency-specific strategic plans, have  established specific procedures or performance metrics to clarify how  they will ensure that the particular needs of consumers are met, in the  midst of competing demands from providers. In particular, they have not  specified how the particular needs of consumers, as distinct from those of  providers, should be determined, and how to assess progress in  addressing those needs:  In an analysis conducted for HHS of its public reporting efforts,  researchers proposed that HHS develop and adopt processes and  criteria to select measures for reporting to consumers as well as for  reporting to more specialized audiences. HHS subsequently  released a framework to guide its public reporting activities in  December 2013. Although the framework acknowledged gaps in  available measures relevant to consumers and made the task of  addressing these gaps a priority, it did not outline a process for  integrating consumer needs into developing or selecting measures,  nor did it define criteria HHS planned to use in selecting measures for  reporting to consumers.
In November 2012, CMS also produced a draft strategic plan that  outlines the agency’s public reporting strategy. One of three goals  contained in the plan is to better meet consumer and other audience  needs, but the plan does not contain any procedures to address how  consumer needs should be determined or metrics to assess whether  identified needs are met. According to CMS officials, the draft plan is  under ongoing review by CMS officials and has not been finalized.
Similarly, CMS’s policies with respect to collaborating with other  entities to align cost and quality measures and its programs to publicly  report Medicare cost information, through its own public releases and  via QEs, have not included procedures or performance metrics that  address the particular needs of consumers.
Establishing procedures to implement goals and performance metrics to  monitor progress is important for accomplishing agency goals, according  to our guidance to federal agencies on effectively implementing  GPRAMA, which describes leading practices for how federal agencies  should assess their performance. According to the guidance,  procedures and metrics are particularly important when a government  agency must respond to multiple priorities and competing demands, such  as CMS faces in meeting the demands of both consumers and providers  in the development and selection of cost and quality measures and in the  reporting of information through its transparency tools. Without such  specific procedures and performance metrics, CMS cannot ensure that its  efforts to make cost and quality information publicly available will meet  consumer needs or help them to make meaningful distinctions among  providers.
section_title: Conclusions
paragraphs: Health care costs vary widely without being related to quality. For some  services, the differences in consumers’ out-of-pocket costs between  providers can be thousands of dollars. However, as transparent  information on both cost and quality is difficult for consumers to obtain— either through transparency tools or calling providers directly—consumers  often do not realize they could obtain better value in their health care  decisions, for example by utilizing high-quality, low-cost providers. With  consumers paying greater shares of their health care costs, it is  particularly important that they have access to relevant and  understandable cost and quality information to make value-based  decisions about services that can be planned in advance, as well as to  better anticipate and plan for their expenses.
Through HHS’s National Quality Strategy, which builds on HHS’s  strategic plan, the department has made health care cost and quality  transparency a priority for itself and its component agencies, including  CMS. CMS’s transparency tools—five Compare websites—are one way  the agency has provided cost and quality information to consumers.  However, CMS addresses multiple strategic aims with these tools,  including both informing consumers and incentivizing providers to  improve the quality of their care. Partly as a result, the tools exhibit critical  weaknesses when assessed in terms of the characteristics that would  make them most effective for consumers. For example, the tools lack  information on topics of considerable relevance to consumers, such as  patient-reported outcome measures and patient out-of-pocket costs.  Additionally, the tools do not organize cost and quality information in a  way that may enable consumers to readily understand and compare  provider performance, or customize how the information is presented to  enable consumers to identify the best providers for aspects of care that  they may find most relevant. While providing relevant and understandable  information on cost and the ability to compare it with provider quality is  inherently complex, consumers need this information to determine if  different providers or settings provide a greater overall value for care they  expect to receive and to determine what their health care expenses will  be in advance.
Although HHS has set goals to promote transparency for consumers,  and, largely through CMS, has taken steps to expand available cost and  quality information, CMS has not established procedures or performance  metrics to ensure that this information is relevant and understandable to  consumers. Furthermore, each of the steps CMS has taken has  limitations that may prevent consumer needs from being met. In  particular, CMS’s process for developing and selecting cost and quality  measures for its tools has been heavily influenced by the concerns of  providers rather than consumers, which helps to account for the relative  lack of cost and quality information in CMS’s tools that consumers would  find relevant. Especially in such situations where agencies are responding  to multiple priorities, leading practices for strategic planning call for  establishing specific procedures and performance metrics to ensure that  each of those priorities receives due attention. Until CMS establishes  such procedures and performance metrics with respect to implementing  their goals for promoting health care transparency, the agency is likely to  continue to have limited effectiveness in conveying relevant and  understandable information on cost and quality to consumers.
section_title: Recommendations for Executive Action
paragraphs: To improve consumers’ access to relevant and understandable  information on the cost and quality of health care services, we  recommend that the Secretary of HHS direct the Administrator of CMS to  take four actions:  1.  Include in the CMS Compare websites, to the extent feasible,  estimated out-of-pocket costs for Medicare beneficiaries for common  treatments that can be planned in advance;  2.  Organize cost and quality information in the CMS Compare websites  to facilitate consumer identification of the highest-performing  providers, such as by listing providers in order based on their  performance;  3.  Include in the CMS Compare websites the capability for consumers to  customize the information presented, to better focus on information  relevant to them; and  4.  Develop specific procedures and performance metrics to ensure that  CMS’s efforts to promote the development and use of its own and  others’ transparency tools adequately address the needs of  consumers.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to HHS for review, and HHS provided  written comments, which are reprinted in appendix III. In its comments,  HHS concurred with each of our recommendations and noted a number  of activities being done to improve transparency of cost and quality  information for consumers. HHS stated its intention to ensure that its  transparency tools, the Compare websites, fully address consumer  priorities. For example, HHS stated that it was committed, to the extent  feasible, to providing estimated out-of-pocket costs to Medicare  beneficiaries for common procedures that can be planned in advance.  However, HHS noted that, as mentioned in our draft report, there are  challenges to collecting all of the relevant cost information. Similarly, HHS  agreed to organize the cost and quality information provided in its  Compare websites to facilitate consumer identification of the highest  performing providers, such as by listing providers in order based on  ratings of the quality of their performance. HHS noted its plans to expand  the use of star ratings, including the ability to sort and filter provider  listings based on those ratings. HHS also agreed with our  recommendation to enable consumers to customize the information  presented to them on the Compare websites. Finally, HHS agreed to  develop specific procedures and performance metrics to ensure that its  transparency tools adequately address the needs of consumers. Although  HHS stated that it had already developed many internal procedures and  performance metrics, it did not identify or describe them in its comments,  nor did we find any such procedures or performance metrics in the  planning documents provided to us by HHS during the course of our  work. HHS also provided us with technical comments, which we  incorporated as appropriate.
As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies of this report to the  Secretary of the Department of Health and Human Services, the  Administrator of the Centers for Medicare & Medicaid Services, and other  interested parties. In addition, the report will be available at no charge on  GAO’s website at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-7114 or at kohnl@gao.gov. Contact points for our Office  of Congressional Relations and Office of Public Affairs can be found on  the last page of this report. Other major contributors to this report are  listed in appendix IV.
section_title: Appendix I: Examples of Variation in Cost and Quality Information Available to Consumers for Selected Procedures
paragraphs: We obtained examples of cost and quality information that were available  to certain consumers in specific circumstances by obtaining information  from selected transparency tools and by contacting providers, acting in  the role of a consumer, to ask for cost and quality information. The  examples we obtained from two selected transparency tools (a health  insurer and Castlight) and from contacting selected providers in two  health care markets by telephone illustrate the variation in cost and  quality information that are available to consumers in specific situations,  such as those who are members of a particular health plan or uninsured.  Specifically, the examples showed variation in cost, but often did not  include enough information on quality for it to be considered alongside  cost (see tables 4-7 below).
section_title: Appendix II: Assessment of CMS Transparency Tools
paragraphs: We assessed the extent to which the Centers for Medicare & Medicaid  Services’ transparency tools include the 15 characteristics of effective  tools we identified through our literature review and interviews with  experts. (See Table 8.)
section_title: Appendix III: Comments from the Department of Health and Human Services
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contacts
section_title: Acknowledgments
paragraphs: In addition to the contact named above, Will Simerl, Assistant Director;   N. Rotimi Adebonojo; Jennie Apter; Rebecca Hendrickson;   Monica Perez-Nelson; Eric Peterson; and Elise Pressma made key  contributions to this report.