Transparency tools are a way to make information on health care cost and quality transparent to consumers and others, and are a key part of HHS’s strategy to improve the quality and affordability of health care. There are multiple ways to assess the cost of health care. For example, cost can be measured based on the amount of money providers set as a “charge” for various individual services, but these charges typically do not represent the actual amounts paid by insurers or consumers. The cost that an insured consumer is responsible for paying to receive services is called an out-of-pocket cost, which depends on the consumer’s individual provider choices and insurance benefit design. In addition, any given episode of care usually involves payments to multiple providers (e.g., surgeons, anesthesiologists, pathologists, etc.), facility fees, and other ancillary fees, and any given cost figure may or may not represent the total costs for an episode of care by including all of these expenses. For a description of the quality measure categories, see Agency for Healthcare Research and Quality, National Quality Measures Clearinghouse, http://www.qualitymeasures.ahrq.gov/tutorial/varieties.aspx, accessed August 5, 2014. The National Quality Measures Clearinghouse also includes other types of clinical quality measures, such as access measures, which measure a patient’s ability to attain timely and appropriate care. In March 2011, HHS first published its National Strategy for Quality Improvement in Health Care (the National Quality Strategy), as required by the Patient Protection and Affordable Care Act (PPACA). The National Quality Strategy builds on priorities HHS previously identified in its strategic plan for fiscal years 2010-2015, which emphasize the need for transparent information to give consumers the means to make more informed choices about their health care. Two of the National Quality Strategy’s overarching goals—better care and affordable care—relate to health care cost and quality transparency. According to the strategy, to achieve better care, patients must be given access to understandable information and decision support tools that help them manage their health and navigate the health care delivery system. To achieve affordable care, systems must be created to make health care cost and quality more transparent to consumers and providers, so they can make better choices and decisions. The strategy also focuses on coordinating and aligning efforts across the public and private sectors, for example by establishing an aligned set of common cost and quality measures by which to assess how well providers and programs support effective care. As part of its efforts to foster greater transparency of information, HHS developed transparency tools for consumers focused on quality, some of which predate PPACA and the development of the National Quality Strategy. Between 1998 and 2010, HHS—through CMS—launched five Compare websites to publicly report certain information, including quality information, based on data submitted by different types of providers participating in the Medicare program (see table 2). For some, but not all, of these provider types, HHS is required to publicly report certain information about provider performance. HHS—primarily through AHRQ—also plays a role in research and dissemination of information on consumer-focused public reporting. AHRQ has supported the research and publication of numerous papers that lay out best practices for public reporting of cost and quality information to consumers and has widely shared the expertise contained in that research through various forums, such as conferences and webinars that support wider transparency efforts. A variety of private sector entities have developed transparency tools that provide cost and quality information to consumers, including health insurance carriers, third-party vendors, and regional collaboratives. These private-sector tools allow consumers to obtain personalized cost estimates, compare different providers, or estimate their out-of-pocket costs before receiving a service. For example, many health plans offer tools that provide cost estimates to enrolled members. Researchers calculated that approximately 70 percent of the privately insured population had access to cost transparency tools in 2013. In addition to tools provided directly by health plans, employers may contract with third- party vendors to provide transparency tools for their employees. Health plans and third-party vendors frequently offer some quality information along with cost estimates to their enrolled members. Often this information is derived from data reported by CMS on its Compare sites and may be combined with additional quality information obtained from a variety of sources that collect and report data on provider quality. Several states also have developed transparency tools.plans and third-party vendors, state tools can be accessed by the general public and are not restricted to members who pay or enroll for services, giving the uninsured access to cost and quality information. Like the private-sector tools, state tools generally draw on quality information from CMS in combination with information from other sources. The two selected consumer transparency tools we reviewed show that some providers are paid thousands of dollars more than others for the same service in the same geographic area, regardless of the quality of such services. Specifically, we found this variation in cost to be present across multiple services, settings, and geographic areas in the information provided by the transparency tools of two different entities (a health insurer and Castlight) that we reviewed. For example, the health insurer reported that in the Denver area, the estimated total cost of a laparoscopic gallbladder surgery in selected ASCs ranged between $3,281 and $18,770 (consumers would pay between $3,281 and $6,954 in estimated out-of-pocket costs) in July 2014. Meanwhile, for the same time period, the health insurer reported that the estimated total cost for the same service in selected hospital outpatient departments in the Denver area ranged from $17,791 to $40,626 (consumers would pay between $6,758 and $11,325 in estimated out-of-pocket costs).Similarly, Castlight reported, in May 2014, that for an MRI of the lower back at selected acute care hospitals in the Indianapolis, Indiana area, the estimated total cost ranged between $277 and $5,184 depending on the provider (with consumers paying between $277 and $2,637 in estimated out-of-pocket costs).variation in information we obtained from selected consumer transparency tools.) (See app. I for more examples of the Information from the two selected transparency tools indicates that the observed cost variation is not tied to variations in quality, regardless of the treatment and geographic area. For example, according to information obtained from Castlight, the estimated total cost of maternity care at selected acute care hospitals in the Boston area that rated more highly on several quality indicators ranged between $6,834 and $21,554 (consumers would pay between $2,967 and $5,000 in estimated out-of- pocket costs). laparoscopic gallbladder surgery showed that a number of hospital outpatient departments in the Denver area that rated more highly on several quality indicators had lower total costs than other hospital outpatient departments that rated less highly on quality. These examples are particularly relevant as researchers have found that many consumers assume that all providers offer good quality care, while others have the misconception that higher-cost providers will provide higher quality of care than lower-cost providers. The quality indicators provided by Castlight for maternity care include the percent of pregnancies for which the baby is delivered at the appropriate developmental age, using an appropriate delivery method, and recommended processes of care. assistance of consumer transparency tools.hospitals we contacted to inquire about the cost of an inguinal hernia repair and diagnostic colonoscopy for an uninsured patient in the Portland, Oregon, and Minneapolis, Minnesota health care markets— locations selected because they have initiatives to promote transparency—we received limited cost information from 54 percent (13) and quality information from 29 percent (7) (see table 3). Transparency tools are most effective if they both provide information relevant to consumers and convey that information in a way that consumers can readily understand. The information on cost and quality that consumers find relevant makes up just a portion of the information available for inclusion in transparency tools. For example, research shows that most consumers do not find the information derived from many specific process-of-care measures that clinicians have developed and used to guide quality improvement efforts useful. Rather, consumers are most likely to respond to information that applies to their personal circumstances, including, for example, information on the specific procedures consumers are considering, on providers who would be available to perform those procedures, and on cost estimates that take account of their particular insurance coverage. Consumers also seek information that helps them to make meaningful distinctions among providers in terms of cost and quality. The research we reviewed has found that consumers value quality measures that show differences in clinical outcomes and patient experiences, and cost measures that show differences in out-of-pocket expenses. In addition, consumers want to know the source of cost and quality information because this source information helps them determine their level of confidence in that information. Eight of the 15 characteristics of effective transparency tools we identified address the extent to which a tool provides substantive quality and cost information of relevance to consumers. Specifically, the research we reviewed shows that more effective transparency tools: 1. Review a broad range of services so that more consumers’ particular needs are included. The more services that are covered by a transparency tool (or set of tools), the more likely it is that the tool will have information relevant to the particular services of interest to any given consumer. It is especially important to include services that are predictable and non-urgent in a transparency tool, because these services are most likely to afford consumers the opportunity to evaluate cost and quality information before receiving the service. 2. Cover a broad range of providers. Transparency tools that provide information for all or most of the available providers in a given geographic area, regardless of network status or practice setting, give consumers more information about their full range of options. For example, for procedures that can be conducted in either a hospital outpatient department or ASC, it helps consumers to provide comparable information for both settings, so that consumers can choose from a larger number of providers that offer those procedures. 3. Describe key differences in clinical quality of care, particularly patient-reported outcomes. Assessments of the clinical quality of care that have been shown to have particular relevance to consumers are those that relate to long-term outcomes of the care experienced by other patients. Often this is best addressed by patient-reported outcomes, which tell consumers the eventual outcome of treatments, as reported by previous patients of a particular provider. For example, patients receiving hip replacements can be asked, through such patient-reported outcomes, to rate their ability to climb stairs both before and after their procedures, which enables assessments of the procedures’ effects on patients’ mobility. 4. Describe key differences in patient experiences with providers. Another outcome that matters is patients’ assessment of their interactions with providers. Effective transparency tools include information on how past patients evaluated providers in terms of dimensions such as how well nurses communicate with patients, or the responsiveness of clinicians to patients’ needs. 5. Describe key differences in costs, particularly patient out-of- pocket costs. The cost information of greatest relevance to consumers is finding out what they will have to pay for a given service before it occurs. For example, a tool that provides information on the average cost of a selected procedure in a given geographic location may be less relevant to consumers than one that takes into account the specific costs for that procedure and location given that consumer’s specific health insurance coverage. 6. Describe other information related to quality, where appropriate. There may be other quality indicators that could have major significance to consumers for certain types of services. For example, facility inspection results and staffing levels are of particular relevance to nursing home care. 7. Provide timely Information. More recent data are intrinsically more relevant than data that are several years old. Because consumer transparency tools necessarily rely on past data to assess likely cost and quality performance in the future, some lag in collecting, analyzing, and providing data is inevitable. Data that are no more than two years old are generally considered timely. 8. Describe key strengths and limitations of the data. Although the research we reviewed shows that few consumers are inclined to delve into the many methodological issues that concern appropriate techniques for collecting, checking, and analyzing cost and quality data, transparency tools can provide both summary assessments of strengths and limitations for most consumers, and links to more complete explanations for those wanting to pursue these issues in greater detail. Such information, along with identification of the organization responsible for the tool, provides consumers a basis to judge the credibility of the cost and quality information provided. Regardless of its potential relevance to them, consumers only can respond to information about cost and quality if they understand it. Although cost and quality information can be inherently difficult to understand, research suggests ways that transparency tools can make it easier for consumers to do so. One factor that increases the difficulty of understanding such information is the prevalence of misleading preconceptions about the cost and quality of care. For example, researchers find that many consumers incorrectly assume that all hospitals and physicians provide good quality care, while others assume that higher-cost providers will provide higher quality of care than lower- cost providers. As a result, transparency tools that address these misleading preconceptions can better help consumers to understand the information they present. Other challenges identified by researchers that consumers face include absorbing and evaluating large amounts of information about multiple providers across different measures of cost and quality, assessing a provider who does relatively well on some measures of quality and less well on others, and interpreting complex numerical information. Therefore, it is important that transparency tools limit the amount of information that consumers need to pay attention to and make it easy for them to discern overall patterns. One experiment demonstrated that improving the organization and presentation of quality information in transparency tools led to an increased proportion of consumers who could identify the best-performing providers for a given dimension of quality. Specifically, with the improvements, the proportion of consumers who could identify the best-performing providers increased from 18 percent to 76 percent. Seven of the 15 characteristics of effective consumer transparency tools we identified focus on the extent to which a tool presents its information in a way that enables the consumer to grasp and interpret it. Specifically, the research we reviewed shows that more effective transparency tools: 1. Use plain language with clear graphics. Effective consumer transparency tools use labels and descriptions that make sense to consumers who typically are unfamiliar with clinical terminology and who often have difficulty interpreting numerical information. Graphics, including symbols, can help to readily convey information on relative provider performance, especially when they are designed to display a summary assessment of that performance as part of the symbol itself, for example one that incorporates the words “superior” or “poor”. 2. Explain purpose and value of quality performance ratings to consumers. Effective consumer transparency tools address prevalent misleading preconceptions by providing consumers coherent explanations of how different quality measures relate to the aspects of quality that consumers find relevant. These explanations work best when they link individual measures to overarching categories indicating what is being achieved, such as effectiveness of care, safety, or patient-focused care. 3. Summarize related information and organize data to highlight patterns and facilitate consumer interpretation. Two techniques that consumer tools can use to help consumers make sense of large amounts of information are (a) combining information from multiple related measures into summary or composite scores, and (b) structuring presentation of the data in ways that make patterns evident. For example, listing providers in rank order on selected cost and quality dimensions greatly simplifies identification of high and low performers. 4. Enable consumers to customize information selected for presentation to focus on what is most relevant to them. Consumers differ in the priority they assign to different aspects of quality. Tools that enable consumers to customize which quality information is presented help consumers filter out information of lesser consequence to them, and hone in on the information that they find most compelling. For example, one consumer may choose to focus on providers’ capacity to communicate well with patients, while another may focus on providers’ rates of complications and infections. 5. Enable consumers to compare quality performance of multiple providers in one view. Transparency tools are most effective when they present side-by-side assessments of providers’ performance on a given aspect of cost or quality, so consumers can most easily compare providers. 6. Enable consumers to assess cost and quality information together. Consumers cannot make judgments about the value of the care offered by providers unless they can consider both cost and quality in relation to each other. For example, transparency tools can enable consumers to rank order available providers first on selected measures of quality, and then, within the high-quality group, show those with lower costs. 7. Enable easy use and navigation of the tool. Unless consumers can quickly find information of interest to them, they are likely to quickly dismiss the potential utility of a consumer transparency tool and move on. Extensive testing with consumers can help public and private entities providing transparency tools to develop intuitive, user-friendly approaches to website navigation and for manipulating how the data are presented. The CMS transparency tools we evaluated are limited in the relevance and understandability of cost and quality information they provide to consumers. Based on the characteristics we identified concerning information provided to consumers by effective transparency tools, we found that CMS’s tools demonstrate a number of the characteristics of effective transparency tools, such as timeliness of data; however, the tools lack relevant information on cost and key differences in quality of care. These limitations hinder their relevance and usefulness by consumers, particularly consumers’ ability to make meaningful distinctions among providers. (See Appendix 2 for our assessment of how CMS tools fare on all 15 characteristics of effective tools identified through our literature review and interviews with experts.) With respect to cost information, none of the tools contain information on the specific costs that patients would incur under Medicare—such as the out-of-pocket costs to a consumer for a full episode of care. Therefore, they do not allow consumers to combine cost and quality information to assess the value of health care services, or anticipate and plan for expenses related to non-emergency procedures. For example, a consumer may wish to compare the costs of similar high-quality providers. Depending on the service a consumer is planning to receive, a consumer may be able to find some indications of provider quality in CMS transparency tools, but would not find relevant information on cost. One CMS official stated that CMS does not provide expected patient out-of- pocket costs because the agency does not have information on what beneficiaries would pay when they have coverage other than, or in addition to, traditional fee-for-service Medicare, as many beneficiaries have.of what Medicare beneficiaries likely would pay for different treatments and procedures, based on payment levels the program has set for each provider and the cost-sharing provisions that apply under the traditional fee-for-service Medicare program. For some Medicare beneficiaries, this information would provide an estimate of their out-of-pocket costs. For those with supplemental insurance coverage, it would not provide full out- However, CMS has the information necessary to create estimates of-pocket costs, but could be useful as an indicator of the maximum amounts they would have had to pay without supplemental coverage. We found that all of the CMS transparency tools provide some clinical quality information relevant to consumers, but they often lack condition- specific information for the type of non-urgent procedures that consumers can most readily plan for in advance. For example, some information on hip and knee replacements is included in Hospital Compare, but limited information is available in Hospital Compare for many other common medical procedures such as a colonoscopy. In addition, with the exception of Hospital Compare, none of CMS’s transparency tools currently provide information on patient-reported outcomes, which have been shown to be particularly relevant to consumers considering common elective medical procedures, including hip and knee replacements. CMS officials stated that the department plans to expand over time the cost and quality information reported in the CMS transparency tools. For example, CMS officials described their efforts to develop patient-reported outcome measures to include in Hospital Compare, beginning with hip and knee replacements. They also plan to expand the information in Hospital Compare on costs to the Medicare program to cover a number of specific conditions, such as heart attacks, pneumonia, and stroke. However, CMS officials stated that the department has no plans to add estimated consumer out-of-pocket costs related to specific medical procedures to any of its transparency tools. Based on the characteristics we identified regarding how effective transparency tools make cost and quality information understandable to consumers, we found limitations in how CMS transparency tools present information about comparative provider performance. Specifically, the tools have substantial limitations in their use of clear language and symbols, in summarizing and organizing information to highlight patterns for consumers, and in enabling consumers to customize how information is presented. Although the CMS tools demonstrate other characteristics of effective transparency tools, such as explaining the purpose and value of the performance information reported in the tools, and providing the capacity to compare up to three providers side by side, the limitations we identified may hinder consumers’ ability to understand and use the information provided. (See Appendix 2 for our assessment of how CMS tools compared to the 15 characteristics of effective tools identified through our literature review and interviews with experts.) The tools generally do not use clear language and symbols. Although consumers can follow links to a separate page to obtain plain-language explanations of quality measures, the labels for the actual results of the tools often use fairly technical terms. For example, one quality measure for heart attack patients is labeled “heart attack patients given fibrinolytic medication within 30 minutes of arrival.” In addition, many of the quality measures are reported in numerical form, most often as percentages. With the exception of the “five-star” ratings currently limited to Nursing Home Compare, none of the other CMS tools use symbols to help consumers interpret the meaning of the information provided. Additionally, the CMS tools also generally do not summarize results for consumers—with the exception of Nursing Home Compare—or organize data to highlight patterns. The Nursing Home Compare tool’s five-star rating system provides overall assessments of performance in three major categories—health inspections, quality measures, and staffing— each of which summarizes a set of individual measures in those categories that the tool also reports. CMS officials told us that the agency In plans to expand this five-star rating system to its other tools by 2015.addition, CMS generally has not structured the presentation of the information in its tools in a way that helps consumers detect patterns in provider performance. For example, consumers may choose up to three providers to compare, but must do so from lists in which the providers are sequenced in alphabetical order or by distance from a geographic location, not in rank order according to provider performance.consumers have to sort through the information themselves if they want to identify the top performing providers. In addition, the CMS tools provide consumers very limited ability to customize what information is presented, which may not be sufficient to allow consumers to focus on information most relevant to them. the CMS tools typically allow consumers to filter providers for consideration based on their geographic location and whether the provider offers one or more of a particular set of services. Expanding the options available to consumers to customize the information presented on cost and quality according to their individual situations and priorities would allow them to reduce the amount of information that they must review to identify providers that meet their needs. For the purposes of our review, customizing information refers to the ability of consumers to select from the quality measures offered in the tool those that are relevant to their personal situation. organizing the tools’ information in rank order according to provider performance. CMS has taken limited steps in three key areas to expand information on cost and quality and increase transparency to consumers and others. Although both HHS, through its department-wide strategy, as well as CMS, through agency-specific plans, have clearly articulated the goal of enhancing cost and quality transparency, these plans lack the procedures and performance metrics needed to ensure that the particular needs of consumers will be met. CMS has taken steps to expand cost and quality information in three key areas, although we identified some limitations in each step based on the research we reviewed and experts we interviewed. Developing and selecting measures. CMS funds and directs, primarily through contracts, the development of new cost and quality measures that can then be reported in both CMS and state and private sector transparency tools. It also selects among available measures for reporting information in its own tools. Through these two activities CMS influences the extent to which relevant cost and quality information are made available to consumers. Although CMS’s transparency tools are intended to provide consumers with the cost and quality information that they need to assess their health care options, the tools also serve a different role for providers. According to research conducted to support CMS public reporting efforts, CMS uses the tools to create an incentive for providers to improve the quality of care they deliver. By publically reporting how providers compare to their peers, CMS intends providers to be able to identify shortcomings in their performance that can be improved. However, research we reviewed demonstrates that information used to motivate providers can be different from information needed by consumers. Consequently, researchers who have assessed CMS’s tools have raised the question of whether CMS can ensure that tools intended to influence provider quality can also provide consumers with the cost and quality information they need. According to consumer transparency experts we interviewed and research we reviewed, CMS’s processes for measure development and selection do not adequately address consumer needs. For example, according to a summary of CMS-sponsored consumer testing conducted between 2001 and 2013 for the development and support of the CMS Compare websites, consumer involvement generally comes late in the measure development process. Moreover, CMS’s consumer testing has focused on assessing the ability of consumers to interpret measures developed for use by clinicians, rather than to develop or select measures that specifically address consumer needs. CMS officials report several ways that they have recently begun to incorporate consumer input into the agency’s measure development and selection process, although it is not yet clear if these efforts will ensure that measures are focused on consumers. For example: In 2013, CMS updated its requirements for measure development contracts to address the needs of consumers. Specifically, contractors are required to include patient or caregiver participation on technical expert panels, which are a part of the measure development process. According to one CMS official, the requirements do not specifically dictate how contractors must involve these consumers, but rather asks contractors to consider meaningful ways to include them. They also do not specify how contractors will be evaluated on their ability to meet consumer needs or specifically define expected results. According to the CMS official, only two measure contracts have taken effect since the requirements were updated, so it will take time to determine the extent to which they may improve the development of cost and quality measures of particular relevance to consumers. CMS also has begun to solicit consumer input in the selection of measures included in its transparency tools. In 2011, HHS convened the Measure Applications Partnership (MAP), a consensus-based group that includes representatives of consumer as well as provider and other stakeholder organizations, to furnish CMS with input on the selection of measures to include in its transparency tools and payment programs. Although the inclusion of consumer organizations in the MAP gives consumers an opportunity to influence measure selection, both a CMS official and a MAP member said that the concerns of provider organizations, which command greater technical expertise on measurement methodology and therefore are better prepared to advocate for their own selection preferences, tend to take priority. Aligning cost and quality measurement. CMS, along with several other HHS agencies, collaborates with multiple consumer organizations, state and private sector entities that also collect and report cost and quality data, to work toward agreement on an aligned core set of cost and quality measures. According to HHS’s National Quality Strategy, providers are often asked to submit quality and other information to multiple payers, including Medicare, and having a consistent, or aligned, set of measures can help facilitate the collection of this information. Transparency experts have indicated that such alignment could not only reduce the providers’ reporting burden, but it could also lead to more consistent information on provider performance. However, according to a CMS official and an expert involved in the alignment efforts, the extent to which this promise is fulfilled remains to be seen, both in terms of the number and breadth of the measures for which an agreed alignment is reached, as well as how effectively the agreement ensures uniform application of the aligned measures. In addition, according to experts we interviewed, the extent to which this movement toward measure alignment benefits consumers will depend on the extent to which the agreed-upon set of measures are relevant to consumer concerns. Releases of claims data. In addition to the quality data published through the Compare sites, CMS has begun to publicly report information on provider costs based on Medicare fee-for-service claims data. Specifically, beginning in 2013, CMS began releasing large data sets containing partially aggregated payment data on individual physicians and hospitals. According to experts we interviewed, the release of these data help to promote the concept of transparency. However, these experts noted that the lack of patient-level information in the files prevents even rudimentary risk adjustment—adjustments based on differences in patients—which precludes using these data to supply cost and claims- data-based quality information for state and private sector transparency tools. CMS has released more detailed patient-level claims data to CMS- approved qualified entities (QE). Established under PPACA, QEs are organizations that are given access to Medicare claims data to enable evaluation of the performance of providers and suppliers.CMS officials, by having access to more detailed patient-level data, QEs—selected based in part on their expertise in analyzing complex claims data—could help make available to consumers information on quality measures that use claims data, such as mortality and readmissions for specific conditions or procedures. HHS has begun to According to share these data with 13 organizations that have been approved to become QEs, according to CMS officials, but the organizations have only just begun to publicly share new cost or quality information based on these data. Therefore, it is too early to tell to what extent the QE program will expand the cost and quality information available to consumers. Although HHS has included transparency as part of its strategic plan, and set a specific goal to report on measures that meet consumer needs, it has not established specific procedures or performance metrics to ensure that the cost and quality information that it collects and publicly reports accomplish this goal. A key part of HHS’s strategic plan to ensure optimal health care is to give the public the means to make more informed health care choices by improving transparency and providing public access to HHS’s data on provider performance, among other things. Furthermore, HHS has set a specific goal to report on measures that are important to consumers and other key audiences in ways that can be easily understood and readily acted upon. However, despite this goal, neither HHS, through its department-wide strategy, nor CMS, through its agency-specific strategic plans, have established specific procedures or performance metrics to clarify how they will ensure that the particular needs of consumers are met, in the midst of competing demands from providers. In particular, they have not specified how the particular needs of consumers, as distinct from those of providers, should be determined, and how to assess progress in addressing those needs: In an analysis conducted for HHS of its public reporting efforts, researchers proposed that HHS develop and adopt processes and criteria to select measures for reporting to consumers as well as for reporting to more specialized audiences. HHS subsequently released a framework to guide its public reporting activities in December 2013. Although the framework acknowledged gaps in available measures relevant to consumers and made the task of addressing these gaps a priority, it did not outline a process for integrating consumer needs into developing or selecting measures, nor did it define criteria HHS planned to use in selecting measures for reporting to consumers. In November 2012, CMS also produced a draft strategic plan that outlines the agency’s public reporting strategy. One of three goals contained in the plan is to better meet consumer and other audience needs, but the plan does not contain any procedures to address how consumer needs should be determined or metrics to assess whether identified needs are met. According to CMS officials, the draft plan is under ongoing review by CMS officials and has not been finalized. Similarly, CMS’s policies with respect to collaborating with other entities to align cost and quality measures and its programs to publicly report Medicare cost information, through its own public releases and via QEs, have not included procedures or performance metrics that address the particular needs of consumers. Establishing procedures to implement goals and performance metrics to monitor progress is important for accomplishing agency goals, according to our guidance to federal agencies on effectively implementing GPRAMA, which describes leading practices for how federal agencies should assess their performance. According to the guidance, procedures and metrics are particularly important when a government agency must respond to multiple priorities and competing demands, such as CMS faces in meeting the demands of both consumers and providers in the development and selection of cost and quality measures and in the reporting of information through its transparency tools. Without such specific procedures and performance metrics, CMS cannot ensure that its efforts to make cost and quality information publicly available will meet consumer needs or help them to make meaningful distinctions among providers. Health care costs vary widely without being related to quality. For some services, the differences in consumers’ out-of-pocket costs between providers can be thousands of dollars. However, as transparent information on both cost and quality is difficult for consumers to obtain— either through transparency tools or calling providers directly—consumers often do not realize they could obtain better value in their health care decisions, for example by utilizing high-quality, low-cost providers. With consumers paying greater shares of their health care costs, it is particularly important that they have access to relevant and understandable cost and quality information to make value-based decisions about services that can be planned in advance, as well as to better anticipate and plan for their expenses. Through HHS’s National Quality Strategy, which builds on HHS’s strategic plan, the department has made health care cost and quality transparency a priority for itself and its component agencies, including CMS. CMS’s transparency tools—five Compare websites—are one way the agency has provided cost and quality information to consumers. However, CMS addresses multiple strategic aims with these tools, including both informing consumers and incentivizing providers to improve the quality of their care. Partly as a result, the tools exhibit critical weaknesses when assessed in terms of the characteristics that would make them most effective for consumers. For example, the tools lack information on topics of considerable relevance to consumers, such as patient-reported outcome measures and patient out-of-pocket costs. Additionally, the tools do not organize cost and quality information in a way that may enable consumers to readily understand and compare provider performance, or customize how the information is presented to enable consumers to identify the best providers for aspects of care that they may find most relevant. While providing relevant and understandable information on cost and the ability to compare it with provider quality is inherently complex, consumers need this information to determine if different providers or settings provide a greater overall value for care they expect to receive and to determine what their health care expenses will be in advance. Although HHS has set goals to promote transparency for consumers, and, largely through CMS, has taken steps to expand available cost and quality information, CMS has not established procedures or performance metrics to ensure that this information is relevant and understandable to consumers. Furthermore, each of the steps CMS has taken has limitations that may prevent consumer needs from being met. In particular, CMS’s process for developing and selecting cost and quality measures for its tools has been heavily influenced by the concerns of providers rather than consumers, which helps to account for the relative lack of cost and quality information in CMS’s tools that consumers would find relevant. Especially in such situations where agencies are responding to multiple priorities, leading practices for strategic planning call for establishing specific procedures and performance metrics to ensure that each of those priorities receives due attention. Until CMS establishes such procedures and performance metrics with respect to implementing their goals for promoting health care transparency, the agency is likely to continue to have limited effectiveness in conveying relevant and understandable information on cost and quality to consumers. To improve consumers’ access to relevant and understandable information on the cost and quality of health care services, we recommend that the Secretary of HHS direct the Administrator of CMS to take four actions: 1. Include in the CMS Compare websites, to the extent feasible, estimated out-of-pocket costs for Medicare beneficiaries for common treatments that can be planned in advance; 2. Organize cost and quality information in the CMS Compare websites to facilitate consumer identification of the highest-performing providers, such as by listing providers in order based on their performance; 3. Include in the CMS Compare websites the capability for consumers to customize the information presented, to better focus on information relevant to them; and 4. Develop specific procedures and performance metrics to ensure that CMS’s efforts to promote the development and use of its own and others’ transparency tools adequately address the needs of consumers. We provided a draft of this report to HHS for review, and HHS provided written comments, which are reprinted in appendix III. In its comments, HHS concurred with each of our recommendations and noted a number of activities being done to improve transparency of cost and quality information for consumers. HHS stated its intention to ensure that its transparency tools, the Compare websites, fully address consumer priorities. For example, HHS stated that it was committed, to the extent feasible, to providing estimated out-of-pocket costs to Medicare beneficiaries for common procedures that can be planned in advance. However, HHS noted that, as mentioned in our draft report, there are challenges to collecting all of the relevant cost information. Similarly, HHS agreed to organize the cost and quality information provided in its Compare websites to facilitate consumer identification of the highest performing providers, such as by listing providers in order based on ratings of the quality of their performance. HHS noted its plans to expand the use of star ratings, including the ability to sort and filter provider listings based on those ratings. HHS also agreed with our recommendation to enable consumers to customize the information presented to them on the Compare websites. Finally, HHS agreed to develop specific procedures and performance metrics to ensure that its transparency tools adequately address the needs of consumers. Although HHS stated that it had already developed many internal procedures and performance metrics, it did not identify or describe them in its comments, nor did we find any such procedures or performance metrics in the planning documents provided to us by HHS during the course of our work. HHS also provided us with technical comments, which we incorporated as appropriate. As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies of this report to the Secretary of the Department of Health and Human Services, the Administrator of the Centers for Medicare & Medicaid Services, and other interested parties. In addition, the report will be available at no charge on GAO’s website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-7114 or at kohnl@gao.gov. Contact points for our Office of Congressional Relations and Office of Public Affairs can be found on the last page of this report. Other major contributors to this report are listed in appendix IV. We obtained examples of cost and quality information that were available to certain consumers in specific circumstances by obtaining information from selected transparency tools and by contacting providers, acting in the role of a consumer, to ask for cost and quality information. The examples we obtained from two selected transparency tools (a health insurer and Castlight) and from contacting selected providers in two health care markets by telephone illustrate the variation in cost and quality information that are available to consumers in specific situations, such as those who are members of a particular health plan or uninsured. Specifically, the examples showed variation in cost, but often did not include enough information on quality for it to be considered alongside cost (see tables 4-7 below). We assessed the extent to which the Centers for Medicare & Medicaid Services’ transparency tools include the 15 characteristics of effective tools we identified through our literature review and interviews with experts. (See Table 8.) In addition to the contact named above, Will Simerl, Assistant Director; N. Rotimi Adebonojo; Jennie Apter; Rebecca Hendrickson; Monica Perez-Nelson; Eric Peterson; and Elise Pressma made key contributions to this report.

Summary: The cost and quality of health care services can vary significantly, with high cost not necessarily indicating high quality. As consumers pay for a growing proportion of their care, they have an increased need for cost and quality information before they receive care, so they can plan and make informed decisions. Transparency tools can provide such information to consumers and others. GAO was asked to study cost and quality information for consumers. This report examines (1) information on cost and quality available to consumers from selected transparency tools, (2) characteristics of effective transparency tools, (3) limitations, if any, in the effectiveness of CMS transparency tools, and (4) CMS efforts to expand cost and quality information available through transparency tools. GAO analyzed information from two private tools—selected because they had both cost and quality information—and CMS's five transparency tools, reviewed research to identify best practices for conveying information to consumers, interviewed CMS and HHS officials and subject matter expects, and reviewed CMS and HHS planning documents and relevant criteria for effective planning in the federal government. Results obtained from two selected private consumer transparency tools GAO reviewed—websites with health cost or quality information comparing different health care providers—show that some providers are paid thousands of dollars more than others for the same service in the same geographic area, regardless of the quality of such services. For example, the cost for maternity care at selected acute care hospitals in Boston, all of which rated highly on several quality indicators, ranged between $6,834 and $21,554 in July 2014. Transparency tools are most effective if they provide information relevant to consumers and convey information in a way that consumers can readily understand. GAO identified key characteristics of effective transparency tools through a literature review and interviews with experts. The information that is most relevant to consumers relates directly to their personal circumstances, such as information on specific procedures they are considering, and allows them to make meaningful distinctions among providers based on their performance. Characteristics of such relevant information include describing key differences in quality of care and costs, particularly for what consumers are likely to pay out of pocket based on their specific circumstances. In addition, effective transparency tools must take specific steps to make the information they present understandable by consumers. For example, tools must enable consumers to discern patterns by summarizing related information and allowing consumers to customize information to focus on what is most relevant to them. The Centers for Medicare & Medicaid Services (CMS) operates five transparency tools—Nursing Home Compare, Dialysis Facility Compare, Home Health Compare, Hospital Compare and Physician Compare—that are limited in their provision of relevant and understandable cost and quality information for consumers. In particular, GAO found that the tools lack relevant information on cost and provide limited information on key differences in quality of care, which hinders consumers' ability to make meaningful distinctions among providers based on their performance. Because none of the tools contain information on patients' out-of-pocket costs, they do not allow consumers to combine cost and quality information to assess the value of health care services or anticipate the cost of such services in advance. Additionally, GAO found substantial limitations in how the CMS tools present information, such as, in general, not using clear language and symbols, not summarizing and organizing information to highlight patterns, and not enabling consumers to customize how information is presented. CMS, part of the Department of Health and Human Services (HHS), has taken some steps to expand access to cost and quality information for consumers, but has not established procedures or metrics to ensure the information it collects and reports meets consumer needs. Both HHS and CMS have set goals to report on measures that meet consumer needs. However, CMS's process for developing and selecting cost and quality measures for its tools has been heavily influenced by the concerns of providers rather than consumers. Without procedures or metrics focusing on consumer needs, CMS cannot ensure that these efforts will produce cost and quality information that is relevant and understandable to consumers seeking to make meaningful distinctions. GAO recommends that HHS's CMS take steps to improve the information in its transparency tools and develop procedures and metrics to ensure that tools address consumers' needs. HHS concurred with the recommendations and provided technical comments that were incorporated as appropriate.