Employers have been the driving force behind the growing move to compare health care providers and plans on the basis of their performance. These employers have worked both individually and collaboratively with providers, health plans, and government to produce information that will allow them to assess the quality of the care they purchase. Health plans have been publishing reports comparing their performance to their peers or to a national standard. State governments have published comparative information, often focused on specific procedures performed in hospitals. Although the federal government was responsible for the first widespread public disclosure of hospital performance data in 1987, it discontinued this practice in 1993. As a payer of health care services on behalf of Medicare and Medicaid beneficiaries, the Health Care Financing Administration (HCFA) lags behind others in making performance data public. Report cards can include a variety of performance indicators, either structural, process, or outcome based. Structural indicators measure the resources and organizational arrangements in place to deliver care, such as the ratio of nurses to inpatient beds. Process indicators measure the physician and other provider activities carried out to deliver the care, such as the rates of childhood immunization. Outcome indicators measure the results of the physician and other provider activities, such as mortality, morbidity, and customer satisfaction. In 1989, a group of employers initiated one of the most significant efforts to identify uniform and standardized performance indicators. This effort resulted in the creation of a performance measurement system known as the Health Plan Employer Data and Information Set (HEDIS). Several business coalitions and health care organizations used the first HEDIS measures in 1991. The nonprofit National Committee for Quality Assurance (NCQA) has led the effort to revise the measures, issuing HEDIS 2.0 in 1993 and HEDIS 2.5 in 1995. Current HEDIS measures focus on process indicators. (See table 1 for a list of some key HEDIS measures.) Using HEDIS as a base, some employers have begun to distribute to their employees educational materials that include outcome measures. For example, the California Public Employees’ Retirement System (CalPERS) recently distributed to its employees a performance report about the health plans it offers. Although it had furnished some comparative information to its employees in previous years, the information generally featured cost and benefits. CalPERS’ May 1995 Health Plan Quality/Performance Report is its first effort at distributing comprehensive information that includes both specific quality performance indicators and member satisfaction survey results. The quality performance data are based on HEDIS indicators measuring health maintenance organizations’ (HMO) success with providing childhood immunizations, cholesterol screening, prenatal care, cervical and breast cancer screening results, and diabetic eye exams. Employee survey results include employee satisfaction with physician care, hospital care, and the overall plan, and the results of a question asking whether members would recommend the plan to a fellow employee or friend. Some employers are using third-party health care accrediting organizations to measure health plan performance using structural indicators. These employers are requiring the health plans they contract with to be accredited by organizations such as NCQA and the Joint Commission on Accreditation of Healthcare Organizations. Furthermore, some accrediting agencies publicize their accreditation decisions, which allows employers and individual consumers to consider accreditation status in their health care purchasing decisions. For example, a consortium of employers has elected to exclude a Florida HMO from new business with its employer-sponsored health plans because of the HMO’s failure to obtain accreditation. Health plans have published comparative information intended to assist individual consumers in their health care choices and health care providers in their quality improvements. For example, in 1993, Kaiser Permanente Northern California Region released a report on 102 performance measures divided into the following categories: childhood health, maternal care, cardiovascular disease, cancer, common surgical procedures, other adult health, and mental health/substance abuse. (See fig. 1.) Although Kaiser was one of the first health plans to publish this kind of information, an increasing number of health plans are now providing similar information. Health plans have been exploring new ways to make information readily available and understandable to individual consumers. For example, on September 15, 1995, HealthPartners, Inc., will initiate a consumer-oriented program using touch-screen computers. Initially, at least 50 computers will be installed permanently at 50 employer sites, and at least 100 computers will be rotated among other employers. This will allow employees to obtain details about any one of the plans’ primary care sites, such as its physicians’ credentials, on-site services offered, and specialists to which its physicians refer. Because health plan members are expected to enroll in a specific care delivery system—a set of primary care sites with affiliated specialists—HealthPartners will furnish data about each care system to help plan members make a decision about which one to join. Currently these data include preventive screening rates and patient satisfaction measures. HealthPartners anticipates expanding the availability of touch-screen computers to more public spaces, such as shopping malls, after physician concerns about data confidentiality and other matters are resolved. The states have also been active in providing information about provider performance to the public. Forty states have mandated the collection, analysis, and public distribution of health care data, such as hospital use, charges or cost of care, effectiveness of health care, and performance of hospitals. For example, Pennsylvania has released four report cards on the hospitals and physicians in the state performing coronary artery bypass graft surgery (CABG) since 1992. Providing both costs and mortality rates, the reports are publicized through the local media and are available free to consumers. (See fig. 2.) In 1987, HCFA initially publically released hospital mortality information, but did so only in response to a request under the Freedom of Information Act (5 U.S.C. 552). The published information, collected as part of HCFA’s oversight efforts, included the observed and expected mortality rates for Medicare beneficiaries in each hospital that performed CABG surgery. HCFA published the information annually until 1993, when the HCFA Administrator discontinued the reports. He cited problems with the reliability of HCFA’s methods to adjust the data to account for the influence of patient characteristics on the outcomes. HCFA has not published any other information about the performance of Medicare providers. HCFA’s responsibility to Medicare beneficiaries in the selection and oversight of Medicare contract HMOs is similar to that of employers to their employees in selecting health plans. However, HCFA does not routinely provide beneficiaries the results of its monitoring reviews or other performance-related information such as HMO disenrollment rates. In August 1995, we recommended that HCFA publish (1) comparative performance data it collects on HMOs such as complaint rates, disenrollment rates, and rates and outcomes of appeals and (2) the results of its investigations or any findings of noncompliance by HMOs. Our recommendation that HCFA publish performance data was consistent with the views of experts we interviewed about the federal government’s role in ensuring that Medicare beneficiaries receive quality care. These experts cited the need for gathering health plan information such as (1) performance measures, (2) patient satisfaction, and (3) assurances that basic organizational standards have been met. Furthermore, they believed that when the information is obtained, it should be shared with beneficiaries to assist them in their health care purchasing decisions. Although HCFA has not been publishing data on Medicare providers, it is collaborating with others to publish performance information about Medicaid providers. HCFA has been participating with NCQA and the American Public Welfare Association on behalf of the State Medicaid Agencies Directors Group to tailor HEDIS to the particular needs of state Medicaid agencies, health plans that serve Medicaid recipients, and the recipients themselves. In July 1995 the work group released the first draft of Medicaid HEDIS and is expected to release a final version of the document in Fall 1995 after considering comments received. Like HEDIS, many of the most recent initiatives to provide data involve a partnership between private and public players. For example, a more recent public/private initiative that includes some of the major employers involved in developing HEDIS is the Foundation for Accountability (FAcct), created in June 1995. At a meeting of the Jackson Hole Group, some of the nation’s largest employers and HCFA, together representing more than 80 million people, or almost a third of the U.S. population, agreed to combine their expertise and purchasing power. This action grew out of employer frustration with current performance data that focus on plan and provider structure and process rather than outcomes of care. FAcct intends to recommend measures of health care quality that can be easily understood by the general public so that people can make informed decisions when choosing a health plan. FAcct also hopes to encourage the common adoption of these standards to establish uniformity and minimize health plan reporting burdens as well as develop a means of educating diverse audiences about the significance and applications of health plan accountability. Experts have noted that studies performed to determine how consumers make decisions when no comparative information on quality has been available may not be helpful in determining what information consumers would actually use. Adding to the conclusions of numerous researchers that individual consumers give more weight to information from acquaintances than to expert opinion, researchers at Brandeis University reported in 1994 that Massachusetts state employees they surveyed valued information about quality but did not value report card information. From this apparent contradiction, the researchers concluded that survey respondents view quality as something other than what is described in report cards. In 1995, NCQA reported that almost all consumers participating in focus groups NCQA sponsored stated that they would use better evaluative information if it were available to them. In addition, when NCQA provided participants with sample report cards, NCQA noted that in every group, participants were able to critically evaluate the information, raising the same questions about the validity of the data that experts debate. In 1994, we reported that while performance measures or report cards could be a useful tool to educate consumers about the health care that plans provide, the report cards being developed may not reflect the needs of some users. Employers have been the primary users of information comparing quality of care; little is known about the extent to which this information is meeting individual consumers’ needs. The sections that follow discuss in more detail the results of our efforts to determine, from the consumers’ perspectives, the extent to which they use quality of care information in making health care choices and the types of information consumers find useful in arriving at decisions. Many of the employers and individual consumers of health care we talked with are increasingly using information that compares the quality of care furnished by health care providers or health plans to make purchasing decisions and to encourage providers and plans to improve the quality of their care. However, some of those we interviewed told us they are not using the information because they are unaware that it exists, they have not been able to find it in some markets, they believe the available information does not meet their needs, or they lack the resources or time to find and use the information. Further, they stated that the information would be more useful if their concerns about the reliability and validity of the information were addressed. “we’d like to get some kind of value-based decision for purchasing health care. The pure pricing arrangements, the deals . . . have not really been a complete answer for us. Those arrangements don’t address quality, and we’re coming to believe that that’s got to be the cornerstone of your health care plan.” “I think that they [HMOs] should be in the business of comparing hospitals, picking out the high-quality, cost-effective providers; that’s what I’m paying them to do. I just want to make sure they’re doing it, and feel comfortable that they’re doing it.” These employers used the comparative data as a “red flag,” signaling a possible decline in quality. For example, one large southeastern self-insured employer stated that he watched for trends in performance measures that might serve as a warning that a problem was developing. Some smaller employers reported that they had neither the resources nor the time to find or use report cards but wanted the information to be available to the insurance agents or purchasing alliance staff they relied on to make health insurance recommendations. Employers told us that they are also using the data as a tool to market a specific plan to their employees or to negotiate contract terms with the insurance carriers. Numerous employers told us that providing employees with data comparing quality of care was particularly helpful in convincing their workers that managed care plans do not compromise the quality of care provided. Employers stated that they use the data to influence providers and plans to improve quality. For example, one employer told us that during contract negotiations, data were used comparing hospitals on specific procedures, such as hysterectomies, to encourage hospitals to reduce unnecessary surgeries. The individual consumers we talked with in Pennsylvania, California, and Minnesota who had requested and received specific report cards generally used the information and found it to be very helpful in making health care purchasing decisions. These consumers received either (1) information about patient outcomes for physicians and hospitals performing specific procedures or (2) information on a specific plan. More specifically, individual consumers in Pennsylvania and California reported using the procedure-specific reports to select the best surgeon or hospital because they or someone in their family anticipated having the surgery described in the report, select the best surgeon or hospital for procedures other than those described in the report, review the ranking of the surgeon who had performed their surgery before they had obtained the report, ask more informed questions of their doctors, increase their general knowledge, provide advice to others, or satisfy their curiosity. Individual consumers using a plan-specific report card told us that they used the information to select a health plan or to increase their knowledge about the health plan chosen by their employer, such as the services provided or the financial health of the plan. Consumers using either a plan- or procedure-specific report card who had no choice of provider reported that the information gave them reassurance. Although most individual consumers we interviewed found the report card helpful, some did not. Some consumers reported that they did not use the information because it focused on one procedure or health plan, or because it was limited to a specific state or area. Other consumers told us that they were unaware the information existed until after they or a family member no longer needed it. For example, a Pennsylvania woman stated that she wished she had known about this information before her mother died after heart surgery, because it might have helped her select a provider. Both employers and individual consumers echoed many of the same concerns expressed by health care experts and previously reported by us that comparative information may not be measuring what it is intended to measure. Experts have varying beliefs about what information should be included in a report card because of acknowledged difficulties with the reliability and validity of data sources and systems designed to measure quality. Areas of concern for purchasers we interviewed focused on risk adjustment, age of data, subjectivity, and bias. More specifically, consumers, both corporate and individual, questioned whether procedure-specific data were properly adjusted to account for differences in patient characteristics that might contribute to adverse outcomes. They were skeptical about whether factors such as age, severity of condition, and functional status could be accounted for to ensure that outcomes were an accurate reflection of provider quality. We have reported previously that severity-adjusted performance measurement systems are in a relatively early stage of development and may not provide information for accurately comparing hospitals’ performance. We concluded that additional information and methodological improvements are needed to provide more useful data on which to base purchasing decisions. “they are to reassure the public, but they can’t be used to make health care decisions because they are too general and outdated from the time the data was gathered until the decision is made.” Some consumers stated that selecting a health care provider is a subjective decision that is difficult to quantify. In the words of a Pennsylvania consumer, while the report card was a good publication, “it is limited by trying to objectify something that will always be subjective.” For example, consumers differ in what they want from a provider. Some consumers mentioned that it is more important for some patients to feel at ease with their doctor than it is for others. Although many consumers stated that they wanted information on customer satisfaction, others felt it was of limited value because “just because you’re happy with your doctor doesn’t mean I would be happy with him or her.” Another individual consumer questioned a patient’s ability to assess a doctor’s medical knowledge, technical skills, and ability. “Quality is in the eyes of the beholder . . . . It is not appropriate for the employer to place a value on one outcome over another. It is up to the patient to place that value. Is it more important that I be alive but it’s okay if I’m hurt or I’d rather die than be ?” “you know I think that a big part of the problem, and we’re guilty of it too, is imposing our own tastes or beliefs on other people . . . . In health care we do a lot of deciding of what’s good for people on the basis of our own beliefs, and the issues that a $9 an hour person are not the same ones that I’m contending with . . . . The highly paid person may not have any problem in going out of network—may be able to afford to go to Mayo’s [the Mayo clinic] and decide, ‘hey, that’s where I’m going with this problem. I’m not going to stick around .’ Whereas somebody on the shop floor has got to stay in [city deleted].” Individual consumers questioned the objectivity of the health care data produced and distributed by the provider or plan. Many consumers stated they would be less likely to believe the information if it is gathered and reported by the provider or plan rather than an independent third party. For example, one individual stated that “an unscrupulous provider could make sure they hit home runs on all of these particular items [the quality measurements] . . . .” Individual consumers who requested and received report card information from health plans used terms such as “self-serving,” “one-sided,” and “nontrustworthy” to describe the report. These respondents saw the purpose of the reports as a provider’s public relations effort to “blow its own horn” or use the report as a “marketing tool” rather than to provide information to the consumer. Consumers we interviewed want more information than they currently have. Both employers and individual consumers want information that emphasizes outcomes rather than process or structure measures of quality. They want standardized information that allows them to compare providers and plans. Few employers we interviewed are sharing unpublished data with employees, and they differ from one another on whether or not they believe their employees would use it to make decisions. Individual consumers generally stated that they wanted reports on quality to make decisions, but many emphasized that such reports would never be the sole source of information; they would only augment the advice of others. When emphasizing that they want information on the outcome of health care provided, consumers are asking for a measure that allows them to select providers who will improve their health status or that of their employees. For example, in describing the need for outcome data, one employer stated that rather than just knowing how many women received mammography screening for breast cancer, he wanted to know if the number of women who died or were incapacitated from breast cancer was being reduced. A major northeastern food manufacturer used outcomes to relate quality assurance in health care to its manufacturing quality assurance program to explain that “outcome data . . . is the only way to measure quality . . . . Once you have the outcome, you can go back and look at the processes themselves.” A large West Coast employer stated that what the company really wants is information on health status. “What we’d like as a measure is we’d like to know that the plan has improved the health status of the population served . . . . That might be different for some subpopulations. So, moving much more to population-based approaches.” “in general, you’re looking for quality and you’re looking for value, so maybe more of a functional analysis. There is some subjective information that needs to be obtained along with the length of stay and cost of stay and some of these other factors that we’re just not getting yet . . . . You need to do a kind of functional analysis as well, to say 30 days after that angioplasty was that patient back at work, and were they working 40 hours per week, and were they doing their job . . . . How’s your quality of life after you’ve had this?” “the number one thing people ask . . . when they’re considering an HMO . . . is not like gee, ‘Am I going to get that mammogram,’ it’s ‘What if I get sick, am I going to die, are they going to take care of me?’” Both employers and individual consumers stated that although data reflecting the outcomes may be the best measure of provider quality, it is very difficult to know whether outcomes result from quality of care or factors such as the patient’s condition or lifestyle choices. “the government should prescribe some standards and force providers to adhere to these standards in the publishing of information. The government should say, ‘You’re going to code this disease this way, and you do it consistently and uniformly . . . .’” Individual consumers stated that the way the information was presented was very important to them. For example, some wanted to have providers or plans compared side-by-side on one or two pages. Consumers using the procedure-specific reports uniformly praised the table format that provided this kind of direct comparison. (See fig. 2 for an example of a table providing a comparison of providers from a procedure-specific report.) Some individual consumers wanted the information to cover a wider geographic area, and others emphasized the need for community-specific data. For example, some Pennsylvania consumers stated that the report card for that state pertained only to providers in Pennsylvania. Consumers living in Philadelphia would like to have had this type of information for surrounding states because their providers, while close to their homes, were located in other states. The same concern came up in a midwestern city that bordered two states. “hav some report card concepts that the employees could understand the information, user friendly . . . consistent . . . . I want to have a tool for the employees to make that decision. If the employees are making that decision, they are going to change the marketplace. They are going to improve the quality of the system because the doctors and the hospitals are going to have to alter their practices because of the information that has been gathered and is presented and understood by employees. They are then making intelligent decisions as far as where to get their health care . . . . Empower the people to make the decision.” “if we are going to have value-based purchasing which would drive a competitive marketplace in health care, we have to involve consumers who make the ultimate choice. Therefore the information has to be relevant for them.” “I think it’s not speaking to how they make decisions. I think we’d overwhelm them . . . . Also, I don’t know if the data we’d be giving them would be the complete picture.” “I’ve been in health care benefits for 15 years. I don’t know how to make the choice. What happens to poor Harry the Huffer working on the shop floor when you give him . . . the morbidity in this hospital is here, and you know the readmission rate is this, and the reinfection rate is this, and the guy says, ‘I don’t know what I should do.’ Because what they do to our counselors is say, ‘I don’t want to make choices.’” Nevertheless, employees we interviewed disagreed with those employers who said that employees would not use the information. Employees’ concerns included issues of validity and reliability such as risk assessment and accuracy rather than their ability to understand the data. Most of the individual consumers who had requested the published reports found them to be easy to understand, using terms such as “clear,” “concise,” and “well organized.” They found the charts and tables particularly useful. For those who had some problems understanding the reports, additional assistance was useful. For example, a Pennsylvania consumer who had been unable to fully understand the published report on her own had no trouble after it was explained by the state agency that had produced the report. Another Pennsylvania consumer stated that the first report card she received was difficult to understand but that by the third report she received, she found it very useful. Many individual consumers emphasized that published information would never be the sole source of data for their health care decisions but would be used in addition to other information such as personal consultation with their physician, friends, family members, or coworkers. Data comparing health care plans and providers helped the consumers we interviewed make their health care purchasing decisions. However, performance reports have not yet achieved their fullest potential. Consumers said they needed more reliable and valid data, more readily available and standardized information, and a greater emphasis on outcome measures. Meeting the information needs of individual consumers continues to lag behind meeting the employer needs. Attention must be paid to ensuring that individual consumers have access to health care data. While employers themselves have initiated efforts to cooperate with one another, few we interviewed are making complete health care data available to assist individual consumers in making purchasing decisions. Relevant stakeholders have not yet addressed the issues of disseminating performance data to individual consumers so that they can make responsive, informed decisions about their health care coverage. We are sending copies of this report to interested congressional committees and other interested parties. We will make copies available to others on request. This report was prepared under the direction of Carlotta C. Joyner, Associate Director. Other major contributors to this report include Sandra K. Isaacson, Assistant Director; Susan Lawes; Lise Levie; Lesia Mandzia; and Janice Raynor. Please call me on (202) 512-6806, or Dr. Joyner on (202) 512-7002, if you have any questions. To obtain information on how consumers use data comparing the quality of health care providers or health plans and what information they want when making health care purchasing decisions, we interviewed both employers and individual consumers. To obtain the view of employers, we interviewed officials at over 60 businesses. The size of these businesses ranged from under 5 employees to over 100,000 employees. These employers were selected on the basis of the following criteria: (1) size of workforce (small, medium, and large); (2) geographic variability; and (3) variation in whether or not they used published report cards. “Small” employers were defined as those with fewer than 50 employees, “medium” as having 50 to 499, and “large” as 500 or more employees. Because the businesses were not randomly selected, their experiences and opinions cannot be generalized to all employers. We also interviewed a major private sector management consulting firm that supplies comparative health care data to employers. To obtain the views of individual consumers who had received a report card, we conducted telephone interviews during January, February, and March 1995 with 153 consumers who had requested and received published report cards to determine how they used the information (see table I.1). The report cards they received were published by either California or Pennsylvania state agencies or by health maintenance organizations in California and Minnesota. These report cards were selected because they were the most recently available in which the issuing entity had a record of requesters and the state or HMO was willing to assist in the study. The consumers we talked with had received this information sometime during 1993 and 1994. The reports published by the state agencies contained only procedure-specific indicators, while the health plan reports focused on various plan and procedure quality indicators related to the individual health plans. Kaiser Permanente (HMO) Although we attempted to contact all 1,087 individuals who had requested the report cards issued by those states or health plans, many of these individuals did not choose to participate, could not recall receiving the information, or had requested the information for reasons other than making health care purchasing decisions, such as for school or work. Because we spoke with only a small number of individuals who had requested information for consumer-related purposes and they were not chosen at random, their experiences and opinions cannot be generalized to the entire consumer population that requested report card information. We also conducted interviews with seven groups of employees around the country who may not have had previous experience with such reports. We conducted these interviews with employees from four federal government agencies and three private corporations—manufacturing, sales, and service. These employees were selected because their employers offered them more than one health insurance plan to choose from when making their health care insurance purchasing decisions. The number of participants in each group ranged from 8 to 10 and included employees with varying marital, family, and age status as well as employees enrolled in both indemnity and managed care plans. Medicare: Increased HMO Oversight Could Improve Quality and Access to Care (GAO/HEHS-95-155, Aug. 3, 1995). Employer-Based Health Plans: Issues, Trends, and Challenges Posed by ERISA (GAO/HEHS-95-167, July 25, 1995). Medicare: Enhancing Health Care Quality Assurance (GAO/T-HEHS-95-224, July 27, 1995). Health Care: Employers Urge Hospitals to Battle Costs Using Performance Data Systems (GAO/HEHS-95-1, Oct. 3, 1994). Health Care Reform: “Report Cards” Are Useful but Significant Issues Need to Be Addressed (GAO/HEHS-94-219, Sept. 29, 1994). Access to Health Insurance: Public and Private Employers’ Experience With Purchasing Cooperatives (GAO/HEHS-94-142, May 31, 1994). Managed Health Care: Effect on Employers’ Cost Difficult to Measure (GAO/HRD-94-3, Oct. 19, 1993). The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (301) 258-4066, or TDD (301) 413-0006. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (301) 258-4097 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.

Summary: Pursuant to a congressional request, GAO provided information on health care quality issues, focusing on: (1) how consumers use health care performance reports that contain comparative data on the quality of health care providers; and (2) what information consumers consider most important. GAO found that: (1) employers and individuals use information that measures and compares the quality of health care furnished by providers and health plans when making purchasing decisions; (2) consumers want performance reporting efforts to continue and are requesting that more data be made publicly available; (3) consumers want standardized and comparable health care information to assess health care providers' or health plans' performance; (4) many employers get health care performance data through business coalitions, consultants, and their own data collection efforts; and (5) although employers have begun cooperating with one another to enhance their purchasing decisions, few employers make health care data available to their employees.