Medicare: Per Capita Method Can Be Used to Profile Physicians and Provide Feedback on Resource Use
Released: 2009-10-26T00:00:00

Letter

Background

We, MedPAC, and the Congressional Budget Office (CBO) have all suggested that CMS profile physician resource use and provide feedback to physicians as a step toward improving the efficiency of care financed by Medicare. In July 2008, Congress passed MIPPA, which directed the Secretary of HHS to establish a program by January 1, 2009, to provide physicians confidential feedback on the Medicare resources used to provide care to beneficiaries. MIPPA gave HHS the flexibility to measure resource use on a per capita basis, an episode basis, or both. In response to the MIPPA mandate, CMS is pursuing its Physician Resource Use Measurement and Reporting Program. (See table 1.)

Key Decisions in Physician Profiling

When profiling physicians on their resource use, five key decisions must be made: Which resource use measurement methodology to use. There are two main profiling methodologies: per capita and episode-based. Using both types of measures of resource use may provide more meaningful results by more fully capturing the relevant characteristics of a physician’s practice patterns.

How to account for differences in patient health status. Accounting for differences in patient health status, a process sometimes referred to as risk-adjustment, is an important and challenging aspect of physician profiling. Because sicker patients are expected to use more health care resources than healthier patients, we believe the health status of patients must be taken into account to make meaningful comparisons among physicians. There are various risk-adjustment methods and the suitability of a given method will depend on characteristics of the physicians to be profiled and their patients.

How to attribute resource use to physicians. Important attribution decisions include whether to assign a patient’s resource use to the single physician who bears the greatest responsibility for the resource use, to all physicians who bore any responsibility, or to all physicians who met a given threshold of responsibility, such as providing a certain percentage of the expenditures or volume of services. A single attribution approach may not be applicable for all types of measures or for all types of physician specialties.

What benchmark(s) to use. Physician profiling involves comparing physicians’ resource use to a benchmark. There are differing opinions on what are the most appropriate and meaningful comparative benchmarks.

How to determine what is a sufficient sample size to ensure meaningful comparisons. The feasibility of using resource use measures to compare physicians’ performance depends, in part, on two factors: the availability of enough data on each physician to compute a resource use measure and a sufficient number of physicians to provide meaningful comparisons. It is important to calculate resource use measures only for physicians with sufficient sample sizes in order to address concerns that a physician’s profile may be distorted by a few aberrant cases. There is no consensus on what sample size is adequate to ensure meaningful measures.

CMS’s Resource Use Measurement and Reporting Program

Responding to the MIPPA mandate to establish a physician feedback program by January 1, 2009, CMS began in April 2008 to develop its program for reporting to physicians on their resource use. In the first phase of the program, CMS identified eight priority conditions and disseminated approximately 310 Resource Use Reports to physicians in selected specialties who practiced in one of 13 geographic areas. The reports generally included both per capita and episode-based resource use measures that were calculated according to five different attribution rules. The reports also contained multiple cost benchmarks relative to physicians in the same specialty and geographic area. In Phase II, CMS is proposing to expand the program by adding quality measures and reporting on groups of physicians as a mechanism for addressing small sample size issues.

Per Capita Profiling Method Shows Specialist Physicians’ Practice Patterns Relatively Stable Over 2 Years; Patients of High Resource Use Physicians Used More Institutional Services Than Other Patients

Using a per capita profiling method, we found that from 2005 to 2006, specialist physicians showed considerable stability in their practice patterns, as measured by resource use—greater stability than their patients, despite high patient turnover. We also found that our per capita method can differentiate specialists’ patterns of resource use with respect to different types of services, such as institutional services, which were a major factor in beneficiaries’ resource use. In particular, patients of high resource use physicians used more institutional services than patients of low resource use physicians.

Specialist Physicians’ Resource Use More Stable Than Beneficiaries’ Resource Use

Using a per capita method to profile specialist physicians, we found that their practice patterns, as measured by the level of their resource use, was relatively stable over 2005 and 2006 by comparison with individual beneficiaries’ resource use (see figure 1). This is true despite the fact that our measure of physicians’ resource use is derived from their patients’ resource use and that the specific patients whom physicians see are not always the same from year to year. Among the physicians we studied, less than one-third of patients seen by study physicians in 2005 were also seen by the same physician in 2006. This stability suggests that per capita resource use is a reasonable approach for profiling physicians, because it reflects distinct patterns of a physician’s resource use, not the particular population of beneficiaries seen by a physician in a given year.

We divided both physician and beneficiary resource use into five groups of approximately equal size (quintiles) and found that, on average across the four metropolitan areas and four specialties, 58 percent of physicians and 30 percent of beneficiaries were in the same quintile of resource use in 2005 and 2006. The pattern was even more pronounced for the top resource use quintile: 72 percent of physicians and 35 percent of beneficiaries remained in that quintile. If the level of physicians’ and beneficiaries’ resource use was purely random, only 20 percent would be expected to have remained in the same quintile.

We also examined the stability of physicians’ resource use by specialty and found a similar pattern, although not to the same extent in all specialties. The average percentage of physicians who were in the same resource use quintile in 2005 and 2006 ranged from 48 percent for orthopedic surgeons to 60 percent for internists. Resource use in the top quintile was more stable and ranged from 69 percent for diagnostic radiologists to 74 percent for internists. (See table 2.)

In each of the four metropolitan areas, physicians showed greater stability in their resource use than individual beneficiaries, although the percentages varied. For example, the percentage of physicians remaining in the top quintile ranged from 68 percent in Phoenix to 76 percent in Miami. For beneficiaries, the percentage in the top quintile ranged from 31 percent in Phoenix to 39 percent in Miami. (See table 3.)

The greater stability of physicians’ resource use compared to beneficiaries’ resource use could be due to their individual practice styles, as well as to a range of other factors, such as participation in formal or informal referral networks. These networks have a range of providers, including other physicians, who treat their patients and refer them for treatment, testing, and admissions to hospitals.

Beneficiary Use of Institutional Services Varies by Physician Resource Level

Beneficiaries seen by high resource use physicians generally were heavier users of institutional services than those seen by lower resource use physicians, and institutional services accounted for more than one-half of total patient expenditures. This pattern was consistent across three of the four specialties we studied, with orthopedic surgery being the exception.

Institutional services were the major driver of Medicare expenditures for beneficiaries in physicians’ practices, accounting on average for 54 percent of expenditures. Services provided by a particular physician in our study directly to that physician’s patients accounted for only 2 percent of total expenditures or about $350 for each beneficiary in a physician’s practice. All other services—those provided by other physicians, home health care, hospice care, outpatient services, and durable medical equipment—accounted for the remaining 44 percent of expenditures. (See fig. 2.)

Expenditures for institutional services for a physician’s patients grew as the level of physician resource use increased. Dividing the level of physician resource use into quintiles, we examined the relationship of physicians’ resource use and expenditures for services provided to their patients. Average expenditures for institutional services increased more steeply by physician resource quintile than expenditures for all other services.

The four specialties all exhibited this pattern of increasing beneficiary expenditures for institutional services accompanying increasing physician resource use, although for orthopedic surgery the increase was small. The increase in average beneficiary expenditures for all other services that accompanied increasing physician resource use was similar for three of the four specialties and was steeper for internal medicine.

We also examined the average number of physicians seen by the Medicare beneficiaries we studied and found that it was positively associated with increasing physician resource use. Overall, the number of physicians seen increased from an average of about 13 physicians per beneficiary in the lowest quintile of resource use to more than 23 in the highest. The increase in the number of physicians seen was accompanied by an increase in average beneficiary expenditures for institutional services that was steeper than the rise in other services.

Research Literature, Health Insurers, and Specialists Identified Considerations in Developing Physician Feedback Reports on Resource Use

Through our review of selected literature and interviews with officials of health insurance companies, specialty societies, and profiling experts, we identified several key considerations in developing reports to provide feedback to physicians on their performance, including their per capita resource use. We also drew on information from these sources to develop an example of how per capita measures could be presented in a physician feedback report.

Key Considerations in Developing Physician Feedback Reports Include Content, Design, Dissemination Strategy, and Transparency

We identified four key considerations in developing reports to provide feedback to physicians (see table 4).

Our review of selected literature suggested that a physician feedback report should contain three basic elements: an explanation of the information contained in the report (which we will discuss in the context of transparency), measures describing the performance of the physician or physicians to whom the report is directed, and comparative benchmarks.

Measures. Both the selected literature we reviewed and the officials we interviewed supported including measures of quality along with measures of cost, and ensuring that measures are actionable by providing information that can help physicians improve their performance. The officials we interviewed were divided as to whether these measures should reflect physicians’ performance at the individual level or the group level.

Quality measures. All five of the insurers we contacted were profiling physicians in terms of quality and cost, and four of the five had adopted a model code for physician ranking programs that called for rankings to be based on quality as well as cost. Most of the specialty society officials we interviewed also called for the inclusion of quality measures in physician feedback reports, and some cautioned that focusing solely on costs could create perverse incentives—for example, encouraging physicians to reduce inappropriately the level of care provided to patients. The lack of widely accepted, claims-based quality measures for some specialties has limited the number of specialties some insurers profile. For example, at the time of our interview, one insurer was profiling physicians in only one specialty (cardiology) while planning to begin profiling other specialties within a year.

Actionable measures. According to one research report we reviewed, little research has been done to determine how the reporting of global scores—such as an overall per capita cost rank—influences physician behavior, but experts on physician profiling and a broad array of stakeholders, including physicians and insurance company officials, agreed that performance data should be disaggregated into enough categories to enable physicians to identify practice patterns to change. According to some profiling experts, resource use reports must pinpoint physicians’ overuse and misuse of resources, and identify practices that add costs but do not improve desired outcomes. Similarly, specialty society officials we interviewed emphasized the importance of including measures that focus on areas in which the physician has control.

Individual versus group measures. Another measurement consideration is whether physicians in group practices should be profiled as individuals or as a group. The insurers we contacted took varying approaches. In some cases, the approach was driven by contracting arrangements, with insurers constructing group profiles for physicians with whom they had group contracts. One insurance company official pointed out that profiling at the group level allows more physicians to be profiled, as it increases the data available to construct a profile. Another official advocated profiling at the individual level because he believes physicians are more interested in assessments of individual performance. Officials of the four specialty societies generally saw some merit to both approaches, but some underscored the difficulty of identifying group affiliations or noted that groups are not necessarily homogeneous enough for a group assessment to be appropriate.

Comparative benchmarks. One consideration addressed by multiple publications we reviewed was the kind of benchmark to which physicians’ performance should be compared. For example, a physician’s performance may be compared to (1) an evidence-based standard, (2) a standard based on professional judgment, such as the consensus standards endorsed by the National Quality Forum, or (3) to a statistical norm, such as the average for a physician’s peers locally or nationally.

Although studies we reviewed offered conflicting evidence as to whether including peer comparisons in physician feedback reports increases their effectiveness, some profiling experts and specialty society officials believe comparative information is useful and of interest to physicians. In the literature we reviewed, for example, one profiling expert suggested that such comparisons can motivate behavior change by taking advantage of physicians’ desire to perform at least as well as their peers; another stated that performance statistics are not meaningful to physicians without peer comparisons.

A physician’s peer group can be defined in various ways. According to one study, some organizations that provide performance feedback to physicians have found comparisons within specialty and locality most useful to and most frequently requested by physicians. Representatives of some of these organizations said physicians find local information more relevant because it reflects the practice patterns of their geographic area. All five insurers we contacted compare physicians to others in the same market and specialty; one of the five also compares physicians to peers nationwide on some measures. In contrast, officials of all four specialty societies recommended comparisons at the national level, with officials of one society stating that there is no scientific basis for regional variations in practice patterns. There was less agreement about whether physicians should be compared to others in their specialty or to a more narrowly defined group. Officials of one specialty society advocated comparisons at the subspecialty level in recognition of the variation in resource use patterns among subspecialists. Another official pointed out that such comparison groups could be difficult to define because physicians in some specialties tend to have multiple subspecialties. Because views differ on appropriate comparison groups, one hospital-owned healthcare alliance plans to incorporate in its physician reports a customizable feature that will allow users to select the peer comparison they wish to see.

Comparisons to physicians’ own past performance (trend data) are commonly presented in feedback reports, and the majority of physicians surveyed in one study found these comparisons useful.

Report Design

The selected literature we reviewed offered little hard evidence on how feedback reports should be designed to engage physicians’ interest or to prove their comprehension of the material. However, researchers and profiling experts offered some comments and suggestions based either on their experience with clinical performance measurement or on an analysis of the literature on consumer behavior and its possible implications for physician reporting (see table 5).

The amount and combination of material that should be included in a single report is an important consideration. According to one publication that summarized a review of multiple feedback reports, some organizations issue separate reports on efficiency/cost and effectiveness/clinical quality, in part to avoid diluting the impact of either set of measures. Others believe a single report gives physicians a more complete picture of their performance.

Officials of the three insurers we contacted that routinely issued feedback reports to physicians said that their companies produced summary reports, typically one to two pages in length, containing high-level information, but also made more detailed information, such as patient- level data, available to physicians. One insurer’s summary report consisted of one page of cost efficiency measures and one page of effectiveness measures. The cost efficiency page presented average cost per episode of care by service category for the physician and the physician’s peer group, as well as the ratio of the two, in both tabular and graphic form. The effectiveness page presented process-of-care measures for selected conditions, including cardiovascular disease and asthma. Company officials said summary reports were limited to two pages to accommodate physicians’ attention spans and that the two sets of measures were presented separately to discourage attempts to link the two. Specialty society officials agreed reports should be short—most proposed one to two pages—and strongly recommended that information be presented graphically to the extent possible. One official, noting that physicians are very visually oriented, recommended feedback reports consisting mainly of easily understood graphics.

The selected literature we reviewed, our interviews with specialty society officials, and existing physician feedback reports suggested reports can be kept short by segmenting some information into separate documents—for example, a cover letter that explains the report’s purpose, a description of the profiling methodology, a set of frequently asked questions, and a list of definitions.

Report Dissemination

Some key considerations with respect to report dissemination are which physicians should receive reports, how frequently to issue reports, and whether to issue reports in hardcopy or electronically.

Which physicians should receive feedback reports. One major decision is whether to issue reports to all physicians for whom performance measures can be calculated or only to a subset who fail to meet certain performance standards—a decision that may involve weighing reporting costs against potential impacts. None of the studies we reviewed directly addressed this issue, but all of the specialty society officials we interviewed advised sending reports to all or nearly all physicians, rather than just to poor performers. They gave several reasons: to provide positive recognition to physicians who are performing well; to avoid singling out certain physicians as poor performers, especially on the basis of excess costs over which they have little control; and to create opportunities for voluntary peer-to-peer learning among physicians who are at different points along the performance spectrum. Similarly, all three of the insurers that routinely issued feedback reports sent them to all physicians for whom they had performance measures.

Frequency of reporting. According to one book we reviewed, organizations that provide feedback to physicians should do so more than once a year to give physicians an opportunity to improve their performance in a timely manner. However, because of the time needed to gather sufficient data to identify trends and patterns of performance, many organizations provide feedback no more than twice a year. Of the two insurers that told us how frequently they issued feedback reports, one did so annually and the other at least every 6 months. Officials of the latter company said the frequency of their reporting was limited by the number of claims in their dataset and suggested that CMS would not face the same limitations.

Hardcopy versus electronic dissemination. Reports can be disseminated in hardcopy through various channels, such as the mail, or electronically, through e-mail or a Web site. One literature scan we reviewed cited certain advantages of electronic formats such as Web- based applications. Specifically, they allow users to organize information as they choose and are well suited to presenting data from the general to the specific, which facilitates information processing. Although this report noted some concerns about physicians’ access to the Internet, according to a report based on a national survey of physicians in December 2002 and January 2003, almost all respondents said they had Internet access, and most said they considered it important for patient care.

Of the three insurers that routinely issued feedback reports, two issued them electronically and one issued them in hardcopy. Officials of the latter company said that staff typically hand-delivered the reports to physicians during on-site visits in order to discuss the results. Officials of most of the specialty societies we contacted did not advocate one dissemination mode over the other, but some noted that organizations that issue reports electronically must confront certain challenges, such as ensuring that security features do not make access difficult, addressing the lack of high- speed Internet service in some areas, and determining whether to send reports by e-mail or to instruct physicians to access them on the Internet. One specialty society official recommended using both modes of dissemination to accommodate different preferences.

Transparency

Both the selected literature we reviewed and our interviews with officials from insurance companies and specialty societies underscored the importance of ensuring transparency regarding the purpose of the report and the methodology and data used to construct performance measures.

Purpose. According to one literature scan, feedback reports should explicitly state their purpose—for example, to reduce costs, improve quality, or simply to provide information—and should highlight any items for which the physician will be held accountable.

Methodology. Two important considerations are where to provide information about methodology—whether in the report itself or through some other mechanism, such as a Web page—and how much technical detail to provide. Some of the insurers we contacted provide information on-line about their profiling methodologies, including details about measures, attribution of care to physicians, risk adjustment, and statistical issues. In addition, some of the officials we interviewed said that company staff will meet with physicians to explain the profiling methodology, if requested. For example, officials of one company said that it has on staff four profiling experts, mostly nurses, in addition to about 20 medical directors who can answer physicians’ questions.

Specialty society officials we interviewed highlighted a potential trade-off between providing enough information in the report to persuade physicians of the validity of the measures and keeping the report concise enough to maintain physicians’ interest. All of the officials we interviewed agreed that physicians should have access to details about the methodology; some suggested this information might best be disseminated through a Web site. Explaining how the data are risk-adjusted to account for differences in physicians’ patient populations was cited by specialty society officials as particularly important.

Data. Another consideration is ensuring transparency with regard to the data used in profiling—making patient-level detail available so physicians can reconcile performance measures with their own information about their practices. All five of the health insurers we contacted provided opportunities for physicians to examine patient-level data and file appeals before results are made public, although their processes or policies for doing so varied (see table 6).

Officials of one of the two insurers that made detailed data available on- line said their company previously sent hardcopy reports to physicians, but learned from medical office managers that they would prefer an on- line format that could be manipulated to facilitate physician comparisons. Officials of the other insurer said that their company planned to make the data available in a manipulatable format soon. Most of the specialty society officials we interviewed agreed that patient-level data should be made available to physicians, but some predicted that few physicians would access them. Two interviewees suggested practice size would probably be a factor; one added that physicians in smaller groups would likely lack the resources and skills to analyze the data.

Per Capita Measures Can Be Presented in a Physician Feedback Report

Drawing upon lessons culled from the literature and our interviews, we developed a mock report that illustrates how per capita measures could be included in a physician feedback report. Such a report could also include other measures such as quality measures and episode-based resource use measures. We included two types of per capita measures—risk-adjusted cost ranks and risk-adjusted utilization rates—each presented with local and national comparative benchmarks. To provide further context, we also included per capita measures showing how the average Medicare costs of patients the physician treated at least once were distributed among service categories, and the percentage of those costs that were for services directly provided by the physician to whom the report is directed. We kept the mock report under two pages and included minimal text, while ensuring transparency by indicating the availability of methodology details and supporting data. To accommodate physicians’ differing dissemination preferences, we designed the mock report to be available in both electronic and hardcopy formats. (See fig. 3.)

Specialty society officials who vetted a draft of the mock report made several recommendations. Some recommendations centered on taking advantage of electronic capabilities, such as adding hovers to define key terms (see fig. 4), creating interactive features to let physicians explore “what if” scenarios, and including links to educational materials and specialty guidelines. Officials also recommended adding information on pharmaceutical costs, a category we did not include because not all beneficiaries are enrolled in a Medicare Part D prescription drug plan.

A patient’s risk adjusted cost rank is calculated by comparing the patient’s Medicare costs to all other Cityville patients with similar risk scores and represents how unexpectedly expensive or inexpensive the patient’s Medicare-covered care was. Your rank is the average rank of all patients you treated at least once. See Glossary for more details.

All providers: All providers: 100% of total 100% of total ($13,422) ($13,422)

You: You: 11% of category 11% of category ($1,449) ($1,449)

More generally, specialty society officials said that they particularly liked the graphs and charts in our mock report. One official added that our report was easier to understand than other reports he had seen and that he thought it would get physicians’ attention. Another official commented how the presented per capita measures could give physicians insight on the care their patients are receiving that they were not previously aware of—a perspective other cost measures could not provide. However, multiple officials said the measures as presented were too broad to be actionable and might not seem relevant to physicians, as most physicians feel responsible only for the costs of services they directly order or provide, not for the total cost of patients’ care. Two officials suggested that these per capita measures would have more value in health care systems that emphasized coordination of care.

Potential Influence of Feedback Regarding Medicare Costs on Physician Behavior Is Uncertain

Our review of available literature on the effectiveness of physician feedback suggests that feedback alone generally has no more than a moderate influence on physician behavior. However, the potential influence of feedback from CMS regarding Medicare costs is uncertain, and may be greater than that of feedback from other sources, because Medicare reimbursement typically represents a larger share of physicians’ practice revenues than that from other insurers.

In general, studies examining the effect of feedback on physicians’ behavior have found it to have a small to moderate effect. Factors that appear to influence the effectiveness of feedback include its source, frequency, and intensity. For example, one review of the literature concluded that physicians were more likely to be influenced by reports from a source they expected to continue monitoring their performance. This review also found that repeated feedback over a period of several years may be more likely to get physicians’ attention. Another review reported that the intensity of the feedback appeared to influence its effectiveness. The review cited individual, written feedback containing information about costs or numbers of tests, but no personal incentives, as among the least intensive, and therefore likely to be among the least effective approaches.

Consistent with the literature we reviewed, most of the insurance company officials we interviewed questioned whether providing performance feedback to physicians would have a significant impact on the physicians’ behavior in the absence of other incentives. While all five insurers profiled physicians, none used the results solely to provide feedback. Officials of four of the five insurance companies said that to affect physicians’ behavior, profiling results must be made public, thus influencing patients’ choice of physicians, or linked to monetary incentives, as in pay-for-performance arrangements. However, officials of one company disagreed, stating that feedback alone can affect physicians’ behavior if the reports show how they rank against their peers and make clear what behavior they need to change to improve their efficiency. These officials also said that the impact of feedback could depend on the size of physicians’ practices and whether they have the resources to review the reports and the management structure to affect changes.

Whether the experiences of private insurers or the lessons from the literature on the influence of feedback will hold in the case of the Medicare program is uncertain. A survey conducted in 2004-2005 found that, for most physicians, Medicare represented more than one-quarter of practice revenue, and for 17 percent of physicians, the proportion was more than one-half. Because physicians typically contract with a dozen or more health insurance plans, few, if any, of these plans are likely t o represent as large a share of physicians’ practice revenue as Medicare. Hence, the impact of feedback from CMS might be greater than that from other sources. In addition, one profiling expert suggested that physicians might expect feedback from CMS to be only the first step in efforts to influence physicians’ behavior—to be followed, for example, by public reporting of profiling results. This perspective comports with recommendations in our earlier report. providing feedback on a confidential basis would be an appropriate first step. One said it would allow time to test the profiling methodology and gauge physicians’ reactions; the other said it would provide an opportunity for physicians to vet the measures and identify any errors.

See GAO-07-307. suggestions for enhancing its effectiveness. Other suggestions can be drawn from the literature we reviewed. These suggestions included: providing advance notice of feedback reports (through presentations, letters, or other communications) to help ensure that physicians open and read the reports; working through credible intermediaries, such as medical societies or locally prominent physicians, to assure physicians that the feedback process is reasonable and legitimate; providing opportunities for physicians to discuss the reports through videoconferences, teleconferences, or on-line discussion groups; and offering in-person follow up, possibly drawing on the resources of the Medicare Quality Improvement Organizations.

Involving physicians in the development of a feedback system may also enhance its effectiveness. One literature scan concluded that physician involvement in system design was vital for obtaining physician buy-in. Information from insurers suggested that, although physicians may not always be involved in initial development of feedback systems, their feedback can prompt modifications. Some insurance officials we interviewed described an iterative process involving ongoing communication with physicians and continuous modification of reports and systems. For example, officials of one insurance company said that the company did not seek initial input from physicians—in the belief that they would not have been able to provide much input without a complete understanding of the data and methodology—but took into account physicians’ responses to earlier, less formal systems. Officials of other companies described various mechanisms for obtaining physicians’ perspectives, including formal physician advisory councils, regular meetings with officials of national medical societies, and town hall meetings with physicians at the local level.

Concluding Observations

Profiling physicians to improve efficiency is used by some private insurance companies and, at the direction of Congress, is being adopted by the Medicare program. We believe that a per capita methodology is a useful approach to profiling physicians on their practice efficiency and could be part of a feedback program that could also include quality measures and episode-based resource use measures.

Our findings are consistent with those of our previous report on physician profiling in which, through analysis of physician practice patterns, we determined that CMS could use profiling to improve the efficiency of Medicare. Despite a more diverse mix of physician specialties in our present analysis, and with certain exceptions noted in our findings, we found substantial consistency in certain patterns we observed across metropolitan areas and specialties. We also found consistency across time in that physicians who showed high resource use in one year tended to stay high in the subsequent year.

Agency and Professional Association Comments and Our Evaluation

We provided a draft of this report to the HHS for comment and received written comments from CMS, which are reprinted in appendix II. We also solicited comments on the draft report from representatives of the American Academy of Orthopaedic Surgeons (AAOS), the American College of Cardiology (ACC), the American College of Physicians, and the American College of Radiology. We received oral comments from the first two.

CMS Comments

Our draft report did not include any recommendations for CMS to respond to. CMS broadly agreed with each of our three findings: CMS agreed that the per capita methodology is a useful approach to measuring physicians’ resource use and noted that per capita measurement is one of the cost of care measures included in CMS’s Physician Resource Use Management and Reporting Program. CMS also agreed that the consistency of our per capita measure across years is an important finding and stated that the agency intends to examine measure consistency in the ongoing administration of its program.

CMS found the attention in our report to considerations for developing a physician feedback system to be particularly helpful. CMS listed several examples of how its program already addresses many of these considerations and is in the process of addressing others. We agree with CMS that some of the approaches described in our report would require significant resources and recognize that CMS will need to investigate how to balance the trade-offs between different approaches in order to best leverage its resources.

CMS agreed that physician feedback may have a moderate influence on physician behavior. CMS further stated its commitment to developing meaningful, actionable, and fair measurement tools for physician resource use that, along with quality measures, will provide a comprehensive assessment of performance. We continue to believe that providing physicians feedback on their performance could be a promising step toward encouraging greater efficiency in Medicare; however, we are still concerned that efforts to achieve greater efficiency that rely solely on physician feedback without financial or other incentives will be suboptimal.

CMS also provided technical comments, which we incorporated as appropriate.

The representatives of AAOS and ACC raised no major issues with regard to the substance of the report. The AAOS representative said that the report captured well the key aspects of physician profiling and the key considerations in developing physician feedback reports. The ACC representatives endorsed the overall approach of a feedback report consisting of a high-level summary accompanied by additional sections with greater detail and a separate document that explains the methodology in detail. The representatives of both groups said that physicians should be provided feedback on both quality and resource use, but differed on whether they should be presented in the same report. Both groups also stressed that physicians should only be compared to physicians within their specialty or subspecialty.

Both the AAOS and the ACC representatives commented on the design of our mock report. Both said that the measures of physician resource use by type of service and the benchmark comparisons were easy to understand. They had difficulty, however, in understanding a related measure that shows the physician’s share of payments by service category. We did not alter our mock report in response to these comments, but believe that the concerns they expressed should be taken into account by organizations designing physician feedback reports.

The representatives of both groups stressed the importance of risk adjustment in the measurement of physician resource use and suggested that we include a fuller explanation of risk adjustment techniques in our report. We did not expand our explanation of such techniques because they are not the focus of this report; however, we acknowledge the important role played by risk adjustment techniques in constructing physician feedback reports on resource use.

As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies of this report to the Acting Administrator of CMS, committees, and others. The report will also be available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff have any questions, please contact me at (202) 512-7114 or steinwalda@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of the report. GAO staff who made major contributions to this report are listed in appendix III.

Appendix I: Methodology

This appendix describes the per capita methodology that we used to measure beneficiaries’ and physicians’ Medicare fee-for-service (FFS) resource use. We focused our analysis on four diverse specialties: a medical specialty (cardiology), a diagnostic specialty (diagnostic radiology), a primary care specialty (internal medicine), and a surgical specialty (orthopedic surgery). We included diagnostic radiologists in our study because they are less amenable to episode grouping, the major alternative to per capita profiling of physicians. We limited our analysis to physicians in these specialties who practiced in one of four areas: Miami, Fla.; Phoenix, Ariz.; Pittsburgh, Pa.; and Sacramento, Calif. We chose these areas for their geographic diversity, range in average Medicare spending per beneficiary, and number of physicians in each of the four specialties. Our results apply only to the four specialties in the four metropolitan areas we studied.

To conduct our analysis, we obtained 2005 and 2006 Centers for Medicare & Medicaid Services (CMS) data from the following sources: (1) Medicare claims files that include data on physician, durable medical equipment, skilled nursing, home health, hospice, and hospital inpatient and outpatient services; (2) Denominator File, a database that contains enrollment and entitlement status information for all Medicare beneficiaries in a given year; (3) Hierarchical Condition Category (HCC) files that summarize Medicare beneficiaries’ diagnoses; (4) files summarizing the institutional status of beneficiaries; and (5) Unique Physician Identification Number Directory, which contains information on physicians’ specialties.

Adjustment for Differences in Patient Health Status

In order to develop a resource use measure that accounts for differences in health status between beneficiaries, we developed a risk adjustment model that uses an individual’s diagnoses during the year to estimate the total Medicare FFS expenditures expected for the individual in that year. As our inputs to the model, we used the same 70 HCCs as those in the model CMS uses to set managed care capitation rates. HCCs are a way of summarizing an individual’s diagnoses into major medical conditions, such as vascular disease or severe head injury. To estimate our model, we used HCC and expenditure data for 2005 and 2006 five percent national samples of Medicare FFS beneficiaries.

Methodology Used to Determine Beneficiaries’ Resource Use

For all Medicare FFS beneficiaries who received at least one service in 2005 or 2006 from a physician located in any of our four metropolitan areas and who also did not meet our exclusion criteria (see footnote 5), we used our risk adjustment model to estimate their total expected Medicare FFS expenditures. Based on their expected expenditures, we placed beneficiaries into 1 of 25 discrete risk categories. The categories were ordered in terms of health status from healthiest (category 1) to sickest (category 25). Next, within each risk category and metropolitan area, we ranked beneficiaries from 1 to 100 by their total actual annual Medicare expenditures, such that the average beneficiary in a given risk category and metropolitan area had a rank of 50. We used this rank as our risk-adjusted measure of beneficiary resource use.

To examine the stability of beneficiaries’ resource use, we divided the 2005 and 2006 beneficiary populations into five ascending groups of nearly equal size (quintiles) based on the level of their resource use. We then identified beneficiaries in each of the four metropolitan areas who saw a physician in their area in 2005 and again in 2006. We measured the stability of beneficiaries’ resource use as the percentage of beneficiaries who remained in the same quintile in 2006 that they were in during 2005. In addition, we determined the percentage of beneficiaries who remained in the highest resource quintile.

Methodology Used to Determine and Compare Physicians’ Resource Use

For the purposes of this study, we defined a physician’s practice as all Medicare FFS beneficiaries who did not meet our exclusion criteria and who had at least one evaluation and management visit with the physician during the calendar year for cardiologists, internists, and orthopedic surgeons, or who received any service from the physician for diagnostic radiologists. To ensure that a physician’s resource use measure would not be overly influenced by a few patients with unusually high or low Medicare expenditures, we excluded physicians with small practices— those who treated fewer than 100 of the Medicare patients in our study during the year. For all physicians, we calculated the average beneficiary resource use rank of the patients in their practices, which ranged from a low of 26.0 to a high of 91.8 in 2006. Next, within each metropolitan area and specialty, we ranked physicians on the basis of this average from 1 to 100 such that the average measure of physician resource use was 50. We used this rank as our measure of physician resource use. This measure reflects how expensive a physician’s patients are compared to the patients of other physicians in the same specialty and area after adjusting for differences in patient health status. For example, a cardiologist in Miami is only compared to other cardiologists in Miami.

To examine physicians’ resource use, we divided the physicians into five ascending groups (quintiles) of nearly equal size based on the measure of their resource use described above. In the same manner as we measured the stability of beneficiaries’ resource use, we measured the stability of physicians’ resource use by determining the percentage of them who remained in the same physician resource use quintile from 2005 to 2006. We also measured the degree of turnover in the patients seen by physicians by computing the percentage of patients seen in 2005 by each physician that were also seen by the same physician in 2006.

We examined utilization patterns by physician resource use quintile by decomposing the 2006 Medicare expenditures of physicians’ patients into those for institutional services (inpatient hospital and skilled nursing care), those for services provided directly by the physician to his or her patients, and those for all other services—outpatient hospital, home health care, hospice care, durable medical equipment, and all other Part B services of Part B providers and suppliers. We also measured the number of physicians seen by a physicians’ patients by physician resource use quintile.

Although our measure of a beneficiary’s resource use is independent of the beneficiary’s health status, there was an association between physician resource use and the mix of healthy and sick patients in physicians’ practices—physicians who ranked high in terms of resource use also treated a larger proportion of beneficiaries who were in poor health than did physicians who ranked low in resource use. However, the resource use of all their patients was also consistently higher than that of low resource use physicians’ patients regardless of patient health status. Figure 5 shows the average resource use of beneficiaries in five health status categories across the five physician resource use quintiles. For example, patients in the healthiest category who were treated by physicians in the highest resource use quintile had an average resource use rank of 74, whereas similarly healthy patients treated by physicians in the lowest quintile had average resource use rank of 53. This ordering of the differences in patient resource use by the level of physician resource use is repeated across all health categories. It indicates that physicians have consistent patterns of resource use with respect to all of their patients, regardless of their patients’ health status.

The mix of healthy and sick patients in physicians’ practices did not affect the positive relationship we found between average institutional expenditures per beneficiary and physician resource use level. Within each beneficiary health category, the patients of high resource use physicians had average institutional expenditures that exceeded those of the patients of physicians with lower resource use. Similar analyses showed that patient mix did not affect (1) the positive relationship between physicians’ resource use and the average number of physicians seen by their patients, (2) the positive relationship between physicians’ resource use and expenditures for all other services provided their patients, and (3) the steeper rise in the use of institutional services by physicians’ patients with increasing physician resource use as compared to the rise in the use of all other services.

Appendix II: Comments from the Centers for Medicare & Medicaid Services

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

A. Bruce Steinwald, (202) 512-7114, or steinwalda@gao.gov.

Staff Acknowledgments

In addition to the contact named above, Phyllis Thorburn, Assistant Director; Alison Binkowski; Nancy Fasciano; Richard Lipinski; Drew Long; Jessica Smith; Maya Tholandi; and Eric Wedum made key contributions to this report.

Bibliography

Balas, E. Andrew, Suzanne Austin Boren, Gordon D. Brown, Bernard G. Ewigman, Joyce A. Mitchell, and Gerald T. Perkoff. “Effect of Physician Profiling on Utilization: Meta-analysis of Randomized Clinical Trials.” Journal of General Internal Medicine, vol. 11, no. 10 (1996): 584-590.

Beckman, Howard B., Anthony L. Suchman, Kathleen Curtin, and Robert A. Greene. “Physician Reactions to Quantitative Individual Performance Reports.” American Journal of Medical Quality, vol. 21, no. 3 (2006): 192- 199.

Beckman, Howard B., Thomas Mahoney, and Robert A. Greene. Current Approaches to Improving the Value of Care: A Physician’s Perspective. The Commonwealth Fund. November 2007.

Hartig, J.R., and Jeroan J. Allison. “Physician Performance Improvement: An Overview of Methodologies.” Clinical and Experimental Rheumatology, vol. 25, supplement 47 (2007): S50-S54.

Jamtvedt, Gro, Jane M. Young, Doris Tove Kristoffersen, Mary Ann O’Brien, and Andrew D. Oxman. “Audit and Feedback: Effects on Professional Practice and Health Care Outcomes.” Cochrane Database of Systematic Reviews, no. 2 (2006).

Jamtvedt, Gro, Jane M. Young, Doris Tove Kristoffersen, Mary Ann O’Brien, and Andrew D. Oxman. “Does Telling People What They Have Been Doing Change What They Do? A Systematic Review of the Effects of Audit and Feedback.” Quality & Safety in Health Care, vol. 15, no. 6 (2006): 433-436.

Kiefe, Catarina I., Jeroan J. Allison, O. Dale Williams, Sharina D. Person, Michael T. Weaver, and Norman W. Weissman. “Improving Quality Improvement Using Achievable Benchmarks for Physician Feedback: A Randomized Controlled Trial.” JAMA, vol. 285, no. 22 (2001): 2871-2879.

Marder, Robert J., Mark A. Smith, and Richard A. Sheff. “Changing Physician Practice: Providing Physicians Useful Feedback.” In Effective Peer Review: A Practical Guide to Contemporary Design, 2nd ed., 153- 164. Marblehead, Mass.: HCPro, Inc., 2007.

Micklitsch, Christine N. and Theresa A. Ryan-Mitlyng. Physician Performance Management: Tool for Survival and Success. Englewood, Colo.: Medical Group Management Association, 1996.

Mold, James W., Cheryl A. Aspy, and Zsolt Nagykaldi. “Implementation of Evidence-Based Preventive Services Delivery Processes in Primary Care: An Oklahoma Physicians Resource/Research Network (OKPRN) Study.” Journal of the American Board of Family Medicine, vol. 21, no. 4 (2008): 334-344.

Nathanson, Philip. “Influencing Physician Practice Patterns,” Topics in Health Care Financing, vol. 20, no. 4 (1994): 16-25.

Pacific Business Group on Health. Advancing Physician Performance Measurement: Using Administrative Data to Assess Physician Quality and Efficiency. September 2005.

Paxton, E. Scott, Barton H. Hamilton, Vivian R. Boyd, and Bruce L. Hall. “Impact of Isolated Clinical Performance Feedback on Clinical Productivity of an Academic Surgical Faculty.” Journal of American College of Surgeons, vol. 202, no. 5 (2006): 737-745.

Teleki, Stephanie S., Rebecca Shaw, Cheryl L. Damberg, and Elizabeth A. McGlynn. Providing Performance Feedback to Individual Physicians: Current Practice and Emerging Lessons. RAND Health Working Paper Series. July 2006.

Van Hoof, Thomas J., David A. Pearson, Tierney E. Giannotti, Janet P. Tate, Anne Elwell, Judith K. Barr, and Thomas P. Meehan. “Lessons Learned from Performance Feedback by a Quality Improvement Organization.” Journal for Healthcare Quality, vol. 28, no. 3 (2006): 20-31.

Veloski, Jon, James R. Boex, Margaret J. Grasberger, Adam Evans, and Daniel B. Wolfson. “Systematic Review of the Literature on Assessment, Feedback and Physicians’ Clinical Performance: BEME Guide No. 7.” Medical Teacher, vol. 28, no. 2 (2006): 117-128.