Physician groups with at least 200 physicians were eligible to apply for the PGP Demonstration and 10 were selected by CMS. (See table 1.) CMS’s technical review panel evaluated each applicant based on its organizational structure, operational feasibility, geographic location, and demonstration implementation strategy. Collectively, the 10 participating physician groups are all multispecialty practices comprising more than 6,000 physicians who provide care for more than 220,000 Medicare FFS beneficiaries. While all the participants have at least 200 physicians, group practice size varies widely, ranging from 232 to 1,291 physicians. Except for the Marshfield Clinic, all participants identified themselves as integrated delivery systems that include, in addition to their group practice, other health care entities such as hospitals, surgical centers, or laboratories. Nearly all of the participants have nonprofit tax status, except for the Everett Clinic and the Integrated Resources for the Middlesex Area (IRMA), which are for profit. Overall, a majority of the 10 participants are located in small cities and serve either predominantly rural or suburban areas. These participants provide care over wide geographic areas by using satellite physician group office locations, ranging from 10 to 65 physician group office locations. Under the PGP Demonstration’s design, participating physician groups are eligible to earn annual cost-savings bonuses for generating Medicare program savings. Participants that received cost-savings bonuses were also eligible to receive additional bonuses for meeting certain quality targets. Both the cost-savings and quality bonuses are in addition to payments physicians receive under Medicare FFS. There are three main steps in CMS’s bonus payment methodology to determine which participants are awarded bonus payments and the amount of these bonuses: (1) determination of eligibility for performance bonus payments, (2) determination of the size of the bonus pool, and (3) determination of actual bonus payments earned. (See fig. 1.) For the first step of the bonus payment methodology, to determine eligibility for receiving bonus payments, participating physician groups had to generate savings greater than 2 percent of their target expenditure amounts, relative to a comparison group of beneficiaries intended to have similar characteristics. CMS stated that the purpose of the 2 percent savings threshold was to further account for the possibility of random fluctuations in expenditures rather than actual savings. CMS also stated that it used separate comparison groups for each of the participants to distinguish the effect of the demonstration’s incentive payments from trends among Medicare beneficiaries unrelated to the demonstration. Operationally, Medicare beneficiaries were assigned to the comparison groups or to the participating physician groups retrospectively at the conclusion of each performance year, using Medicare claims data sent to CMS by providers following the delivery of care. As a part of the process of selecting beneficiaries for each comparison group that were similar to those served by the participating physician group they were being compared with, CMS ensured that beneficiaries (1) resided in the same geographic service areas as the beneficiaries assigned to the corresponding physician group; (2) had received at least one office or outpatient service, referred to as an evaluation and management (E&M) service, in that performance year; and (3) had not received any E&M services from the corresponding physician group that year or had been assigned to the participant’s group of beneficiaries in any previous performance year. For step two, determining the size of the bonus pools, participating physician groups that generated savings beyond the 2 percent threshold were eligible to receive up to 80 percent of those savings as potential bonuses. The remaining 20 percent, and all other savings not awarded to the participants, were retained by the Medicare program. In the third step, the determination of actual bonus amounts earned, eligible participating physician groups could receive up to the full amount available in their bonus pools as cost-savings bonuses and quality-of-care bonuses. Specifically, for PY1, 70 percent of the bonus pool was awarded as a cost-savings bonus to participants who met the 2 percent cost-savings threshold, and up to 30 percent of the bonus pool could have been awarded as a quality-of-care bonus, for those who met the cost-savings threshold. The quality-of-care bonus was awarded to participants that met or exceeded various quality-of-care targets within an area of clinical focus selected by CMS, in collaboration with other organizations and the participating physician groups. In PY1, CMS focused on diabetes management, and required participants to meet targets on a set of 10 diabetes measures, including whether a beneficiary received an eye exam or foot exam. To meet the quality-of-care target for each of the diabetes measures, a participant had to either improve its performance by a certain amount relative to its baseline performance or meet a national set of performance measures, referred to as HEDIS® measures, established by the National Committee for Quality Assurance (NCQA). Participants could also receive a prorated share of the quality-of-care bonus, based on success meeting some, but not all, of the quality-of-care targets. While the bonus payment methodology will remain the same throughout the demonstration, CMS added other quality-of-care measures and increased the relative significance of the quality-of-care measures in PY2 and PY3. In PY2, quality-of-care measures pertaining to CHF and coronary artery disease (CAD) were added to the existing diabetes measures. In PY3, quality-of-care measures pertaining to the management of hypertension and screening for breast and colorectal cancer were added to the existing diabetes, CHF, and CAD measures. The proportion of the bonus pool dedicated to meeting the quality-of-care targets—30 percent in PY1—also increased in each performance year. For PY2, the potential quality-of-care bonus increased to 40 percent of the potential bonus pool, and the proportion of the bonus pool that will be paid as a cost-savings bonus decreased to 60 percent. For PY3, the cost-savings and quality-of- care bonuses each will constitute 50 percent of the total bonus paid. In July 2007, CMS reported that in PY1, 2 of the 10 participating physician groups earned bonuses for achieving cost-saving and quality-of-care targets, while all 10 participants achieved 7 or more of the 10 quality-of- care targets. The Marshfield Clinic and the University of Michigan Faculty Group Practice received performance bonus payments of approximately $4.6 million and $2.8 million, respectively, in PY1. The Marshfield Clinic generated approximately $6 million in Medicare savings in PY1, above the 2 percent threshold established by CMS. Of this $6 million bonus pool, the Medicare program retained approximately $1.2 million, and Marshfield Clinic earned $3.4 million for the cost-savings component of the bonus and $1.2 million for meeting 9 of the 10 quality-of-care targets. The University of Michigan Faculty Group Practice generated approximately $3.5 million in savings in PY1 above the 2 percent threshold. Medicare retained approximately $700,000 and the University of Michigan Faculty Group Practice earned nearly $2 million for the cost-savings component of the bonus, and just over $800,000 for meeting 9 of the 10 quality-of-care targets. Of the remaining eight participating physician groups that did not earn cost-savings bonuses in PY1, all performed well in meeting the quality-of- care targets. Specifically, all eight of these participants achieved 7 or more of the 10 quality-of-care targets, with two participants meeting all 10 quality-of-care targets and two others achieving 9 of the targets. In addition, six of the participants came close to achieving the 2 percent threshold for the cost-savings component of the performance bonus payment in PY1. These six groups reduced their Medicare spending growth rates compared to their comparison group, but not beyond the 2 percent threshold. (See fig. 2.) While the number of pay-for-performance programs—programs in which a portion of a provider’s payment is based on their performance against defined measures—has increased in recent years, this growth has occurred largely in the private sector by commercial health plans. MedVantage reported that 107 pay-for-performance programs were in place as of November 2005, up from 84 the year before. Of these 107 pay-for-performance programs, 21 were public sector programs, of which 10 were Medicare programs. Currently, CMS has 5 programs, including the PGP Demonstration, that are demonstrations testing alternative physician payment methods. (See table 2.) Among these 5 physician pay-for-performance demonstrations, 4, including the PGP Demonstration, test physician pay-for-performance methods by offering incentives to physicians for meeting clinical performance standards, while 1 focuses on aligning financial incentives between hospitals and physicians. The PGP Demonstration was the first of CMS’s Medicare demonstrations to test physician pay-for-performance. Participants in CMS’s Medicare Health Care Quality Demonstration, projected to begin in 2008, may elect to use this overall design and bonus payment methodology from the PGP Demonstration. Among CMS’s other pay-for- performance demonstrations that are not physician related, is the Premier Hospital Quality Incentive Demonstration, a hospital-specific pay-for- performance demonstration for more than 260 hospitals in the Premier Inc., system. Under this demonstration, CMS provides bonus payments for hospitals with the highest levels of performance in five clinical conditions, including acute myocardial infarction. A recent study examining this demonstration concluded that among hospitals receiving performance bonuses, patients did not have a significant improvement in quality-of-care of care or outcomes for acute myocardial infarction. The participating physician groups implemented care coordination programs to achieve cost savings and improved their management processes to meet quality improvement targets CMS set for particular diabetes measures in PY1. More specifically, management process improvements included enhancing information technology (IT) systems, incorporating more team-based approaches, and improving administrative processes. Despite early positive indicators in cost savings, the full impact of programs implemented for the PGP Demonstration, particularly in care coordination, is largely unknown because many programs were not in place for all 12 months of the first performance year. The participating physician groups implemented 47 programs, which were either new or expansions of existing programs, to achieve cost savings and meet the CMS-set diabetes quality-of-care targets, with each participant implementing from 2 to 9 programs. (See app. II for a complete list of new and expanded programs implemented for the PGP Demonstration.) More specifically, participants focused nearly three-quarters of their new and expanded programs on care coordination—programs that manage the care of a small number of chronically ill and frail elderly patients who account for a disproportionately large share of overall costs. (See fig. 3.) The remaining one-quarter of programs focused on patient education, medication-related issues, improving administrative processes, and other initiatives. Among the 47 programs, participating physician groups devoted the largest portion of their program resources to care coordination programs designed to reduce hospitalizations by improving post-acute care. Our analysis showed that for 9 of the 10 participants at least half of demonstration-specific, full-time equivalents (FTE) were devoted to care coordination programs. (See table 3.) Participants told us they selected care coordination programs that provided post-acute care because they believed these programs would reduce future hospitalizations and yield the most cost savings in the shortest amount of time. For example, both Billings Clinic and Park Nicollet Health Services used a telephonic interactive voice response (IVR) system to monitor patients’ health status at home following a hospitalization or another significant health event. Approximately half of the care coordination programs were case- management programs that targeted high-cost, high-risk patients with multiple medical conditions, while the other half were disease- management programs that treated patients with a specific disease, such as CHF. Seven participants focused on case-management programs by using care managers for patients with multiple medical conditions to reduce hospitalizations. For example, an official from the Dartmouth- Hitchcock Clinic stated that the clinic’s primary strategy for the PGP Demonstration was to reduce hospitalizations and readmissions through more effective discharge planning, such as calling patients at home following their hospital discharge and encouraging them to schedule follow-up appointments with their physicians. Three participants committed the majority of their resources to disease-management programs; two of the three participants told us they focused on CHF because it is a costly disease to treat and would therefore generate savings within the first performance year. Other diseases, such as diabetes, could take several years to generate cost savings. CHF and diabetes are two of the most common chronic diseases among Medicare beneficiaries, according to recent health policy research. All 10 participating physician groups reported that their care coordination programs were making progress in both achieving cost savings and providing broader benefits to their programs and communities. In particular, four participants reported declines in hospitalizations for patients enrolled in their CHF care coordination programs. For example, Park Nicollet Health Services reported a 61 percent reduction in hospitalizations for patients enrolled in its CHF care-management program, which utilized an IVR to interact with patients on a daily basis. Park Nicollet representatives estimated this program saved $4,680 yearly, on average, for each patient enrolled in the program. Because the demonstration included other Medicare and non-Medicare patients, its benefits extended beyond the patients assigned to Park Nicollet for the demonstration. Further, several participants stated that collaboration and information sharing among the 10 participants on designing and implementing programs and analyzing data resulted in improvements to their demonstration programs, which broadly benefit their organizations. Representatives from St. John’s Health Systems stated that creating a care- coordination program had additional benefits, including the adoption of such programs by other health systems and physician groups throughout the community. Despite early positive indicators of cost savings, the full impact of programs implemented for the PGP Demonstration, particularly in care coordination, is largely unknown for a variety of reasons, including that many programs were not in place for all 12 months of the first performance year. Only 1 of the 10 participants had all of its programs in place for all 12 months of PY1. For example, the Marshfield Clinic had a case-management program operational for all 12 months in PY1, and a disease-management program operational for 4 months. By the beginning of PY2, only 6 of the 10 participants had all of their care coordination programs operational. Officials from participating physician groups stated that program implementation delays were caused by program complexity, the process of gaining management approval for significant program start- up costs, and educating physicians about the programs. In addition, two participants stated that because their care coordination programs were phased in throughout the first two performance years, PY3 may be the first year that the full impact of these programs is realized. To meet the quality-of-care targets set by CMS on diabetes management, participating physician groups improved their management processes by investing in IT, creating team-based approaches, and improving administrative processes, particularly for diabetes management, the quality-of-care target for PY1. To earn the maximum bonus, participants that met the 2 percent cost-savings target had to further meet a quality-of- care improvement target in a particular clinical area. The measures selected by CMS for each performance year of the demonstration focused on chronic conditions prevalent in the Medicare population that are treated in primary care. In PY1, CMS selected diabetes management as the focus for quality improvement for the demonstration participants. See table 4 for a categorization of how the participants worked to improve quality, specifically for diabetes, by using physician feedback, patient registries, team-based approaches, and improved documentation. All participating physician groups made new investments in IT, by adding features to existing EHR systems or using technology to track physicians’ performance on the quality-of-care measures set by CMS. For example, Marshfield Clinic implemented electronic alerts in its EHR system to remind clinical staff to provide care, such as immunizations. Participants primarily used electronic methods for physician feedback as a tool for physicians to track their performance and that of their peers to improve their internal operations and patient care. For example, Geisinger Health System’s physician feedback system provided physicians with access to monthly reports for each physician, which compared each physician’s performance in meeting the quality-of-care measures. According to administrators from Geisinger, this transparent approach fostered positive competition among its physicians to improve quality of care. Participants also invested in IT by creating electronic patient or disease-specific databases or lists referred to as patient registries to better identify patients eligible for enrollment in diabetes programs. The St. John’s Health System, which did not have an EHR system, created an electronic patient registry to track patients with diabetes and to alert physicians to provide certain tests. Six of the participating physician groups relied to a greater extent on a team-based approach to improve care processes. Using a team-based approach, participants expanded the roles and responsibilities of nonphysician staff such as nurses, medical assistants, and care managers so that they worked more effectively with physicians to deliver quality care. Although the demonstration required additional quality reporting, officials from two of the participating physician groups stated that they were able to treat the same number of patients in a day. For example, Dartmouth-Hitchcock Clinic used care managers who were nurses to maximize the effectiveness of patients’ office visits. These staff scheduled lab tests in advance of patients’ office visits when appropriate, developed patient action plans, and communicated with physicians before and after patients’ arrivals. Physicians from Dartmouth-Hitchcock told us that the time they spent with patients had become more effective because of this new approach. Four participating physician groups improved their administrative processes by creating better documentation methods for diabetes-related tests and exams. They created worksheets, derived from patients’ medical records, to ensure that patients received diabetes tests, such as foot and eye exams. In addition to improving documentation, these initiatives served as reminders to physicians to complete diabetes-related tests and exams and also reduced the burden of data collection for reporting purposes. For example, IRMA created paper forms that were added to patient records to collect data on tests as they were conducted. In addition to improving documentation, these forms were intended to relieve some of the burden of collecting data for smaller practices within the organization. IRMA physicians also received paper worksheets at the point of care to help monitor and track care provided to their diabetes patients. CMS’s design for the PGP Demonstration was generally a reasonable approach for rewarding participating physician groups for cost-savings and quality performance. However, the demonstration design created a particular challenge for CMS in providing timely performance feedback and bonus payments to the participants which, if received more quickly, may have enabled them to improve their programs. CMS’s design for the PGP Demonstration was generally a reasonable methodological approach for determining whether the actions taken by the participants resulted in cost savings and improvements in quality, and rewarding participants as appropriate. In particular, three aspects of the PGP Demonstration design were consistent with established methodological practices considered effective: a rigorous research study design to isolate the effects of the demonstration’s incentives, a risk- adjustment approach to adjust for changes in patient health status, and a quality component to help ensure that participating physician groups did not achieve cost savings at the expense of quality. CMS used a rigorous research design to enable it to isolate the effectiveness of the actions taken by each of the participants in the demonstration. Specifically, CMS used a modified “pre-test/post-test” control group design that is generally viewed by experts as an effective way to control for some of the most common threats to internal validity, in this case the ability of the study design to measure the true effects of CMS’s incentive payments. The features of CMS’s study design included a separate comparison group for each participant to distinguish the effects of the demonstration’s incentives from unrelated spending trends in the participants’ service areas. Comparison groups’ beneficiaries are from the participants’ geographic service areas and, as such, are affected by the same local market trends as the participants. In addition, the study design included a baseline period, before the demonstration began, that helped to control for trends that may have occurred without demonstration-related interventions. A standard pre-test/post-test control group design would have randomly assigned beneficiaries to either a comparison group or a participant’s beneficiaries. To avoid having to restrict or control beneficiaries’ choice of providers and health care services, and to continue to operate within the Medicare FFS system while the demonstration was in place, CMS modified this standard approach. Rather than assigning beneficiaries randomly at the start of the demonstration to participant or comparison groups, the agency retrospectively assigned beneficiaries at the end of each year based on the beneficiaries’ natural use of outpatient E&M services. CMS also used a rigorous risk-adjustment approach to adjust for changes in patients’ health-care status. Without these adjustments, CMS could not have been reasonably assured that changes in spending growth were not attributable to changes in patients’ health care status, and the severity and complexity of diagnosis. For the PGP Demonstration, CMS tailored the CMS-Hierarchical Condition Category (HCC) model, the risk-adjustment model that it currently uses to make capitation payments to Medicare managed care plans. This model accounts for changes in the health status of beneficiaries. Furthermore, CMS incorporated a quality component into the research design, which helped ensure that participants would not achieve cost savings at the expense of quality. The quality-of-care measures CMS selected were based on a consensus of experts and were developed in collaboration with the American Medical Association and quality assurance organizations and with input from the participants. In addition, CMS has placed an increased emphasis on quality in its bonus payment methodology for future years. By PY3, half of the available bonus pool will be awarded based on each participant’s success in meeting quality-of-care metrics in six clinical areas: diabetes, CHF, CAD, management of hypertension, and screening for breast cancer and colorectal cancer. While CMS’s research design for the PGP Demonstration was generally a reasonable approach, it also created some challenges for the participating physician groups. Challenges resulting from the demonstration design included providing timely performance feedback and bonus payments and the use of a uniform 2 percent savings threshold that may have disadvantaged certain participants. Participants also raised other concerns related to the demonstration design that were related to their local markets. Overall, participants did not receive performance feedback or bonus payments for their PY1 efforts until after the beginning of the third performance year. Specifically, CMS provided participants with performance feedback and bonus payments regarding their efforts in PY1 in three phases beginning 12 months after the end of PY1. In April 2007, CMS provided each participant with a cost-savings summary report displaying its success in controlling Medicare expenditures for PY1 and the size of its cost-savings bonus pool. (See fig. 4.) A little over 2 months later, CMS provided each participant with a detailed settlement sheet displaying its individual cost-savings and quality-of-care bonuses for PY1. It was not until July 2007, 15 months after the end of PY1, that the two participants that earned a demonstration bonus for PY1—the Marshfield Clinic and the University of Michigan Faculty Group Practice—received their bonus payments of $3.4 million and $2.1 million, respectively. CMS officials explained that generating feedback for the participating physician groups required 15 months because the demonstration design depended on the time-consuming process of retrospectively analyzing Medicare beneficiaries’ claims and chart-based data. Specifically, CMS officials stated that the process of calculating participants’ cost-savings bonuses required at least 12 months after the conclusion of the first performance year—6 months to accrue claims data that were sufficiently complete and a second 6 months to analyze and calculate the bonus amounts. In addition, they stated that the calculation of the quality-of- care bonus required an additional 3 months to audit and reconcile chart- based data with claims-based data pertaining to the 10 diabetes quality-of- care measures. CMS officials stated that to calculate the cost-savings bonus they chose to use a claims file that was 98 percent complete because they wanted to ensure that the feedback they provided to participants was accurate. CMS officials also stated that the time frames for providing performance feedback and bonus payments to participants in PY1 will be the same for PY2 and PY3. Officials from all 10 participating physician groups expressed concern about the length of time CMS took to provide them with performance feedback and bonus payments. Several participants stated that they had difficulty making adjustments to their programs and improving their overall performance because of delayed feedback and payments. One official from the Novant Medical Group stated that the 15-month time lag in receiving bonus payments would prevent the organization from reinvesting these resources into demonstration-related programs and improving them for subsequent performance years. In addition, two of the participants told us that other pay-for-performance programs they have participated in have used payment methodologies that yielded more timely performance feedback or bonus payments. For example, officials from the University of Michigan Faculty Group Practice indicated that a pay-for- performance program sponsored by Blue Cross Blue Shield of Michigan provided them twice a year with feedback for meeting certain quality-of- care targets. In response to these concerns, CMS has been working to provide each participating physician group with a quarterly Medicare patient claims data set related to beneficiaries they served. Initially, data sets were provided quarterly and focused on identifying patients with chronic conditions who had a hospital admission or emergency room visit. In July 2007, CMS provided each participating group with a data set on hospital inpatient, outpatient, and physician information consisting of the Medicare claims of beneficiaries likely to be included in the PY2 cost-savings calculations. In September 2007, CMS responded to participants’ requests for quarterly claims data that would allow them to assess their cost- savings performance during the performance year, by providing them with a revised data set. CMS’s most recent data set included Medicare inpatient, outpatient, and physician claims data for beneficiaries likely to be included in the year-end cost-savings calculation. The data set includes information on these beneficiaries for the first quarter of PY3. CMS noted that it will not provide equivalent information pertaining to comparison group beneficiaries because these data are too time-consuming to assemble. While CMS’s provision of ongoing quarterly data sets to participants is timelier than the information provided before, most participants told us they do not have the necessary resources to analyze these data sets in a timely manner. This lack of timely actionable data could hinder participants’ ability to adjust their programs on a more “real-time” basis. Officials from only 2 of the 10 participants told us they would able to analyze and use the quarterly data sets CMS provided. Consequently, the data sets are not as useful as providing CMS-generated quarterly reports, similar to the final reports CMS provided on participants’ progress in achieving cost-savings and quality-of-care targets. CMS may not be able to provide quarterly reports that include comparison group trends or provide quality-of-care data that rely on chart-based data because of complexity and cost. However, CMS could provide participants with estimates on their growth in per-beneficiary expenditures each quarter, as well as changes in the profile of the beneficiaries who are likely to be assigned to the participants. CMS could also use more readily available claims data to provide quarterly estimates of participants’ progress in meeting the quality-of-care targets. In PY1, for example, that would have included reporting progress on 4 of the 10 claims-based quality targets on diabetes, such as whether a beneficiary received an eye exam. CMS officials said they adopted the use of a uniform 2 percent threshold to ensure that savings generated really were due to demonstration-related programs. Just as CMS used individual comparison groups for each participant, CMS could have used separate savings thresholds that more closely reflected the market dynamics of each participant’s overall area instead of a uniform savings threshold chosen based on historical data averaged across the 10 participants. However, use of different thresholds for each participant, according to CMS officials, would have been complex and would have generated additional administrative burden in processing bonus payments. Nevertheless, the use of a uniform savings threshold—2 percent—that all participating physician groups had to achieve before becoming eligible for a bonus payment may have made earning bonus payments more challenging for particular providers, specifically those with already low Medicare spending growth rates or those that had comparison group providers with low spending growth rates. Participants with low relative spending may have had difficulty generating annual Medicare savings of greater than 2 percent compare to those participants with high spending growth rates before the demonstration began, some participants argued. Supporting this concern is the wide variation in the amount participants spent per beneficiary in the year prior to the demonstration, which ranged from $6,426 to $11,520, after adjusting for health status. In addition, participants with comparison groups that had relatively low spending growth may have faced more of a challenge reducing their spending below 2 percent of their comparison groups’ spending than participants with comparison groups that had higher relative spending growth. In fact, both participants that received a bonus, Marshfield Clinic and the University of Michigan Faculty Group Practice, were measured against comparison groups with high relative spending growth rates—the 2 largest among the 10 participants. While their success cannot necessarily be attributed to the high relative spending of their comparison groups, the high spending growth of the comparison groups against which they were measured may have had some effect. In addition, several participating physician groups noted selected concerns particular to their local markets. For example, officials from one participating physician group expressed concern that CMS did not adequately adjust for the conversion of several hospitals in their market to critical access hospitals (CAH), which generally receive higher Medicare payments. Participant officials noted that their physician group treat more patients from these hospitals, which resulted in a higher spending trend and lower likelihood of obtaining a cost-savings bonus. CMS stated that the agency will examine this issue as part of its evaluation at the conclusion of the PGP Demonstration. Several participating physician groups were also concerned that their groups had more beneficiaries with specialist visits relative to their comparison groups. As a result, participants providing more specialty care may have had less control over the health outcomes of these beneficiaries. However, analyses conducted by CMS showed that these participants provided 80 percent or more—a predominant share—of the E&M services for most of the beneficiaries assigned to them in PY1, regardless of specialty, and had meaningful opportunities to influence beneficiary health care expenditures. CMS officials stated that they will continue examining this issue and other related issues brought to their attention by the participants as part of their evaluation of the demonstration. The large size of the 10 participating physician groups compared with the majority of physician practices operating in the U.S. gave the participants certain size-related advantages that might make broadening the payment approach used in the demonstration to more participants challenging. The 10 participating physician groups had significantly higher numbers of physicians, higher annual medical revenues, and higher numbers of supporting staff, and were more likely to be multispecialty practices than most practices in the U.S. Specifically, the participating physician groups generally had three unique size-derived advantages: institutional affiliations that allowed greater access to financial capital, access to and experience using EHR systems, and experience prior to the PGP Demonstration with pay-for-performance programs. While all the participating physician groups in the demonstration have 200 or more physicians in their practices, significantly less than 1 percent of the approximately 234,000 physician practices in the U.S. in 2005 had 151 or more physicians in their practice. (See fig. 5.) By contrast, practices with only 1 or 2 physicians comprised 83 percent of all practices. Furthermore, while all 10 participants were multispecialty practices, 68 percent of the all practices in the U.S. were single-specialty practices, which are generally smaller organizations. The 10 participating physician groups were also large compared with other physician practices in terms of annual medical revenues and nonphysician staff. Participants generated an average of $413 million in annual medical revenues in 2005 from patients treated by their group practice, far greater than the revenues generated by single specialty practices in the U.S. Only about 1 percent of single specialty practices had revenues greater than $50 million. In addition, these physician groups and their affiliated entities, such as hospitals, employed approximately 3,500 nonphysician FTEs, over 100 times more than the average single-specialty practice. Their larger relative size gave the 10 physician groups participating in the PGP Demonstration three size-related advantages over smaller physician practices, which may have better prepared them to participate in the demonstration’s payment model and implement programs encouraged by the demonstration. First, participants typically had institutional affiliations with an integrated delivery system, a general hospital, or a health insurance entity. Specifically, 9 of the 10 participating physician groups were a part of an integrated delivery system, 8 were affiliated with a general hospital, and 5 were affiliated with an entity that marketed a health insurance product. In contrast, a representative of the Medical Group Management Association estimated that approximately 15 percent of all physician practices in the U.S. have an affiliation with a general hospital. As a result of these affiliations, participating physician groups generally have greater access to the relatively large amounts of financial capital needed to initiate or expand programs. On average, each participating physician group invested $489,354 to initiate and expand its demonstration-related programs and $1,265,897 in operating expenses for these programs in PY1. (See app. III.) Specifically for individual programs, participants reported spending an average of $190,974 to initiate and $409,332 to operate case management programs in PY1, almost twice the spending associated with any other type of program. (See table 5.) Several participants reported that the majority of their individual program expenditures were labor costs for care managers. Officials from several participating physician groups said that smaller practices might have difficulty implementing similar programs because they may not have the financial resources to do so. The second advantage the 10 large participating physician groups had over smaller physician practices is a greater likelihood of having or acquiring EHR systems, which was essential in participants’ ability to gather data and track progress in meeting quality-of-care targets. Eight of the 10 participating physician groups had an EHR in place before the demonstration began, and the 2 other participants, out of necessity, developed alternative methods for gathering patient data electronically specifically for the demonstration, such as creating patient registries. In contrast, only an estimated 24 percent of all physician practices in the United States had either a full or partial EHR in 2005, and large practices were more likely to have EHRs than small practices. These systems enable physician group practices with multiple locations, such as the 10 participating physician groups, to share patient information and other administrative resources across a wide geographic area. Health care information technology experts believe that the primary reason smaller physician practices have not implemented EHRs is their cost, estimated at $15,000 and $50,000 per physician. In addition, experts estimated that annual maintenance costs add between 15 and 25 percent of the initial per physician investment. Furthermore, experts noted that small practices tend to pay more per physician for EHR systems than larger physician practices because larger physician practices are better able to spread the fixed costs of these systems across more physicians. Finally, the third size-related advantage that most of the 10 participating physician groups had over smaller physician practices was the larger groups’ experience with other pay-for-performance systems prior to participating in the PGP Demonstration. Overall, 8 of the 10 participants had previous experience with pay-for-performance programs initiated by private or public sector organizations. This experience may have eased their adjustment to the PGP Demonstration and afforded them greater initial and overall success. For example, the University of Michigan Faculty Group Practice’s participation in a pay-for-performance system sponsored by Blue Cross Blue Shield of Michigan offered the physician group incentives to upgrade its chronic care infrastructure. In addition to experience with pay-for-performance programs, generally, the majority of the 10 participating physician groups had experience with specific elements of pay-for-performance, such as physician bonus compensation methods and physician feedback processes. Representatives from some of the participating physician groups stated that their exposure to one of these various elements of pay-for-performance elements prior to the PGP Demonstration may have enabled their organizations to adjust to the demonstration more rapidly. The care coordination programs used by the participating physician groups show promise in achieving cost savings and improving patient outcomes for Medicare beneficiaries. As a result of the demonstration, participating physician groups generated several different approaches for coordinating patient care across inpatient and physician settings for high- risk and high-cost patients, such as those with CHF, and better managing patients with diabetes, the quality-of-care target for the demonstration set by CMS. Additional years of the demonstration may be needed, however, for CMS to collect and analyze the information necessary to fully evaluate the effectiveness of these care coordination programs and their potential for cost savings in this demonstration. Only one participant had all of its care coordination programs operational for all 12 months of PY1, and participants did not receive timely feedback from CMS on their progress until PY3 had already begun. While CMS’s demonstration design was generally reasonable, the lengthy time CMS took to provide participating physician groups with performance feedback and bonus payments may limit more widespread use in other demonstrations, or use as an alternative method for paying physicians group in Medicare FFS. The lack of timely and actionable performance feedback also hinders participants’ ability to improve their programs in response to data. Providing performance feedback and bonus payments to participants more than 12 months after the end of the measurement period precludes physician groups from adjusting their program strategies on a more “real-time” basis. CMS has recently taken action to provide participants with quarterly claims data sets on their beneficiaries for PY3, but most participants indicated they would have difficulty analyzing such data to determine their progress in achieving cost-savings and quality-of-care targets. Measuring participants’ performance using a comparison group where beneficiaries were retrospectively assigned after the end of a performance year, as used in this demonstration, may be impractical for more widespread use beyond this demonstration because physician groups cannot accurately predict on an ongoing basis whether they would be able to generate cost-savings and receive bonus payments. In addition, the use of a uniform savings threshold amount, such as the 2 percent used in this demonstration, raises questions about whether this approach provides a disincentive for physician groups that have lower spending. Physician groups with fewer than 200 physicians—the vast majority of practices in the United States—may also have more difficulty than larger practices, such as the participants in this demonstration, absorbing the start-up and annual-operating costs of the care coordination programs and implementing them without EHR systems that many groups believed were necessary to achieve cost savings while maintaining and improving the quality of care. As the PGP Demonstration continues, data will become available to CMS to determine how much influence factors such as the delay in the start-up of participating physician groups’ care coordination programs and its decision to use a uniform 2 percent threshold, and other factors, may have had on participants’ ability to earn bonus payments. Consequently, it is understandably too early to determine the success of the PGP demonstration, but evidence so far indicates that the care coordination programs initiated by the participants show promise, but that the wider applicability of the payment methodology used in the demonstration may be more limited. We recommend that the Administrator of CMS provide participating physician groups with interim summary reports that estimate participants’ progress in achieving cost-savings and quality-of-care targets. CMS reviewed a draft of this report and provided comments, which appear in appendix IV. CMS stated that it appreciated our thoughtful analysis and that our report would provide additional insight into performance year one results and complement its ongoing evaluation efforts. CMS agreed with the intent of our recommendation. CMS stated that it was developing a new quarterly report and refined data set to aid the physician groups in monitoring their performance, coordinating care, and improving quality. CMS stated that these reports would address a key limitation of existing quarterly data sets—that most physician groups do not have the necessary resources to analyze the data sets in a timely manner. We agree that this information would be helpful in improving performance feedback to physician groups, which would allow them to adjust their program strategies on a more “real-time” basis. As the demonstration continues, we encourage CMS to continue its efforts to improve performance feedback to the physician groups participating in the PGP demonstration. We are sending copies of this report to the Administrator of CMS. We will provide copies to others on request. In addition, this report is available at no charge on the GAO web site at http://www.gao.gov. If you or your staff have questions about this report, please contact me at (202) 512-7114 or kingk@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff members who made key contributions to this report are listed in appendix V. For the first performance year, we examined three objectives: (1) what actions the participating physician groups took to achieve cost savings and meet the diabetes quality-of-care targets selected by the Centers for Medicare & Medicaid Services (CMS), (2) the extent to which the demonstration design was a reasonable approach to rewarding participating physician groups for cost savings and quality performance, and (3) potential challenges involved in broadening the payment approach used in the demonstration from the 10 large participating physician groups to other physician groups and nongroup practices. For each of our reporting objectives, we analyzed data we collected, by written questionnaire, and supplemented this information with interviews in person and by telephone and site visits to 5 of the 10 locations. We sent questionnaires to individuals CMS identified as points of contact at each of the participating physician groups. These individuals were often physicians or administrative staff tasked with overseeing their physician group’s demonstration efforts. All 10 participants completed and returned our questionnaire. The questionnaire contained three sections. The first section gathered standardized information about the practice’s general characteristics, including organizational structure, size, institutional affiliations, and the extent to which it used electronic health records systems. The second section gathered information about the programs participants used as a part of the Physician Group Practice (PGP) Demonstration. This section confirmed summary statements the individual practices described in their original applications to CMS or in other documents we obtained from CMS’s contractor, Research Triangle Institute (RTI), and provided an opportunity for the group to add new programs, if needed. Summary statements detailed the purpose, type, and characteristics of each program. We also asked participants whether each of their programs was created specifically for the demonstration, was a preexisting program, or was an expansion of a preexisting program. In addition, we asked officials from these physician groups to identify the start-up costs and the annual operating costs of these programs. We also asked about the extent to which the physician groups believe smaller physician practices could implement similar programs. The third section of the questionnaire gathered information about how the participating physician groups compensated their physicians and how any demonstration bonus dollars they may earn would be distributed to individual physicians within the group. We also conducted site visits or telephone interviews with staff of all 10 participating physician groups. Five of these interviews were site visits, which we chose to reflect geographic diversity (region of country and urban/rural), size, and ownership status, among other factors. We conducted site visits to Geisinger Health System in Pennsylvania, Park Nicollet Health Services in Minnesota, Marshfield Clinic in Wisconsin, Billings Clinic in Montana, and the Everett Clinic in Washington. We collected the same information by telephone from the other participants in the demonstration. For these in-person or telephone interviews, we interviewed the demonstration project managers, physicians, care managers, finance officials, and information technology staff. To identify programs used by the participating physician groups to achieve cost savings and meet the CMS-set diabetes quality-of-care targets, we analyzed data we collected by written questionnaires and interviews, supplemented with information we obtained at site visits to 5 of the 10 participants. We included in our analysis new programs or expansions of existing programs created in response to the demonstration. To determine the extent to which the demonstration’s design was reasonable, we analyzed documents on the overall research design and bonus payment methodology obtained from CMS, analyzed data collected through the questionnaire, and used interviews we conducted with the participating physician groups and CMS and its contractor RTI. We also reviewed and analyzed CMS-contracted documents on the design of PGP Demonstration. To determine the potential challenges involved in broadening the payment approach used in the demonstration to other physician groups, we compared selected characteristics of the 10 participating physician groups to physician practices in the United States, using data primarily from the Medical Group Management Association’s (MGMA) annual survey. We also used data we collected from the questionnaire and from our interviews with officials from the physician group practices. We also interviewed experts on health information technology systems. We assessed the reliability of the information we obtained about participating physician group practices and the data we used to compare them to other physician groups in the U.S. in several ways. First, we checked the internal consistency of the information we obtained from the physician groups with information from RTI’s PGP Demonstration site visit reports and CMS’s 2006 Report to Congress on the PGP Demonstration. We verified the information we collected from the questionnaire with detailed follow-up interviews with officials from all 10 participants. Second, we spoke to the survey director for the 2005 MGMA survey to ensure that we used the information from that survey appropriately and that we understood any data limitations. In addition, we compared the data we used on U.S. group practices from the 2005 MGMA survey with data from the 2005 National Ambulatory Medical Care Survey and determined that the results were largely consistent and adequate for our purposes. Third, on the basis of this comparison and discussions with experts knowledgeable about the data, we used broad categories to describe the data. We determined that the data used in our analysis were sufficiently reliable for the purposes of this report. We conducted our work from May 2006 through December 2007 in accordance with generally accepted government auditing standards. Billings Clinic Cancer treatment center—care coordination (disease management) Patients received coordinated cancer care including screening, prevention education, and infusion treatments. Chronic obstructive pulmonary disease (COPD) management—care coordination (disease management) Care managers worked with COPD patients to avoid functional decline, offer preventive services such as immunizations, and treat complications early. Community crisis center—care coordination (case management) High-risk patients with chronic psychiatric conditions were redirected from the emergency room to the psychiatric center for treatment. A patient registry was used to identify diabetes patients and create patient report cards, which displayed treatment confirmation and treatment gaps. Heart failure clinic—care coordination (disease management) Care managers monitored an automated system, which recorded patients’ answers to health status questions, and intervened if necessary. Hospitalist program—care coordination (case management) Hospitalists worked with internists and family practitioners to improve the communication and care provided to patients at hospital discharge. Electronic prescription system better reconciled patients’ medications between inpatient and outpatient settings to reduce adverse events. Nursing home staff educated on consulting a Billings geriatrician before admitting patient to the hospital. To coordinate nursing home and hospital care, physician assistants were assigned to patients entering the hospital’s emergency room from local nursing homes. Cancer care/palliative care—care coordination (case management) Care managers assisted cancer patients and families in coordinating, planning end-of-life care. Health coaching—care coordination (case management) Care managers helped patients follow hospital post- discharge instructions, make physician appointments, and take the correct medications and dosages. Physicians received feedback through intranet, met with management on identified issues. Forms were placed on the front of patients’ medical charts to remind physicians of CAD quality-of-care measures. A director position was created and charged with coordinating program interventions and with overseeing all care for Medicare patients. Hypertension management program— care coordination (disease management) A patient registry was used to identify patients with hypertension and remind physicians to measure and document patients’ blood pressure. Program name and program type Palliative care program—care coordination (case management) Care managers provided patients and families end-of-life planning information on quality-of-life issues, alternative living options, and in-home support. Patient care coordination—care coordination (case management) Care managers coordinated inpatient and outpatient care, helping to ensure proper discharge planning and schedule follow-up appointments with physicians. COPD management—care coordination (disease management) Care managers worked with patients to monitor their health status and to encourage patients to visit their physicians when necessary. Congestive heart failure management— Care coordination (disease management) Care managers monitored patients’ health status through a voice recognition system and contacted patients when their health status became problematic. Diabetes disease management—care coordination (disease management) Care managers worked with patients to educate them on managing diabetes. Moderate risk case management—care coordination (case management) Care managers worked with patients to reduce risk factors associated with potential future hospitalizations. Postacute case management—care coordination (case management) Care managers contacted patients after a hospital discharge to ensure that home health services were received, correct medications taken, etc. Anticoagulation—care coordination (case management) Pharmacists and physicians worked with patients during a hospitalization to ensure prescriptions were correct and to assist in the transition from inpatient to outpatient care. Cancer care management—care coordination (disease management) Care managers worked with colon, breast, and lung cancer patients and their physicians to ensure that evidence-based treatment guidelines are followed for psychological, nutritional, and palliative care. Chronic care management—care coordination (case management) Care managers educated patients admitted to the hospital on disease self-management and proper medication use. Congestive heart failure (CHF)—care coordination (disease management) Care managers helped patients understand and follow their post-discharge instructions. Diabetes disease management—care coordination (disease management) Care managers worked with diabetes patients to coordinate care across providers, provide in-person patient education, and remind patients of appointments. Certified diabetes educators assisted patients in understanding diabetes self management tools. Heart smart program—care coordination (disease management) Care managers provided case management services to cardiac patients enrolled in home care services. HomeMed program—care coordination (case management) Frail, elderly patients with multiple conditions receive a telemedicine device in their home that monitors vital signs. Program name and program type Anticoagulation—care coordination (case management) Care managers worked with patients to ensure that dosages of the anticlotting drug Warfarin were adjusted properly. Also educated patients on recognizing factors that can influence anticoagulation such as diet, activity, other medications, and other illnesses. CHF management—care coordination (disease management) Care managers called patients to check on health status, schedule physician visits, and answer questions. Disease management, compass—care coordination (disease management) Care managers assisted patients with medication management, appointments, and physician referral. Outpatient case management—care coordination (case management) Care managers helped high-risk, high-cost patients recently discharged from the hospital schedule follow-up physician visits and learn of available resources. Physicians and staff were educated on how to talk to patients about palliative care. Physicians and clinical staff were educated on evidence- based guidelines for chronic disease management. Transition of care program—care coordination (case management) Nurses contacted patients after hospital discharges to ensure patients made follow-up physician appointments. Diabetes care management—care coordination (disease management) Care managers educated newly diagnosed diabetes patients on diabetes self management. During a 30-minute office visit, patients received an evaluation of needs, health education, diagnoses, prevention measures, and fitness counseling. Heart failure care coordination—care coordination (disease management) Care managers monitored patients’ medication usage and dietary regimes through an interactive voice response system. Patients telephoned a call center where nurses directed them to care based on their symptoms. Case management systems—care coordination (case management) Care managers provided care to high-risk patients including coordination of inpatient and outpatient care services and guidance on following treatment plans. Disease management—care coordination (disease management) Care managers managed, educated, and coached patients with chronic conditions. CHF program (disease management) Nurses assessed the health status of heart failure patients. Low-income patients were assisted in obtaining free medications from pharmaceutical companies. Program name and program type Complex care coordination—care coordination (case management) Care managers monitor patients with multiple chronic diseases and educate them on self management. Post-discharge transitional care—care coordination (case management) Care managers provided education, medication counseling, guidance on post-acute care treatment, and assistance with making and getting to post-discharge appointments. Start-up investment expenditures for Integrated Resources for the Middlesex Area were not available. In addition to the contact named above, Thomas Walke, Assistant Director; Jennie Apter; Kelly Barar; Zachary Gaumer; and Jennifer Rellick made key contributions to this report.

Summary: Congress mandated in 2000 that the Centers for Medicare & Medicaid Services (CMS) conduct the Physician Group Practice (PGP) Demonstration to test a hybrid payment methodology for physician groups that combines Medicare fee-for-service payments with new incentive payments. The 10 participants, with 200 or more physicians each, may earn annual bonus incentive payments by achieving cost savings and meeting quality targets set by CMS in the demonstration that began in April 2005. In July 2007, CMS reported that in the first performance year (PY1), 2 participants earned combined bonuses of approximately $7.4 million, and all 10 achieved most of the quality targets. Congress mandated that GAO evaluate the demonstration. GAO examined, for PY1, the programs used, whether the design was reasonable, and the potential challenges in broadening the payment approach used in the demonstration to other physician groups. To do so, GAO reviewed CMS documents, surveyed all 10 groups, and conducted interviews and site visits. All 10 participating physician groups implemented care coordination programs to generate cost savings for patients with certain conditions, such as congestive heart failure, and initiated processes to better identify and manage diabetes patients in PY1. However, only 2 of the 10 participants earned a bonus payment in PY1 for achieving cost savings and meeting diabetes quality-of-care targets. The remaining 8 participants met most of the quality targets, but did not achieve the required level of cost savings to earn a bonus. Many of the participants' care coordination programs were not in place for all of PY1. CMS's design for the PGP Demonstration was generally a reasonable approach for rewarding participating physician groups for achieving cost-savings and quality-of-care targets, but created challenges. CMS's decision to use comparison groups, adjust for Medicare beneficiaries' health status, and include a quality component in the design helped ensure that bonus payments were attributable to demonstration-specific programs and that cost-savings were not achieved at the expense of quality. However, the design created challenges. For example, neither bonuses nor performance feedback for PY1 were given to participants until after the third performance year had begun. CMS provides participants with quarterly claims data sets, but most participants report they do not have the resources to analyze these data sets and generate summary reports on their progress and areas for improvement. The large relative size of the 10 participating physician groups (all had 200 or more physicians) compared with most U.S. physician practices (less than 1 percent had more than 150 physicians) gave the participants certain size-related advantages that may make broadening the payment approach used in the demonstration to other physician groups and non-group practices challenging. Their larger size provided the participants with three unique size-related advantages: institutional affiliations that allowed greater access to financial capital, access to and experience with using electronic health records systems, and prior experience with pay-for-performance programs.