The Pioneer ACO Model's overall goal is to improve the delivery of Medicare services by reducing expenditures while preserving or enhancing the quality of care for patients. Beginning in 2012, CMS contracted with ACOs for a 3-year period and subsequently offered a 2-year contract extension to ACOs that completed the first three years.The ACOs are expected to meet the goal of the model, in part, by coordinating the care they provide to Medicare beneficiaries and engaging beneficiaries in their own care. CMS designed the model for organizations with experience in providing coordinated care to beneficiaries at a lower cost to Medicare. Another goal of the Pioneer ACO Model is to help inform potential future changes to the agency's permanent ACO program, the Medicare Shared Savings Program (MSSP), which began about 3 months after the Pioneer ACO Model. MSSP ACOs share less financial risk with CMS than Pioneer ACOs, as many are not responsible for paying CMS for any losses that they may generate during their contract period. CMS established eligibility requirements for participation in the Pioneer ACO Model through the request for applications and the contract between CMS and Pioneer ACOs. The requirements include the following: Organizational structure. ACOs must be structured to allow the organization to partner with groups of providers including ACO professionals such as physicians or physician assistants, to accept joint responsibility for the cost and quality-of-care outcomes for a specified group of patients. For example, ACOs may be structured as ACO professionals in a group practice or as partnerships between ACO professionals and hospitals. Care improvement plan. ACOs must implement a care improvement plan, as they described in their applications. These plans include a range of care strategies such as providing remote patient monitoring to beneficiaries with chronic illnesses and engaging beneficiaries through shared decision making. Beneficiary protections. ACOs must ensure that their providers and suppliers make all Medicare-covered services available to beneficiaries and that they do not inhibit beneficiaries' freedom of choice to obtain health services from providers or suppliers not participating in the model. The ACOs annually provide CMS with a list of the providers and suppliers that have elected to participate as Pioneer providers or suppliers. Quality performance standards. ACOs must completely and accurately report quality data annually to CMS for 33 measures that address four quality domains. The four domains are (1) patient experiences of care, (2) care coordination and patient safety, (3) preventive health care, and (4) disease management for at-risk populations, such as beneficiaries with diabetes. ACOs must also meet performance standards for quality. In the first year (2012), CMS defined the quality performance standard as completely and accurately reporting all of the quality measures, regardless of the ACO's scores on the measures. Beginning in 2013, CMS required that ACOs score a minimum level for at least 70 percent of the quality measures within each of the four quality domains. CMS determined a minimum performance level for each quality measure, based on performance benchmarks.quality apply to all participating ACOs. CMS's oversight and evaluation responsibilities are broadly defined in the contract between CMS and the Pioneer ACOs and in regulation. CMS is responsible for monitoring beneficiary service use and investigating any unusual service use patterns to assess, for example, whether ACOs may be compromising beneficiary care. CMS is also responsible for monitoring whether ACOs may be avoiding at-risk beneficiaries. CMS may use a range of methods to conduct this monitoring, including analyzing beneficiary and provider complaints, and may investigate patterns suggesting that an ACO has avoided at-risk beneficiaries. In addition, CMS's oversight role includes monitoring ACOs' compliance with the quality performance standards. CMMI's Seamless Care Models Group is responsible for carrying out the agency's oversight responsibilities for the model. CMS is responsible for conducting an evaluation of the model's financial and quality performance results and making the evaluation findings available to the public in a timely manner. conduct the evaluation and has chosen to focus the evaluation on eight research areas, based on a conceptual model outlining the pathways in which various factors can affect ACO performance results. The eight research areas are (1) Medicare service use and expenditures, (2) unintended consequences of ACOs, (3) beneficiary access to care, (4) ACOs' care coordination activities, (5) quality of care, (6) health care markets served by ACOs, (7) ACO characteristics, and (8) ACO attrition. See 42 U.S.C. SS 1315a(b)(4). The law does not specify a timeline for making these findings available to the public. Under this provision, CMS is responsible for evaluating models to test innovative payment and service delivery, such as the Pioneer ACO Model. Taking into account the evaluation findings, the Secretary of Health and Human Services may expand the duration and scope of the Pioneer ACO Model. Medicare beneficiaries are assigned by CMS to Pioneer ACOs based on their prior use of primary care services. CMS refers to this as "alignment." ACOs are responsible for the annual expenditures of their aligned beneficiaries. CMS determines through an analysis of Medicare claims data which beneficiaries have received the plurality of their primary care services from primary care providers affiliated with an ACO in the prior three years. The ACO's financial performance is based on the annual expenditures of its aligned beneficiaries for services covered by Medicare Parts A and B, which include hospital stays, outpatient services, physician visits, and skilled nursing facility (SNF) stays. To assess financial performance, CMS includes the expenditures for services provided by the ACO as well as by non-ACO Medicare providers since aligned beneficiaries may continue to obtain services from providers that are not affiliated with the ACO. Pioneer ACOs chose one of five payment arrangements with CMS that specified the type of risk sharing and the sharing rates, that is, the percentage of savings or losses that the ACO shared with CMS. The type of risk sharing is one- or two-sided. Under one-sided risk sharing, the ACO may receive a payment from CMS if it generates a minimum amount of savings but does not owe CMS a payment if it generates losses. In comparison, an ACO owes CMS a payment if it generates a minimum amount of losses under two-sided risk sharing, and is eligible to receive a payment from CMS if it produces savings. Four of the five arrangements required two-sided risk sharing in the first and second years; the other arrangement allowed for one-sided risk sharing, but only in the first year. Half of the ACOs (16 of 32) that participated in the first year selected the arrangement with one-sided risk sharing in the first year, and half (16 of 32) selected arrangements with two-sided risk sharing. The sharing rate specifies the maximum amount of savings that the ACO can share with CMS and the maximum amount of losses that the ACO may share with CMS. The sharing rates increase from the first to the second year in each of the payment arrangements. (See table 1.) CMS determines whether each Pioneer ACO has generated savings, losses, or neither by comparing the actual expenditures for their aligned beneficiaries for each year against their spending benchmarks. Each ACO's spending benchmark is based on the baseline expenditures for the ACO's aligned beneficiaries. Specifically, the spending benchmark incorporates each ACO's actual expenditures for their aligned beneficiaries from 2009 to 2011 and the Medicare national growth rate. CMS subtracts the ACO's expenditures for each year from the ACO's spending benchmark, and if the ACO's expenditures are lower than the benchmark by at least a minimum amount, the ACO has produced shared savings. In contrast, if the ACO's expenditures exceed the benchmark by at least a minimum amount, the ACO has generated shared losses. CMS calculates a dollar amount for each ACO's final annual payment if the ACO generates shared savings or losses. To perform this calculation, CMS multiplies the amount of shared savings or losses by the ACO's final sharing rate. For shared savings, CMS calculates the final sharing rate by multiplying the ACO's sharing rate by its total quality score. As a result, ACOs with higher total quality scores will have higher final sharing rates for savings and thus, will receive a higher portion of any shared savings. To calculate the final sharing rate for losses, CMS first adds 40 percent to the ACO's sharing rate and then subtracts the product of the sharing rate and the ACO's total quality score. quality scores will have lower final sharing rates for losses and thus, will owe CMS a lower portion of any shared losses. The total quality score is calculated with the 33 quality measures that the ACOs report to CMS each year. ACOs earn from 0 to 2 points for each measure, depending on their level of performance relative to the performance benchmarks CMS established. The total quality score is a percentage of the maximum number of points that an ACO can earn for the measures combined. The maximum total quality score is 100 percent. As an example of a final sharing rate for an ACO with savings, an ACO with a sharing rate of 50 percent and a quality score of 80 percent would have a final sharing rate of 40 percent (0.50 x 0.80 = 0.40). In this example, CMS would pay the ACO an amount equal to 40 percent of the shared savings it generated. (See fig. 1.) CMS applies a ceiling to this calculation equal to the sharing rate established in the ACO's payment arrangement. Fewer than half of the ACOs that participated in the Pioneer ACO Model in the first two years earned shared savings in each year, although the ACOs overall produced net shared savings. The 23 ACOs that participated in the model both years had significantly higher quality scores in the second year than in the first year for 67 percent of the quality measures that they reported to CMS. Fewer than half of the ACOs that participated in the Pioneer ACO Model in 2012 and 2013--the first two years of the model--earned savings that were shared with CMS. Of the 32 ACOs that participated in 2012, 13 (about 41 percent) produced about $139 million in total shared savings. Of the 23 ACOs that participated in 2013, 11 (48 percent) produced about $121 million in total shared savings. The amount of shared savings that the 13 ACOs produced in 2012 ($139 million) and the amount the 11 ACOs produced in 2013 ($121 million) each represent about 4 percent of the total expenditures for the ACOs that produced shared savings in each year. The average amount of shared savings that the ACOs produced each year was about $11 million (per ACO with shared savings). CMS provided payments to these ACOs for about 56 percent of the total shared savings each year. For example, in 2012, CMS paid 13 ACOs $77 million of the $139 million that they produced in shared savings. The average payment amount that CMS made to ACOs that produced shared savings was about $6 million in each year. One of the 32 Pioneer ACOs (3 percent) that participated in the first year produced losses that were shared with CMS, and 6 of the 23 participating ACOs (26 percent) produced shared losses in the second year. The total amount of shared losses that the ACO produced in 2012 was $5.1 million, and in 2013 the 6 ACOs produced about $23 million in total shared losses. On average, ACOs with shared losses in 2013 produced $3.8 million each in shared losses, with a range from $2.2 million to $6.3 million. In 2013, ACOs with shared losses paid or are expected to pay CMS about $11 million, an amount equal to about 48 percent of the $23 million in shared losses that they produced. The remaining ACOs did not produce shared savings or shared losses in either year. Eighteen of the 32 ACOs (56 percent) did not produce shared savings or losses in 2012.losses in 2013. (See table 2.) The 23 ACOs that participated in the Pioneer ACO Model in both 2012 and 2013 had significantly higher quality scores in the second year than in the first year for two-thirds of the quality measures (22 of the 33, or 67 percent) that they reported to CMS. We observed significantly higher scores for measures in each of the four quality domains: (1) patient experiences of care; (2) care coordination and patient safety; (3) preventive health care; and (4) disease management for at-risk populations. ACOs demonstrated the most improvement in the disease management for at-risk populations' domain. That is, we found that the ACOs had higher scores in 2013 than in 2012 for 83 percent of the measures (10 of the 12) in this domain. For example, ACOs increased the percentage of beneficiaries with a diagnosis of hypertension whose blood pressure was adequately controlled, from about 65 percent in 2012 to 74 percent in 2013. We observed no significant differences between ACOs' scores in 2012 and 2013 for 10 of the 33 quality measures (30 percent), but we found a statistically significant decline in quality for one measure. Specifically, the rate of hospital admissions for beneficiaries with congestive heart failure was higher in 2013 than in 2012. Table 3 shows the average quality scores in 2012 and 2013 and the quality measures for which we observed significant differences in scores from 2012 to 2013. (See app. I for a summary of the distribution of quality scores in 2012 and 2013.) CMS oversees Pioneer ACOs by monitoring the service use of their aligned beneficiaries and the quality of care provided by the ACOs, and by investigating provider and beneficiary complaints about ACOs. As provided for by law, CMS has reported its evaluation findings publicly for the first year of the Pioneer ACO Model in 2013, and these findings addressed two of the eight research areas that CMS established for the evaluation. CMS oversees Pioneer ACOs by monitoring the service use of their aligned beneficiaries, pursuant to the contract between CMS and ACOs and CMS regulation. CMS monitors beneficiaries' use of services quarterly by reviewing the expenditure and utilization reports that a CMS contractor produces for each ACO, according to CMS officials. The reports include the baseline expenditures for each ACO and expenditures by the type of services that the ACO's aligned beneficiaries have received, such as physician and SNF services. As of February 2015, CMS officials indicated that they had examined two reports about potentially discrepant trends in beneficiaries' use of services. In one case, an ACO raised a concern with CMS that its negative financial performance in the first year did not reflect the actual service use of its aligned beneficiaries. CMS investigated the service use for the beneficiaries aligned to this ACO and observed a sharp increase in expenditures during one time period. CMS officials consulted with the agency's Office of the Actuary to further investigate this trend and determined that a national claims processing error had occurred, but that the correction had not been implemented properly in the affected ACO's geographic region. CMS officials and its contractors corrected the error, and determined that the error did not affect other ACOs in the region. In the second case, an ACO stated that the service use data included in an expenditure and utilization report for the first year of the model could be inaccurate. The ACO believed the data were inaccurate because the service use in the report was higher than the service use of aligned beneficiaries as tracked by the ACO. CMS officials investigated expenditures over time and by service type for the ACO's beneficiaries, compared its expenditures to state and national populations, and determined that the ACO's beneficiaries had a significant increase in SNF service use. The analysis the ACO had presented to CMS included inpatient service use but not SNF use, according to CMS officials. CMS also oversees Pioneer ACOs by monitoring their compliance with the model's quality performance standards, consistent with the contract between CMS and the ACOs and CMS regulation. CMS officials review the annual quality reports that a CMS contractor produces for each ACO, according to agency officials. The quality reports include information about the ACO's performance for each of the 33 quality measures and state whether the ACO achieved the minimum performance standard in each of the four quality domains. CMS determined that one ACO did not meet the quality performance standards in the second year of the model, because it did not meet the minimum standard in the care coordination and patient safety domain. The ACO achieved a score of 40 percent for this quality domain instead of the required minimum score of 70 percent. As a result, CMS required the ACO to submit a corrective action plan to CMS. The plan, provided to CMS in October 2014, outlines steps the ACO will take to ensure future compliance with the quality standards, according to CMS officials. CMS and the ACO discussed and reviewed the submitted corrective action plan in November 2014. CMS officials told us they also review the performance levels for the quality measures to assess whether ACOs may have compromised beneficiary care. That is, they compare the ACOs' scores to the benchmarks for each of the individual quality measures to evaluate the ACOs' performance. For example, each ACO scored over 80 out of 100 in 2013 for the measure reflecting access to specialists--such as surgeons and cardiologists. Further, each ACO's quality score fell into the two highest performance levels, according to CMS's benchmarks. CMS also investigates complaints about Pioneer ACOs that the agency receives from Medicare beneficiaries and providers as part of its monitoring efforts. As of February 2015, CMS officials indicated that it had completed or had begun investigating three complaints. CMS has completed its investigation of a provider complaint that it received from the Department of Health and Human Services' Office of Inspector General in March 2014. In this case, according to CMS officials, a provider alleged that an ACO was inhibiting beneficiaries' choice of home health providers. CMS officials spoke with the ACO in June 2014 and determined that the complaint was unsubstantiated. CMS made this determination after the ACO demonstrated that it had comprehensive procedures in place to avoid restricting beneficiaries' choice of home health providers. CMS is currently investigating two other complaints, one from a beneficiary and the other from a provider. In the first case, CMS received a beneficiary complaint in August 2014 in which the beneficiary alleged that an ACO stinted on care and provided inadequate medical care. CMS officials stated that they are coordinating with representatives from a CMS regional Quality Improvement Organization and CMS's Center for Program Integrity to investigate this complaint, including conducting a full medical chart review. In the second case, CMS is investigating a provider complaint from a SNF alleging that an ACO had placed undue pressure on the SNF to participate in the ACO. CMS officials met with the trade association that submitted the complaint on behalf of the SNF in September 2014, and a CMS contractor has initiated discussions with other SNFs that are affiliated with the ACO under investigation. Through these discussions, CMS officials indicated that they plan to determine whether the ACO misrepresented any information about the Pioneer ACO Model. CMS officials told us that they occasionally receive general queries related to Pioneer ACOs from their regional offices and have asked staff in the regional offices to investigate the queries. Based on its monitoring efforts, CMS has no substantiated evidence suggesting that beneficiary care has been compromised, as of February 2015. For example, CMS has not determined that ACOs have stinted on the care that they provide to beneficiaries or have avoided providing care to at-risk beneficiaries. As provided for by law, CMS has reported its evaluation findings publicly for the first year of the Pioneer ACO Model. The reported findings addressed two of the eight research areas that CMS established for the evaluation--Medicare service use and expenditures and ACO characteristics. CMS issued a public report in November 2013 that included findings related to these two research areas. For example, CMS reported that none of the ACO characteristics it tested, such as organization type, was significantly related to an ACO's ability to reduce expenditures in the first year of the model, and that most of the ACOs that reduced expenditures had higher Medicare expenditures than their comparison groups prior to the start of the Pioneer ACO Model. CMS planned to issue the report in the summer of 2013, and intended to include results for more of the research areas, according to agency officials. However, the release of the report was delayed until November 2013 because of delays in securing the CMS contractor's access to Medicare claims data. The delay also limited the scope of the findings for which CMS could report, according to CMS officials, and these data access issues have since been resolved. In 2015, CMS also plans to report additional findings for the first year of the model. For example, CMS plans to report findings related to quality of care. beneficiaries, (2) access to care, (3) beneficiary quality of care, (4) health care markets, and (5) ACO characteristics. CMS officials added that although they have not made such findings public, they have shared preliminary second year findings internally for five of the eight research areas and their analysis is ongoing for the other three research areas. (See table 4.) The Department of Health and Human Services (HHS) reviewed a draft of this report and provided written comments, which are reprinted in appendix II. In its comments HHS emphasized the Pioneer ACO Model's goal to reduce Medicare costs while providing beneficiaries better care through greater care coordination. HHS also provided technical comments, which we incorporated as appropriate. As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the Secretary of Health and Human Services and other interested parties. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-7114 or KingK@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III. This appendix presents information on the distribution of scores for the 33 quality measures that 23 Pioneer ACOs reported to CMS in 2012 and 2013. (See table 5.) We used the Wilcoxon signed-rank test, a nonparametric test, to analyze the differences in ACOs' quality scores from 2012 to 2013. The signed-rank test determines whether the differences between the median scores for the 2 years are statistically significant. In addition to the contact named above, Martin T. Gahart, Assistant Director; Yesook Merrill, Assistant Director; George Bogart; Pamela Dooley; Toni Harrison; and Roseanne Price made key contributions to this report.

Summary:
ACOs were established in Medicare to provide incentives to physicians and other health care providers to better coordinate care for beneficiaries across care settings such as doctors' offices, hospitals, and skilled nursing facilities. The Pioneer ACO Model was established as a result of the Patient Protection and Affordable Care Act of 2010 creating CMMI in CMS to test new models of service delivery in Medicare. Thirty-two ACOs joined the model in 2012, the first year. Under the model, CMS rewards ACOs that lower their growth in health care spending while meeting performance standards for quality of care. GAO was asked to review the results of the Pioneer ACO Model and CMS's oversight of the ACOs. In this report GAO (1) describes the financial and quality results for the first two years of the model and (2) examines how CMS oversees and evaluates the model. To do this work, GAO analyzed data from CMS on the financial and quality results for each ACO for 2012 and 2013 (the first two years of the model). GAO analyzed ACOs' expenditures, spending benchmarks, the amount of shared savings and losses, and payment amounts for shared savings or losses. GAO also reviewed relevant laws, regulations, and documents describing CMS's oversight and evaluation role and interviewed CMS officials about the agency's oversight and evaluation activities. Health care providers and suppliers voluntarily form accountable care organizations (ACO) to provide coordinated care to patients with the goal of reducing spending while improving quality. Within the Centers for Medicare & Medicaid Services (CMS), the Center for Medicare & Medicaid Innovation (CMMI) began testing the Pioneer ACO Model in 2012. Under this model, ACOs can earn additional Medicare payments if they generate savings, which are shared with CMS, but must pay CMS a penalty if their spending is higher than expected. ACOs must report quality data to CMS annually and meet quality performance standards. GAO found that fewer than half of the ACOs earned shared savings in 2012 and in 2013, although overall the Pioneer ACO Model produced net shared savings in each year. Specifically, Forty-one percent of the ACOs produced $139 million in total shared savings in 2012, and 48 percent produced $121 million in total shared savings in 2013. In 2012 and 2013 CMS paid ACOs $77 million and $68 million, respectively, for their shared savings. The Pioneer ACO Model produced net shared savings of $134 million in 2012 and $99 million in 2013. GAO also found that ACOs that participated in both years had significantly higher quality scores in 2013 than in 2012 for 67 percent of the quality measures. CMS oversees the use of Medicare services by beneficiaries receiving their care from ACOs and the quality of care that ACOs provide, consistent with the contract between CMS and ACOs and CMS regulation, and has reported publicly on findings from its evaluation of the model. CMS reviews reports on each ACO's service use, expenditures, and quality performance and investigates complaints about ACOs. As of February 2015, CMS officials said the agency had investigated two potentially discrepant trends in service use. CMS determined that one ACO did not meet the quality performance standards in 2013, and, as a result, CMS is requiring it to implement an action plan to ensure future compliance. Based on its monitoring efforts, CMS has no substantiated evidence suggesting that beneficiary care has been compromised, as of February 2015. CMS reported publicly on its evaluation findings, as provided for by law, in 2013. CMS included in this initial report findings related to Medicare service use and expenditures and ACO characteristics--two of the eight research areas that it established for the evaluation. CMS officials told GAO that the agency has shared preliminary findings within CMS for five of the six remaining areas and that it plans to report publicly on additional findings in 2015. In commenting on a draft of this report, the Department of Health and Human Services (HHS) emphasized the overall goal of the Pioneer ACO Model. HHS also provided technical comments that GAO incorporated as appropriate.