Medicare: Results from the First Two Years of the Pioneer Accountable Care Organization Model
Published: 2015-04-22T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: ACOs were established in Medicare to provide incentives to physicians and other health care providers to better coordinate care for beneficiaries across care settings such as doctors' offices, hospitals, and skilled nursing facilities. The Pioneer ACO Model was established as a result of the Patient Protection and Affordable Care Act of 2010 creating CMMI in CMS to test new models of service delivery in Medicare. Thirty-two ACOs joined the model in 2012, the first year. Under the model, CMS rewards ACOs that lower their growth in health care spending while meeting performance standards for quality of care.
GAO was asked to review the results of the Pioneer ACO Model and CMS's oversight of the ACOs. In this report GAO (1) describes the financial and quality results for the first two years of the model and (2) examines how CMS oversees and evaluates the model.
To do this work, GAO analyzed data from CMS on the financial and quality results for each ACO for 2012 and 2013 (the first two years of the model). GAO analyzed ACOs' expenditures, spending benchmarks, the amount of shared savings and losses, and payment amounts for shared savings or losses. GAO also reviewed relevant laws, regulations, and documents describing CMS's oversight and evaluation role and interviewed CMS officials about the agency's oversight and evaluation activities.
section_title: What GAO Found
paragraphs: Health care providers and suppliers voluntarily form accountable care organizations (ACO) to provide coordinated care to patients with the goal of reducing spending while improving quality. Within the Centers for Medicare & Medicaid Services (CMS), the Center for Medicare & Medicaid Innovation (CMMI) began testing the Pioneer ACO Model in 2012. Under this model, ACOs can earn additional Medicare payments if they generate savings, which are shared with CMS, but must pay CMS a penalty if their spending is higher than expected. ACOs must report quality data to CMS annually and meet quality performance standards.
GAO found that fewer than half of the ACOs earned shared savings in 2012 and in 2013, although overall the Pioneer ACO Model produced net shared savings in each year. Specifically,
Forty-one percent of the ACOs produced $139 million in total shared savings in 2012, and 48 percent produced $121 million in total shared savings in 2013.
In 2012 and 2013 CMS paid ACOs $77 million and $68 million, respectively, for their shared savings.
The Pioneer ACO Model produced net shared savings of $134 million in 2012 and $99 million in 2013.
GAO also found that ACOs that participated in both years had significantly higher quality scores in 2013 than in 2012 for 67 percent of the quality measures.
CMS oversees the use of Medicare services by beneficiaries receiving their care from ACOs and the quality of care that ACOs provide, consistent with the contract between CMS and ACOs and CMS regulation, and has reported publicly on findings from its evaluation of the model. CMS reviews reports on each ACO's service use, expenditures, and quality performance and investigates complaints about ACOs. As of February 2015, CMS officials said the agency had investigated two potentially discrepant trends in service use. CMS determined that one ACO did not meet the quality performance standards in 2013, and, as a result, CMS is requiring it to implement an action plan to ensure future compliance. Based on its monitoring efforts, CMS has no substantiated evidence suggesting that beneficiary care has been compromised, as of February 2015. CMS reported publicly on its evaluation findings, as provided for by law, in 2013. CMS included in this initial report findings related to Medicare service use and expenditures and ACO characteristics—two of the eight research areas that it established for the evaluation. CMS officials told GAO that the agency has shared preliminary findings within CMS for five of the six remaining areas and that it plans to report publicly on additional findings in 2015.
In commenting on a draft of this report, the Department of Health and Human Services (HHS) emphasized the overall goal of the Pioneer ACO Model. HHS also provided technical comments that GAO incorporated as appropriate.

section_title: Letter
paragraphs: Medicare beneficiaries often receive care from multiple unrelated  providers. Such fragmented care can result in higher costs when  duplicative services are provided, and in lower quality for example, when  beneficiaries experience poor transitions between settings—including  doctors’ offices, hospitals and nursing homes. Further, Medicare  payments are not generally linked to improving care coordination and  quality outcomes. Accountable care organizations (ACO) are  organizations of health care providers and suppliers that come together  voluntarily to provide coordinated care to a defined group of patients with  the goal of reducing spending while improving quality. ACOs that  accomplish these goals may receive additional payments from Medicare,  and ACOs that are successful in coordinating the care of Medicare  beneficiaries and reducing costs could help slow the growth in Medicare  spending while improving quality.
The Patient Protection and Affordable Care Act created the Center for  Medicare & Medicaid Innovation (CMMI) within the Centers for Medicare  & Medicaid Services (CMS) to test innovative payment and service  delivery models to reduce Medicare and Medicaid program expenditures  while preserving or enhancing the quality of care. CMMI is testing the Pioneer ACO Model, in which ACOs can earn  additional Medicare payments if they achieve savings for Medicare and  meet quality performance targets, but are penalized for poor performance  by having to pay CMS a penalty if their spending is higher than expected.
Pub. L. No. 111-148, §§ 3021(a), 10306, 124 Stat. 119, 389, 939 (2010) (adding § 1115A  to the Social Security Act); (codified at 42 U.S.C. § 1315a).
Beginning in 2012, CMS contracted with 32 ACOs to provide services to  Medicare beneficiaries for a 3-year period as part of the Pioneer ACO  Model. In the first year of the model, Pioneer ACOs provided services to  approximately 670,000 beneficiaries. Beneficiaries may continue to obtain  Medicare services from providers not participating in the model.
There is debate among health policy analysts and other stakeholders  about the potential for ACOs, including those in the Pioneer ACO Model,  to achieve significant savings for the Medicare program. For example,  some health policy analysts and other stakeholders believe that the  Pioneer ACO Model has the potential to achieve significant savings and  improvements in quality over time, as demonstrated by early successes  of certain Pioneer ACOs. In contrast, others have expressed concern that  some Pioneer ACOs have increased Medicare spending more than  expected, and that the model may not be sustainable since, as of  December 2014, 13 of the original 32 Pioneer ACOs had withdrawn. In  addition, some health policy analysts have raised concerns that some  ACOs may be restricting Medicare beneficiaries’ access to necessary  services that are expensive, or avoiding beneficiaries with greater health  care needs altogether, in order to reduce expenditures.
CMS is responsible for overseeing Medicare beneficiaries’ use of services  in the Pioneer ACO Model and ACOs’ compliance with the model’s  quality-of-care standards, as broadly defined in the contract between  CMS and Pioneer ACOs and in applicable regulation. Under applicable  law, CMS is also responsible for evaluating the model’s effect on  expenditures and quality performance results and making the evaluation  findings available to the public in a timely manner.
You asked us to describe the results of the Pioneer ACO Model and the  oversight that CMS provides of the ACOs participating in the model. In  this report, we (1) describe the financial and quality performance results  for the first two years of the Pioneer ACO Model and (2) examine how  CMS oversees and evaluates the Pioneer ACO Model.
To describe the financial and quality performance results for the first two  years of the Pioneer ACO Model, we analyzed 2012 and 2013 CMS data  for each ACO in the model. These data were the most recently available  data at the time of our analysis. We used these data to determine the  number of ACOs that had savings or losses shared with Medicare and the  average amount and range of shared savings or losses for each year,  and whether the ACO continued to participate in the model. We analyzed  the average amount and range of payments that CMS made to ACOs  based on their shared savings and the average amount and range of  payments that ACOs made to CMS for their shared losses, for each year.  We also analyzed ACOs’ total quality scores and their actual and  expected expenditures per Medicare beneficiary for each year. We also  used these data to compare ACOs’ scores for the individual quality  measures in the first year to ACOs’ scores in the second year for the  ACOs that participated in the model both years. To determine whether the  differences we observed in quality scores were statistically significant, we  used the Wilcoxon signed-rank test, a nonparametric statistical test. We  reviewed the CMS data for reasonableness and consistency, including  screening for outliers. We also reviewed documentation about the CMS  data and spoke with CMS officials about steps taken to ensure data  reliability. Based on this review, we determined that the data used in this  report were sufficiently reliable for our purposes.
To examine how CMS oversees and evaluates the Pioneer ACO Model,  we reviewed relevant laws, regulations, and documents describing CMS’s  oversight and evaluation responsibilities and activities, and we  interviewed CMS officials. The documents we reviewed included CMS’s  Pioneer ACO Model Innovation Agreement, which provides the template  for contracts between CMS and each ACO and outlines CMS’s oversight  role. We also reviewed an agency plan for evaluating the model, which  includes the evaluation’s research areas, objectives, and study design, as  well as reports that describe findings from CMS’s evaluation of the model.  We interviewed CMS officials to obtain information about the agency’s  ongoing and planned oversight and evaluation activities. We focused on  CMS’s oversight of beneficiaries’ use of Medicare services and the quality  of care that ACOs provide to beneficiaries, as well as CMS’s progress in  publicly reporting information about the evaluation results.
We conducted this performance audit from December 2014 to April 2015  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings based  on our audit objectives.
subsections: section_title: Background
subsections: section_title: Timeline and Goal of Pioneer ACO Model
paragraphs: The Pioneer ACO Model’s overall goal is to improve the delivery of  Medicare services by reducing expenditures while preserving or  enhancing the quality of care for patients. Beginning in 2012, CMS  contracted with ACOs for a 3-year period and subsequently offered a   2-year contract extension to ACOs that completed the first three years.The ACOs are expected to meet the goal of the model, in part, by  coordinating the care they provide to Medicare beneficiaries and  engaging beneficiaries in their own care. CMS designed the model for  organizations with experience in providing coordinated care to  beneficiaries at a lower cost to Medicare. Another goal of the Pioneer  ACO Model is to help inform potential future changes to the agency’s  permanent ACO program, the Medicare Shared Savings Program  (MSSP), which began about 3 months after the Pioneer ACO Model.  MSSP ACOs share less financial risk with CMS than Pioneer ACOs, as  many are not responsible for paying CMS for any losses that they may  generate during their contract period.
section_title: Pioneer ACO Model Participation Requirements
paragraphs: CMS established eligibility requirements for participation in the Pioneer  ACO Model through the request for applications and the contract between  CMS and Pioneer ACOs. The requirements include the following:
Organizational structure. ACOs must be structured to allow the  organization to partner with groups of providers including ACO  professionals such as physicians or physician assistants, to accept  joint responsibility for the cost and quality-of-care outcomes for a  specified group of patients. For example, ACOs may be structured as  ACO professionals in a group practice or as partnerships between  ACO professionals and hospitals.
Care improvement plan. ACOs must implement a care improvement  plan, as they described in their applications. These plans include a  range of care strategies such as providing remote patient monitoring  to beneficiaries with chronic illnesses and engaging beneficiaries  through shared decision making.
Beneficiary protections. ACOs must ensure that their providers and  suppliers make all Medicare-covered services available to  beneficiaries and that they do not inhibit beneficiaries’ freedom of  choice to obtain health services from providers or suppliers not  participating in the model. The ACOs annually provide CMS with a list  of the providers and suppliers that have elected to participate as  Pioneer providers or suppliers.
Quality performance standards. ACOs must completely and  accurately report quality data annually to CMS for 33 measures that  address four quality domains. The four domains are (1) patient  experiences of care, (2) care coordination and patient safety,   (3) preventive health care, and (4) disease management for at-risk  populations, such as beneficiaries with diabetes. ACOs must also  meet performance standards for quality. In the first year (2012), CMS  defined the quality performance standard as completely and  accurately reporting all of the quality measures, regardless of the  ACO’s scores on the measures. Beginning in 2013, CMS required that  ACOs score a minimum level for at least 70 percent of the quality  measures within each of the four quality domains. CMS determined a  minimum performance level for each quality measure, based on  performance benchmarks.quality apply to all participating ACOs.
section_title: CMS’s Oversight and Evaluation Responsibilities
paragraphs: CMS’s oversight and evaluation responsibilities are broadly defined in the  contract between CMS and the Pioneer ACOs and in regulation. CMS is  responsible for monitoring beneficiary service use and investigating any  unusual service use patterns to assess, for example, whether ACOs may  be compromising beneficiary care. CMS is also responsible for monitoring  whether ACOs may be avoiding at-risk beneficiaries. CMS may use a  range of methods to conduct this monitoring, including analyzing  beneficiary and provider complaints, and may investigate patterns  suggesting that an ACO has avoided at-risk beneficiaries. In addition,  CMS’s oversight role includes monitoring ACOs’ compliance with the  quality performance standards. CMMI’s Seamless Care Models Group is  responsible for carrying out the agency’s oversight responsibilities for the  model.
CMS is responsible for conducting an evaluation of the model’s financial  and quality performance results and making the evaluation findings  available to the public in a timely manner. conduct the evaluation and has chosen to focus the evaluation on eight  research areas, based on a conceptual model outlining the pathways in  which various factors can affect ACO performance results. The eight  research areas are (1) Medicare service use and expenditures,   (2) unintended consequences of ACOs, (3) beneficiary access to care,   (4) ACOs’ care coordination activities, (5) quality of care, (6) health care  markets served by ACOs, (7) ACO characteristics, and (8) ACO  attrition.
See 42 U.S.C. § 1315a(b)(4). The law does not specify a timeline for making these  findings available to the public. Under this provision, CMS is responsible for evaluating  models to test innovative payment and service delivery, such as the Pioneer ACO Model.  Taking into account the evaluation findings, the Secretary of Health and Human Services  may expand the duration and scope of the Pioneer ACO Model.
section_title: Beneficiary Alignment and Medicare Expenditures in the Pioneer ACO Model
paragraphs: Medicare beneficiaries are assigned by CMS to Pioneer ACOs based on  their prior use of primary care services. CMS refers to this as “alignment.”  ACOs are responsible for the annual expenditures of their aligned  beneficiaries. CMS determines through an analysis of Medicare claims  data which beneficiaries have received the plurality of their primary care  services from primary care providers affiliated with an ACO in the prior  three years. The ACO’s financial performance is based on the annual  expenditures of its aligned beneficiaries for services covered by Medicare  Parts A and B, which include hospital stays, outpatient services, physician  visits, and skilled nursing facility (SNF) stays. To assess financial  performance, CMS includes the expenditures for services provided by the  ACO as well as by non-ACO Medicare providers since aligned  beneficiaries may continue to obtain services from providers that are not  affiliated with the ACO.
section_title: Pioneer ACO Model Payment Arrangements
paragraphs: Pioneer ACOs chose one of five payment arrangements with CMS that  specified the type of risk sharing and the sharing rates, that is, the  percentage of savings or losses that the ACO shared with CMS. The type  of risk sharing is one- or two-sided. Under one-sided risk sharing, the  ACO may receive a payment from CMS if it generates a minimum amount  of savings but does not owe CMS a payment if it generates losses. In  comparison, an ACO owes CMS a payment if it generates a minimum  amount of losses under two-sided risk sharing, and is eligible to receive a  payment from CMS if it produces savings. Four of the five arrangements  required two-sided risk sharing in the first and second years; the other  arrangement allowed for one-sided risk sharing, but only in the first year.  Half of the ACOs (16 of 32) that participated in the first year selected the  arrangement with one-sided risk sharing in the first year, and half (16 of  32) selected arrangements with two-sided risk sharing. The sharing rate  specifies the maximum amount of savings that the ACO can share with  CMS and the maximum amount of losses that the ACO may share with  CMS. The sharing rates increase from the first to the second year in each  of the payment arrangements. (See table 1.)
section_title: Methodology to Determine ACOs’ Shared Savings, Losses, and Final Payment
paragraphs: CMS determines whether each Pioneer ACO has generated savings,  losses, or neither by comparing the actual expenditures for their aligned  beneficiaries for each year against their spending benchmarks. Each  ACO’s spending benchmark is based on the baseline expenditures for the  ACO’s aligned beneficiaries. Specifically, the spending benchmark  incorporates each ACO’s actual expenditures for their aligned  beneficiaries from 2009 to 2011 and the Medicare national growth rate.  CMS subtracts the ACO’s expenditures for each year from the ACO’s  spending benchmark, and if the ACO’s expenditures are lower than the  benchmark by at least a minimum amount, the ACO has produced shared  savings. In contrast, if the ACO’s expenditures exceed the benchmark by  at least a minimum amount, the ACO has generated shared losses.
CMS calculates a dollar amount for each ACO’s final annual payment if  the ACO generates shared savings or losses. To perform this calculation,  CMS multiplies the amount of shared savings or losses by the ACO’s final  sharing rate. For shared savings, CMS calculates the final sharing rate by  multiplying the ACO’s sharing rate by its total quality score. As a result,  ACOs with higher total quality scores will have higher final sharing rates  for savings and thus, will receive a higher portion of any shared savings.  To calculate the final sharing rate for losses, CMS first adds 40 percent to  the ACO’s sharing rate and then subtracts the product of the sharing rate  and the ACO’s total quality score. quality scores will have lower final sharing rates for losses and thus, will  owe CMS a lower portion of any shared losses. The total quality score is  calculated with the 33 quality measures that the ACOs report to CMS  each year. ACOs earn from 0 to 2 points for each measure, depending on  their level of performance relative to the performance benchmarks CMS  established. The total quality score is a percentage of the maximum  number of points that an ACO can earn for the measures combined. The  maximum total quality score is 100 percent. As an example of a final  sharing rate for an ACO with savings, an ACO with a sharing rate of   50 percent and a quality score of 80 percent would have a final sharing  rate of 40 percent (0.50 x 0.80 = 0.40). In this example, CMS would pay  the ACO an amount equal to 40 percent of the shared savings it  generated. (See fig. 1.)
CMS applies a ceiling to this calculation equal to the sharing rate established in the  ACO’s payment arrangement.
section_title: Fewer Than Half of the Pioneer ACOs Earned Shared Savings, and Scores Increased for Two- Thirds of the Quality Measures
paragraphs: Fewer than half of the ACOs that participated in the Pioneer ACO Model  in the first two years earned shared savings in each year, although the  ACOs overall produced net shared savings. The 23 ACOs that  participated in the model both years had significantly higher quality scores  in the second year than in the first year for 67 percent of the quality  measures that they reported to CMS.
subsections: section_title: Fewer Than Half of Pioneer ACOs Earned Shared Savings in First and Second Years, and the Pioneer ACO Model Produced Net Shared Savings
paragraphs: Fewer than half of the ACOs that participated in the Pioneer ACO Model  in 2012 and 2013—the first two years of the model—earned savings that  were shared with CMS. Of the 32 ACOs that participated in 2012, 13  (about 41 percent) produced about $139 million in total shared savings.  Of the 23 ACOs that participated in 2013, 11 (48 percent) produced about  $121 million in total shared savings. The amount of shared savings that  the 13 ACOs produced in 2012 ($139 million) and the amount the 11  ACOs produced in 2013 ($121 million) each represent about 4 percent of  the total expenditures for the ACOs that produced shared savings in each  year. The average amount of shared savings that the ACOs produced  each year was about $11 million (per ACO with shared savings). CMS  provided payments to these ACOs for about 56 percent of the total  shared savings each year. For example, in 2012, CMS paid 13 ACOs   $77 million of the $139 million that they produced in shared savings. The  average payment amount that CMS made to ACOs that produced shared  savings was about $6 million in each year.
One of the 32 Pioneer ACOs (3 percent) that participated in the first year  produced losses that were shared with CMS, and 6 of the 23 participating  ACOs (26 percent) produced shared losses in the second year. The  total amount of shared losses that the ACO produced in 2012 was   $5.1 million, and in 2013 the 6 ACOs produced about $23 million in total  shared losses. On average, ACOs with shared losses in 2013 produced  $3.8 million each in shared losses, with a range from $2.2 million to   $6.3 million. In 2013, ACOs with shared losses paid or are expected to  pay CMS about $11 million, an amount equal to about 48 percent of the  $23 million in shared losses that they produced. The remaining ACOs  did not produce shared savings or shared losses in either year. Eighteen  of the 32 ACOs (56 percent) did not produce shared savings or losses in  2012.losses in 2013. (See table 2.)
section_title: ACOs Had Significantly Higher Quality Scores in the Second Year for Two- Thirds of the Quality Measures
paragraphs: The 23 ACOs that participated in the Pioneer ACO Model in both 2012  and 2013 had significantly higher quality scores in the second year than  in the first year for two-thirds of the quality measures (22 of the 33, or   67 percent) that they reported to CMS. We observed significantly higher  scores for measures in each of the four quality domains: (1) patient  experiences of care; (2) care coordination and patient safety;   (3) preventive health care; and (4) disease management for at-risk  populations. ACOs demonstrated the most improvement in the disease  management for at-risk populations’ domain. That is, we found that the  ACOs had higher scores in 2013 than in 2012 for 83 percent of the  measures (10 of the 12) in this domain. For example, ACOs increased  the percentage of beneficiaries with a diagnosis of hypertension whose  blood pressure was adequately controlled, from about 65 percent in 2012  to 74 percent in 2013. We observed no significant differences between  ACOs’ scores in 2012 and 2013 for 10 of the 33 quality measures   (30 percent), but we found a statistically significant decline in quality for  one measure. Specifically, the rate of hospital admissions for  beneficiaries with congestive heart failure was higher in 2013 than in  2012. Table 3 shows the average quality scores in 2012 and 2013 and  the quality measures for which we observed significant differences in  scores from 2012 to 2013. (See app. I for a summary of the distribution of  quality scores in 2012 and 2013.)
section_title: CMS Oversees Beneficiary Service Use and Quality, and Has Reported Some of Its Evaluation Findings Publicly
paragraphs: CMS oversees Pioneer ACOs by monitoring the service use of their  aligned beneficiaries and the quality of care provided by the ACOs, and  by investigating provider and beneficiary complaints about ACOs. As  provided for by law, CMS has reported its evaluation findings publicly for  the first year of the Pioneer ACO Model in 2013, and these findings  addressed two of the eight research areas that CMS established for the  evaluation.
subsections: section_title: CMS Oversees Pioneer ACOs by Monitoring Beneficiary Service Use and Quality of Care, and Investigates Complaints about Them
paragraphs: CMS oversees Pioneer ACOs by monitoring the service use of their  aligned beneficiaries, pursuant to the contract between CMS and ACOs  and CMS regulation. CMS monitors beneficiaries’ use of services  quarterly by reviewing the expenditure and utilization reports that a CMS  contractor produces for each ACO, according to CMS officials. The  reports include the baseline expenditures for each ACO and expenditures  by the type of services that the ACO’s aligned beneficiaries have  received, such as physician and SNF services. As of February 2015,  CMS officials indicated that they had examined two reports about  potentially discrepant trends in beneficiaries’ use of services. In one case,  an ACO raised a concern with CMS that its negative financial  performance in the first year did not reflect the actual service use of its  aligned beneficiaries. CMS investigated the service use for the  beneficiaries aligned to this ACO and observed a sharp increase in  expenditures during one time period. CMS officials consulted with the  agency’s Office of the Actuary to further investigate this trend and  determined that a national claims processing error had occurred, but that  the correction had not been implemented properly in the affected ACO’s  geographic region. CMS officials and its contractors corrected the error,  and determined that the error did not affect other ACOs in the region. In  the second case, an ACO stated that the service use data included in an  expenditure and utilization report for the first year of the model could be  inaccurate. The ACO believed the data were inaccurate because the  service use in the report was higher than the service use of aligned  beneficiaries as tracked by the ACO. CMS officials investigated  expenditures over time and by service type for the ACO’s beneficiaries,  compared its expenditures to state and national populations, and  determined that the ACO’s beneficiaries had a significant increase in SNF  service use. The analysis the ACO had presented to CMS included  inpatient service use but not SNF use, according to CMS officials.
CMS also oversees Pioneer ACOs by monitoring their compliance with  the model’s quality performance standards, consistent with the contract  between CMS and the ACOs and CMS regulation. CMS officials review  the annual quality reports that a CMS contractor produces for each ACO,  according to agency officials. The quality reports include information  about the ACO’s performance for each of the 33 quality measures and  state whether the ACO achieved the minimum performance standard in  each of the four quality domains. CMS determined that one ACO did not  meet the quality performance standards in the second year of the model,  because it did not meet the minimum standard in the care coordination  and patient safety domain. The ACO achieved a score of 40 percent for  this quality domain instead of the required minimum score of 70 percent.  As a result, CMS required the ACO to submit a corrective action plan to  CMS. The plan, provided to CMS in October 2014, outlines steps the  ACO will take to ensure future compliance with the quality standards,  according to CMS officials. CMS and the ACO discussed and reviewed  the submitted corrective action plan in November 2014. CMS officials told  us they also review the performance levels for the quality measures to  assess whether ACOs may have compromised beneficiary care. That is,  they compare the ACOs’ scores to the benchmarks for each of the  individual quality measures to evaluate the ACOs’ performance. For  example, each ACO scored over 80 out of 100 in 2013 for the measure  reflecting access to specialists—such as surgeons and cardiologists.  Further, each ACO’s quality score fell into the two highest performance  levels, according to CMS’s benchmarks.
CMS also investigates complaints about Pioneer ACOs that the agency  receives from Medicare beneficiaries and providers as part of its  monitoring efforts. As of February 2015, CMS officials indicated that it had  completed or had begun investigating three complaints. CMS has  completed its investigation of a provider complaint that it received from  the Department of Health and Human Services’ Office of Inspector  General in March 2014. In this case, according to CMS officials, a  provider alleged that an ACO was inhibiting beneficiaries’ choice of home  health providers. CMS officials spoke with the ACO in June 2014 and  determined that the complaint was unsubstantiated. CMS made this  determination after the ACO demonstrated that it had comprehensive  procedures in place to avoid restricting beneficiaries’ choice of home  health providers. CMS is currently investigating two other complaints, one  from a beneficiary and the other from a provider. In the first case, CMS  received a beneficiary complaint in August 2014 in which the beneficiary  alleged that an ACO stinted on care and provided inadequate medical  care. CMS officials stated that they are coordinating with representatives  from a CMS regional Quality Improvement Organization and CMS’s  Center for Program Integrity to investigate this complaint, including  conducting a full medical chart review. In the second case, CMS is  investigating a provider complaint from a SNF alleging that an ACO had  placed undue pressure on the SNF to participate in the ACO. CMS  officials met with the trade association that submitted the complaint on  behalf of the SNF in September 2014, and a CMS contractor has initiated  discussions with other SNFs that are affiliated with the ACO under  investigation. Through these discussions, CMS officials indicated that  they plan to determine whether the ACO misrepresented any information  about the Pioneer ACO Model. CMS officials told us that they  occasionally receive general queries related to Pioneer ACOs from their  regional offices and have asked staff in the regional offices to investigate  the queries.
Based on its monitoring efforts, CMS has no substantiated evidence  suggesting that beneficiary care has been compromised, as of February  2015. For example, CMS has not determined that ACOs have stinted on  the care that they provide to beneficiaries or have avoided providing care  to at-risk beneficiaries.
section_title: CMS Has Reported Evaluation Findings Publicly for the Pioneer ACO Model on Medicare Service Use and Expenditures and ACO Characteristics
paragraphs: As provided for by law, CMS has reported its evaluation findings publicly  for the first year of the Pioneer ACO Model. The reported findings  addressed two of the eight research areas that CMS established for the  evaluation—Medicare service use and expenditures and ACO  characteristics. CMS issued a public report in November 2013 that  included findings related to these two research areas. For example,  CMS reported that none of the ACO characteristics it tested, such as  organization type, was significantly related to an ACO’s ability to reduce  expenditures in the first year of the model, and that most of the ACOs that  reduced expenditures had higher Medicare expenditures than their  comparison groups prior to the start of the Pioneer ACO Model. CMS  planned to issue the report in the summer of 2013, and intended to  include results for more of the research areas, according to agency  officials. However, the release of the report was delayed until November  2013 because of delays in securing the CMS contractor’s access to  Medicare claims data. The delay also limited the scope of the findings for  which CMS could report, according to CMS officials, and these data  access issues have since been resolved.
In 2015, CMS also plans to report additional findings for the first year of the model. For  example, CMS plans to report findings related to quality of care. beneficiaries, (2) access to care, (3) beneficiary quality of care, (4) health  care markets, and (5) ACO characteristics. CMS officials added that  although they have not made such findings public, they have shared  preliminary second year findings internally for five of the eight research  areas and their analysis is ongoing for the other three research areas.  (See table 4.)
section_title: Agency Comments
paragraphs: The Department of Health and Human Services (HHS) reviewed a draft of  this report and provided written comments, which are reprinted in  appendix II. In its comments HHS emphasized the Pioneer ACO Model’s  goal to reduce Medicare costs while providing beneficiaries better care  through greater care coordination. HHS also provided technical  comments, which we incorporated as appropriate.
As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to the Secretary of Health  and Human Services and other interested parties. In addition, the report  will be available at no charge on the GAO website at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-7114 or KingK@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix III.
section_title: Appendix I: Quality Scores for Pioneer Accountable Care Organizations (ACO) in 2012 and 2013
paragraphs: This appendix presents information on the distribution of scores for the 33  quality measures that 23 Pioneer ACOs reported to CMS in 2012 and  2013. (See table 5.) We used the Wilcoxon signed-rank test, a  nonparametric test, to analyze the differences in ACOs’ quality scores  from 2012 to 2013. The signed-rank test determines whether the  differences between the median scores for the 2 years are statistically  significant.
section_title: Appendix II: Comments from the Department of Health and Human Services
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Martin T. Gahart, Assistant  Director; Yesook Merrill, Assistant Director; George Bogart;   Pamela Dooley; Toni Harrison; and Roseanne Price made key  contributions to this report.