GAO-12-409R Quality Bonus Payment Demonstration

United States Government Accountability Office

Washington, DC 20548

March 21, 2012

The Honorable Orrin G. Hatch

Ranking Member

Committee on Finance

United States Senate

Subject: Medicare Advantage: Quality Bonus Payment Demonstration Undermined 

by High Estimated Costs and Design Shortcomings

Dear Senator Hatch:

The Medicare Advantage (MA) program, an alternative to the original Medicare feefor-service (FFS) program, provides health care coverage to Medicare beneficiaries 

through private health plans offered by organizations under contract with the Centers 

for Medicare & Medicaid Services (CMS). In 2011, about a quarter of all Medicare 

beneficiaries were enrolled in approximately 3,300 MA plans sponsored by 175 MA 

organizations. These organizations generally offer beneficiaries one or more plans to 

choose from—with different coverage, premiums, and cost sharing features—in the 

areas they serve. Also, MA plans may provide additional benefits not offered under 

Medicare FFS, such as reduced cost sharing or vision and dental care coverage. In 

2010, Medicare payments to MA plans totaled about $115 billion, or roughly 

25 percent of all Medicare spending for Part A and Part B.1

CMS determines the amount to pay an MA plan by comparing its bid—the MA plan’s 

projected revenue requirements for providing the traditional Medicare benefit 

package to an average beneficiary—to a benchmark derived from the average 

amount of Medicare FFS spending in the plan’s service area.2

 

1

Medicare Part A includes inpatient hospital, skilled nursing, and some home health services.

Medicare Part B includes physicians’ services, outpatient care, and durable medical equipment. MA 

plans must cover Medicare Part A and Part B benefits except hospice care. 

Most plans receive 

their bid amount plus a rebate based on the difference between their bid and their 

benchmark. The rebate must be used to reduce premiums, reduce cost sharing, or 

offer additional benefits. To help Medicare beneficiaries select an MA plan in their 

area, CMS rates MA contractors on a 5-star scale, with 5 stars indicating the highest 

2

An MA plan’s benchmark is the enrollment-weighted average of the maximum amount Medicare will 

pay in each county within the plan’s service area.
2 GAO-12-409R Quality Bonus Payment Demonstration

quality.3 Plans’ overall star ratings indicate their performance relative to that of all 

other plans on about 50 measures of clinical quality, patient experience, and 

contractor performance.4

The 2010 Patient Protection and Affordable Care Act as amended (PPACA) aligned 

benchmarks more closely with Medicare FFS spending and provided incentives for 

plans to achieve high star ratings.5 PPACA tied the new benchmarks to a 

percentage of average FFS spending in each county and caps them at the prePPACA level. The new benchmarks would be phased in from 2012 to 2017 by 

blending them with the old benchmarks. PPACA also tied MA payments to plans’ 

star ratings in two ways. First, PPACA introduced differential rebates—varying from 

50 to 70 percent—based on plans’ star ratings.6 Second, PPACA provided that plans 

with 4 or more stars receive a bonus that adds 1.5 percent to the PPACA portion of 

their blended benchmark in 2012, 3 percent in 2013, and 5 percent in 2014 and 

beyond.7 CMS’s Office of the Actuary (OACT) estimated that PPACA’s payment 

reforms would reduce Medicare payments to MA plans by $145 billion over 9 years 

and would cause plans to offer less generous benefit packages.8 OACT also 

projected that MA enrollment in 2017 would be half as much as it would have been 

in PPACA’s absence.

Rather than implement PPACA’s bonus structure, CMS announced in November 

2010 that it would conduct a nationwide demonstration from 2012 through 2014 to 

test an alternative method for calculating and awarding bonuses. Compared with 

PPACA, the MA Quality Bonus Payment Demonstration extends the bonuses to 

plans with 3 or more stars, accelerates the phase-in of the bonuses for plans with 4 

or more stars, and increases the size of the bonuses in 2012 and 2013. According to 

CMS, the goal of the demonstration is to test whether a scaled bonus structure 

would lead to larger and faster annual quality improvement for plans at various star 

rating levels compared with what would have occurred under PPACA.

 

3

Although multiple MA plans are typically included in a contract, CMS assigns star ratings at the 

contract level. As a result, every plan covered under the same contract receives the same star rating.

According to CMS estimates, in 2012, about 9 percent of MA beneficiaries are in 5-star plans, about 

20 percent are in 4-star and 4.5-star plans, and about 58 percent are in 3-star and 3.5-star plans. The 

remaining MA beneficiaries are in plans that have fewer than 3 stars or that do not have a star rating.

4

CMS can revise the methodology for calculating star ratings and can also change the cut points for 

the levels of the measures that constitute the ratings. On December 20, 2011, CMS proposed 

changing the 2013 star rating methodology by adding four new measures and modifying seven 

current measures. It was beyond the scope of this report to assess the 5-star quality rating system.

5

For purposes of this report, references to PPACA include the amendments made by the Health Care 

and Education Reconciliation Act of 2010. Pub. L. No. 111-148, §§ 3201-02, 124 Stat. 119, 442, 454 

(2010). Pub. L. No. 111-152, § 1102, 124 Stat. 1029, 1040 (2010).

6

Prior to 2012, if a plan’s bid was less than the benchmark, the plan received a rebate equal to 

75 percent of the difference between the bid and the benchmark. The new rebates will be phased in 

from 2012 through 2014. In 2012, the rebate equals the sum of two-thirds of the old rebate amount 

and one-third of the new rebate amount. In 2013, the rebate will equal the sum of one-third of the old 

rebate amount and two-thirds of the new rebate amount.

7

New plans and plans with low enrollment are also eligible for bonus payments. The bonus 

percentages are doubled for plans in qualifying counties.

8

See OACT, Estimated Financial Effects of the “Patient Protection and Affordable Care Act,” as 

Amended (Baltimore, Md.: Apr. 22, 2010).
3 GAO-12-409R Quality Bonus Payment Demonstration

You asked us to review the cost and design of the MA Quality Bonus Payment 

Demonstration. In this report, we examined (1) cost estimates that have been 

developed for the demonstration; (2) the extent to which the demonstration conforms 

to the principles of budget neutrality; (3) how the demonstration compares in 

budgetary impact, size, and scope with other Medicare demonstrations; and (4) the 

extent to which the design of the demonstration will allow CMS to achieve its stated 

research goal. On November 18, 2011, we presented information on these 

objectives to committee staff (see enc. I). This report contains the results of our 

review and a recommendation that the Secretary of the Department of Health and 

Human Services (HHS) cancel the demonstration and allow the quality bonus 

payment system established by PPACA to take effect.

To examine cost estimates that have been developed for the demonstration, we 

obtained published and unpublished analyses and interviewed representatives from 

OACT and the Kaiser Family Foundation. We also conducted interviews with 

Medicare Payment Advisory Commission representatives and other health care 

researchers. To assess the extent to which the demonstration conforms to the 

principles of budget neutrality, we reviewed information on the budget neutrality 

policy for Medicare demonstrations and conducted interviews with officials from 

CMS and the Office of Management and Budget (OMB). To compare the 

demonstration’s budgetary impact, size, and scope with those of other Medicare 

demonstrations, we reviewed OMB cost estimates published in the President’s 

Budgets from fiscal years 1996 through 2012, as well as CMS documents. To 

assess the extent to which the design of the demonstration will allow CMS to 

achieve its goal, we reviewed CMS’s announcements and evaluation plans as well 

as literature on evaluating Medicare demonstrations.

We conducted this performance audit from October 2011 to March 2012 in 

accordance with generally accepted government auditing standards. Those 

standards require that we plan and perform the audit to obtain sufficient, appropriate 

evidence to provide a reasonable basis for our findings and conclusions based on 

our audit objectives. We believe that the evidence obtained provides a reasonable 

basis for our findings and conclusions based on our audit objectives.

In summary, we found the following:

• OACT has estimated that the MA Quality Bonus Payment Demonstration will cost 

$8.35 billion over 10 years, most of which will be paid to 3-star and 3.5-star 

plans. About $5.34 billion of OACT’s cost estimate is attributed to quality bonus 

payments more generous than those prescribed in PPACA, specifically to 

(1) higher bonuses for 4-star and 5-star plans, (2) new bonuses for 3-star and 

3.5-star plans, (3) applying bonuses to plans’ entire blended benchmarks, and 

(4) allowing plans’ benchmarks to exceed their pre-PPACA levels.9

 

9

Under PPACA, about one-third of MA beneficiaries would be covered by contracts eligible for a 

bonus in 2012 and 2013. In contrast, under the demonstration, about 90 percent of MA beneficiaries 

will be covered by contracts eligible for a bonus in 2012 and 2013.

Most of the 

remaining projected demonstration spending stems from higher MA enrollment 

because the bonuses enable MA plans to offer beneficiaries more benefits or 
4 GAO-12-409R Quality Bonus Payment Demonstration

lower premiums. Taken together, the expanded bonuses and higher MA 

enrollment mainly benefit average performing plans—those receiving 3-star and 

3.5-star ratings.10 In addition, OACT estimated that the demonstration will offset 

more than one-third of the reduction in MA payments projected to occur under 

PPACA during the demonstration years. The largest annual offset will occur in 

2012—71 percent—followed by 32 percent in 2013 and 16 percent in 2014.

• The MA Quality Bonus Payment Demonstration does not—and is not required by 

law to—conform to the principles of budget neutrality. OMB officials told us that 

they considered the costs of the demonstration in the context of other 

administrative actions in the Medicare program that are expected to generate 

savings. However, they did not confirm whether specific offsets were identified to 

account for the total costs of the demonstration.

• The MA Quality Bonus Payment Demonstration dwarfs all other Medicare 

demonstrations—both mandatory and discretionary—conducted since 1995 in its 

estimated budgetary impact and is larger in size and scope than many of them.11

Our review of CMS and OMB data shows that the estimated budgetary impact of 

the demonstration, adjusted for inflation, is at least seven times larger than that 

of any other Medicare demonstration conducted since 1995 and is greater than 

the combined budgetary impact of all of those demonstrations. While the 

demonstration is similar in size and scope to some Part D demonstrations, it is 

unlike many Medicare pay-for-performance demonstrations in that it is 

implemented nationwide and allows all eligible plans or providers to participate.12

• The design of the demonstration precludes a credible evaluation of its 

effectiveness in achieving CMS’s stated research goal—to test whether a scaled 

bonus structure leads to larger and faster annual quality improvement compared 

with what would have occurred under PPACA. Because of the timing of data 

collection—all of the performance data used to determine the 2012 bonus 

payments and nearly all of the data used to determine the 2013 bonus payments 

were collected before the demonstration’s final specifications were published—

the demonstration’s incentives to improve quality can have a full impact only in 

2014. In addition, the demonstration’s bonus percentages are not continuously 

scaled—in 2014, plans with 4, 4.5, and 5 stars will all receive the same 5 percent 

bonus—and its bonus payments do not consistently offer better incentives than 

PPACA to achieve high star ratings in 2013 and 2014. Moreover, because the 

demonstration lacks a direct comparison group, it may not be possible to isolate 

 

10An analysis by the Kaiser Family Foundation estimated that the total cost of the demonstration 

would be about $3 billion in 2012. The authors indicated that expanded bonus payments for 3-star 

and 3.5-star plans would account for a majority of the cost—about $2 billion. G. Jacobson, T. 

Neuman, A. Damico, and J. Huang, Medicare Advantage Plan Star Ratings and Bonus Payments in 

2012 (Washington, D.C.: Kaiser Family Foundation, November 2011).

11The estimated budgetary impact refers to the difference between the total costs of the 

demonstration and the total costs that would occur in its absence.

12The Medicare Part D program provides voluntary, outpatient prescription drug coverage for eligible 

individuals.
5 GAO-12-409R Quality Bonus Payment Demonstration

its effects, and any effects that are observed could be attributable, at least in 

part, to other MA payment and policy changes.

Conclusion

Estimated to cost more than $8 billion, the MA Quality Bonus Payment 

Demonstration offsets a significant portion of PPACA’s MA payment reductions 

during its 3-year time frame. At the same time, the design shortcomings of the 

demonstration may undermine its ability to achieve CMS’s stated research goal—to

test whether a scaled bonus structure leads to larger and faster annual quality 

improvement compared with what would have occurred under PPACA. Rather than 

rewarding only high performing plans, most of the additional payments made under 

the demonstration will accrue to average performing plans. In addition, the 

demonstration’s ability to test an alternative quality improvement incentive structure 

is compromised by its design. The reliance on predemonstration performance data, 

the absence of an appropriate comparison group of MA plans, and design features 

that are inconsistent with its research goal make it unlikely that the demonstration 

will produce meaningful results.

Recommendation for Executive Action

The Secretary of HHS should cancel the MA Quality Bonus Payment Demonstration 

and allow the MA quality bonus payment system established by PPACA to take 

effect. If, at a future date, the Secretary finds that this system does not adequately 

promote quality improvement, HHS should determine ways to modify the system, 

which could include conducting an appropriately designed demonstration.

Agency Comments and Our Evaluation

We obtained written comments on a draft of this report from HHS, which are 

reprinted in enclosure II. HHS did not concur with our recommendation to cancel the 

MA Quality Bonus Payment Demonstration and our finding regarding its design.

After reviewing HHS’s response, we determined that our recommendation is 

warranted and our finding is sound.

Regarding its disagreement with our recommendation, HHS contended that the MA 

quality bonus payment system established by PPACA would not provide an 

immediate incentive for many plans to improve the quality of care delivered to MA 

beneficiaries. It stated that the demonstration addresses this concern by providing 

quality improvement incentives for plans throughout the star ratings continuum. We 

found that bonuses paid in 2012 and 2013 under both PPACA and the 

demonstration would primarily reward past performance, with the demonstration 

doing so far more generously. Moreover, we found that PPACA’s bonus structure in 

2014 provides many plans better incentives than the demonstration to achieve 

higher star ratings. In fact, plans improving from 3.5 to 4 stars would generally 

receive a larger increase in their bonus payment under PPACA. Therefore, we 

maintain that HHS should implement and evaluate the MA quality bonus payment 

system established by PPACA before it considers ways to modify that system.
6 GAO-12-409R Quality Bonus Payment Demonstration

HHS also disagreed with our finding that the design features of the demonstration 

are inconsistent with CMS’s research goal. The agency stated that the 

demonstration provides an incrementally larger quality bonus for each increase in a 

plan’s star rating with the exception of bonuses to 4-star, 4.5-star, and 5-star plans in 

2014. However, as we stated in our report, 4-star and 4.5-star plans receive the 

same bonus percentage as each other in all 3 years of the demonstration.

Furthermore, we found that CMS’s decision to provide all plans with a bonus at least 

as great as the one they would have received under PPACA results in a bonus 

structure that is not continuously scaled and, therefore, conflicts with its stated 

research goal of testing whether a scaled bonus structure leads to larger and faster 

annual quality improvement.

In addition, HHS disagreed with our finding that the timing of data collection 

precludes a credible evaluation of the demonstration. The agency noted that CMS’s 

evaluation contractor will compare the impact of the demonstration—as measured 

by MA plans’ 2012 and 2013 star ratings—to what would have occurred under 

PPACA—as shown in their 2014 star ratings. However, such a comparison fails to 

distinguish between predemonstration and demonstration performance. As we 

stated in our report, the 2012 star ratings were based on data collected almost 

entirely before the demonstration’s final specifications were published in April 2011 

and, therefore, cannot be used to measure the demonstration’s impact. Moreover, 

this comparison confuses the chronological order of events by using the 2014 star 

ratings to represent what would have occurred under PPACA. The 2014 star ratings 

will be based on data collected during the demonstration and, therefore, will reflect 

the demonstration’s incentives. The agency acknowledged this point in stating that 

improvements in plan quality during the demonstration would affect star ratings prior 

to the end of the demonstration in 2014, as well as in future years.

Finally, HHS disagreed with our finding that the demonstration lacks an appropriate 

comparison group. The agency stated that CMS’s evaluation contractor will 

determine the demonstration’s impact on quality improvement by comparing MA 

plans’ performance with that of non-MA plans—specifically, managed care plans 

contracting with Medicare under section 1876 cost contracts, Medicaid plans, and 

commercial plans. We do not believe that these plans constitute an appropriate 

comparison group because they may serve different populations, may follow 

different regulations and policies, and may have different incentives to improve 

quality.

HHS also provided technical comments that we incorporated as appropriate.

– – – – – 

As agreed with your office, unless you publicly announce the contents of this report 

earlier, we plan no further distribution until 30 days from the report date. At that time, 

we will send copies to the CMS Administrator and other interested congressional 

committees. In addition, the report will be available at no charge on the GAO website 

at http://www.gao.gov. 
7 GAO-12-409R Quality Bonus Payment Demonstration

If you or your staff have any questions regarding this report, please contact me at 

(202) 512-7114 or cosgrovej@gao.gov. Contact points for our Offices of 

Congressional Relations and Public Affairs may be found on the last page of this 

report. Individuals making key contributions to this report include Rosamond Katz, 

Assistant Director; Sandra George; David Grossman; and Eric Wedum.

Sincerely yours,

James C. Cosgrove

Director, Health Care
Enclosure I

8 GAO-12-409R Quality Bonus Payment Demonstration

MEDICARE ADVANTAGE: Quality Bonus Payment 

Demonstration Undermined by High Estimated Costs and 

Design Shortcomings

Presentation to staff of the

Senate Committee on Finance

November 18, 2011

(updated)
Enclosure I

9 GAO-12-409R Quality Bonus Payment Demonstration

Introduction

• In 2008, the Centers for Medicare & Medicaid Services (CMS) implemented a 

5-star quality rating system—with 5 stars indicating the highest quality—for 

Medicare Advantage (MA) plans as a tool to help beneficiaries make enrollment 

decisions.* 

• This system assigns an overall star rating derived from measures of clinical quality, 

patient experience, and contract performance. CMS publishes MA plans’ ratings 

on the Medicare website and uses them in its oversight of plans. 

• The 2010 Patient Protection and Affordable Care Act as amended (PPACA) 

expanded the use of star ratings by requiring CMS to award bonus payments to 

MA plans with at least 4 stars beginning in 2012.

*CMS assigns a star rating for each MA contract rather than for individual plans. Contracts typically include 

multiple plans.
Enclosure I

10 GAO-12-409R Quality Bonus Payment Demonstration

Introduction (cont.)

• In November 2010, CMS announced that it would conduct a nationwide 

demonstration from 2012 through 2014 to test an alternative method for calculating 

and awarding quality bonus payments to MA plans. 

• CMS is conducting the demonstration under section 402(a)(1)(A) of the Social 

Security Amendments of 1967, as amended.* 

• The MA Quality Bonus Payment Demonstration 

 extends the bonuses to plans with 3 and 3.5 stars,

 accelerates the phase-in of the bonuses for plans with 4 or more stars, and

 increases the size of the bonuses in 2012 and 2013 compared with PPACA.

• When the demonstration terminates at the end of 2014, MA plans will transition to 

the quality bonus payment system established by PPACA. 

*This section allows CMS to test whether changes in Medicare payment methodologies increase the efficiency 

and economy of Medicare services through the creation of additional incentives without adversely affecting the 

quality of such services.
Enclosure I

11 GAO-12-409R Quality Bonus Payment Demonstration

Introduction (cont.)

• CMS’s stated research goal for the demonstration is to test whether a scaled 

bonus structure leads to larger and faster annual quality improvement for plans at 

various star rating levels compared with what would have occurred under PPACA. 

• According to CMS, the demonstration was designed to reflect several principles, 

such as the following: 

 Providing a strong incentive for MA plans to improve performance at various star rating 

levels.

 Creating a difference between the bonus payment percentages for 4-star and 5-star plans 

to test whether the difference moves plans to achieve 5-star ratings.

 Ensuring that all plans will receive a bonus at least as great as the bonus they would have 

received under PPACA.
Enclosure I

12 GAO-12-409R Quality Bonus Payment Demonstration

Objectives

This report examines

1. cost estimates that have been developed for the demonstration; 

2. the extent to which the demonstration conforms to the principles of 

budget neutrality; 

3. how the demonstration compares in budgetary impact, size, and scope 

with other Medicare demonstrations; and 

4. the extent to which the design of the demonstration will allow CMS to 

achieve its stated research goal.
Enclosure I

13 GAO-12-409R Quality Bonus Payment Demonstration

Scope and Methodology

• To examine cost estimates of the demonstration, we obtained published and 

unpublished analyses and interviewed representatives from CMS’s Office of the 

Actuary (OACT) and the Kaiser Family Foundation. We also conducted interviews 

with Medicare Payment Advisory Commission representatives and other health 

care researchers. 

• To assess the extent to which the demonstration conforms to the principles of 

budget neutrality, we reviewed information on the budget neutrality policy for 

Medicare demonstrations and conducted interviews with officials from CMS and 

the Office of Management and Budget (OMB). 

• To compare the demonstration’s budgetary impact, size, and scope with those of 

other Medicare demonstrations, we reviewed OMB cost estimates published in the 

President’s Budgets from fiscal years 1996 through 2012, as well as CMS 

documents. 

• To assess the extent to which the design of the demonstration will allow CMS to 

achieve its goal, we reviewed CMS’s announcements and evaluation plans as well 

as literature on evaluating Medicare demonstrations.
Enclosure I

14 GAO-12-409R Quality Bonus Payment Demonstration

Background

• In 2011, about a quarter of all Medicare beneficiaries were enrolled in 

approximately 3,300 MA plans included in 555 contracts. In 2010, MA payments 

totaled about $115 billion, or roughly 25 percent of all Medicare spending for 

Part A and Part B.* 

• Since 2006, payments to each MA plan have been determined by the plan’s bid—

the projected cost of providing Medicare Part A and Part B benefits—and a 

benchmark—the maximum amount Medicare will pay for those benefits in the 

plan’s service area. 

 If the plan’s bid is higher than the benchmark, Medicare pays the plan its benchmark and 

enrollees pay the remainder in their monthly premium.

 If the bid is lower than the benchmark, the plan receives its bid and a portion of the 

difference as a rebate, which must be used to reduce premiums, reduce cost sharing, or 

provide additional benefits, such as vision or dental coverage. Prior to 2012, this portion 

was set at 75 percent.

*Medicare Part A services include inpatient hospital, skilled nursing, and some home health services. Medicare Part B services include 

physicians’ services, outpatient care, and durable medical equipment.
Enclosure I

15 GAO-12-409R Quality Bonus Payment Demonstration

Background (cont.)

PPACA changed the way MA plan payments are set in the following ways:

1. Tying benchmarks to projected county-level fee-for-service (FFS) spending.

From 2012 to 2017, benchmarks will be a blend of the pre-PPACA and PPACA 

amounts. The PPACA benchmark formula will be phased in over 2, 4, or 6 years 

depending on the difference between counties’ pre-PPACA and PPACA 

amounts. 

2. Tying rebates to performance beginning in 2012. By 2014, rebates will be set at 

• 70 percent for plans with 4.5 or 5 stars, 

• 65 percent for plans with 3.5 or 4 stars, and

• 50 percent for all other plans. 

3. Establishing bonus payments for plans with 4 or more stars. Bonuses will be 

phased in from 2012 through 2014 and will be applied only to the PPACA 

portion of the blended benchmark. 
Enclosure I

16 GAO-12-409R Quality Bonus Payment Demonstration

Background (cont.)

Quality Bonus Payment Characteristics under PPACA and the Demonstration

PPACA

MA Quality Bonus

Payment Demonstration

How many stars do MA plans need to be eligible for bonuses? 4 or more stars 3 or more stars

What portion of the benchmark are bonuses applied to? Only the PPACA portion The entire blended benchmark

May the blended benchmark exceed the pre-PPACA amount? No Yes

Are rebates tied to star ratings? Yes Yes

Are MA plans in qualifying counties eligible for double bonuses? Yes Yes

Are new MA plans eligible for bonuses? Yes Yes

Are MA plans with low enrollment eligible for bonuses? Yes Yes in 2012 and 2013

Source: GAO analysis of CMS information.

Notes: New MA plans are defined as those in a contract offered by a parent organization that has not had an MA contract in the previous 3 years. These plans lack 

sufficient data to calculate an overall star rating.

MA plans with low enrollment are defined as those in a contract that lacked a sufficient number of enrollees to reliably measure plan performance.

MA plans qualify for double bonuses in counties that meet three conditions: (1) have lower-than-average projected FFS county spending for 2012, (2) had at least 25 

percent of beneficiaries residing in the county enrolled in an MA plan as of December 2009, and (3) were designated as large urban floor counties–those whose 

benchmarks are based on their being located in a Metropolitan Statistical Area with a population of more than 250,000 in 2004. 
Enclosure I

17 GAO-12-409R Quality Bonus Payment Demonstration

 

Background (cont.)

Quality Bonus Payment Percentages under PPACA and the Demonstration

Percentage of benchmark

Overall star rating

PPACA

MA Quality Bonus Payment 

Demonstration

2012 2013 2014 2012 2013 2014

5 stars 1.5 3 5 5 5 5 

4 or 4.5 stars 1.5 3 5 4 4 5 

3.5 stars 0 0 0 3.5 3.5 3.5

3 stars 0 0 0 3 3 3 

Fewer than 3 stars 0 0 0 0 0 0 

New plan 1.5 2.5 3.5 3 3 3.5

Low enrollment plan * * * 3 3 **

Source: GAO analysis of CMS data.

Notes: Under PPACA, the bonus payment percentages would be applied only to the PPACA portion of the blended benchmark. Under the demonstration, the bonus 

percentages will be applied to the entire blended benchmark. Under both PPACA and the demonstration, MA plans in qualifying counties will be eligible for double 

bonuses.

*In the final rule for MA program changes in 2012, CMS stated that low enrollment plans will qualify for bonuses in 2012 and subsequent years. However, CMS did not 

specify the bonus payment percentages for low enrollment plans.

**In the February 2012 call letter for the MA Quality Bonus Payment Demonstration, CMS did not specify the bonus payment percentages for low enrollment plans in 

2014. 
Enclosure I

18 GAO-12-409R Quality Bonus Payment Demonstration

Background (cont.)

• Under PPACA, about one-third 

of MA enrollees would be 

covered by contracts eligible 

for a bonus in 2012 and 2013.

• In contrast, under the

demonstration, about 90 

percent of MA enrollees will be 

covered by contracts eligible 

for a bonus in 2012 and 2013.

Percentage of Enrollees in MA Contracts Eligible for Quality 

Bonus Payments under PPACA and the Demonstration,

2012 and 2013

Percentage

1.5%

Bonus

2.5%

Bonus

3%

Bonus

3%

Bonus

3%

Bonus

3.5%

Bonus

3.5%

Bonus

4%

Bonus

4%

Bonus

5%

Bonus

5%

Bonus

0%

20%

40%

60%

80%

100%

PPACA MA Quality Bonus

Payment

Demonstration

PPACA MA Quality Bonus

Payment

Demonstration

2012 2013

Source: GAO analysis of CMS data.

Notes: The distribution of contracts is weighted by prior enrollment data and is subject to 

change if there are shifts in enrollment. MA plans in qualifying counties may receive 

double bonuses, which are not reflected in this chart. Low enrollment plans are included 

only in the distributions for the MA Quality Bonus Payment Demonstration.
Enclosure I

19 GAO-12-409R Quality Bonus Payment Demonstration

Results

1. The current estimated cost for the MA Quality Bonus Payment 

Demonstration exceeds $8 billion, most of which is paid to 3-star and 

3.5-star plans.

2. The MA Quality Bonus Payment Demonstration does not—and is not 

required by law to—conform to the principles of budget neutrality. 

3. The MA Quality Bonus Payment Demonstration dwarfs all other 

Medicare demonstrations—both mandatory and discretionary—

conducted since 1995 in its estimated budgetary impact and is larger 

in size and scope than many of them.

4. The design of the demonstration precludes a credible evaluation of its 

effectiveness in achieving CMS’s stated research goal.
Enclosure I

20 GAO-12-409R Quality Bonus Payment Demonstration

Finding 1

The current estimated cost for the MA Quality Bonus Payment 

Demonstration exceeds $8 billion, most of which is paid to 3-star 

and 3.5-star plans.
Enclosure I

21 GAO-12-409R Quality Bonus Payment Demonstration

1: OACT’s current estimate of the demonstration’s cost over 10 years is 

over $8 billion, most of it occurring during 2012 to 2014*

• Most of the cost is concentrated in the 3 demonstration years.  $6.83 billion

 Higher bonuses for 4-star and 5-star plans, new $5.34 billion

bonuses for 3-star and 3.5-star plans, applying 

bonuses to plans’ entire blended benchmarks, and allowing 

plans’ benchmarks to exceed their pre-PPACA levels

account for most of the cost in the demonstration years.

 Net cost of higher enrollment in MA as beneficiaries $1.49 billion

switch from FFS because bonuses enable plans to 

offer more benefits or lower premiums

• The remaining cost is almost entirely due to continued higher  $1.52 billion

enrollment in MA in the post-demonstration period.**

Total 10-year cost (2012-2021)  $8.35 billion 

*The budgetary impact of the demonstration is in comparison to the implementation of PPACA.

**A small portion of the remaining cost, $40 million, stems from plans whose star ratings are

 assumed to increase as a result of the demonstration.
Enclosure I

22 GAO-12-409R Quality Bonus Payment Demonstration

1: Bonuses for and additional enrollment in 3-star and 3.5-star plans will 

likely account for most of the cost of the demonstration

• Of the $5.34 billion of projected demonstration spending attributed to expanded 

bonuses, the majority is likely to go to 3-star and 3.5-star plans. 

 This result is due, in part, to a difference in the number of 4-star and 5-star plans in 2013 

and 2014 assumed to exist under the demonstration and PPACA scenarios.

• Most of the remaining projected demonstration spending attributed to higher MA 

enrollment (both during and after the demonstration) is also likely to go to 3-star 

and 3.5-star plans. 

 OACT assumes that the additional MA enrollment stimulated by the demonstration will be 

distributed similarly to the current distribution of enrollees—which is concentrated among 

3-star and 3.5-star plans.
Enclosure I

23 GAO-12-409R Quality Bonus Payment Demonstration

1: Kaiser Family Foundation’s estimate of the cost of the 

demonstration in 2012 exceeds that of OACT

Billions of dollars Kaiser Family 

Foundation estimate*

OACT estimate

New bonus payments to 3-star and 

3.5-star plans

$2.0** $1.8**

Higher bonus payments to 4-star and 

5-star plans

1.0 0.7

Net cost of FFS beneficiaries switching to 

MA plans

Not included*** 0.2

Total $3.0 $2.7

*G. Jacobson, T. Neuman, A. Damico, and J. Huang, Medicare Advantage Plan Star Ratings and Bonus Payments in 2012

(Washington, D.C.: Kaiser Family Foundation, November 2011).

**Includes new plans and low enrollment plans.

***Kaiser’s methodology holds constant other factors, such as changes in enrollment and plans’ bids.
Enclosure I

24 GAO-12-409R Quality Bonus Payment Demonstration

1: Estimated cost of the demonstration offsets a substantial portion of 

the MA payment reductions projected to result from PPACA during the 

demonstration years

• OACT estimates that the demonstration will offset a significant portion of PPACA’s 

MA payment reductions in the demonstration years: 

 71 percent in 2012

 32 percent in 2013

 16 percent in 2014

• Overall, OACT estimates that the demonstration will offset more than one-third of 

PPACA’s projected reductions in MA payments during the demonstration years. 
Enclosure I

25 GAO-12-409R Quality Bonus Payment Demonstration

Finding 2

The MA Quality Bonus Payment Demonstration does not—and is 

not required by law to—conform to the principles of budget 

neutrality. 
Enclosure I

26 GAO-12-409R Quality Bonus Payment Demonstration

2: Although the demonstration is not budget neutral, OMB officials

considered the cost of the demonstration in a programwide context

• Medicare demonstrations conducted under section 402(a)(1)(A) of the Social 

Security Amendments of 1967, as amended, are not required to be budget 

neutral—whereby the total costs of a demonstration cannot exceed the total costs 

in its absence—but OMB generally does require these demonstrations to be budget 

neutral. 

• OMB officials told us that they considered the costs of the MA Quality Bonus 

Payment Demonstration in the context of other administrative actions in the 

Medicare program that are expected to generate savings, such as an adjustment to 

skilled nursing facility payment rates. 

• However, OMB officials did not confirm whether specific offsets were identified to 

account for the total costs of the demonstration.
Enclosure I

27 GAO-12-409R Quality Bonus Payment Demonstration

Finding 3

The MA Quality Bonus Payment Demonstration dwarfs all other 

Medicare demonstrations—both mandatory and discretionary—

conducted since 1995 in its estimated budgetary impact and is 

larger in size and scope than many of them.
Enclosure I

28 GAO-12-409R Quality Bonus Payment Demonstration

3: The MA Quality Bonus Payment Demonstration has the greatest 

estimated budgetary impact of any Medicare demonstration conducted 

since 1995

According to CMS and OMB cost 

estimates, the estimated budgetary 

impact of the MA Quality Bonus 

Payment Demonstration, adjusted 

for inflation, is 

• at least seven times larger than 

any other Medicare demonstration 

conducted since 1995 and

• greater than the estimated 

budgetary impact of all other 

Medicare demonstrations

conducted since 1995 combined.

Top Five Medicare Demonstrations by Estimated Budgetary 

Impact, 1995 to 2021

Outlays in billions of 2011 dollars

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

8.0

9.0

MA Quality 

Bonus Payment 

Demonstration

Social Health 

Maintenance 

Organization I 

Demonstration

Demonstration to 

Limit Annual 

Changes in Part 

D Premiums

Replacement 

Drug 

Demonstration

Demonstration to 

Transition 

Enrollment of 

Low-Income 

Subsidy 

Beneficiaries

Source: GAO analysis of CMS and OMB data.

Notes: Estimated budgetary impact refers to the difference between the total costs of the 

demonstration and the total costs that would occur in its absence. Estimates shown are in 

constant 2011 dollars.
Enclosure I

29 GAO-12-409R Quality Bonus Payment Demonstration

3: The MA Quality Bonus Payment Demonstration is larger in size and 

scope than other Medicare pay-for-performance demonstrations

Unlike other Medicare pay-forperformance demonstrations, 

the MA Quality Bonus Payment 

Demonstration 

• is implemented nationwide and

• allows all eligible plans or 

providers to participate.

Size and Scope of Medicare Pay-for-Performance Demonstrations

Demonstration name

Number of 

affected states

Number of 

participants

MA Quality Bonus Payment Demonstration 50 All MA plans

Premier Hospital Quality Incentive 

Demonstration

38 About 225 hospitals

End-Stage Renal Disease (ESRD) Disease 

Management Demonstration

13 3 providers

Physician Group Practice Demonstration 10 10 physician groups

Cancer Prevention and Treatment 

Demonstration for Ethnic and Racial 

Minorities

7 6 organizations

Care Management for High-Cost 

Beneficiaries Demonstration

7 6 care management 

organizations

Home Health Pay-for-Performance 

Demonstration

7 569 home health 

agencies

Health Care Quality Demonstration 5 3 organizations

Acute Care Episode Demonstration 4 5 organizations

Care Management Performance 

Demonstration

4 Almost 700 

physician groups

Hospital Gainsharing Demonstration 2 2 hospitals

Physician Hospital Collaboration 

Demonstration

1 1 hospital 

consortium

Source: GAO analysis of CMS information. 
Enclosure I

30 GAO-12-409R Quality Bonus Payment Demonstration

3: However, the large size and scope of the MA Quality Bonus Payment 

Demonstration is similar to some Medicare Part D demonstrations*

Like the MA Quality Bonus 

Payment Demonstration, some 

Part D Demonstrations have 

• been implemented nationwide 

and

• allowed all eligible plans to 

participate.

Size and Scope of the MA Quality Bonus Payment Demonstration 

Compared to Medicare Part D Demonstrations

Demonstration name

Number of 

affected states

Eligible 

participants

MA Quality Bonus Payment Demonstration 50 All MA plans

Demonstration to Limit Annual Changes in 

Part D Premiums

50 All Part D plans

Demonstration to Revise the Part D LowIncome Benchmark Calculation50 All Part D plans

Demonstration to Transition Enrollment of 

Low Income Subsidy Beneficiaries

50 All Part D plans

Source: GAO analysis of CMS information.

*The Medicare Part D program provides voluntary, outpatient prescription drug coverage for eligible individuals.
Enclosure I

31 GAO-12-409R Quality Bonus Payment Demonstration

Finding 4

The design of the MA Quality Bonus Payment Demonstration 

precludes a credible evaluation of its effectiveness in achieving 

CMS’s stated research goal. 
Enclosure I

32 GAO-12-409R Quality Bonus Payment Demonstration

4: The timing of data collection limits the extent to which the 

demonstration’s impact on quality improvement can be measured

• The timing of data collection for the 

measures constituting the star ratings will 

compromise an evaluation of the 

demonstration’s impact on quality 

improvement in the first 2 years. 

 In 2012, bonus payments will be based on data 

collected entirely before the demonstration was 

announced in November 2010.

 In 2013, bonus payments will be based on data 

collected almost entirely before the final 

specifications of the demonstration were 

announced in April 2011.

• Accordingly, 2014 is the only year for which 

plans can respond comprehensively to the 

demonstration’s quality improvement 

incentives. 

Data Collection Timeline for the Measures Constituting the 

5-Star Quality Rating System

Demonstration 

year

Release date of 

applicable star ratings Data collection period*

2012 November 10, 2010

(2011 ratings)

January 2009 – June 2010

2013 October 18, 2011

(2012 ratings)

January 2010 – June 2011

2014 Fall 2012

(2013 ratings)

To be determined**

Source: GAO analysis of CMS information.

*Data collection time frames for individual measures vary within this overall period.

**GAO assumes that the dates for the data collection period for 2013 star ratings will 

be similar to those of the 2 preceding years.
Enclosure I

33 GAO-12-409R Quality Bonus Payment Demonstration

4: The demonstration’s design is inconsistent with CMS’s stated 

research goal 

Research goal Design features that conflict with research goal

Test whether a scaled 

bonus structure leads to 

larger and faster annual 

quality improvement for 

plans at various star 

rating levels compared 

with what would have 

occurred under PPACA

• The demonstration’s bonus percentages are not continuously 

scaled.

 In 2012 and 2013, the size of the bonus percentage for 4-star and 

4.5-star plans is the same—4 percent.

 In 2014, the size of the bonus percentage for 4-star, 4.5-star, and 5-star 

plans is the same—5 percent.

• The demonstration’s bonus percentages in 2013 and 2014 do not 

offer all plans better incentives than PPACA to achieve higher 

star ratings.

 Most plans improving from 3.5 to 4 stars would receive a larger increase 

in their bonus payment under PPACA.
Enclosure I

34 GAO-12-409R Quality Bonus Payment Demonstration

• Comparison groups are necessary to isolate the effects of a demonstration from 

other factors in an unbiased manner. However, the demonstration lacks a direct 

comparison group because all MA plans are participating. 

• The effects of the demonstration may be explained, at least in part, by other MA 

payment and policy changes that may encourage plans to improve quality, such as 

the following: 

 PPACA’s method of tying rebates to star ratings. For example, in 2014, PPACA rewards 

plans that improve from 3 to 3.5 stars with a 30 percent increase in their rebate amount and 

plans that improve from 4 to 4.5 stars with a 7.7 percent increase in their rebate amount.

 CMS’s proposed rule establishing procedures under which the agency may terminate 

contracts with MA plans that achieve fewer than 3 stars in 3 consecutive years.*

• CMS officials acknowledged that it will not be possible to isolate the effects of the 

demonstration from other MA payment and policy changes. 

*CMS issued this proposed rule in October 2011. If finally adopted, this proposal is expected to take effect beginning in contract year 

2013. Under this proposal, CMS could terminate MA contracts beginning in contract year 2015 based on their 2013, 2014, and 2015 star 

ratings.

4: Isolating the effects of the demonstration may not be possible
Enclosure II

 35 GAO-12-409R Quality Bonus Payment Demonstration

Comments from the Department of Health and Human Services
Enclosure II

36 GAO-12-409R Quality Bonus Payment Demonstration
Enclosure II

37 GAO-12-409R Quality Bonus Payment Demonstration

(290990)
This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (www.gao.gov). Each weekday afternoon, 

GAO posts on its website newly released reports, testimony, and 

correspondence. To have GAO e-mail you a list of newly posted products, 

go to www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. 

Visit GAO on the web at www.gao.gov. 

Contact:

Website: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Please Print on Recycled Paper.