Medicare Advantage: Changes Improved Accuracy of Risk Adjustment for Certain Beneficiaries
Released: 2012-01-09T00:00:00

Letter

Background

Research has shown that CMS’s method of risk adjusting payments to MA plans to reflect beneficiary health status has become more accurate over time by including more comprehensive information on beneficiaries’ health status. Before 2000, CMS risk adjusted MA payments based only on beneficiaries’ demographic data. From 2000 to 2003, CMS risk adjusted MA payments using a model that was based on beneficiaries’ demographic characteristics and primary inpatient diagnosis associated with the principal reason for an inpatient stay. In 2004, CMS began risk adjusting payments to MA plans based on beneficiaries’ demographic characteristics and major medical conditions, using a set of models called the CMS-Hierarchical Condition Category (CMS-HCC) risk-adjustment models. HCCs are a way of summarizing an individual’s diagnoses into major medical conditions, such as vascular disease or severe head injury. CMS developed and used criteria to determine which HCCs to include in the models. Certain HCCs that did not meet these criteria, such as HCCs that CMS considered particularly discretionary— susceptible to variable or inappropriate coding by providers—were excluded from the models.

The revised community model included two modifications to the current community model: it incorporated a revised set of HCCs and was calibrated with more recent data. To revise the HCCs, CMS worked with a panel of clinical experts to regroup diagnoses into HCCs, and it also reassessed which HCCs to include. This regrouping and reassessment increased the number of HCCs in the model from 70 in the current community model to 87 in the revised community model. Some of the new HCCs in the revised model were previously excluded because they were considered particularly discretionary. Two examples are dementia with and without complications. At the time CMS was determining which HCCs to include in the revised community model, CMS believed that benefits in improved accuracy of payment adjustments from including HCCs for dementia in the revised model outweighed the risks of introducing HCCs for which coding could be discretionary or subject to coding variation. CMS officials intended to mitigate this risk by closely monitoring the coding of dementia by comparing plans’ coding of dementia with benchmarks previously established by CMS.

In March 2011, CMS published an evaluation of the CMS-HCC risk- adjustment models, as required by the 2010 Patient Protection and Affordable Care Act, which presented results on the extent to which the current and revised community models accurately estimated average actual expenditures for selected beneficiary groups. The evaluation found that compared with the current community model, the revised community model, on average, was better at estimating health care expenditures for all FFS community beneficiaries. The evaluation also examined the extent to which the revised community model improved accuracy for certain beneficiary groups. In particular, the evaluation found that the revised community model generally produced small changes in accuracy for beneficiaries with multiple chronic conditions, with the greatest increase in accuracy occurring for beneficiaries with 10 or more HCCs. For beneficiaries with dementia, the evaluation indicated that the revised community model—which, unlike the current community model, included two HCCs for dementia—estimated expenditures more accurately. CMS did not evaluate the performance of the revised community model for beneficiaries who received the Part D LIS and were not dually eligible for Medicare and Medicaid nor did it evaluate the performance of the C-SNP new enrollee model.

In April 2011, revised coding guidelines for Alzheimer’s disease dementia were issued, which raised new concerns for CMS that diagnosing Alzheimer’s disease dementia may be more discretionary. The revised guidelines, developed under the leadership of the National Institutes of Health and the Alzheimer’s Association, expand the definition of dementia caused by Alzheimer’s disease to include mild cognitive impairment and allow clincians to diagnose patients with this pre-Alzheimer’s disease impairment. According to CMS officials, the revised coding guidelines for Alzheimer’s disease dementia increase the risk of including HCCs for dementia in the revised model because they may lead to coding variation or gaming. CMS plans to reassess whether to include HCCs for dementia in the revised model and may decide to include dementia for certain applications of the model and not others.

Effect of Revised Community Model on Payment Accuracy Varied for High-Risk Groups Studied

In our comparison of the current and revised community models, the revised community model slightly reduced the accuracy of MA payment adjustments for beneficiaries with multiple chronic conditions—one of three high-risk groups in our study. The revised community model also slightly reduced accuracy for beneficiaries with a single or no chronic conditions. Specifically, the revised community model reduced the accuracy for beneficiaries with at least two chronic conditions by $164, which was about 1 percent of average actual expenditures. Even with the reduced accuracy, for these beneficiaries the revised community model estimates, on average, were within $169 of actual expenditures, while the current community model estimates were within $5. For beneficiaries with a single or no chronic conditions, the revised community model reduced accuracy by $94, or 2 percent of average actual expenditures.

While the revised community model reduced the accuracy of MA payment adjustments for beneficiaries with multiple chronic conditions as a whole, this model improved the accuracy for the 4 percent of community FFS beneficiaries with six or more chronic conditions by $727 or 2 percent of average actual expenditures. However, the revised community model still underestimated expenditures for this group by $608, about 2 percent of average actual expenditures. (See fig. 1.)

For our second high-risk study group—beneficiaries with low income—as well as for beneficiaries who were not in this group, the revised community model produced MA payment adjustments of a similar magnitude to those produced by the current community model. Specifically, estimates from the revised model and current model differed by only $5 (less than 0.1 percent of average actual expenditures). Both models estimated expenditures for this group of low-income beneficiaries that were within $80 of actual expenditures. Within the low-income group, estimates from the revised and current community models were similar for beneficiaries who received the Part D LIS but were not dually eligible for Medicare and Medicaid, with the revised community model slightly reducing accuracy by $28, or less than 0.5 percent of average actual expenditures. Both the current and revised community models underestimated expenditures for these beneficiaries by about $450 ($435 and $463, respectively). The estimates were substantially more accurate for beneficiaries who were dual-eligibles. This greater accuracy for dual- eligibles relative to beneficiaries who received the Part D LIS but were not dually eligible reflects the design of the models: that both the current and revised community models account for whether a beneficiary is dually eligible and neither model accounts for whether a beneficiary received the Part D LIS. (See fig. 2.)

The revised community model produced a substantial improvement in the accuracy of health care expenditure estimates, and therefore MA payment adjustments, for the approximately 5 percent of beneficiaries diagnosed with dementia. Specifically, the revised community model improved the accuracy of estimated health care expenditures for beneficiaries with dementia by $2,674, or about 16 percent of average actual expenditures. Estimates from the revised model matched actual expenditures exactly for beneficiaries with dementia, reflecting the inclusion in the model of two new HCCs for dementia (dementia with complications and dementia without complications). The magnitude of the improvement in accuracy was greater for beneficiaries diagnosed with dementia with complications and less for beneficiaries diagnosed with dementia without complications. For the 95 percent of FFS community beneficiaries without dementia, the revised community model improved accuracy by $129, or about 2 percent of average actual expenditures. (See fig. 3.)

C-SNP New Enrollee Model Substantially Improved Accuracy of MA Payment Adjustments, but Considerable Inaccuracy Remains for Certain Groups

Compared with the general new enrollee model, the C-SNP new enrollee model substantially improved the accuracy of estimated health care expenditures, and therefore of MA payment adjustments, for C-SNP- eligible new enrollees but still underestimated expenditures for certain Specifically, the C-SNP new enrollee groups by considerable amounts.model underestimated expenditures for C-SNP-eligible new enrollees by $1,461, while the general new enrollee model underestimated expenditures for this group by $3,914—an improvement in accuracy of $2,453, or about 25 percent of average actual expenditures. The amount by which accuracy improved was similar across 14 severe or disabling chronic conditions: $2,402 to $2,723 (a range which represented 7 to 24 percent of average actual expenditures). This result reflects the design of the C-SNP new enrollee model, which increases the expenditure estimates from the general new enrollee model by an amount that does not depend on beneficiaries’ medical conditions. Despite the improved accuracy both on average and for each of the 14 conditions, the C-SNP new enrollee model still underestimated expenditures for beneficiaries who had certain conditions, such as end-stage liver disease or stroke, by more than $15,000. (See fig. 4.)

Although the accuracy of MA payment adjustments with the C-SNP new enrollee model improved, both on average and by condition, the results varied depending on the number of severe or disabling conditions the new enrollees had. The C-SNP new enrollee model reduced the accuracy of expenditure estimates for the lowest-cost group of C-SNP-eligible new enrollees—those who were diagnosed with only 1 of the 14 severe or disabling chronic conditions. Specifically, the overestimate of health care expenditures for this group increased from $1,739 with the general new enrollee model to $4,160 with the C-SNP new enrollee model—a reduction in accuracy of $2,421, or about 62 percent of average actual expenditures. On the other hand, the C-SNP new enrollee model improved the accuracy of estimated health care expenditures of C-SNP eligible new enrollees with 4 or more severe or disabling conditions by $2,521, or about 8 percent of actual average expenditures. However, the C-SNP new enrollee model still underestimated expenditures for this group by over $20,000. (See fig. 5.)

Concluding Observations

Accurately adjusting payments to MA plans is important to help ensure that these plans have the same financial incentive to enroll and care for beneficiaries regardless of their health status or the resources they are expected to consume. Our analysis showed that compared with the current community model, the effect of CMS’s revised community model on payment accuracy varied for the high-risk groups we studied. Specifically, we found that the revised model resulted in slight reductions in accuracy, on average, for beneficiaries diagnosed with multiple chronic conditions, similar levels of accuracy for beneficiaries with low income, and a substantial improvement in accuracy for beneficiaries with dementia. Recent revisions to the coding guidelines for Alzheimer’s disease dementia increased CMS’s concerns that there may be more coding variation or gaming for dementia. Whether CMS decides to implement the revised community model that includes HCCs for dementia will depend on CMS’s assessment of the advantage of more accurate payment adjustment for beneficiaries with dementia compared with the disadvantage of a potential increase in the discretionary coding of dementia. In addition, our analysis showed that compared with the general new enrollee model, the C-SNP new enrollee model substantially improved the accuracy of payment adjustments for new enrollees diagnosed with severe or disabling conditions, on average. However, the model still considerably underestimated expenditures for new enrollees diagnosed with four or more severe or disabling conditions, which could place plans that disproportionately enroll these beneficiaries at a relative financial disadvantage.

Agency Comments and Our Evaluation

CMS provided written comments on a draft of this report, which are reprinted in appendix III.

In its comments, CMS suggested that the report include an assessment of the overall accuracy of the current risk adjustment model. While we agree that an overall assessment of the model’s accuracy would be useful, such an analysis was not within the scope of our work and would have required additional data.

CMS stated that the report places considerable focus on the C-SNP new enrollee model even though there were fewer than 10,000 C-SNP new enrollees in 2010. One of the study’s main objectives was to compare the accuracy of CMS’ C-SNP model for new enrollees with the general new enrollee model. Moreover, we note that 50 percent of the 2007 FFS new enrollee population was eligible to enroll in a C-SNP.

In addition, CMS suggested that the report should include an evaluation of whether the differences in the current and revised risk adjustment models are statistically significant or meaningful. We did not test for statistical differences because we used a large sample, and note that CMS also did not test for statistical differences in its evaluation of the risk adjustment model.allow readers to make their own judgments as to whether these differences are large enough to be meaningful.

We do report dollar differences in the estimates to Finally, CMS commented that the community risk-adjustment models do not include a coefficient for beneficiaries who received the Part D LIS but were not dually eligible because CMS found that such a coefficient was small and not statistically significant. We have included a footnote to that effect in the report.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the Secretary of HHS, interested congressional committees, and others. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-7114 or cosgrovej@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix IV.

Appendix I: Additional Potential Changes to the Medicare Advantage Risk-Adjustment Models

The Centers for Medicare & Medicaid Services (CMS) has been conducting ongoing research on the Medicare Advantage (MA) risk- adjustment models to improve the models’ accuracy in estimating expenditures for beneficiary groups. A major focus of CMS’s research is modifying the models to ensure that they accurately estimate expenditures for high-cost beneficiaries and therefore encourage high- quality care for them. The modifications that CMS proposed in 2010—to revise the diagnoses included in the community model and to introduce the chronic condition special needs plan new enrollee model—are results of this ongoing research.

The following tables present some of these potential changes to the MA risk-adjustment models that CMS considered and describe CMS’s rationale for not implementing them. The tables group the potential changes into three categories: table 1 presents the changes that involved adding new variables to the models, table 2 describes new information sources that CMS considered using, and table 3 presents the changes that involved changing the models’ structure.

Appendix II: Scope and Methodology

This appendix describes the scope and methodology we used to address our two objectives: to compare (1) the accuracy with which the current and revised community models adjust Medicare Advantage (MA) payments to account for differences in beneficiaries’ expected health care expenditures and (2) the accuracy with which the general and chronic condition special needs plan (C-SNP) new enrollee models adjust MA payments to account for differences in beneficiaries’ expected health care expenditures.

Comparing the Accuracy of Current and Revised Community Models

To compare the accuracy with which the current and revised community models would adjust MA payments, we computed the amount by which health care expenditure estimates from the current and revised community models were above or below actual expenditures. We calculated the accuracy of the average estimated health expenditures for a particular group of beneficiaries by subtracting the group’s average actual annualized Medicare expenditures in 2007 from the group’s average estimated expenditures for 2007.community model an improvement in accuracy of MA payment adjustments if the magnitude of the over- or underestimate of health care expenditures from the revised community model was smaller than the over- or underestimate from the current community model.

For our analysis, we used versions of the current and revised community models that were calibrated with the same data (Medicare fee-for-service (FFS) data for 2006 and 2007) and therefore differed only in the hierarchical condition categories (HCC) that were included in the models. Because we compared versions of the current and revised community models calibrated on the same data, our results reflect only the impact of the clinical revisions to the HCCs and not the recalibration that the Centers for Medicare & Medicaid Services (CMS) performs on a periodic basis.

Our study population consisted of a 5 percent nationally representative, random sample of 2007 Medicare FFS community beneficiaries—FFS beneficiaries who, if they had enrolled in an MA plan, would have had their 2007 payments adjusted by the community model. We used Medicare FFS data because health care expenditure data were not available for MA beneficiaries, and we used data from 2007 because this was the most recent version of these data available at the time we began our study. For each beneficiary, we obtained the inputs to the community model (selected demographic characteristics and medical diagnoses from the previous year),portion of the year for which plan payments for the beneficiary would have actual annualized 2007 FFS expenditures, and the been adjusted using the community model. We then used these inputs and the versions of the community models described above to estimate Medicare’s 2007 health care expenditures on behalf of each beneficiary.

Within our 5 percent sample of community beneficiaries, we separately compared the accuracy of the current and revised community models for three high-risk beneficiary groups. We defined these three groups as follows based on their characteristics in the previous year: assigned to at least two of the HCCs in the revised community model, dually eligible for Medicare and Medicaid or received the Part D low- assigned to at least one of the two HCCs in the revised community model associated with dementia: dementia with complications and dementia without complications.

For each high-risk group, we also examined the accuracy of the current and revised community models for beneficiaries who were not in that group to provide context for our findings. For example, in addition to examining the models’ accuracy for community beneficiaries with multiple chronic conditions, we also examined accuracy for beneficiaries with a single chronic condition or no chronic conditions.

Comparing the Accuracy of General and C-SNP New Enrollee Models

To compare the accuracy with which the general and C-SNP new enrollee models adjust payments for C-SNP-eligible new enrollees, we measured the accuracy of the general and C-SNP new enrollee models using the same method we used for comparing the accuracy of the current and revised community models. Similar to our community model analysis, we used versions of the general and C-SNP new enrollee models that were derived using Medicare FFS data for 2006 and 2007. To identify our study population, we started with a 5 percent nationally representative, random sample of 2007 Medicare FFS new enrollees— those beneficiaries who were not enrolled in Medicare Part B during the entire previous calendar year. We then restricted this sample of new enrollees to those who, in 2007, were assigned to HCCs associated with at least 1 of 14 severe or disabling chronic conditions that can be targeted by C-SNPs. This methodology provided us with an estimate of the new enrollees who would have been eligible to enroll in a C-SNP in 2007. We compared the accuracy of the general and C-SNP new enrollee models for these new enrollees (1) overall, (2) by condition, and (3) by number of conditions.

Data Reliability and Limitations

We assessed the reliability of the Medicare data we used for this report by reviewing relevant documentation, performing data checks, and interviewing CMS officials knowledgeable about the CMS-Hierarchical Condition Category (CMS-HCC) risk-adjustment models. We checked our data in three major ways. First, we verified that the beneficiaries in the 5 percent community and new enrollee sample files met their respective inclusion criteria. Second, we verified that we were using the CMS-HCC risk-adjustment models correctly by checking the values of the estimated expenditures and assigned HCCs for several beneficiaries. Third, for beneficiary groups in our analysis that CMS also included in its evaluation, we compared our results with those published by CMS. We determined that the data were sufficiently reliable for the purposes of our study.

Our study has three limitations. First, the accuracy of actual plan payments will depend on the characteristics of beneficiaries who enroll in MA, which may be different from the characteristics of our sample of beneficiaries who were eligible but who did not enroll in an MA plan. Second, we used the same beneficiary sample to examine the accuracy of the community models as CMS used to develop the community models. As a result, for community beneficiary groups defined by characteristics included in one or more of the models, our estimated expenditures will match actual expenditures exactly. More generally, our estimates of the accuracy of health care expenditure estimates from the community models will be more accurate than estimates using data for a different beneficiary sample or for beneficiary data from later years. Third, our sample of C-SNP-eligible new enrollees may not be representative of all new enrollees with severe or disabling chronic conditions because we identified these beneficiaries using HCCs and did not analyze underlying diagnoses. However, given that there is currently variation in how C- SNPs determine beneficiary eligibility, we believe our results are a reasonable estimate of the accuracy of the general and C-SNP new enrollee models for the population of C-SNP eligible new enrollees.

We conducted this performance audit from November 2009 through November 2011 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix III: Comments from the Centers for Medicare & Medicaid Services

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Christine Brudevold, Assistant Director; Alison Binkowski; William Black; Andrew Johnson; Richard Lipinski; Elizabeth Morrison; Merrile Sing; and James Walker, Jr. made key contributions to this report.