Medicare Advantage: Changes Improved Accuracy of Risk Adjustment for Certain Beneficiaries
Published: 2011-12-09T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Centers for Medicare & Medicaid Services (CMS) pays plans in Medicare Advantage (MA)--the private plan alternative to Medicare fee-for-service (FFS)--an amount per beneficiary that is adjusted to reflect beneficiary health status. This adjustment, called risk adjustment, helps ensure that health plans have the same financial incentive to enroll and care for beneficiaries regardless of their health status. In 2010, CMS announced plans to revise the major medical conditions included in its principal risk-adjustment model--the community model--and add a model for new enrollees in chronic condition special needs plans (C-SNP), which target beneficiaries with certain severe or disabling chronic conditions. CMS began using the C-SNP new enrollee model in 2011, in place of the general new enrollee model, to adjust MA payments for new Medicare beneficiaries who enroll in a C-SNP. GAO was asked to examine the accuracy of these models for high-risk beneficiaries. Using data for a nationally representative sample of 2007 FFS beneficiaries, GAO computed the amount that expenditure estimates were above or below actual expenditures for 2007, the most recent data available at the time. GAO compared the accuracy of the current and revised community models for three high-risk groups: beneficiaries with multiple chronic conditions, with low income, and with dementia. GAO compared the accuracy of the general and C-SNP new enrollee models for new enrollees eligible to enroll in a C-SNP.
section_title: What GAO Found
paragraphs: The effect of CMS's revised community model on payment accuracy varied for the high-risk groups studied. Specifically, compared with the current community model, the revised community model slightly reduced the accuracy of MA payment adjustments for beneficiaries with multiple chronic conditions by $164, or about 1 percent of average actual expenditures. For beneficiaries with low income, the accuracy of the revised and the current community models was similar: estimates differed by $5, or less than 0.1 percent of average actual expenditures. For beneficiaries with dementia, the revised community model substantially improved the accuracy of MA payment adjustments by $2,674, or about 16 percent of average actual expenditures.
Compared with the general new enrollee model, the C-SNP new enrollee model substantially improved the accuracy of MA payment adjustments for new enrollees with C-SNP conditions, but considerable inaccuracy in the model's estimates remains for certain groups. The amount by which accuracy improved was similar across 14 severe or disabling chronic conditions: about $2,500. This reflects the design of the C-SNP new enrollee model, which increases expenditure estimates from the general new enrollee model by an amount that does not depend on beneficiaries' medical conditions. However, the C-SNP new enrollee model still underestimated expenditures for C-SNP-eligible new enrollees, on average, by about $1,500 and by more than $15,000 for beneficiaries who had certain conditions, such as end-stage liver disease or stroke. The C-SNP new enrollee model's results varied depending on the number of severe or disabling conditions the beneficiary had. Specifically, the model reduced the accuracy of estimated expenditures for new enrollees with only 1 severe or disabling condition by about 62 percent of average actual expenditures but improved the accuracy for those with 4 or more conditions by about 8 percent. However, the C-SNP new enrollee model still underestimated expenditures for beneficiaries with 4 or more conditions by over $20,000.
Accurate risk adjustment is particularly important for certain high-risk beneficiary groups that are more challenging and costly to treat and may benefit particularly from the coordination of care MA plans can provide. The decision to implement the revised community model that adjusts for dementia will depend on CMS's assessment of the advantages of more accurate payment adjustment for beneficiaries with dementia compared with the potential increase in the discretionary coding of dementia because of revised coding guidelines for Alzheimer's disease dementia published in April 2011. Additionally, while the introduction of the C-SNP new enrollee model improved the accuracy of payment adjustments for eligible new enrollees, on average, the model still considerably underestimated expenditures for certain groups, which could place plans that disproportionately enroll beneficiaries in these groups at a relative financial disadvantage.
In its comments on a draft of this report, CMS suggested that GAO assess the overall accuracy of the current risk adjustment model. GAO did not assess overall model accuracy because such an analysis was not within the scope of GAO's work and would have required additional data.

section_title: Letter
paragraphs: In 2010, the federal government spent approximately $114 billion on the  Medicare Advantage (MA) program, which covered nearly a quarter of all  Medicare beneficiaries. The MA program is an alternative to the original  Medicare fee-for-service (FFS) program, in which private health  insurance plans offer health care coverage to Medicare beneficiaries. MA  plans are required to enroll all eligible Medicare beneficiaries who apply,  regardless of their health status. The Centers for Medicare & Medicaid  Services (CMS), which administers Medicare, pays MA plans a monthly  amount per beneficiary that is adjusted to reflect beneficiary health  status—a process known as risk adjustment. Accurate risk adjustment  helps ensure that health plans have the same financial incentive to enroll  and care for beneficiaries regardless of their health status and avoids the  creation of a financial advantage or disadvantage for health plans solely  on the basis of the health status of enrolled beneficiaries.
Risk-adjustment models are designed to accurately estimate average health care  expenditures for groups of beneficiaries with similar characteristics, but not necessarily to  accurately estimate expenditures for each individual in those groups. adjusted using expenditure estimates from CMS’s community model. The  community model uses a beneficiary’s demographic information and  major medical conditions from a base year to estimate health care  expenditures during the following year. CMS uses a different model,  called the general new enrollee model, for most beneficiaries with less  than 1 complete calendar year of Medicare Part B enrollment. This  model estimates expenditures solely based on beneficiaries’ demographic  information.
Some research has shown that CMS’s community risk-adjustment model  systematically overestimates health care expenditures for some  beneficiary groups and underestimates expenditures for others. Of  particular interest is the degree of the model’s accuracy in estimating  health care expenditures for certain high-risk beneficiary groups, which  are more challenging and costly to treat and may benefit particularly from  the coordination of care MA plans can provide. These groups include  beneficiaries with multiple chronic conditions, low income levels, and  dementia. For instance, one study found that the community risk- adjustment model underestimated expenditures for beneficiaries with  dementia by nearly 16 percent. The potentially inaccurate estimation of  Medicare expenditures for high-risk beneficiary groups may be especially  relevant for new enrollees in chronic condition special needs plans   (C-SNP)—plans that only enroll beneficiaries with at least 1 of 15 severe  or disabling chronic conditions—because the general new enrollee risk- adjustment model does not account for the severe or disabling  condition(s) that an individual must have to enroll in a C-SNP. While  there are relatively few C-SNP new enrollees—fewer than 10,000 in  2010—accurate risk-adjustment is especially important for this group of  beneficiaries as they may benefit particularly from the coordination of care  C-SNPs can provide.
CMS has been conducting ongoing research on how to improve the  accuracy of its MA risk-adjustment models and, in 2010, announced its  plans to implement two major changes.  First, CMS proposed revising  the major medical conditions included in the community model. CMS  initially planned to implement this revised community model in 2011 but  has since announced that the model would not be implemented in 2011   Second, CMS created a new risk-adjustment model to adjust  or 2012.payments for new enrollees who enrolled in C-SNPs. Like the general  new enrollee model, the C-SNP new enrollee model only uses  beneficiaries’ demographic characteristics to estimate expenditures.  However, to account for the relatively higher expected health care  expenditures for C-SNP new enrollees compared with the average new  enrollee in Medicare, the C-SNP new enrollee model estimates higher  expenditures than the general new enrollee model for beneficiaries with  the same set of demographic characteristics. CMS began adjusting MA  payments using the C-SNP new enrollee model in 2011.
Given the importance of accurate risk adjustment, you asked us to  examine how accurately the MA risk-adjustment models estimate health  care expenditures, especially for high-risk beneficiary groups. This report  compares (1) the accuracy with which the current and revised community  models adjust MA payments to account for differences in beneficiaries’  expected health care expenditures and (2) the accuracy with which the  general and C-SNP new enrollee models adjust MA payments to account  for differences in beneficiaries’ expected health care expenditures.
To compare the accuracy with which the current and revised community  models would adjust MA payments, we computed the amount by which  health care expenditure estimates from the current and revised  community models were above or below actual expenditures. We  calculated the accuracy of the average estimated health expenditures for  a particular group of beneficiaries by subtracting the average actual  annualized Medicare expenditures in 2007 from the average estimated  annual expenditures for 2007. We considered the revised model an  improvement in accuracy of MA payment adjustments if the magnitude of  the over- or underestimate of health care expenditures from the revised  model was smaller than the over- or underestimate from the current  model. For example, if the revised model overestimated expenditures by  $100 and the current model underestimated expenditures by $200, this  indicated that the revised model improved accuracy because the  magnitude of the over- or underestimate was smaller than the magnitude  for the current model.
For this analysis, we used versions of the current and revised community  models that differed only in the diagnoses that were included in the  models. We analyzed data on annualized Medicare expenditures and  diagnostic and demographic information for a 5 percent nationally  representative, random sample of 2007 Medicare FFS community  beneficiaries, which was the most recent version of these data  available. The beneficiaries in our sample are those who, if they had  enrolled in an MA plan, would have had their 2007 payments adjusted by  the community model. We used data on beneficiaries in Medicare FFS  because health care expenditure data were not available for MA  beneficiaries.
Using these data, we compared the accuracy of the current and revised  community models for three high-risk beneficiary groups separately. We  defined these three groups as follows based on their characteristics in the  previous year: diagnosed with multiple chronic conditions among those included in  the revised community model;  low income, which we defined as being dually eligible for Medicare  and Medicaid or receiving the Part D low-income subsidy (LIS); diagnosed with dementia with or without complications.
About 36 percent of 2007 FFS community beneficiaries had multiple  chronic conditions, about 23 percent had low income, and about   5 percent had dementia. About 50 percent of 2007 FFS community  beneficiaries had at least one of these characteristics. We also examined  the accuracy of the current and revised community models for  beneficiaries who were not in each group to provide context for our  findings.
To compare the accuracy with which the general and C-SNP new  enrollee models adjust payments, we measured the accuracy of the  general and C-SNP new enrollee models using the same method we  used for comparing the accuracy of the current and revised community  models. We used data on annualized Medicare FFS expenditures and  diagnostic and demographic information for a 5 percent nationally  representative, random sample of 2007 Medicare FFS new enrollees— those beneficiaries who were not enrolled in Medicare during the entire  previous calendar year. We then restricted this 5 percent sample to those  who, in 2007, were assigned to groups of diagnoses—called hierarchical  condition categories (HCC)—that are associated with at least 1 of 14  severe or disabling chronic conditions that can be targeted by C-SNPs.  This methodology provided us with an estimate of the new enrollees who  would have been eligible to enroll in a C-SNP in 2007. Our resulting  sample of C-SNP-eligible new enrollees represented about 50 percent of  2007 FFS new enrollees and about 4 percent of all FFS beneficiaries in  2007. Although C-SNP new enrollees made up less than 2 percent of all  MA new enrollees and less than 0.1 percent of all MA beneficiaries in  2010, we found that about half of all 2007 FFS new enrollees would have  been eligible to enroll in a C-SNP. We compared the accuracy of the  general and C-SNP new enrollee models for these new enrollees   (1) overall, (2) by condition, and (3) by the number of conditions.
We assessed the reliability of the Medicare data we analyzed for this  report by reviewing relevant documentation, performing data checks to  ensure that the data were reasonable and consistent, comparing the data  to published sources, and interviewing CMS officials knowledgeable  about the MA risk-adjustment models. We determined that the data were  sufficiently reliable for the purposes of our study. Our study has three  limitations. First, the accuracy of actual plan payments will depend on the  characteristics of beneficiaries who enroll in MA, which may be different  from the characteristics of our sample of beneficiaries who were eligible  but who did not enroll in MA. Second, we used the same beneficiary  sample to examine the accuracy of the community models that CMS used  to develop them. As a result, for community beneficiary groups defined by  characteristics included in one or both of the models, our estimated  expenditures will match actual expenditures exactly. More generally, our  estimates of the accuracy of health care expenditure estimates from the  community models will be more accurate than estimates using data for a  different beneficiary sample or for beneficiary data from later years. Third,  our sample of C-SNP eligible new enrollees may not be representative of  all new enrollees with severe or disabling chronic conditions because we  identified these beneficiaries using major medical conditions and did not  analyze individual diagnoses. However, given that there is currently  variation in how C-SNPs determine beneficiary eligibility, we believe our  results are a reasonable estimate of the accuracy of the general and   C-SNP new enrollee models for the population of C-SNP-eligible new  enrollees. (See app. II for more detail on our scope and methodology.)
We conducted this performance audit from November 2009 through  November 2011 in accordance with generally accepted government  auditing standards. Those standards require that we plan and perform the  audit to obtain sufficient, appropriate evidence to provide a reasonable  basis for our findings and conclusions based on our audit objectives. We  believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: Research has shown that CMS’s method of risk adjusting payments to  MA plans to reflect beneficiary health status has become more accurate  over time by including more comprehensive information on beneficiaries’  health status. Before 2000, CMS risk adjusted MA payments based only  on beneficiaries’ demographic data. From 2000 to 2003, CMS risk  adjusted MA payments using a model that was based on beneficiaries’  demographic characteristics and primary inpatient diagnosis associated  with the principal reason for an inpatient stay. In 2004, CMS began risk  adjusting payments to MA plans based on beneficiaries’ demographic  characteristics and major medical conditions, using a set of models called  the CMS-Hierarchical Condition Category (CMS-HCC) risk-adjustment  models. HCCs are a way of summarizing an individual’s diagnoses into  major medical conditions, such as vascular disease or severe head  injury. CMS developed and used criteria to determine which HCCs to  include in the models. Certain HCCs that did not meet these criteria,  such as HCCs that CMS considered particularly discretionary— susceptible to variable or inappropriate coding by providers—were  excluded from the models.
The revised community model included two modifications to the current  community model: it incorporated a revised set of HCCs and was  calibrated with more recent data. To revise the HCCs, CMS worked with a  panel of clinical experts to regroup diagnoses into HCCs, and it also  reassessed which HCCs to include. This regrouping and reassessment  increased the number of HCCs in the model from 70 in the current  community model to 87 in the revised community model. Some of the new HCCs in the revised model were previously excluded because they  were considered particularly discretionary. Two examples are dementia  with and without complications. At the time CMS was determining which  HCCs to include in the revised community model, CMS believed that  benefits in improved accuracy of payment adjustments from including  HCCs for dementia in the revised model outweighed the risks of  introducing HCCs for which coding could be discretionary or subject to  coding variation. CMS officials intended to mitigate this risk by closely  monitoring the coding of dementia by comparing plans’ coding of  dementia with benchmarks previously established by CMS.
In March 2011, CMS published an evaluation of the CMS-HCC risk- adjustment models, as required by the 2010 Patient Protection and  Affordable Care Act, which presented results on the extent to which the  current and revised community models accurately estimated average  actual expenditures for selected beneficiary groups. The evaluation  found that compared with the current community model, the revised  community model, on average, was better at estimating health care  expenditures for all FFS community beneficiaries. The evaluation also  examined the extent to which the revised community model improved  accuracy for certain beneficiary groups. In particular, the evaluation found  that the revised community model generally produced small changes in  accuracy for beneficiaries with multiple chronic conditions, with the  greatest increase in accuracy occurring for beneficiaries with 10 or more  HCCs. For beneficiaries with dementia, the evaluation indicated that the  revised community model—which, unlike the current community model,  included two HCCs for dementia—estimated expenditures more  accurately. CMS did not evaluate the performance of the revised  community model for beneficiaries who received the Part D LIS and were  not dually eligible for Medicare and Medicaid nor did it evaluate the  performance of the C-SNP new enrollee model.
In April 2011, revised coding guidelines for Alzheimer’s disease dementia  were issued, which raised new concerns for CMS that diagnosing  Alzheimer’s disease dementia may be more discretionary. The revised  guidelines, developed under the leadership of the National Institutes of  Health and the Alzheimer’s Association, expand the definition of dementia  caused by Alzheimer’s disease to include mild cognitive impairment and  allow clincians to diagnose patients with this pre-Alzheimer’s disease  impairment. According to CMS officials, the revised coding guidelines  for Alzheimer’s disease dementia increase the risk of including HCCs for  dementia in the revised model because they may lead to coding variation  or gaming. CMS plans to reassess whether to include HCCs for dementia  in the revised model and may decide to include dementia for certain  applications of the model and not others.
section_title: Effect of Revised Community Model on Payment Accuracy Varied for High-Risk Groups Studied
paragraphs: In our comparison of the current and revised community models, the  revised community model slightly reduced the accuracy of MA payment  adjustments for beneficiaries with multiple chronic conditions—one of  three high-risk groups in our study. The revised community model also  slightly reduced accuracy for beneficiaries with a single or no chronic  conditions. Specifically, the revised community model reduced the  accuracy for beneficiaries with at least two chronic conditions by $164,  which was about 1 percent of average actual expenditures. Even with the  reduced accuracy, for these beneficiaries the revised community model  estimates, on average, were within $169 of actual expenditures, while the  current community model estimates were within $5. For beneficiaries with  a single or no chronic conditions, the revised community model reduced  accuracy by $94, or 2 percent of average actual expenditures.
While the revised community model reduced the accuracy of MA payment  adjustments for beneficiaries with multiple chronic conditions as a whole,  this model improved the accuracy for the 4 percent of community FFS  beneficiaries with six or more chronic conditions by $727 or 2 percent of  average actual expenditures. However, the revised community model still  underestimated expenditures for this group by $608, about 2 percent of  average actual expenditures. (See fig. 1.)
For our second high-risk study group—beneficiaries with low income—as  well as for beneficiaries who were not in this group, the revised  community model produced MA payment adjustments of a similar  magnitude to those produced by the current community model.  Specifically, estimates from the revised model and current model differed  by only $5 (less than 0.1 percent of average actual expenditures). Both  models estimated expenditures for this group of low-income beneficiaries  that were within $80 of actual expenditures. Within the low-income group,  estimates from the revised and current community models were similar  for beneficiaries who received the Part D LIS but were not dually eligible  for Medicare and Medicaid, with the revised community model slightly  reducing accuracy by $28, or less than 0.5 percent of average actual  expenditures. Both the current and revised community models  underestimated expenditures for these beneficiaries by about $450 ($435  and $463, respectively). The estimates were substantially more accurate  for beneficiaries who were dual-eligibles. This greater accuracy for dual- eligibles relative to beneficiaries who received the Part D LIS but were not  dually eligible reflects the design of the models: that both the current and  revised community models account for whether a beneficiary is dually  eligible and neither model accounts for whether a beneficiary received the  Part D LIS. (See fig. 2.)
The revised community model produced a substantial improvement in the  accuracy of health care expenditure estimates, and therefore MA  payment adjustments, for the approximately 5 percent of beneficiaries  diagnosed with dementia. Specifically, the revised community model  improved the accuracy of estimated health care expenditures for  beneficiaries with dementia by $2,674, or about 16 percent of average  actual expenditures. Estimates from the revised model matched actual  expenditures exactly for beneficiaries with dementia, reflecting the  inclusion in the model of two new HCCs for dementia (dementia with  complications and dementia without complications). The magnitude of the  improvement in accuracy was greater for beneficiaries diagnosed with  dementia with complications and less for beneficiaries diagnosed with  dementia without complications. For the 95 percent of FFS community  beneficiaries without dementia, the revised community model improved  accuracy by $129, or about 2 percent of average actual expenditures.  (See fig. 3.)
section_title: C-SNP New Enrollee Model Substantially Improved Accuracy of MA Payment Adjustments, but Considerable Inaccuracy Remains for Certain Groups
paragraphs: Compared with the general new enrollee model, the C-SNP new enrollee  model substantially improved the accuracy of estimated health care  expenditures, and therefore of MA payment adjustments, for C-SNP- eligible new enrollees but still underestimated expenditures for certain   Specifically, the C-SNP new enrollee  groups by considerable amounts.model underestimated expenditures for C-SNP-eligible new enrollees by  $1,461, while the general new enrollee model underestimated  expenditures for this group by $3,914—an improvement in accuracy of  $2,453, or about 25 percent of average actual expenditures. The amount  by which accuracy improved was similar across 14 severe or disabling  chronic conditions: $2,402 to $2,723 (a range which represented 7 to   24 percent of average actual expenditures). This result reflects the design  of the C-SNP new enrollee model, which increases the expenditure  estimates from the general new enrollee model by an amount that does  not depend on beneficiaries’ medical conditions. Despite the improved  accuracy both on average and for each of the 14 conditions, the C-SNP  new enrollee model still underestimated expenditures for beneficiaries  who had certain conditions, such as end-stage liver disease or stroke, by  more than $15,000. (See fig. 4.)
Although the accuracy of MA payment adjustments with the C-SNP new  enrollee model improved, both on average and by condition, the results  varied depending on the number of severe or disabling conditions the  new enrollees had. The C-SNP new enrollee model reduced the accuracy  of expenditure estimates for the lowest-cost group of C-SNP-eligible new  enrollees—those who were diagnosed with only 1 of the 14 severe or  disabling chronic conditions. Specifically, the overestimate of health care  expenditures for this group increased from $1,739 with the general new  enrollee model to $4,160 with the C-SNP new enrollee model—a  reduction in accuracy of $2,421, or about 62 percent of average actual  expenditures. On the other hand, the C-SNP new enrollee model  improved the accuracy of estimated health care expenditures of C-SNP  eligible new enrollees with 4 or more severe or disabling conditions by  $2,521, or about 8 percent of actual average expenditures. However, the  C-SNP new enrollee model still underestimated expenditures for this  group by over $20,000. (See fig. 5.)
section_title: Concluding Observations
paragraphs: Accurately adjusting payments to MA plans is important to help ensure  that these plans have the same financial incentive to enroll and care for  beneficiaries regardless of their health status or the resources they are  expected to consume. Our analysis showed that compared with the  current community model, the effect of CMS’s revised community model  on payment accuracy varied for the high-risk groups we studied.  Specifically, we found that the revised model resulted in slight reductions  in accuracy, on average, for beneficiaries diagnosed with multiple chronic  conditions, similar levels of accuracy for beneficiaries with low income,  and a substantial improvement in accuracy for beneficiaries with  dementia. Recent revisions to the coding guidelines for Alzheimer’s  disease dementia increased CMS’s concerns that there may be more  coding variation or gaming for dementia. Whether CMS decides to  implement the revised community model that includes HCCs for dementia  will depend on CMS’s assessment of the advantage of more accurate  payment adjustment for beneficiaries with dementia compared with the  disadvantage of a potential increase in the discretionary coding of  dementia. In addition, our analysis showed that compared with the  general new enrollee model, the C-SNP new enrollee model substantially  improved the accuracy of payment adjustments for new enrollees  diagnosed with severe or disabling conditions, on average. However, the  model still considerably underestimated expenditures for new enrollees  diagnosed with four or more severe or disabling conditions, which could  place plans that disproportionately enroll these beneficiaries at a relative  financial disadvantage.
section_title: Agency Comments and Our Evaluation
paragraphs: CMS provided written comments on a draft of this report, which are  reprinted in appendix III.
In its comments, CMS suggested that the report include an assessment  of the overall accuracy of the current risk adjustment model. While we  agree that an overall assessment of the model’s accuracy would be  useful, such an analysis was not within the scope of our work and would  have required additional data.
CMS stated that the report places considerable focus on the C-SNP new  enrollee model even though there were fewer than 10,000 C-SNP new  enrollees in 2010. One of the study’s main objectives was to compare the  accuracy of CMS’ C-SNP model for new enrollees with the general new  enrollee model. Moreover, we note that 50 percent of the 2007 FFS new  enrollee population was eligible to enroll in a C-SNP.
In addition, CMS suggested that the report should include an evaluation  of whether the differences in the current and revised risk adjustment  models are statistically significant or meaningful. We did not test for  statistical differences because we used a large sample, and note that  CMS also did not test for statistical differences in its evaluation of the risk  adjustment model.allow readers to make their own judgments as to whether these  differences are large enough to be meaningful.
We do report dollar differences in the estimates to  Finally, CMS commented that the community risk-adjustment models do  not include a coefficient for beneficiaries who received the Part D LIS but  were not dually eligible because CMS found that such a coefficient was  small and not statistically significant. We have included a footnote to that  effect in the report.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to the Secretary of HHS,  interested congressional committees, and others. In addition, the report  will be available at no charge on the GAO website at http://www.gao.gov.  If you or your staff have any questions about this report, please contact  me at (202) 512-7114 or cosgrovej@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made major contributions to  this report are listed in appendix IV.
section_title: Appendix I: Additional Potential Changes to the Medicare Advantage Risk-Adjustment Models
paragraphs: The Centers for Medicare & Medicaid Services (CMS) has been  conducting ongoing research on the Medicare Advantage (MA) risk- adjustment models to improve the models’ accuracy in estimating  expenditures for beneficiary groups. A major focus of CMS’s research is  modifying the models to ensure that they accurately estimate  expenditures for high-cost beneficiaries and therefore encourage high- quality care for them. The modifications that CMS proposed in 2010—to  revise the diagnoses included in the community model and to introduce  the chronic condition special needs plan new enrollee model—are results  of this ongoing research.
The following tables present some of these potential changes to the MA  risk-adjustment models that CMS considered and describe CMS’s  rationale for not implementing them. The tables group the potential  changes into three categories: table 1 presents the changes that involved  adding new variables to the models, table 2 describes new information  sources that CMS considered using, and table 3 presents the changes  that involved changing the models’ structure.
section_title: Appendix II: Scope and Methodology
paragraphs: This appendix describes the scope and methodology we used to address  our two objectives: to compare (1) the accuracy with which the current  and revised community models adjust Medicare Advantage (MA)  payments to account for differences in beneficiaries’ expected health care  expenditures and (2) the accuracy with which the general and chronic  condition special needs plan (C-SNP) new enrollee models adjust MA  payments to account for differences in beneficiaries’ expected health care  expenditures.
subsections: section_title: Comparing the Accuracy of Current and Revised Community Models
paragraphs: To compare the accuracy with which the current and revised community  models would adjust MA payments, we computed the amount by which  health care expenditure estimates from the current and revised  community models were above or below actual expenditures. We  calculated the accuracy of the average estimated health expenditures for  a particular group of beneficiaries by subtracting the group’s average  actual annualized Medicare expenditures in 2007 from the group’s  average estimated expenditures for 2007.community model an improvement in accuracy of MA payment  adjustments if the magnitude of the over- or underestimate of health care  expenditures from the revised community model was smaller than the  over- or underestimate from the current community model.
For our analysis, we used versions of the current and revised community  models that were calibrated with the same data (Medicare fee-for-service  (FFS) data for 2006 and 2007) and therefore differed only in the  hierarchical condition categories (HCC) that were included in the models.  Because we compared versions of the current and revised community  models calibrated on the same data, our results reflect only the impact of  the clinical revisions to the HCCs and not the recalibration that the  Centers for Medicare & Medicaid Services (CMS) performs on a periodic  basis.
Our study population consisted of a 5 percent nationally representative,  random sample of 2007 Medicare FFS community beneficiaries—FFS  beneficiaries who, if they had enrolled in an MA plan, would have had  their 2007 payments adjusted by the community model. We used  Medicare FFS data because health care expenditure data were not  available for MA beneficiaries, and we used data from 2007 because this  was the most recent version of these data available at the time we began  our study. For each beneficiary, we obtained the inputs to the community  model (selected demographic characteristics and medical diagnoses from  the previous year),portion of the year for which plan payments for the beneficiary would have   actual annualized 2007 FFS expenditures, and the  been adjusted using the community model. We then used these inputs  and the versions of the community models described above to estimate  Medicare’s 2007 health care expenditures on behalf of each beneficiary.
Within our 5 percent sample of community beneficiaries, we separately  compared the accuracy of the current and revised community models for  three high-risk beneficiary groups. We defined these three groups as  follows based on their characteristics in the previous year: assigned to at least two of the HCCs in the revised community model, dually eligible for Medicare and Medicaid or received the Part D low- assigned to at least one of the two HCCs in the revised community  model associated with dementia: dementia with complications and  dementia without complications.
For each high-risk group, we also examined the accuracy of the current  and revised community models for beneficiaries who were not in that  group to provide context for our findings. For example, in addition to  examining the models’ accuracy for community beneficiaries with multiple  chronic conditions, we also examined accuracy for beneficiaries with a  single chronic condition or no chronic conditions.
section_title: Comparing the Accuracy of General and C-SNP New Enrollee Models
paragraphs: To compare the accuracy with which the general and C-SNP new  enrollee models adjust payments for C-SNP-eligible new enrollees, we  measured the accuracy of the general and C-SNP new enrollee models  using the same method we used for comparing the accuracy of the  current and revised community models. Similar to our community model  analysis, we used versions of the general and C-SNP new enrollee  models that were derived using Medicare FFS data for 2006 and 2007.  To identify our study population, we started with a 5 percent nationally  representative, random sample of 2007 Medicare FFS new enrollees— those beneficiaries who were not enrolled in Medicare Part B during the  entire previous calendar year. We then restricted this sample of new  enrollees to those who, in 2007, were assigned to HCCs associated with  at least 1 of 14 severe or disabling chronic conditions that can be targeted  by C-SNPs. This methodology provided us with an estimate of the new  enrollees who would have been eligible to enroll in a C-SNP in 2007.  We compared the accuracy of the general and C-SNP new enrollee  models for these new enrollees (1) overall, (2) by condition, and (3) by  number of conditions.
section_title: Data Reliability and Limitations
paragraphs: We assessed the reliability of the Medicare data we used for this report  by reviewing relevant documentation, performing data checks, and  interviewing CMS officials knowledgeable about the CMS-Hierarchical  Condition Category (CMS-HCC) risk-adjustment models. We checked our  data in three major ways. First, we verified that the beneficiaries in the   5 percent community and new enrollee sample files met their respective  inclusion criteria. Second, we verified that we were using the CMS-HCC  risk-adjustment models correctly by checking the values of the estimated  expenditures and assigned HCCs for several beneficiaries. Third, for  beneficiary groups in our analysis that CMS also included in its  evaluation, we compared our results with those published by CMS. We  determined that the data were sufficiently reliable for the purposes of our  study.
Our study has three limitations. First, the accuracy of actual plan  payments will depend on the characteristics of beneficiaries who enroll in  MA, which may be different from the characteristics of our sample of  beneficiaries who were eligible but who did not enroll in an MA plan.  Second, we used the same beneficiary sample to examine the accuracy  of the community models as CMS used to develop the community  models. As a result, for community beneficiary groups defined by  characteristics included in one or more of the models, our estimated  expenditures will match actual expenditures exactly. More generally, our  estimates of the accuracy of health care expenditure estimates from the  community models will be more accurate than estimates using data for a  different beneficiary sample or for beneficiary data from later years. Third,  our sample of C-SNP-eligible new enrollees may not be representative of  all new enrollees with severe or disabling chronic conditions because we  identified these beneficiaries using HCCs and did not analyze underlying  diagnoses. However, given that there is currently variation in how C- SNPs determine beneficiary eligibility, we believe our results are a  reasonable estimate of the accuracy of the general and C-SNP new  enrollee models for the population of C-SNP eligible new enrollees.
We conducted this performance audit from November 2009 through  November 2011 in accordance with generally accepted government  auditing standards. Those standards require that we plan and perform the  audit to obtain sufficient, appropriate evidence to provide a reasonable  basis for our findings and conclusions based on our audit objectives. We  believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our audit objectives.
section_title: Appendix III: Comments from the Centers for Medicare & Medicaid Services
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Christine Brudevold, Assistant  Director; Alison Binkowski; William Black; Andrew Johnson; Richard  Lipinski; Elizabeth Morrison; Merrile Sing; and James Walker, Jr. made  key contributions to this report.