There are four major steps in the contract-level RADV audit process as reported by CMS: MA contract selection. CMS selects 30 MA organization contracts for contract-level RADV audits, which agency officials stated provides a sufficient representation of contracts (about 5 percent) without imposing unreasonable costs on the agency. An MA organization may have more than one contract selected for a contract-level RADV audit. CMS selects contracts based on diagnosis coding intensity, which the agency defines for each contract as the average change in the risk score component specifically associated with the reported diagnoses for the beneficiaries covered by the contract. That is, increases in coding intensity measure the extent to which the estimated medical needs of the beneficiaries in a contract increase from year to year; thus, contracts whose beneficiaries appear to be getting “sicker” at a relatively rapid rate, based on the information submitted to CMS, will have relatively high coding intensity scores. Contracts with the highest increases in coding intensity are those with beneficiaries whose reported diagnoses increased in severity at the fastest rates. CMS officials stated that the agency adopted this selection methodology to (1) focus the contract-level RADV audits on MA organization contracts that might be more likely to have submitted diagnoses that are not supported by the medical records and (2) provide additional oversight of contracts with the most aggressive coding. To be eligible for a contract-level audit, MA contracts must have had at least three pair-years of data that can be used to distinguish a change in disease risk scores from one year to the next; that is, the contract must have been in place for at least 4 years of continuous payment activity plus the audit year. For each pair year, CMS’s coding intensity calculation excludes beneficiaries not enrolled in the same contract or not eligible for Medicare in consecutive years. CMS ranks contracts by coding intensity and divides them into three categories: high, medium, and low. CMS then randomly selects contracts for audit: 20 from the high category, 5 from the medium category, and 5 from the low category. According to CMS officials, this strategy ensures contracts with the highest coding intensity— considered high risk for improper payments by CMS—have a higher probability for audit while keeping all contracts at risk for review. MA beneficiary sampling. After CMS selects 30 MA contracts to audit, the agency selects the beneficiaries whose medical records will be the focus of review. Up to 201 beneficiaries are chosen from each contract based on the individuals’ risk scores using a stratified random sample: 67 beneficiaries from each of the three risk score groups (highest one-third of risk scores, the middle one-third, and the lowest third). Medical record collection and review. After selecting beneficiaries for review, CMS requests supporting medical record documentation for all diagnoses submitted to adjust risk in the payment year. The MA organization may submit up to five medical records per audited diagnosis. CMS contractors review the submitted medical records to determine if the records support the diagnoses submitted by the MA organizations. If the initial reviewer determines that a diagnosis is not supported, a second reviewer reviews the case. Payment error calculation and extrapolation. When medical record review is completed, CMS extrapolates a payment error rate to the entire contract beginning with contract-level audits of 2011 payments. Each beneficiary’s payment error is multiplied by a sampling weight and the number of months the beneficiary was enrolled in the MA contract during the payment year. After these beneficiary-level payment errors are summed, the amount CMS will seek to recover will be reduced by (1) using the lower limit of a 99 percent confidence interval based on the sample and (2) reducing the recovery amount by a FFS adjuster amount that estimates payment errors that would have likely occurred in FFS claims data. Once the recovery amount is finalized, CMS releases contract-level RADV audit finding reports to each audited MA organization, which may dispute the results of medical record review or appeal the audit findings. Beginning with the contract-level RADV audits of 2011 payments, CMS will collect extrapolated overpayments from MA organizations once all appeals are final. Recovery auditors have been used in various industries, including health care, to identify and collect overpayments. The Medicare Prescription Drug, Improvement, and Modernization Act of 2003 directed CMS to test the use of RACs to identify overpayments and underpayments through a postpayment review of FFS medical claims and recoup overpayments. The Tax Relief and Health Care Act of 2006 required CMS to implement a permanent national recovery audit contractor program by January 1, 2010 and to compensate RACs using a contingency fee structure under which the RACs are paid from recovered overpayments. The Patient Protection and Affordable Care Act expanded the recovery audit program initiated in Medicare FFS to MA plans under Part C, among other things. In future contract-level RADV audits, CMS also will review diagnoses submitted through MA encounter data. While CMS previously collected diagnoses from MA organizations, in 2012 the agency also began collecting encounter data from MA organizations similar to that submitted on FFS claims. CMS requires MA organizations to submit, via the Encounter Data System, encounter data weekly, biweekly, or monthly depending on their number of enrollees. Encounter data include diagnosis and treatment information recorded by providers for all medical services and may either originate from claims that providers submit to MA organizations for payment or from MA organizations’ medical record review. CMS started including the diagnosis information from MA encounter data from 2014 dates of service when calculating 2015 enrollee risk scores. While coding intensity scores can be helpful in assessing the likelihood of improper payments for MA contracts, results from the CMS contract-level RADV audits of 2007 payments indicate that the coding intensity scores CMS calculated were not strongly correlated with the percentage of unsupported diagnoses within a contract. The fact that this correlation is not strong reduces the likelihood that contracts selected for audit would be those most likely to yield large amounts of improper payments and hampers CMS’s goal of using the audits to recover improper payments. In addition, internal control standards for federal agencies state that agencies should use and communicate quality information in achieving program goals. Figure 1 shows, for example, that CMS reported that the percentage of unsupported diagnoses (36.0 percent) among the high coding intensity contracts it audited was nearly identical to the percentage of unsupported diagnoses (35.7 percent) among the medium coding intensity contracts audited. In addition, 7 contracts in the high coding intensity group had unsupported diagnosis rates below 30 percent, including the contract with the highest coding intensity score. Several shortcomings in CMS’s methods for calculating coding intensity could have weakened the correlation between the degree of coding intensity and the percentage of improper payments. These shortcomings and their potential effects are as follows. CMS’s coding intensity calculation may be based on noncomparable coding intensity scores across contracts because (1) the years of data used for each contract may not be the same and (2) coding intensity scores are not standardized to control for year-to-year differences. First, although CMS officials stated that the agency requires at least three pair- years of data for each contract, the agency includes data from all available years for each contract, which may vary between contracts. Because the growth in risk scores was lower in the MA program in earlier years among beneficiaries that continuously enrolled in the program, CMS’s inconsistent standard of years measured for each contract would tend to calculate higher coding intensity scores for contracts that entered the MA market during periods of higher risk score growth. Among beneficiaries who enrolled in MA in consecutive years, the growth in average risk scores was 0.106 from 2004 through 2006, 0.119 from 2006 through 2010, and 0.132 from 2010 through 2013. Second, CMS officials stated that the agency does not standardize its coding intensity data relative to a measure of central tendency. Because CMS’s coding intensity calculation does not account for the expected increase in risk scores during each period of growth, changes in risk scores may be more volatile from year to year than they would likely be if standardized or indexed to a measure of central tendency. CMS’s coding intensity calculation does not distinguish between the diagnoses that were likely coded by providers and the diagnoses that were likely revised by MA organizations. MA organizations may receive diagnoses from providers that are related to services rendered to MA beneficiaries. Because these diagnoses are submitted by providers, the medical records they create may be more likely to support these diagnoses compared with diagnoses that are subsequently coded by the MA organization through medical record chart reviews. For future years, CMS has an available method to distinguish between diagnoses likely submitted by providers to MA organizations and diagnoses that were likely later added by MA organizations. CMS’s Encounter Data System provides a way for MA organizations to designate supplemental diagnoses that the organization added or revised after conducting medical record review. CMS has not outlined plans for incorporating encounter data into its contract selection methodology, even though the encounter data could help target the submitted diagnoses that may be most likely related to improper payments in the future. CMS follows contracts that are renewed or consolidated under a different existing contract within the same MA organization; however, the agency’s coding intensity calculation does not include the prior risk scores of the prior contract in the MA organization’s renewed contract. This may result in overestimated improper payment risk if MA organizations move beneficiaries with higher risk scores—such as those with special needs— into one consolidated contract. CMS’s contract selection methodology did not (1) always target contracts with the highest coding intensity scores, (2) use results from prior contract-level RADV audits, (3) account for contract consolidation, and (4) account for contracts with high enrollment. These shortcomings are impediments to CMS’s goal of recovering improper payments and are counter to federal internal control standards, which require that agencies use quality information to achieve their program goals. For the 2011 contract-level RADV audits, CMS used a contract selection methodology that did not focus on contracts with the highest coding intensity scores. While we found that coding intensity scores are not strongly correlated with diagnostic discrepancies, they are somewhat correlated.