Medicare Advantage: Limited Progress Made to Validate Encounter Data Used to Ensure Proper Payments
Released: 2017-01-19T00:00:00

Letter

Background

Compared with RAPS data, MA encounter data contain more data elements, including data on diagnoses not used for risk adjustment. Specifically, MAOs may enter more diagnoses on each encounter data submission than on each RAPS data submission and may transmit encounter data more frequently than RAPS data. (See table 1.) Although there are a number of differences between RAPS data and encounter data, an important distinction is a shift in who is responsible for identifying diagnoses used for risk adjustment. Under RAPS data submissions, MAOs individually analyze all their claims data and only submit data with diagnoses that are relevant for risk adjustment. In contrast, for encounter data submissions, MAOs transmit data to CMS on all enrollee encounters, regardless of whether the encounter contains diagnoses used for risk adjustment.

In addition, the encounter data submission process involves a number of steps and exchanges of information between providers, MAOs, and CMS. First, MAOs collect encounter data—originating from an enrollee’s medical record—from providers to manage and process reimbursement for health care services and supplies for enrollees. After collecting and reviewing these data, MAOs submit the data to CMS through the Encounter Data System. CMS then processes and checks the data for problems. CMS designed the system to reject and return problematic data to MAOs for corrections. MAOs are expected to work with providers to correct the encounter files and resubmit the data to CMS. (See fig. 1.)

CMS Has Made Limited Progress since July 2014 to Validate MA Encounter Data; Stakeholder Organizations Questioned the Effectiveness of CMS Efforts

Since our July 2014 report, CMS has taken additional steps across several activities to ensure that MA encounter data are complete but has yet to fully address data accuracy. (See fig. 2)

The agency has taken the following steps, which address primarily the completeness of encounter data and provide feedback to MAOs on data submission:

Creating a report card with basic statistics on the completeness of encounter data for MAOs. This step partially fulfills CMS’s Medicaid data validation protocol activity to conduct statistical analyses to assess completeness and accuracy. Analyzing values in specific data elements and generating basic statistics on the volume and consistency of data elements can help detect data validity issues. Agency officials told us they are using the report cards to encourage MAOs to submit data more frequently and completely. The report cards contain the following information: quarterly performance indicators. These indicators relate to submission frequency (such as the percentage of biweekly periods with submitted data), data volume (such as the numbers of submitted encounters per 1,000 enrollees), and data quality (such as rejection rates of data submissions). comparisons with other MAOs. These MAO-specific comparisons display an MAO’s volume of encounters—overall and by service type (professional, inpatient, outpatient, and durable medical equipment)—alongside the regional and national averages for both MA encounter data submissions and Medicare FFS claims for each of the past 3 years.

Developing an automated report for MAOs on diagnoses used for risk adjustment. This step partially fulfills the data validation protocol activity to summarize findings on encounter data completeness and accuracy to provide recommendations to MAOs. The automated report identifies diagnoses from MAO encounter data submissions that CMS will use to calculate risk scores for the next payment year. The report is primarily intended to help MAOs ascertain the basis of enrollee risk scores, though representatives from health insurance trade associations told us that they have also used the automated reports to prepare internal financial projections and compare patient diagnoses between encounter data and RAPS data submissions. MAOs first received these reports in December 2015, and since then, CMS has modified the report layout in response to MAO feedback. According to agency officials, CMS has finalized the initial version of the automated report and is distributing the automated reports to MAOs on a monthly basis. Further, the agency intends to make technical changes as necessary in the future.

According to agency officials, they finalized the protocol for validating MA encounter data in November 2016 and have begun implementing several parts of it. In September 2016, CMS awarded a contract to update the protocol and report annually on the implementation and outcomes of protocol activities.

The stakeholder organizations we interviewed raised several issues with CMS’s recent actions to ensure the completeness of MA encounter data.

The main issues mentioned by several stakeholder organizations included the following:

Errors in identifying diagnoses used for risk adjustment. Representatives from health insurance trade associations we interviewed criticized CMS’s process for identifying diagnoses that are relevant for risk adjustment. First, they stated that MAOs question the integrity of CMS’s data processing. They noted, for example, that the automated reports MAOs receive had missing procedure codes for some encounters where the original data submissions had included them. CMS told us that they are working with MAOs to make needed corrections to these reports. Second, representatives said that MAOs have been unable to replicate CMS’s analyses because CMS has made adjustments to how it identifies diagnoses eligible for risk adjustment using encounter data. As a result, they say, MAOs are unsure whether CMS is properly distinguishing diagnoses that are used for risk adjustment from those that are not used. When asked about this concern, CMS officials noted that they publicly announced how the agency intends to implement the risk adjustment transition in December 2015, and the methodology has not changed.

Inclusion of encounter data elements considered irrelevant. Representatives we interviewed from some health insurance and provider trade associations questioned CMS’s inclusion and checks of data elements the agency does not use for risk adjustment and which they contend are irrelevant for the purposes enumerated in the August 2014 final rulemaking. CMS’s Encounter Data System is designed to reject erroneous information by applying a subset of the edits used to process FFS claims, which are not all relevant to MA encounter data, according to CMS officials. CMS requires MAOs to make corrections and resubmit the data for all rejected encounters. Representatives stated that both MAOs and providers must dedicate significant resources to meet CMS requirements. In particular, they said MAOs must alter their data systems and submit numerous requests to providers for data corrections and medical record reviews. When asked about this comment, CMS officials noted that the agency wants encounter data elements to be comparable to FFS claims data. Although not all of the encounter data elements are used for risk adjustment purposes, CMS noted that they should be reliable, comprehensive, and complete in the event they are used for any authorized purposes.

Technical problems with encounter data submission.

Representatives we interviewed from several health insurance and provider trade associations reported that MAOs have experienced difficulties with certain data submissions. They cited, for example, difficulty with submitting encounters for recurring services, such as physical therapy. While MAOs and providers typically record such services as a single encounter, they must submit multiple separate encounters to CMS for recurring services. Agency officials told us they had not heard about this problem from MAOs. In addition, stakeholder representatives mentioned challenges resulting from frequent, randomly scheduled changes to data submission requirements, which they say generate costs and confusion. CMS stated that technical changes occur quarterly, which is typical for other CMS data collection efforts. Agency officials pointed out that CMS has worked with some of the larger MAOs to address data submission issues.

Inadequate CMS communication with individual MAOs. Although some researchers praised CMS officials for their assistance and support, representatives from several health insurance trade associations told us that their members are dissatisfied with CMS’s communication efforts. Representatives noted that CMS’s webinars are not designed to facilitate a conversation between the agency and MAOs, and that the email address CMS set up to handle questions from MAOs does not produce responses that can be shared across MAOs in a timely fashion. CMS officials told us that its response time for emailed questions on MA encounter data largely depends on the complexity of the issue. They said that the agency has not been able to provide individualized assistance because of resource limitations, but has recently contracted with an external organization to provide on-site assistance to MAOs to improve their data submission processes.

Although CMS has taken several steps to ensure the completeness of MA encounter data, as of October 2016, the agency had yet to take a number of other important steps identified in the agency’s Medicaid protocol. Steps CMS has not taken include the following: establish benchmarks for completeness and accuracy. This step would address the data validation protocol activity to establish requirements for collecting and submitting MA encounter data. Without benchmarks, CMS has no objective standards against which it could hold MAOs accountable for complete and accurate data reporting. conduct analyses to compare against established benchmarks.

This step would address the data validation protocol activity to conduct statistical analyses to ensure accuracy and completeness. Without such analyses, CMS is limited in its ability to detect potentially inaccurate or unreliable data. determine sampling methodology for medical record review and obtain medical records. This step would address the data validation protocol activity to review medical records to ensure the accuracy of encounter data. Without medical record reviews, CMS cannot substantiate the information in MAO encounter data submissions and lacks evidence for determining the accuracy of encounter data. summarize analyses to highlight individual MAO issues. This step would address the data validation protocol activity to provide recommendations to MAOs for improving the completeness and accuracy of encounter data. Without actionable and specific recommendations from CMS, MAOs might not know how to improve their encounter data submissions.

To the extent that CMS is making payments based on data that have not been fully validated for completeness and accuracy, the soundness of billions of dollars in Medicare expenditures remains unsubstantiated. Given the limited progress CMS has made, we continue to believe that the agency should complete all the steps necessary to validate the data before using them to risk adjust payments or for other intended purposes, as we recommended in our July 2014 report.

CMS Has Detailed Plans for Using MA Encounter Data for Risk Adjustment but Not for Other Purposes, Creating Unease among Some Stakeholder Organizations

CMS Plans to Fully Transition to Using MA Encounter Data for Risk Adjustment Purposes by 2020, Generating Mixed Reactions from Some Stakeholder Organizations

Since our July 2014 report, CMS has made progress in defining its objectives for using MA encounter data for risk adjustment purposes and in communicating its plans and time frames to MAOs. Although additional work is needed, CMS has improved its ability to manage a key aspect of the MA program.

In April 2014, CMS announced that it would begin incorporating patient diagnoses from MA encounter data submissions into risk score calculations. For 2015 MAO payments, CMS used encounter data diagnoses as an additional source of diagnoses to compute risk scores. CMS supplemented the diagnoses from each enrollee’s RAPS data file with the diagnoses from each enrollee’s MA encounter data file. For 2016, CMS used a different process that increased the importance of encounter data in computing risk scores. Specifically, CMS calculated risk scores as follows:

CMS determined two separate risk scores for each enrollee. CMS based one risk score on the diagnoses from each enrollee’s RAPS data file and the other risk score on the diagnoses from each enrollee’s encounter data file.

CMS combined the two risk scores, weighting the RAPS risk score by 90 percent and the encounter data risk score by 10 percent.

CMS intends to increase the weight of encounter data in the risk score calculation in the next 4 years so that encounter data will be the sole source of diagnoses by 2020. (See fig. 3.)

While some stakeholder organizations we interviewed supported CMS’s time frame for transitioning from RAPS data to encounter data for risk score calculation, others raised objections to CMS’s planned timeline. Representatives from several stakeholder organizations we interviewed— primarily research firms and a health insurance trade association—said that CMS’s time frame was appropriate and that MAOs had adequate time to adjust their data submission processes. However, representatives from several health insurance and provider trade associations we interviewed said that many MAOs and providers are apprehensive about CMS’s time frame because it does not allow sufficient time for a successful transition. They told us that many MAOs and providers are still configuring their encounter data systems. In a December 2015 memo to all MAOs, CMS noted that, since 2016 will be the fourth year of collecting encounter data, the transition time frame is a reasonable, modest step toward ultimately relying exclusively on encounter data as the source of diagnosis information in risk adjustment. The agency noted that it has worked with MAOs to correct issues with how the agency applies the methodology for identifying diagnoses for risk adjustment is applied.

CMS’s Plans and Time Frames for Using MA Encounter Data for Other Authorized Purposes Remain Undeveloped, Creating Unease among Some Stakeholder Organizations

Although the agency has formulated general ideas of how to use MA encounter data for some purposes besides risk adjustment, CMS has not determined specific plans and time frames for most of the additional purposes for which the data may be used, namely (1) to update risk adjustment models; (2) to calculate Medicare disproportionate share hospital percentages; (3) to conduct quality review and improvement activities; (4) for Medicare coverage purposes; (5) to conduct evaluations and other analysis to support the Medicare program (including demonstrations) and to support public health initiatives and other health care-related research; (6) for activities to support the administration of the Medicare program; (7) for activities to support program integrity; and (8) for purposes authorized by other applicable laws. CMS officials explained that their main priority to date has been to use MA encounter data for calculating risk scores and that they plan to use the data for other purposes at a future time. However, this is inconsistent with federal internal control standards relating to risk assessment and information and communication that call for clearly defining objectives and communicating those objectives to key external organizations.

In addition to articulating plans for using encounter data for risk adjustment, CMS has indicated its interest in using MA encounter data for additional purposes. As of October 2016, CMS has begun planning for two of the eight remaining authorized uses: quality review and improvement activities. In April 2016, CMS awarded a contract to develop quality metrics that represent care coordination using encounter data. As of September 2016, the contractor had developed some plans for using encounter data to develop the metrics but testing and analyzing the data are ongoing. program integrity activities. CMS officials told us they anticipate including MA encounter data in the Fraud Prevention System to help identify abusive billing practices, but have yet to fully develop plans for this proposed use. To date, CMS officials reported that the Center for Program Integrity has begun using encounter data to determine improper payments to providers. It conducted a study of the number of services both paid as FFS claims and submitted as MA encounters. Additionally, it used encounter data to identify MA providers that were not enrolled in Medicare.

For the remaining authorized uses of encounter data, CMS reportedly has developed general ideas, but not specific plans and time frames. For example, CMS officials told us the CMS Innovation Center has plans to use MA encounter data to evaluate three demonstration models. Because these efforts are in their infancy, the officials could not provide details or specific time frames for these applications of encounter data. In addition, CMS has released MA encounter data to the Medicare Payment Advisory Commission and the Department of Health and Human Services’ Office of Inspector General for research purposes using standard protocols for releasing FFS data to those agencies. However, CMS has not yet released the data to other organizations or finalized protocols for doing so.

Stakeholder organizations we interviewed acknowledged that the agency has the authority to collect MA encounter data, but some indicated unease about CMS’s expansion of allowable uses of the data. In addition, some were concerned about the potential for future expansions because CMS has not fully defined its plans and time frames for other applications. In contrast, representatives from both health insurance and provider trade associations and research organizations noted that the authorized purposes are within CMS’s purview and that the agency already uses FFS data for similar purposes. CMS officials told us that some of the authorized uses are purposefully broad because they want to have some flexibility to expand their uses of encounter data in the future.

A common point made by all of the stakeholder organizations we interviewed was the importance of privacy protections for releasing MA encounter data to researchers and other interested parties. Many organizations were concerned that CMS might release commercially sensitive information to external entities. A few also highlighted the importance of protecting patient privacy. To protect proprietary information and patient privacy, stakeholder organizations offered the following suggestions:

Aggregate the data. Representatives from some health insurance trade associations said aggregating the data on a geographic level, such as the county or state level, would generally allow MAOs to remain anonymous. One MAO stated that aggregating the data at the physician group level would be appropriate. In the preamble to its final rule, CMS clarified that payment data released to external entities would be aggregated at the level necessary to protect commercially sensitive information, such as proprietary payment rates between plans and providers.

Deny or delay the release of certain data. Representatives from health insurance and provider trade associations were opposed to releasing encounter data elements—such as payment or utilization data—that could be used for anti-competitive behavior. They proposed delaying the release of encounter data by several years to protect proprietary information. In the preamble to its final rule, CMS stated that such delays in releasing the data to external entities would defeat the purposes of improving transparency in the Medicare program.

Limit data access. Representatives from health insurance and provider trade associations and research organizations emphasized that CMS should implement appropriate safeguards for releasing MA encounter data to external entities similar to those protections used for Medicare FFS data. Representatives from three trade associations argued that encounter data should not be made available to researchers and other interested parties until the data quality is assured. In the preamble to its final rule, CMS stated that making encounter data available to researchers using a process similar to that applied to FFS data would enhance transparency in the MA program.

To the extent that specific plans for using the MA encounter data remain undeveloped, CMS is unable to communicate a set of well-defined objectives to stakeholders. Furthermore, in the absence of planning for all of the authorized uses, the agency cannot be assured that the amount and types of data being collected are necessary and sufficient for specific purposes. Given the agency’s limited progress on developing plans for additional uses of encounter data, we continue to believe that CMS should establish specific plans and time frames for using the data for all intended purposes, in addition to risk adjusting payments to MAOs, as we recommended in our July 2014 report.

Agency Comments

We provided a draft of this report to the Department of Health and Human Services (HHS) for comment. HHS provided technical comments, which we incorporated as appropriate.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the appropriate congressional committees and the Secretary of Health and Human Services. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staffs have any questions about this report, please contact me at (202) 512-7114 or cosgrovej@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Major contributors to this report are listed in appendix I.

Appendix I: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Rosamond Katz (Assistant Director), Manuel Buentello, David Grossman, and Jessica Lin made key contributions to this report. Also contributing were Muriel Brown, Christine Davis, Elizabeth Morrison, and Jennifer Rudisill.