Compared with RAPS data, MA encounter data contain more data elements, including data on diagnoses not used for risk adjustment. Specifically, MAOs may enter more diagnoses on each encounter data submission than on each RAPS data submission and may transmit encounter data more frequently than RAPS data. (See table 1.) Although there are a number of differences between RAPS data and encounter data, an important distinction is a shift in who is responsible for identifying diagnoses used for risk adjustment. Under RAPS data submissions, MAOs individually analyze all their claims data and only submit data with diagnoses that are relevant for risk adjustment. In contrast, for encounter data submissions, MAOs transmit data to CMS on all enrollee encounters, regardless of whether the encounter contains diagnoses used for risk adjustment. In addition, the encounter data submission process involves a number of steps and exchanges of information between providers, MAOs, and CMS. First, MAOs collect encounter data—originating from an enrollee’s medical record—from providers to manage and process reimbursement for health care services and supplies for enrollees. After collecting and reviewing these data, MAOs submit the data to CMS through the Encounter Data System. CMS then processes and checks the data for problems. CMS designed the system to reject and return problematic data to MAOs for corrections. MAOs are expected to work with providers to correct the encounter files and resubmit the data to CMS. (See fig. 1.) Since our July 2014 report, CMS has taken additional steps across several activities to ensure that MA encounter data are complete but has yet to fully address data accuracy. (See fig. 2) The agency has taken the following steps, which address primarily the completeness of encounter data and provide feedback to MAOs on data submission: Creating a report card with basic statistics on the completeness of encounter data for MAOs. This step partially fulfills CMS’s Medicaid data validation protocol activity to conduct statistical analyses to assess completeness and accuracy. Analyzing values in specific data elements and generating basic statistics on the volume and consistency of data elements can help detect data validity issues. Agency officials told us they are using the report cards to encourage MAOs to submit data more frequently and completely. The report cards contain the following information: quarterly performance indicators. These indicators relate to submission frequency (such as the percentage of biweekly periods with submitted data), data volume (such as the numbers of submitted encounters per 1,000 enrollees), and data quality (such as rejection rates of data submissions). comparisons with other MAOs. These MAO-specific comparisons display an MAO’s volume of encounters—overall and by service type (professional, inpatient, outpatient, and durable medical equipment)—alongside the regional and national averages for both MA encounter data submissions and Medicare FFS claims for each of the past 3 years. Developing an automated report for MAOs on diagnoses used for risk adjustment. This step partially fulfills the data validation protocol activity to summarize findings on encounter data completeness and accuracy to provide recommendations to MAOs. The automated report identifies diagnoses from MAO encounter data submissions that CMS will use to calculate risk scores for the next payment year. The report is primarily intended to help MAOs ascertain the basis of enrollee risk scores, though representatives from health insurance trade associations told us that they have also used the automated reports to prepare internal financial projections and compare patient diagnoses between encounter data and RAPS data submissions. MAOs first received these reports in December 2015, and since then, CMS has modified the report layout in response to MAO feedback. According to agency officials, CMS has finalized the initial version of the automated report and is distributing the automated reports to MAOs on a monthly basis. Further, the agency intends to make technical changes as necessary in the future. According to agency officials, they finalized the protocol for validating MA encounter data in November 2016 and have begun implementing several parts of it. In September 2016, CMS awarded a contract to update the protocol and report annually on the implementation and outcomes of protocol activities. The stakeholder organizations we interviewed raised several issues with CMS’s recent actions to ensure the completeness of MA encounter data. The main issues mentioned by several stakeholder organizations included the following: Errors in identifying diagnoses used for risk adjustment. Representatives from health insurance trade associations we interviewed criticized CMS’s process for identifying diagnoses that are relevant for risk adjustment. First, they stated that MAOs question the integrity of CMS’s data processing. They noted, for example, that the automated reports MAOs receive had missing procedure codes for some encounters where the original data submissions had included them. CMS told us that they are working with MAOs to make needed corrections to these reports. Second, representatives said that MAOs have been unable to replicate CMS’s analyses because CMS has made adjustments to how it identifies diagnoses eligible for risk adjustment using encounter data. As a result, they say, MAOs are unsure whether CMS is properly distinguishing diagnoses that are used for risk adjustment from those that are not used. When asked about this concern, CMS officials noted that they publicly announced how the agency intends to implement the risk adjustment transition in December 2015, and the methodology has not changed. Inclusion of encounter data elements considered irrelevant. Representatives we interviewed from some health insurance and provider trade associations questioned CMS’s inclusion and checks of data elements the agency does not use for risk adjustment and which they contend are irrelevant for the purposes enumerated in the August 2014 final rulemaking. CMS’s Encounter Data System is designed to reject erroneous information by applying a subset of the edits used to process FFS claims, which are not all relevant to MA encounter data, according to CMS officials. CMS requires MAOs to make corrections and resubmit the data for all rejected encounters. Representatives stated that both MAOs and providers must dedicate significant resources to meet CMS requirements. In particular, they said MAOs must alter their data systems and submit numerous requests to providers for data corrections and medical record reviews. When asked about this comment, CMS officials noted that the agency wants encounter data elements to be comparable to FFS claims data. Although not all of the encounter data elements are used for risk adjustment purposes, CMS noted that they should be reliable, comprehensive, and complete in the event they are used for any authorized purposes. Technical problems with encounter data submission. Representatives we interviewed from several health insurance and provider trade associations reported that MAOs have experienced difficulties with certain data submissions. They cited, for example, difficulty with submitting encounters for recurring services, such as physical therapy. While MAOs and providers typically record such services as a single encounter, they must submit multiple separate encounters to CMS for recurring services. Agency officials told us they had not heard about this problem from MAOs. In addition, stakeholder representatives mentioned challenges resulting from frequent, randomly scheduled changes to data submission requirements, which they say generate costs and confusion. CMS stated that technical changes occur quarterly, which is typical for other CMS data collection efforts. Agency officials pointed out that CMS has worked with some of the larger MAOs to address data submission issues. Inadequate CMS communication with individual MAOs. Although some researchers praised CMS officials for their assistance and support, representatives from several health insurance trade associations told us that their members are dissatisfied with CMS’s communication efforts. Representatives noted that CMS’s webinars are not designed to facilitate a conversation between the agency and MAOs, and that the email address CMS set up to handle questions from MAOs does not produce responses that can be shared across MAOs in a timely fashion. CMS officials told us that its response time for emailed questions on MA encounter data largely depends on the complexity of the issue. They said that the agency has not been able to provide individualized assistance because of resource limitations, but has recently contracted with an external organization to provide on-site assistance to MAOs to improve their data submission processes. Although CMS has taken several steps to ensure the completeness of MA encounter data, as of October 2016, the agency had yet to take a number of other important steps identified in the agency’s Medicaid protocol. Steps CMS has not taken include the following: establish benchmarks for completeness and accuracy. This step would address the data validation protocol activity to establish requirements for collecting and submitting MA encounter data. Without benchmarks, CMS has no objective standards against which it could hold MAOs accountable for complete and accurate data reporting. conduct analyses to compare against established benchmarks. This step would address the data validation protocol activity to conduct statistical analyses to ensure accuracy and completeness. Without such analyses, CMS is limited in its ability to detect potentially inaccurate or unreliable data. determine sampling methodology for medical record review and obtain medical records. This step would address the data validation protocol activity to review medical records to ensure the accuracy of encounter data. Without medical record reviews, CMS cannot substantiate the information in MAO encounter data submissions and lacks evidence for determining the accuracy of encounter data. summarize analyses to highlight individual MAO issues. This step would address the data validation protocol activity to provide recommendations to MAOs for improving the completeness and accuracy of encounter data. Without actionable and specific recommendations from CMS, MAOs might not know how to improve their encounter data submissions. To the extent that CMS is making payments based on data that have not been fully validated for completeness and accuracy, the soundness of billions of dollars in Medicare expenditures remains unsubstantiated. Given the limited progress CMS has made, we continue to believe that the agency should complete all the steps necessary to validate the data before using them to risk adjust payments or for other intended purposes, as we recommended in our July 2014 report. Since our July 2014 report, CMS has made progress in defining its objectives for using MA encounter data for risk adjustment purposes and in communicating its plans and time frames to MAOs. Although additional work is needed, CMS has improved its ability to manage a key aspect of the MA program. In April 2014, CMS announced that it would begin incorporating patient diagnoses from MA encounter data submissions into risk score calculations.