Medicaid Information Technology: CMS Supports Use of Program Integrity Systems but Should Require States to Determine Effectiveness
Released: 2015-03-02T00:00:00

Letter

Background

The 50 states, the District of Columbia, and the 5 U.S. territories (hereafter referred to collectively as “states”) each administer a state- based Medicaid program. Federal laws authorize both federal and state entities to protect the program from fraud, waste, and abuse. Specifically, various provisions of federal law give CMS the authority to oversee Medicaid program integrity and to set requirements with which state Medicaid programs must comply. CMS oversees the states’ Medicaid programs by providing administrators with guidance related to statutory and regulatory requirements, as well as technical assistance on specific program integrity activities, such as the implementation of supporting information systems.

Further, the Deficit Reduction Act of 2005 established the Medicaid Integrity Program within CMS to support and oversee state program integrity efforts. To carry out its oversight responsibilities, CMS established within that program the Medicaid Integrity Group, which was responsible for conducting comprehensive reviews of states’ Medicaid program integrity activities to assess their compliance with federal program integrity laws and regulations.

Administrators of the 56 state-based programs are responsible for the day-to-day operations, including program integrity activities, of Medicaid. State Medicaid administrators employ the expertise of program integrity analysts to screen providers and determine whether the providers are eligible to enroll in the program. These analysts are also responsible for reviewing claims filed for services before they are paid, and for reviewing claims after they have been paid.

Provider enrollment: When enrolling providers to participate in the program, states are to first verify the providers’ eligibility. As part of the enrollment screening process, state program integrity analysts collect certain information about the providers, which may include the results of any criminal background checks and whether they are identified on lists that exclude or bar them from participating in other states’ Medicaid programs or the federal Medicare program. Any providers who are determined to be ineligible as a result of information obtained through the screening process are excluded from participating in Medicaid.

Prepayment claims review: The states also conduct reviews of claims data submitted by providers prior to payment in attempts to ensure that the claims were filed properly. For example, program integrity analysts conduct reviews to identify errors in individual claims, such as incorrect medical codes, and return claims that are found to have errors to the providers, thus preventing payment of such claims until the errors are corrected. The analysts may also compare claims data to prior incidents of known fraudulent behavior in their efforts to identify providers for further investigation.

Post-payment claims review: Medicaid administrators also are to take steps to identify payments that were made to providers for improperly filed claims. In this regard, states’ program integrity analysts may compare data from multiple paid claims to related provider records as they attempt to identify behaviors consistent with fraudulent activity that had been identified previously. Providers demonstrating such behaviors would then be subjected to additional review by states’ auditors and investigators, who are tasked to take actions intended to recover the amounts reimbursed for improper or fraudulent claims.

Figure 1 presents a simplified illustration of the provider enrollment, prepayment review, and post-payment review activities.

The implementation of information systems is integral to states’ efforts to conduct the program integrity activities covering provider enrollment through post-payment claims review. In this regard, the Social Security provides that, to receive federal funds for Medicaid, Act, as amended,every state must implement a claims processing and information retrieval system to support the administration of the program. For the Medicaid program, this system is the Medicaid Management Information System (MMIS).

In accordance with the act, as amended, and relevant regulations, CMS further defined criteria that states must meet to be approved to receive federal funds, including the implementation of system functionality that supports key Medicaid business areas. Such areas would include program performance management, business relationships, and operations management. Program integrity is a component of the performance management business area.

CMS also defined requirements for implementing MMISs, including various subsystems that support program integrity activities, such as provider screening, claims processing, and utilization reviews.

The MMIS provider subsystem is to be used to enroll and maintain a state’s network of providers for serving the Medicaid beneficiary population. Among other things, this subsystem is to include functionality needed to determine the eligibility of the providers participating in Medicaid. For example, the system is to allow Medicaid program administrators the ability to cross-reference license and sanction information with other states and federal agencies in order to identify providers who may not be eligible to enroll. State Medicaid programs are to define and implement functionality within this subsystem to validate providers’ enrollment based on state- specific criteria, such as license and permit expiration dates.

The MMIS claims processing subsystem is to be used to review data from claims filed by providers before they are paid and is to provide functionality needed to prevent improper payments of claims. For example, when analyzing claims data prior to payment, this subsystem is to be used by Medicaid administrators to identify improperly filed claims through the implementation of prepayment edits—i.e., instructions that system developers code into the subsystem to electronically compare claims data to program requirements in order to assure that claims are filed properly before they are approved for payment. Any claims that do not pass such edits are denied for payment or flagged for additional review by program integrity analysts.

The MMIS surveillance and utilization review subsystem (SURS) is to be used by program integrity analysts when they conduct post- payment reviews of claims in an attempt to detect any that were paid improperly. Specifically, the subsystem provides functionality to analyze data supporting the denial or payment of multiple claims submitted by a particular provider to help identify patterns that may indicate inappropriate provider behavior and, therefore, detect improper payments of claims. For example, payments made to a provider for an unusually large number of services for an uncommon type of procedure over a relatively short period of time could indicate fraudulent behavior on that provider’s part and, therefore, warrant additional review or investigation of the provider’s practices.

Additionally, within their MMIS IT environment, states may implement other components, such as databases and data warehouses, to store the beneficiary, claims, and provider data that are collected for processing and analysis by the system and its subsystems.

Further, in accordance with the Patient Protection and Affordable Care Act (PPACA), CMS identified certain prepayment edits and required state Medicaid administrators to incorporate these edits into their MMIS claims processing subsystem.for identifying incorrect coding on Medicaid claims that, if undetected, could lead to improper payments for ambulatory surgical center services, outpatient hospital services, and durable medical equipment.

Specifically, states are to implement functionality Prepayment edits that provide such functionality were developed through efforts of the National Correct Coding Initiative (NCCI)—a program implemented by CMS in 1996 for the Medicare fee-for-service program. Through this initiative, CMS defined more than a million standard claims processing prepayment edits to identify coding errors that are applicable to state programs. For example, some of the edits can identify pairs of medical billing codes that indicate to program integrity analysts any services that should not be reported together, such as two codes for the same service for the same beneficiary on the same date. In such cases, the first code would be eligible for payment but the second code would be denied. Other NCCI edits required for Medicaid programs are designed to identify procedures that could not be performed during a patient’s visit because they would not be feasible based on anatomic or gender considerations. For example, processing claims data against the edits may identify services such as prenatal treatment for a male patient that would not be a likely or feasible medical service. In addition to the NCCI edits, states may design and implement prepayment edits based on their own program experiences and needs to identify improperly filed claims and prevent payment of such claims.

Provider enrollment and pre- and post-payment claims data review activities, and the MMIS subsystems that support them, were designed primarily to address program integrity goals of states’ delivery of fee-for- service health care to Medicaid beneficiaries. In fee-for-service plans, providers are paid for each service that is delivered; they file claims for reimbursement from Medicaid that include detailed data specific to the service delivered during a patient’s visit.

However, as we noted in May 2014, over the past 15 years, states have more frequently implemented managed care delivery systems for providing health care services for Medicaid beneficiaries.care delivery, beneficiaries obtain some or all of their medical services from organizations of providers that are under contract with the state to provide Medicaid benefits in exchange for a monthly payment. The payments to these managed care organizations are typically made by the state Medicaid programs on a predetermined, per-person basis. While the individual managed care providers do not file claims for reimbursement by Medicaid, the managed care organizations are expected to report data to state Medicaid programs that allow the Medicaid administrators to track the services received by beneficiaries enrolled in managed care. These data are referred to as encounter data and are obtained from claims for With managed reimbursement that providers submit to their managed care organizations for services delivered.

Encounter data are similar to the fee-for-service claims data, but they typically do not include the same level of detail, and specific encounter data elements may be defined differently than they are for claims data. For example, encounter data generally would not include a Medicaid- billed amount for a particular beneficiary’s visit to a provider because the state does not pay the provider directly. In contrast, the data included on a Medicaid fee-for-service claim would include a specific amount for services delivered to a beneficiary during a visit since providers in fee-for- service plans bill and are reimbursed on a service-by-service basis. Thus, all the data needed for analyses by MMISs and other systems that were designed to process fee-for-service claims data will not always be consistent or available from the encounter data that managed care organizations collect and report to state Medicaid program administrators. In contrast to the program integrity reviews conducted when administering fee-for-service plans, which are largely based on pre- and post-payment review of claims data, states’ oversight of managed care organizations often occurs through contracts and reporting requirements.

Ten Selected States Rely on a Variety of Systems to Support Program Integrity

All 10 of the states in our study had implemented MMIS subsystems to support their program integrity efforts. Three states reported that they were operating MMISs that were implemented more than 20 years ago, while 7 states had upgraded their subsystems in the past 13 years, and 2 of those reported having done so in the past 2 years. Further, 7 states had, in the past 10 years, implemented other new and more advanced systems, in addition to their MMISs, to meet specific needs related to enrolling providers and processing claims data. Medicaid administrators in the 9 states that administer fee-for-service plans described a number of ways that they use their various systems to help improve the outcomes of their program integrity efforts, and 4 states reported that they had implemented specific functionality needed to support program integrity activities for their managed care plans.

All 10 States Use MMIS Subsystems to Help Prevent and Detect Improper Payments

Consistent with the requirements defined by CMS, the selected state Medicaid programs use the MMIS provider and claims processing subsystems to perform program integrity activities related to provider enrollment and prepayment review. For example, all 10 of the states have incorporated NCCI edits into their claims processing subsystems, as required by CMS, to help identify and prevent potential improper payments. Six of the states also had developed and implemented prepayment edits other than the required NCCI edits that incorporate additional criteria for conducting prepayment reviews of claims data to help prevent improper payments.

Likewise, nine of the selected states use SURS to help detect payments that may have been made to providers improperly. Medicaid administrators in these states told us that they use this subsystem to identify suspicious patterns of provider behavior that are not evident during the prepayment claims data review. For example, SURS can be used to analyze post-payment data for multiple claims at a time in order to identify suspicious provider billing patterns that are not detectable by the claims processing subsystem, which is used to process one claim at a time.

The following examples describe ways that the selected states have implemented functionality into the required MMIS provider subsystem, claims processing subsystem, and SURS to support their program integrity activities.

California, which administers its Medicaid program as both fee-for- service and managed care, implemented its MMIS claims processing subsystem in the 1980s. The subsystem includes approximately 1,500 prepayment edits that were developed and implemented by the state, and are in addition to, and conducted after claims data pass through, the NCCI edits. These additional edits are applied to claims data prior to payment and are designed to help prevent claims from being paid improperly. According to the state’s Medicaid administrators, these additional prepayment edits were developed based on previous improper provider billing activity identified by the state, and may be used to identify claims for services that exceed limitations, such as for drug costs and uses. Beyond this subsystem, administrators also reported that they use SURS to query post-payment fee-for-service data for claims that were submitted over a period of time to identify suspicious activity and trends, such as spikes in payments to providers for a certain type of service. In such cases, the providers identified by SURS may be subjected to further review by program integrity analysts. For example, the analysts may analyze additional data, such as data on prior paid claims, to determine whether the payments were made improperly or whether the activities could indicate potential fraud.

Maryland’s MMIS claims processing subsystem was implemented in 1984 to analyze fee-for-service claims data and identify errors in claims that could lead to improper payments to providers. Program administrators had also integrated managed care organizations’ encounter data into their SURS so that the data would be available for post-payment reviews of payments made to providers within the managed care organizations.

Mississippi implemented its MMIS in 2003. The state requires its managed care organizations to report the same data that fee-for- service providers report on claims; thus, the program integrity functionality implemented in the state’s MMIS subsystems could be Among other uses, the state relies on used for both types of plans.SURS to examine multiple claims submitted by a provider to identify those whom they suspect are submitting claims improperly. For example, the system can be used to identify patterns of providers’ billing practices that may indicate that they are submitting claims to Medicaid for mental health day treatment when they are actually providing day care services, which are not billable to Medicaid.

North Carolina’s Medicaid administrators implemented their MMIS in 2013. The system includes an automated provider credentialing and enrollment function, along with claims processing functionality that integrates pre-payment edits, business rules, program logic, and other user-defined criteria to help identify potential improper payments in the state’s fee-for-service plan. Program integrity analysts who use the claims processing subsystem are able to select multiple provider- or claims-based criteria for suspending claims so that they can be reviewed prior to payment.

Tennessee implemented its MMIS in 2004 to support the administration of the state’s managed care Medicaid program.According to the program administrator, the MMIS provider subsystem, claims processing subsystem, and SURS are used to collect and process all the data created by the state’s managed care organizations, including provider enrollment and claims data for individual providers. Program integrity staff rely on the claims processing subsystem as they review all providers’ claims data submitted by the managed care organizations, and the subsystem incorporates algorithms and NCCI prepayment edits to identify potential payment of improper claims filed by providers with managed care organizations. By requiring managed care organizations to report detailed claims data, Tennessee administrators are able to use their systems to support program integrity activities as if the state was operating a fee-for-service model, unlike other managed care plans that only collect encounter data.

Texas, which administers its Medicaid program as both fee-for-service and managed care, implemented its MMIS, including the claims processing subsystem and the current SURS, in 2009 to process and screen fee-for-service claims data. The Texas MMIS includes thousands of prepayment edits in addition to the required NCCI edits. Further, the state uses SURS to query post-payment claims data to help identify suspicious activity and trends, such as spikes in payments to providers. In these cases, the providers identified by SURS would be subjected to further review by program integrity analysts or the state’s investigators to determine whether the targeted providers had improperly billed Medicaid.

U.S. Virgin Islands, which administers Medicaid as fee-for-service, implemented its MMIS in 2013 to automate its manual program administration processes. In addition to the required NCCI edits, the territory has incorporated unique prepayments edits in the claims processing subsystem. According to the administrator for the program, the territory also uses SURS to conduct post-payment reviews of claims to detect payments to providers that may have been made improperly.

Virginia, which administers Medicaid as a combination of fee-for- service and managed care, implemented its MMIS in 2003. The provider subsystem includes functionality that can automatically identify providers that have been excluded from other Medicaid programs, Medicare, and other federal programs. The system automatically identifies providers that are required to be revalidated before they are eligible to submit claims and be reimbursed for services covered by Medicaid. Further, the claims processing subsystem includes prepayment edits in addition to the NCCI edits. The state also has implemented a commercial software package that edits fee-for-service claims data after they have been processed by the MMIS but before providers are paid. According to the program administrator, these edits are applied to provide additional assurance that billing codes and other data on the claims are accurate.

CMS’s provider screening and enrollment regulations (42 C.F.R. § 455.414) requires states, beginning March 25, 2011, to complete revalidation of enrollment for all providers, regardless of provider type, at least every 5 years.

Seven States Reported Using Systems in Addition to MMISs to Enhance Support for Program Integrity Activities

Beyond using the required MMIS provider subsystem, claims processing subsystem, and SURS, Medicaid administrators in 7 of the 10 selected states have implemented additional systems and functionality. These include data analytics, claims data verification systems, and an in-home care monitoring system, intended to enhance the outcomes of efforts to prevent and detect improper payments to providers. Specifically, among these 7 states:

California implemented a separate decision support system and data warehouse in 2008 to assist with identifying overpayments or erroneous payments for both fee-for-service claims and managed care encounters that were not detected by the state’s MMIS SURS. For example, state administrators said that the results of the decision support system’s automated analyses are used to identify irregular provider behavior indicated by spikes in payments to providers, which may lead to further analysis by program integrity analysts to identify patterns of fraud, waste, and abuse and, consequently, the detection of improper payments. The warehouse stores historical data on providers and claims that were collected over time from the MMIS databases.

Maryland implemented a new system in 2013 that provides additional information about providers’ behavior to enhance the state’s ability to prevent improper payments to in-home care providers. Specifically, the system allows automated monitoring of the individuals who provide in-home services within the fee-for-service program and is used before claims for the services are submitted to and processed by the MMIS. The state requires these providers to use the system to check in and out via phone when they visit a participant’s home. The care provider can either use a land line at the participant’s house, or, if a land line is not available, a passcode along with a password device, which is issued to the patient and must be kept at the patient’s home. According to the state’s Medicaid administrator, the use of the in-home care system helps program integrity analysts verify that the personal care provider actually visited the patient. Specifically, when a provider checks in at a participant’s home, the system records and integrates data into the provider’s records, which are accessible to the MMIS claims processing subsystem. The claims processing subsystem then automatically compares the provider’s records, which indicate when they visited patients, to their claims data to verify that in-home visits were actually made at the times for which claims were filed. Thus, the in-home care systems can be used to identify any providers that filed claims but did not check in using the system, and help the Medicaid administrators prevent improper payments for claims filed for services that were not delivered.

North Carolina built an additional system of data martstools in 2013, which according to the state’s administrators, is used after MMIS processing has been completed to analyze both paid and denied fee-for-service claims data needed to help detect improper payments. Consequently, the results of the system’s analysis can be used to identify repeated provider billing patterns that were determined to be improper and denied in the past.

Texas implemented an additional data analytics system in 2013 to mine and analyze fee-for-service claims data collected by its MMIS claims processing subsystem and stored in an MMIS database. According to the state’s Medicaid administrators, the system retrieves the data from the MMIS database and provides data warehousing and mining capabilities that allow investigators to query the data in a way that reveals patterns and relationships between data on beneficiaries, providers, and locations and dates of service. The technology is used to establish not simply what happens, but also the relationships that explain why things happen—information that is not provided by the analyses conducted by MMIS subsystems.

Kentucky implemented a commercial-off-the-shelf data analytics system in April 2014 that is used to conduct additional analysis of both fee-for-service claims and managed care encounter data after the data have been analyzed by the MMIS subsystems. The system is used to determine, for example, whether a significant increase in claims or encounters is the result of an increase in a provider’s office size or an indicator of improper billing by the provider. According to Kentucky’s administrators, the enhanced functionality available through this system provides a broader overview of data than the MMIS and enables program integrity analysts to detect interconnections between providers to identify and prevent payments of claims filed as a result of fraudulent activities, such as kickback fees paid from one provider to another for a fake referral.

Mississippi implemented a new data verification system in 2007, which retrieves and processes claims and encounter data that were reported to and stored in the state’s MMIS data base. The system can be used in addition to the MMIS to detect improper payments in both the program’s fee-for-service and managed care plans. The system produces reports for different purposes, such as for post-payment claims reviews and provider audits. For example, a report may identify patterns of mental health providers submitting claims for services not billable to Medicaid, which in turn may raise questions about those providers’ billing patterns and warrant further review by program integrity analysts to determine whether the claims were paid improperly.

Tennessee relies on a data warehouse and analytic capabilities implemented in about 2004 to detect improper payments for services provided through managed care organizations. The state uses the system to conduct analyses of data reported by managed care organizations and stored by the MMIS. The warehouse maintains 5 years of encounter data collected from the program’s managed care organizations, and retrieves current encounter data that were collected and stored using the state’s MMIS claims processing subsystem. The warehouse is mined for data to be used in analyses that could lead to additional audit reviews or investigations. For example, the capabilities are used to identify patterns in providers’ billing practices, based on historical data, that support preliminary analyses of provider referrals received from managed care organizations, as well as referrals developed internally through data mining. The results of the analyses help the state’s Medicaid administrators determine whether to investigate providers for whom suspicious behavior is detected.

Figure 2 illustrates the selected states’ program integrity activities and how the MMIS subsystems and other implemented systems have been integrated to support Medicaid provider enrollment, claims processing prepayment review, and post-payment review.

CMS Provides Data, Technical Guidance, and Funds to Help States Enhance Medicaid Program Integrity, but Most of the Selected States Do Not Measure Systems’ Effectiveness

In accordance with federal laws and agency program integrity plans, CMS takes steps to support states’ efforts to implement information systems that help prevent and detect improper payments in the Medicaid program. In particular, the agency provides states access to various sources of data that it maintains for its own use in administering the Medicare and Medicaid programs, along with technical guidance and training offered through the Medicaid Integrity Institute and other agency components. CMS also reviews and approves states’ requests for federal financial assistance offered through a matching funds program that supports the development, operations, and maintenance of information systems used for Medicaid administration, including program integrity. While the states in our study found these resources useful for improving the outcomes of systems they used to help prevent and detect improper payments, only 3 of the 10 states quantified and measured financial benefits achieved as a result of using the systems. As a result, CMS and the selected states do not have the information needed to determine the effectiveness of the systems.

CMS Provides States Access to Data That Support Systems Implementation

The Patient Protection and Affordable Care Act requires that CMS establish a process to make available to state agencies information about individuals and entities terminated from participating in Medicare, Medicaid, or the Children’s Health Insurance Program. Access to these data is intended to assist states in their efforts to determine whether providers are eligible to participate in Medicaid.

To respond to the requirements of the act, CMS defined an objective in its Comprehensive Medicaid Integrity Plan for fiscal years 2014-2018 to increase state Medicaid agency access to Medicare program integrity data.to data that it maintains about Medicare and Medicaid providers. These data are intended to help Medicaid programs screen providers seeking to participate in Medicaid and to identify potential improper payments during post-payment reviews of claims.

Further, in accordance with its plans, CMS provides states access The 10 states described data that CMS currently makes accessible to Medicaid administrators through four systems that it operates and maintains to support the Medicare and Medicaid programs: the Termination Notification Server, Provider Enrollment Chain and Ownership System (PECOS), Medicare Exclusion Database, and Fraud Investigation Database. States may use their own systems to manually log in and connect with CMS’s systems to conduct online queries of the databases. In these cases, the data received in response to the queries are not automatically integrated into the states’ systems. Alternatively, Medicaid staff may program their MMISs and other systems to automatically access and query CMS’s databases, and then download and integrate response data into their systems for use in automated processes, such as provider screening and prepayment claims review.

In particular, CMS allows states access to the Termination Notification Server, which it established and maintains for sharing information with all state Medicaid programs about the Medicaid providers reported to have been excluded or terminated by any state. Medicaid administrators in six of the selected states told us they use data from the server to support their efforts to screen providers seeking to participate in their Medicaid programs and identify those who may have been terminated from other Medicaid programs for causes such as fraudulent activity.

The states are also given access to PECOS—a system to which Medicare providers submit and update their enrollment data. This Internet-based system is used by the states to obtain information on providers eligible to participate in Medicare and Medicaid. In particular, while the data stored in the system are specific to Medicare providers, they are nonetheless useful to state Medicaid programs because many providers participate in both programs. For example, states may use the data when screening providers during enrollment processes to determine whether a provider has ever been excluded from participation in Medicare and, thus, whether they should be allowed to participate in Medicaid. They also use PECOS data during provider screening to determine whether a Medicare screening has already taken place, thus eliminating the need to screen further for Medicaid participation. Eight of the 10 states reported using PECOS.

Further, states are allowed to access provider termination data via the Medicare Exclusion Database. This database is accessed by users who may download files of monthly provider sanctions and reinstatement data and perform inquiries on excluded providers. Three of the selected states told us that they access data from the Medicare Exclusion Database to help program staff screen providers.

Finally, according to agency officials with the Data & Systems Group for Medicaid, Database—a nationwide data entry and reporting system that the agency established to monitor fraudulent activity and payment suspensions related to Medicare and Medicaid providers. The database was designed to capture data from the point when potential fraud is substantiated to the final resolution of a case. CMS updates the database with information regarding fraud investigations in the Medicare program, and state administrators enter data regarding their own investigations of potential fraud in their state-based Medicaid programs. Administrators in one state told us that they use information obtained from the Fraud Investigation Database.

The Center for Medicaid and CHIP Services Data & Systems Group is responsible for overseeing the collection of information from the states as is necessary for effective administration of the Medicaid program and to ensure program integrity.

CMS Provides States Technical Guidance and Training for Program Integrity Systems Implementation

In carrying out responsibilities under PPACA, CMS provided technical guidance and support to help state Medicaid programs implement information systems. Accordingly, CMS’s Comprehensive Medicaid Integrity Plan defines the agency’s objectives to expand training and other technical support activities offered through its Medicaid Integrity Institute for administrators of state programs.

To address the objectives of the plan, CMS provides states with various types of guidance and training opportunities related to technologies that could be implemented to help identify improperly filed claims. For example, CMS provides technical guidance to states incorporating NCCI edits into their MMIS claims processing systems. The agency describes specifications and instructions for state Medicaid programs to incorporate new or modified NCCI edits into their MMISs on a quarterly basis. CMS also provides states with files that include functionality for performing the edits. According to officials with CMS’s Data & Systems Group, some states have updated their MMISs to incorporate certain capabilities that enable them to download the files from a CMS website and integrate them directly into their IT environment, thus reducing the amount of effort needed to implement the edits into their MMIS. However, states that have not updated their legacy MMISs to enable this capability have to make programming changes in order to implement the edits each quarter of the year.

The updates are provided 15 days prior to the first day of the next quarter, and the states are to implement the updated edits with the next 4 weeks. help states integrate and analyze managed care encounter data. According to CMS officials with Medicaid Integrity Institute, the agency determines state Medicaid administrators’ needs for continuing education based on information collected by surveys administered during training sessions. These opportunities are fully supported with federal funds at no cost to the states.

State Medicaid program staff described various ways that their staff had participated in training activities and collaborations with other states that were conducted by the Medicaid Integrity Institute. For example, they attended sessions in which Medicaid data experts gathered to exchange ideas and develop best practices on topics such as integrating data from various sources, predictive analytics, and working with algorithms to analyze both fee-for-service and managed care data. In particular, Vermont Medicaid staff participate in a CMS Fraud Technical Advisory Group that meets on a regular basis to share information related to, among other things, data sources and ways to access data that could be used to help identify improperly paid claims or aberrant provider behavior. Likewise, Kentucky, Maryland, Tennessee, Texas, and Virginia staff had attended a Data Expert Symposium conducted by the institute in 2014, and North Carolina Medicaid program integrity staff had attended training related to the use of PECOS data in provider enrollment systems.

Medicaid administrators in our study also described ways that their staff had benefited from information obtained from training sessions specifically related to the integration of managed care encounter data with their MMISs and other systems that support efforts to prevent and detect improper payments. For example, Texas Medicaid program integrity staff had attended training on Emerging Trends in Managed Care in February 2012 and a Program Integrity Partnership in Managed Care Symposium in March 2014. These sessions addressed topics related to encounter data such as timeliness, validity, and reliability; use of encounter data in data analytics; and collecting and editing encounter data using MMIS.

Further, the selected states told us that the training sessions and collaborations facilitated by CMS and the Medicaid Integrity Institute had been valuable resources that supported their efforts to implement information systems for program integrity purposes. For example, Maryland’s administrators stated that courses on data usage within analytical systems were helpful in their learning new strategies for developing algorithms that they used to identify potential improper claims. North Carolina and Tennessee administrators said that the institute had provided a venue for discussion between states regarding topics such as the implementation of data analytics and other advanced technologies, along with lessons learned related to the implementation of systems for program integrity purposes and algorithms that can be used to analyze data and help identify fraudulent provider billing patterns. They added that the Medicaid Integrity Institute has been the most helpful resource that CMS has provided in support of states’ efforts to implement information systems for program integrity purposes.

CMS Approves Federal Matching Funds to Assist States with Medicaid Systems Implementation, Including Functionality That Supports Program Integrity

CMS is authorized by federal law to provide matching funds to assist states in their implementation and operation of systems to support the administration of their Medicaid programs, including program integrity efforts. Specifically, Title XIX of the Social Security Act provides for CMS to approve states’ requests for federal matching funds to help finance the design, development, and installation of MMISs and other claims processing and information retrieval systems.

States can request and receive funds to cover up to 90 percent of these costs, depending upon the extent to which their plans for implementing the systems meet certain technical specifications and requirements defined by CMS, including those defined for the implementation of system functionality to support efforts to prevent and detect improper payments.

In addition, CMS is authorized to approve states’ requests for federal matching funds to cover up to 75 percent of the costs associated with the operation and maintenance of the systems.

In order to qualify for federal matching funds, Medicaid programs must first submit Advance Planning Documents that define, among other things, goals, objectives, and cost-benefit analyses of information technology projects relevant to specific business areas, such as provider and program integrity management. They also must certify with CMS that their MMIS and other system implementations meet a set of standards and conditions defined by the agency. For example, Medicaid programs must submit to CMS information specific to each business area, such as business objectives and system review criteria that address state-specific objectives and best practices defined by the agency. The documents and information are used by CMS for evaluation and certification of the states’ MMISs and other information systems relevant to the administration of Medicaid. that support Medicaid administration.federal matching funds these states received.

Selected States Maintain That Systems Are Effective, but Most Do Not Measure Benefits

According to GAO’s IT investment management framework, an organization’s process for investing in information systems should include a structured and proven investment analysis, such as a cost reduction or The results from avoidance, cost and benefit, or return on investment. such an analysis should reflect a consistent and repeatable approach for supporting IT investment decisions and ensuring that the organization is aware of the financial as well as other internal and external effects of operating and maintaining particular systems.

All of the selected states asserted that the MMIS subsystems and data analytics, decision support, and other systems they implemented for program integrity purposes had helped improve the outcomes and efficiency of their efforts to prevent and detect improper Medicaid payments. However, most of the states could not provide any supporting evidence of their systems’ effectiveness. Specifically, Medicaid administrators in seven states could not identify any steps that they had taken to quantify improvements in the outcomes, or otherwise assess the effectiveness, of program integrity efforts attributable to the use of their systems. For example, they did not measure financial benefits associated with increases in the amounts of money they saved or recovered as a result of improvements in their efforts to prevent and detect improper payments that were attributable to the implementation of information systems.

The administrators of these seven state programs that had not taken steps to quantify financial benefits gave several reasons for not having done so. According to the administrators of one state, the amount of effort and time that would be required to calculate return on investment or cost avoidance, along with questionable accuracy of the results, outweighs the usefulness of the information. Another state administrator said that return on investment for a single system could not be calculated because a system is only part of the process for recovering funds lost to improper payments. Still, another told us that it is difficult to calculate a return on investment as a result of using the MMIS for program integrity purposes because the system performs other functions beyond those for program integrity; therefore, it is not possible to break out the costs and benefits of implementing a single function.

However, among the 10 states in our study, 3 had identified ways to measure quantifiable benefits realized as a result of using systems designed to help prevent and detect improper payments. They did so by using information available from existing practices and reporting capabilities of systems that were implemented for program integrity purposes. Specifically, Medicaid administrators for the 3 states demonstrated practices for measuring financial benefits that could provide examples of ways to quantify improvements in outcomes resulting from the use of systems for program integrity purposes, along with lessons learned from the states’ experiences. These 3 states provided documentation discussing the results of efforts that had been taken to assess quantifiable benefits, achieved in the form of cost reduction or avoidance, from implementing their systems for program integrity purposes. For example:

Medicaid administrators in California provided the results of a routine internal audit conducted in 2010 that documented cost reductions totaling about $2 million during a 6-month period in 2010, which they attributed to their ability to supply providers with system-generated reports of comparative billing information. According to the administrators, when these providers were made aware of other providers’ billing patterns and costs, they often modified behaviors to be consistent with others to whom they were compared. As a result, they subsequently billed the state’s Medicaid program for lower costs or for fewer services.

Mississippi administrators provided a report generated by the state’s SURS that identified payments of $10 million for potential improper claims for a specific service in 2010. They stated that the information contained in the report had enabled the state to avoid additional costs in 2011. For example, the system identified payments to providers who had filed claims for mental health services for children when the actual services delivered were for day care, which was not billable to Medicaid. As a result, the Medicaid administrators notified providers that these claims were not acceptable, and then used SURS analytical and reporting capabilities to identify and document a subsequent decrease in the number of such claims filed by mental health providers. Ultimately, the administrators attributed cost avoidance totaling $7.5 million in 1 year to their use of SURS, based on a reduction in those types of improper payments from $10 million in 2010 to $2.5 million in 2011.

Virginia administrators measured over $216 million in cost avoidance achieved during fiscal year 2013 as a result of prepayment claims review activities supported by their claims processing subsystem. For example, they provided calculations of cost avoidance based on the cost of service requests denied as a result of claims processing prepayment edits. The cost was multiplied by the number of denied requests.

For its part, CMS has not required the states to identify and report on the outcomes and effectiveness of systems used for program integrity purposes. As mentioned previously, the agency requires states to document expected costs and benefits for systems when they submit requests for federal financial assistance with investments in new systems or functionality needed to support Medicaid program administration—i.e., the 90 percent matching funds.identify and report financial benefits or other quantifiable measures of effectiveness achieved as a result of using the systems in order to receive continued funding during the operations and maintenance. Therefore, CMS does not know whether the Medicaid systems implemented for program integrity purposes are effective in helping states avoid paying for providers’ claims that may be improper or in recovering funds lost to payment of improper claims.

However, it does not require states to As emphasized in our IT investment management framework, investments can outlive their usefulness and consume resources that begin to outweigh their benefits. Without identifying and measuring the financial benefits (i.e., money saved or recovered) that result from using their MMISs and other systems, CMS and state Medicaid administrators cannot be assured of the systems’ effectiveness in helping to prevent and detect improper payments. Moreover, without having required states to institute consistent and repeatable approaches for measuring and reporting such outcomes, CMS Medicaid officials lack an essential mechanism for ensuring that the federal financial assistance that states receive to help fund the operations and maintenance of these systems is an effective use of resources to support Medicaid program integrity efforts.

Selected States and CMS Have Taken Steps to Overcome Challenges with Using Systems to Analyze Managed Care Data

Even as the selected states rely on their systems to help prevent and detect improper Medicaid payments, five of the seven states in our study that administered Medicaid as both fee-for-service and managed care— North Carolina, Texas, Virginia, California, and Maryland—faced challenges that were specific to the use of their systems for ensuring the These challenges introduced integrity of their managed care programs.limitations in the states’ ability to use their systems to analyze managed care encounter data because of the (1) content of the data reported, (2) quality of the data submitted, or (3) inconsistencies between the ways managed care and fee-for-service data values are defined.

In particular, the encounter data reported by managed care organizations often lack content needed for the states’ systems to conduct analyses that help prevent or detect improper payments. Specifically, while these states collect data from managed care organizations, their Medicaid administrators stated that the data do not include the details needed for their systems to prevent and detect improper payments using the MMIS claims processing subsystem, SURS, or their additional data analytical systems that were implemented to conduct pre- and post-payment reviews of fee-for-service claims data, which do include the detailed data needed. North Carolina and Texas pointed to challenges in their ability to use their systems to analyze managed care encounter data resulting from lack of data content. For example, Texas administrators stated that encounter data submitted by their managed care organizations only indicate the reason for a patient’s visit and whether the provider’s claim was paid; they do not always include data such as diagnostic codes and the specific amounts paid for a visit—data that are needed for their systems to analyze paid claims to detect improper payments.

Further, deficiencies in the quality of encounter data impede these states’ ability to analyze the data to help prevent and detect improper payments for services delivered by managed care organizations. Medicaid administrators in California cited examples of data quality issues that presented challenges to their ability to use systems to support the integrity of Medicaid managed care. Specifically, California Medicaid administrators said that the encounter data being submitted by managed care organizations have historically been inaccurate, unreasonable, incomplete, and untimely. As a result, the data could not be effectively analyzed by the systems to identify patterns in claims and services that may help identify fraudulent or abusive provider behaviors and detect improper payments. Thus, any analyses of such erroneous data could not produce valid or reliable outcomes.

Additionally, differences between the way some data values are defined for managed care encounter and fee-for-service claims data cause problems with using systems to prevent and detect improper payments for managed care services. For example, claims processing subsystems that were designed to process claims data for specific services covered by fee-for-service plans may not properly process encounter data for different services allowed under managed care (but not allowed by fee- for-service). Thus, some of the prepayment edits designed to analyze fee- for-service claims data can provide erroneous results when applied to managed care encounter data. Additionally, managed care encounter data analyzed by SURS during post-payment review could include estimated rather than actual costs associated with services delivered to patients, which would not reflect any providers’ overcharges. Consequently, services for which providers overcharged would not be identifiable by SURS.

Virginia administrators said they had experienced such challenges in using their MMIS claims processing subsystem and SURS for analyzing encounter data to support oversight of managed care plans because of the way encounter data are defined. They told us that differences between the ways fee-for-service and managed care data are defined introduce inconsistencies that may affect the outcomes of the systems’ analyses and, consequently, lead to challenges related not only to the state’s ability to conduct oversight of the managed care organizations’ activities, but also to the amount of work and effort required when updating the state’s MMIS. Additionally, Maryland administrators continued to experience challenges with using their SURS to conduct post-payment reviews of managed care data for this reason. For example, they said that encounter data do not typically include the same values or level of detail as claims data. Therefore, they cannot effectively analyze those data during post-payment review using their SURS, which was designed to process fee-for-service claims data.

To address such challenges, one state—Tennessee—had taken actions that could offer lessons learned based on its having incorporated capabilities that enable the analysis of managed care encounter data using the state’s MMIS claims processing subsystem, SURS, and other systems. The Tennessee administrator described actions taken to address challenges with analyzing encounter data using Medicaid systems that had been designed to process fee-for-service claims data. Specifically, the state began to collect data from managed care organizations so that they could be analyzed by the MMIS claims processing subsystem, SURS, and data analytics systems to help prevent and detect improper payments. Tennessee’s Medicaid administrator told us that, to do so, the state defined the data required from the managed care organizations to include the content and level of detail that would be reported by fee-for-service claims, rather than the less detailed data the organizations had been reporting.

Tennessee also required the organizations to report quality encounter data in a timely manner so that they could be analyzed by the MMIS and other systems. For example, when a managed care organization submits encounter data to the state, the MMIS is used to conduct both system and payment edits. If the data do not pass the edits, they are returned to the managed care organization for corrections to be made. If the data are not corrected and returned within 45 days, the organization is fined a certain amount for each day it is late. When the corrected data are returned, they are further reviewed by analysts who ensure the data needed to conduct analyses are present before the data are stored in the state’s Medicaid data warehouse. By requiring managed care organizations to report detailed data and taking steps to ensure that the data meet quality and timeliness standards, Tennessee’s Medicaid administrators are able to use their MMIS claims processing subsystem and SURS to process the encounter data to detect improper payments in the same way they would analyze claims data. As a result of its effort, the administrators said the state is able to identify potential improper payments made to providers.

For its part, CMS had begun to take steps that could help states overcome challenges related to the collection of detailed, quality data needed to enable analyses of managed care encounter data using MMISs and other systems. For example, since 2010, CMS’s Center for Medicaid and Children’s Health Insurance Program Services, through the offerings of a contractor, has provided technical assistance to states. The contractor published documents and conducted webinars that addressed states’ need to collect the content and level of detail needed to conduct analyses of encounter data using their systems, along with steps that would need to be taken to ensure the quality and consistent definition of data reported by managed care organizations. In November 2013, the contractor published a toolkit on the Medicaid.gov website that identifies steps states should take to collect and validate data needed to conduct program integrity oversight of their Medicaid managed care organizations.

Additionally, courses and symposia that the selected states reported attending included sessions on topics such as collecting and editing encounter data and applying fee-for-service methodologies to the automated analysis of managed care encounter data that are defined differently from claims data. As noted above, 5 of the 10 states reported that they had participated in one or more of the training and data sharing sessions conducted by CMS’s Medicaid Integrity Institute. Furthermore, the Medicaid Integrity Institute included in a March 2014 symposium a presentation by Tennessee’s Medicaid administrator, who described the state’s experiences and successes with defining, collecting, editing, integrating, and analyzing managed care encounter data using the functionality of the MMIS claims processing subsystem and SURS to conduct prepayment and post-payment reviews to prevent and detect improper payments for services delivered by Medicaid managed care organizations. By taking such actions, CMS has continued efforts to support information-sharing activities that could help states address challenges.

Conclusions

States have implemented MMISs and other systems to support the administration of Medicaid, including efforts to ensure the integrity of the program. The 10 selected state Medicaid programs incorporated functionality required by CMS to help prevent and detect improper payments for Medicaid services, and had benefited from the support CMS provides in the form of data, technical guidance and training, or financial assistance. However, the effectiveness of the systems for program integrity purposes is unknown. Only 3 of the states had established methodologies for measuring financial benefits they had achieved based on the implementation of systems to help prevent and detect improper payments. While states are required by CMS to document expected benefits when they request financial support to implement new systems or functionality, they are not required to report actual benefits realized from using the systems when requesting additional funds to operate and maintain the systems. Therefore, the selected states and CMS do not have the information needed to determine whether the use of the systems is effective in helping Medicaid programs avoid paying or recover payments made for improperly filed claims. Until states are able and required to identify and measure quantifiable benefits achieved as a result of using systems to help ensure the integrity of both fee-for-service and managed care programs, CMS cannot determine whether the systems help states save money by improving the outcomes of efforts to prevent and detect improper payments in Medicaid. Consequently, the effectiveness of the systems will remain unknown as the federal government continues to provide potentially billions of dollars in financial assistance each year to support the implementation, operation, and maintenance of information systems intended to support Medicaid program integrity efforts.

Recommendation for Executive Action

To ensure that the federal government’s and states’ investments in information systems result in outcomes that are effective in supporting efforts to save funds through the prevention and detection of improper payments in the Medicaid program, we recommend that the Secretary of HHS direct the Administrator of CMS to require states to measure quantifiable benefits, such as cost reductions or avoidance, achieved as a result of operating information systems to help prevent and detect improper payments. Such measurement of benefits should reflect a consistent and repeatable approach and should be reported when requesting approval for matching federal funds to support ongoing operation and maintenance of systems that were implemented to support Medicaid program integrity purposes.

Agency Comments and Our Evaluation

In written comments on a draft of this report (reprinted in appendix II), HHS stated that it concurred with our recommendation. Further, in its comments, HHS stated that it works with state Medicaid programs to determine the effectiveness of systems that support program integrity functions. The department added that it had taken recent steps to help ensure that states provide post-implementation data to document quantifiable benefits, and is taking additional steps to determine effective methods for continuing to evaluate outcomes of Medicaid program integrity information technology investments.

While the actions that HHS described could be beneficial, our study found that the department and CMS had not defined a consistent and reliable approach for determining quantifiable benefits achieved by states before it approves the use of federal funds to finance the ongoing operations of systems intended to support program integrity efforts. Thus, we believe the full implementation of our recommendation is important to ensure that federal and state investments in information systems result in outcomes that help save funds through the prevention and detection of improper payments in the Medicaid program.

HHS also provided technical comments, which we incorporated into the report as appropriate. Additionally, we obtained and, as appropriate, incorporated technical comments from the state Medicaid administrators who participated in our study.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the Secretary of HHS and interested congressional committees. In addition, the report will be available at no charge on our website at http://www.gao.gov.

If you or your staff have any questions on matters discussed in this report, please contact Valerie C. Melvin at (202) 512-6304 or melvinv@gao.gov or (202) 512-7114, or Carolyn L. Yocom at (202) 512-7114 or yocomc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

The objectives of our review were to determine (1) the types and implementation status of the information systems used by states and territories to support Medicaid administrators’ efforts to prevent and detect improper payments to providers; (2) the extent to which the Centers for Medicare & Medicaid Services (CMS) is making available funds, data sources, and other technical resources to support Medicaid programs’ efforts to implement systems that help prevent and detect improper payments to providers, and the effectiveness of the states’ systems; and (3) key challenges, if any, that Medicaid programs have faced in using IT to enhance program integrity initiatives, and CMS’s actions to support efforts to overcome these challenges.

To address the objectives, we selected a nonprobability, nonrandom sample of the 50 states, 5 U.S. territories, and the District of Columbia. To select the states for our sample, we obtained data on states’ expenditures for systems implementation and program integrity activities for fiscal years 2004 through the first quarter of fiscal year 2014. We collected the data from a CMS database to which the states are required to report Medicaid program expenditures for which they request federal reimbursements. We assessed the reliability of the CMS data by reviewing prior GAO work that had accessed and used the data and prior determinations that the data provided reliable evidence to support findings, conclusions, and recommendations. We also held discussion with CMS officials knowledgeable of the specific types of data recorded in the database. Based on how we intended to use the information, we determined that the data were sufficiently reliable for the purpose of selecting states for our study.

We sorted the data we obtained based on states’ total expenditures for development and maintenance of their Medicaid Management Information Systems (MMIS) and the reported administrative costs for program integrity. We grouped the states, territories, and the District of Columbia according to low, medium, and high levels of spending based on their expenditures reported from fiscal year 2004 through the first quarter of fiscal year 2014. For example, those in the low spending group were the three states and two territories reporting the lowest expenditures, from $574,836 to $66,497,668; those in the medium spending group were the five states that reported expenditures, based on the median of all amounts reported, from $202,724,728 to $240,891,446; and those in the high spending group were the five states that reported the highest expenditures, from $825,026,677 to $2,578,096,036. We calculated the median expenditure for each group and identified the two states directly above and the two states directly below each median, which identified 12 states. Of the 12 states that we identified based on the expenditure data we collected and assessed, we selected 10 states—three low- expenditure states (U.S. Virgin Islands, Tennessee, and Vermont), four middle-expenditure states (Kentucky, Mississippi, Virginia, and Maryland), and three high-expenditure states (North Carolina, Texas, and California). Based on our assessment of the extent to which they met the selection criteria defined within our methodology, we determined that any information collected from these states would be sufficient for our use.

We then developed and administered a questionnaire to collect information regarding the selected states’ use of systems to support program integrity activities in their Medicaid programs, the technical support they received from CMS, and any challenges the states faced regarding their efforts to implement information systems for program integrity purposes. The results of our study are not generalizable to Medicaid programs administered by all states, territories, and the District of Columbia.

To address the first objective, we analyzed information taken from the questionnaire responses about the selected states’ program integrity efforts and supporting systems. We also obtained and analyzed documentation describing the types of systems they used to analyze provider and claims data in support of program efforts to prevent and detect improper payments. To determine the status of the systems, we examined relevant project management documents, including project plans and status reports, that provided information about systems implementation dates and dates any significant enhancements and replacements of the states’ information systems were completed or planned. For the states that were planning significant enhancements, updates, or replacements of systems, we also reviewed requests for proposals issued to potential contractors, along with statements of work for ongoing initiatives, to identify the types of changes or enhancements that the programs had planned to implement.

To address the second objective, we obtained and examined federal legislation, along with relevant agency plans, to identify legal and program requirements for CMS to provide financial support, data, and other technical resources to help states implement information systems for program integrity purposes. We included in our scope resources such as agency guidance and training provided to states and examined documentation that described the funding, data sources, systems, and other technical resources intended to help state Medicaid administrators implement the system functionality they need to prevent and detect improper payments.

To determine how states use the resources, we examined information from the questionnaires and analyzed documentation the states provided describing their use of federal funds and the data sources, systems, and technical training and guidance from CMS. We identified how each of the states used federal financial support to develop and implement new systems, operate existing systems, and fund the staff who use the systems in support of program integrity efforts. We also identified various ways the states integrated the data provided by CMS within their IT environment and individual systems, along with the types of training opportunities and technical support the states used to improve their ability to develop and enhance information systems that effectively support program integrity analysts’ efforts. We examined states’ responses from the questionnaire to determine the extent to which the financial, data, and other technical resources provided by the agency were reported to be useful to states in their efforts to implement new and update existing information systems that support the prevention and detection of improper payments.

To describe the extent to which the use of the systems were effective in improving outcomes of the states’ program integrity initiatives, we reviewed best practices identified in our IT Investment management framework for agencies’ management of IT portfolios, including practices for conducting investment analyses and determining financial and other effects of maintaining systems. We obtained from Medicaid program administrators documentation such as performance plans and audit reports regarding practices for determining the effectiveness of the different types of systems. We identified state programs that had developed methodologies and practices for measuring any quantifiable benefits realized from the use of specific systems. From those states we collected additional documentation that identified ways the states had measured quantifiable benefits, such as return on investments and cost avoidances, attributable to the use of the systems, and compared them to practices identified by our IT investment framework. Specifically, we examined methods and calculations used to determine measures such as amounts of payments withheld because of errors detected by the systems during prepayment review and amounts of improper payments recovered as a result of post-payment review activities supported by the systems. For the states that did not measure quantifiable benefits, we discussed with the Medicaid administrators their reasons for or inability to do so. We used the information collected from the questionnaires and document reviews to develop additional questions and conducted interviews with state Medicaid officials.

Finally, to address the third objective we analyzed information from our questionnaire and interviews about states’ experiences with implementing information systems for program integrity purposes and any challenges they faced in doing so. We identified challenges that were relevant to the role that CMS plays in supporting states’ efforts—i.e., those other than challenges related to state-based issues such as local funding levels, internal data sharing between state entities, and economic conditions. We obtained and reviewed CMS documentation, such as the Medicaid Integrity Program’s descriptions and plans that discussed activities planned and initiated by the agency’s Medicaid program integrity officials to support states’ administration of Medicaid, and compared the intent of such activities to challenges the states identified. We also examined agency schedules and training curricula to determine whether recent and planned training and technical assistance sessions were relevant to challenges the states identified. To identify any other actions CMS had taken or planned to help states address any such challenges, we examined annual reports the agency had provided to Congress that described steps taken over the previous year to address goals and objectives of the Medicaid Integrity Program. In addition, we held discussions with CMS Medicaid officials regarding their efforts and intent to address any known challenges associated with states’ efforts to implement information systems for program integrity purposes.

For each of the objectives, we supplemented the information gained from our documentation reviews by holding discussions with CMS officials and state Medicaid program administrators, including those responsible for implementing information systems used to help program integrity analysts prevent and detect improper payments of Medicaid claims.

We conducted this performance audit from November 2013 to January 2015 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Health and Human Services

Appendix III: GAO Contacts and Staff Acknowledgments

GAO Contacts

Staff Acknowledgments

In addition to the contacts named above, Catina R. Bradley, Assistant Director; Teresa F. Tucker, Assistant Director; Melina I. Asencio; Nicholas A. Bartine; Christopher G. Businski; Debra M. Conner; Rebecca E. Eyler; Stuart M. Kaufman; Thomas E. Murphy; and Daniel K. Wexler made key contributions to this report.