Fraud Detection Systems: Centers for Medicare and Medicaid Services Needs to Ensure More Widespread Use
Released: 2011-07-12T00:00:00

Letter

Background

Medicare is a federal program that provides health insurance coverage for individuals aged 65 and older and for certain disabled persons. It is funded by general revenues, payroll taxes paid by most employees, employers, and individuals who are self-employed, and beneficiary premiums. Medicare consists of four parts. Medicare Part A provides payment for inpatient hospital, skilled nursing facility, some home health, and hospice services, while Part B pays for hospital outpatient, physician, some home health, durable medical equipment, and preventive services. In addition, Medicare beneficiaries have an option to participate in Medicare Advantage, also known as Part C, which pays private health plans to provide the services covered by Medicare Parts A and B. Further, all Medicare beneficiaries may purchase coverage for outpatient prescription drugs under Medicare Part D, and some Medicare Advantage plans also include Part D coverage. In 2010, Medicare covered 47 million elderly and disabled beneficiaries and had estimated outlays of about $509 billion.

CMS uses contractors to help administer the claims processing and payment systems for Medicare. These administrative contractors are responsible for processing approximately 4.5 million claims per workday. The contractors review the claims submitted by providers to ensure payment is made only for medically necessary services covered by Medicare for eligible individuals.

Medicaid is the federal-state program that provides health coverage for acute and long-term care services for over 65 million low-income people. It consists of more than 50 distinct state-based programs that each define eligibility requirements and administer payment for health care services for low-income individuals, including children, families, the aged, and the disabled. Within broad federal requirements, each state operates its Medicaid program according to a state plan. Low-income Americans who meet their state’s Medicaid eligibility criteria are entitled to have payments made on their behalf for covered services. States are entitled to federal matching funds, which differ from state to state but can be up to three- fourths of their costs of this coverage. The amount paid with federal funds is determined by a formula established in law.

CMS oversees the Medicaid program at the federal level, while the states administer their respective programs’ day-to-day operations, such as enrolling eligible individuals, establishing payment amounts for covered benefits, establishing standards for providers and managed care plans, processing and paying for claims and managed care, and ensuring that state and federal health care funds are not spent improperly or diverted by fraudulent providers. The estimated outlays for Medicaid for both the federal and state governments were $408 billion in 2010. Of this cost, approximately $275 billion was incurred by the federal government and $133 billion by the states.

CMS Program Integrity Initiatives

The Health Insurance Portability and Accountability Act (HIPAA) of 1996 established the Medicare Integrity Program to increase and stabilize federal funding for health care antifraud activities. The act appropriated funds for the program as well as amounts for HHS and the Department of Justice to carry out the health care fraud and abuse control program. Subsequent legislation further outlined responsibilities under the Medicare Integrity Program.

Under the Medicare Integrity Program, CMS staff and several types of contractors perform functions to help detect cases of fraud, waste, and abuse, and other payment errors, which include reviews of paid claims to identify patterns of aberrant billing. Among these program integrity contractors are program safeguard contractors, zone program integrity contractors, and Medicare drug integrity contractors. The program safeguard and zone program integrity contractors are responsible for ensuring the integrity of benefit payments for Medicare Parts A and B (including durable medical equipment), as well as the Medi-Medi data match program. Medicare drug integrity contractors are responsible for monitoring fraud, waste, or abuse in the Medicare prescription drug program (i.e., Part D). These contractors work with the HHS Office of the Inspector General (OIG) and law enforcement organizations, such as the Department of Justice, to help law enforcement pursue criminal or civil penalties when fraudulent claims are detected. Table 1 summarizes the origin and responsibilities of the program integrity contractors who help CMS to detect fraud, waste, and abuse.

In addition to provisions of HIPAA and other legislation intended to strengthen Medicare program integrity functions, in 2006 Congress created the Medicaid Integrity Program through the Deficit Reduction Act of 2005. Its goals are to strengthen the national Medicaid audit program and to enhance federal oversight of and support and assistance to state Medicaid programs. The program provides states with technical assistance and support to enhance the federal-state partnership as well as to expand activities that involve data analysis, sharing algorithms of known improper billings, and fraud awareness through education and outreach. Individual states are responsible for ensuring the accuracy of Medicaid payments within their state programs, which can involve using their own staff or contractors to analyze claims to detect improper payments. In addition to the states’ efforts, CMS employs Medicaid program integrity contractors to perform specific activities as part of its efforts to detect fraud, waste, and abuse in the Medicaid program, such as reviewing provider claims payments that have been processed by the states.

Generally, each state Medicaid program integrity unit works independently, using its own data models, data warehouses, and approach to analysis. As a result, Medicaid data are stored in multiple disparate systems and databases throughout the country. Because of the volumes of work, states often augment their in-house capabilities by contracting with companies that specialize in Medicaid claims and utilization reviews. State Medicaid program integrity units target their activities to those providers that pose the greatest financial risk to their Medicaid programs. However, the states have limited methods of identifying Medicaid fraud in neighboring jurisdictions or by providers who move from state to state.

As stated in a July 2007 report by the HHS OIG, the agency intends for program integrity contractors to perform a significant amount of self- initiated, exploratory analysis to seek patterns or instances of fraud and abuse. One of the specific activities undertaken by these contractors is the analysis of claims data to identify improper billing that may indicate fraud or abuse. If the billing appears to be potentially fraudulent or abusive, the contractors take further actions, which can include requesting and reviewing medical records associated with the claims and referring the case to law enforcement.

In 2010, CMS created the Center for Program Integrity to serve as its focal point for all national Medicare and Medicaid program integrity fraud and abuse issues. The new center is responsible for, among other things, collaborating with other CMS components to develop and implement a comprehensive strategic plan, objectives, and measures to carry out the agency’s program integrity mission and goals, and ensure program vulnerabilities are identified and resolved. According to agency documentation describing the program, the center was designed to promote the integrity of the Medicare and Medicaid programs through provider and contractor audits and policy reviews, identification and monitoring of program vulnerabilities, and support and assistance to states; collaborate on the development and advancement of new legislative initiatives and improvements to deter, reduce, and eliminate fraud, waste and abuse; oversee all CMS interactions and collaboration with key stakeholders related to program integrity (e.g., the Department of Justice, HHS OIG, and state law enforcement agencies) for the purposes of detecting, deterring, monitoring, and combating fraud and abuse; and take action against those who commit or participate in fraudulent or other unlawful activities.

CMS’s Use of IT to Help Detect Fraud, Waste, and Abuse

Like financial institutions, credit card companies, telecommunications firms, and other private sector companies that take steps to protect customers’ accounts, CMS uses automated software tools to help predict or detect cases of improper claims and payments. For more than a decade, CMS and its contractors have applied such tools to access data from various sources to analyze patterns of unusual activities or financial transactions that may indicate fraudulent charges or other types of improper payments. For example, to identify unusual billing patterns and to support referrals for prosecution or other action, CMS and program integrity contractor analysts and investigators need, among other things, access to information about key actions taken to process claims as they are filed and specific details about claims already paid. This includes information on claims as they are billed, adjusted, and paid or denied; check numbers on payments of claims; and other specific information that could help establish provider intent. These data, along with data on regional or national trends on claims billing and payment, support the investigation and potential prosecution of fraud cases. Upon completing investigations, the contractors determine whether to refer the investigations as cases to law enforcement officials.

CMS and its program integrity contractors currently use many different means to store and manipulate data and, since the establishment of the agency’s program integrity initiatives in the 1990s, have built multiple databases and developed analytical software tools to meet their individual and unique needs. However, according to CMS, these geographically distributed, regional approaches to data analysis result in duplicate data and limit the agency’s ability to conduct analyses of data on a nationwide basis. Additionally, data on Medicaid claims are scattered among the states in multiple disparate systems and data stores, and are not readily available to CMS. Thus, CMS has been working for most of the past decade to consolidate program integrity data and analytical tools for detecting fraud, waste, and abuse. The agency’s efforts led to the initiation of the IDR program and, subsequently, the One PI program, which are intended to provide CMS and its program integrity contractors with a centralized source that consolidates Medicare and Medicaid data from the many disparate and dispersed legacy systems and databases and a Web-based portal and set of analytical tools by which these data can be accessed and analyzed to help detect cases of fraud, waste, and abuse.

CMS’s Initiative to Develop a Centralized Source of Medicare and Medicaid Data

The CMS Office of Information Services is responsible for agencywide IT management. Its initiative to develop a centralized data warehouse began in 2003 as an element of the agency’s Enterprise Data Modernization strategy. According to agency documentation, the strategy was designed to meet the increasing demand for higher quality and more timely data to support decision making throughout the agency, including identifying trends and discovering patterns of fraud, waste, and abuse. As part of the strategy, the agency established the Data Warehouse Modernization project to develop and implement the technology needed to store long- term data for analytical purposes, such as summary reports and statistical analyses. CMS initially planned for the data warehouse project to be complete by September 30, 2008.

However, in 2006 CMS expanded the scope of the project to not only modernize data storage technology but also to integrate Medicare and Medicaid data into a centralized repository. At that time, program officials also changed the name to IDR, which reflected the expanded scope. The Office of Information Services’ Enterprise Data Group manages the IDR program and is responsible for the design and implementation of the system. The program’s overall goal is to integrate Medicare and Medicaid data so that CMS and its partners may access the data from a single source. Specific goals for the program are to transition from stove-piped, disparate sets of databases to a highly integrated data environment for the enterprise; transition from a claim-centric orientation to a multi-view orientation that includes beneficiaries, providers, health plans, claims, drug code data, clinical data, and other data as needed; provide uniform privacy and security controls; provide database scalability to meet current and expanding volumes of provide users the capability to analyze the data in place instead of relying on data extracts.

According to IDR program officials, CMS envisioned that IDR would become the single repository for the agency’s data and enable data analysis within and across programs. Specifically, IDR was to establish the infrastructure for storing data for Medicare Parts A, B, and D, as well as a variety of other CMS functions, such as program management, research, analytics, and business intelligence. CMS envisioned an incremental approach to incorporating data into IDR. Specifically, program plans provided to the Office of Management and Budget (OMB) by the Office of Information Services in 2006 stated that all Medicare Part D data would be incorporated into IDR by the end of that fiscal year. CMS’s 2007 plans added the incorporation of Medicare Parts A and B data by the end of fiscal year 2007, and Medicaid data for 5 states by the end of fiscal year 2009, 20 states by 2010, 35 by 2011, and all 50 states by the end of fiscal year 2012.

Initial program plans and schedules also included the incorporation of additional data from legacy CMS claims-processing systems that store and process data related to the entry, correction, and adjustment of claims as they are being processed, along with detailed financial data related to paid claims. According to program officials, these data, called “shared systems” data, are needed to support the agency’s plans to incorporate tools to conduct predictive analysis of claims as they are being processed, helping to prevent improper payments. Shared systems data, such as check numbers and amounts related to claims that have been paid, are also needed by law enforcement agencies to help with fraud investigations. CMS initially planned to include all the shared systems data in IDR by July 2008. Figure 1 shows a timeline of initial plans for incorporating data into IDR.

In 2006, CMS’s Office of Financial Management initiated the One PI program with the intention of developing and implementing a portal and software tools that would enable access to and analysis of claims, provider, and beneficiary data from a centralized source. CMS’s goal for One PI was to support the needs of a broad program integrity user community, including agency program integrity personnel and contractors who analyze Medicare claims data, along with state agencies that monitor Medicaid claims. To achieve its goal, agency officials planned to implement a tool set that would provide a single source of information to enable consistent, reliable, and timely analyses and improve the agency’s ability to detect fraud, waste, and abuse. These tools were to be used to gather data about beneficiaries, providers, and procedures and, combined with other data, find billing aberrancies or outliers. For example, as envisioned, an analyst could use software tools to identify potentially fraudulent trends in ambulance services. He or she could gather data about claims for ambulance services and medical treatments, and then use other software to determine associations between the two types of services. If the analyst found claims for ambulance travel costs but no corresponding claims for medical treatment, the analyst may conclude that the billings for those services were possibly fraudulent.

According to agency program planning documentation, the One PI system was to be developed incrementally to provide access to data, analytical tools, and portal functionality in three phases after an initial proof of concept phase. The proof of concept phase was reportedly begun in early 2007 and focused on integrating Medicare and Medicaid data into the portal environment. After its completion, the first development phase focused on establishing a development environment in CMS’s Baltimore, Maryland, data center and, according to program officials, was completed in April 2009. The second and third phases of development were planned in January 2009 to run concurrently and to focus on the technical and analytical aspects of the project, such as building the environment to integrate the analytical tools using data retrieved from IDR, sourcing claims data from the shared systems, conducting data analyses in production, and training analysts who were intended users of the system. CMS planned to complete these two phases and implement the One PI portal and two analytical tools for use by program integrity analysts on a widespread basis by the end of fiscal year 2009.

CMS’s Office of Financial Management engaged contractors to develop the system. Responsibility for and management of the One PI program moved from the Office of Financial Management to the Center for Program Integrity in 2010. Figure 2 illustrates initial plans for One PI.

Prior GAO Reports on Fraud, Waste, and Abuse in the Medicare and Medicaid Programs

In our prior work, we have reported on CMS’s efforts to detect and prevent fraudulent and improper payments in the Medicare and Medicaid programs and on its management of IT to support its mission. For example, as early as 1995, we reviewed IT systems used in the Medicare program to detect and prevent fraud and discussed the availability of other technologies to assist in combating fraudulent billing. We found it was too early to fully document the cost-effectiveness of such systems, although several potential fraud cases were detected by this technology, indicating that these types of systems could provide net benefits in combating fraud. We observed that such technology could ultimately be utilized in the claims-processing environment to delay or even prevent the payment of questionable claims submitted by suspect providers.

We have also reported on weaknesses in CMS’s processes for managing IT investments based upon key practices established in our Information Technology Investment Management framework. Specifically, in 2005, we evaluated CMS’s capabilities for managing its internal investments, described plans the agency had for improving these capabilities, and examined the agency’s process for approving and monitoring state Medicaid Management Information Systems. We found that CMS had not established certain key practices for managing individual IT investments and recommended that the CMS Administrator develop and implement a plan to address the IT investment management weaknesses identified in the report. We also recommended that at a minimum, the agency should update its investment management guide to reflect current investment management processes. CMS subsequently took actions to implement each of our recommendations.

Additionally, our 2007 study of the Medicare durable medical equipment, prosthetics, orthotics, and supplies benefit found that it was vulnerable to fraud and improper payments. We recommended that CMS direct its contractors to develop automated prepayment controls to identify potentially improper claims and consider adopting the most cost-effective controls of other contractors. CMS concurred with the recommendation, but has not yet implemented the prepayment controls that we recommended.

In 2009, we examined the administration of the Medicare home health benefit, which we found to leave the benefit vulnerable to fraud and improper payments. We made several recommendations to the Administrator of CMS, including directing contractors to conduct post- payment medical reviews on claims submitted by home health agencies with high rates of improper billing identified through prepayment review. CMS stated it would consider two of our four recommendations—to amend regulations to expand the types of improper billing practices that are grounds for revocation of billing privileges, and to provide physicians who certify or recertify plans of care with a statement of services received by beneficiaries. CMS neither agreed nor disagreed with our other two recommendations.

Finally, in testifying on Medicare and Medicaid fraud, waste, and abuse in March 2011, we described steps that CMS could take to reduce improper payments and the agency’s recent solicitation for proposals of contracts for the development and implementation of automated tools that support reviews of claims before they are paid. These predictive modeling tools are intended to provide new capabilities to help prevent improper payments of Medicare claims.

IDR and One PI Have Been Developed and Implemented but Without All Planned Data and Widespread Use

CMS has developed and implemented IDR and One PI for use by its program integrity analysts, but IDR does not include all the data the agency planned to have incorporated by the end of 2010, and One PI is being used by a limited number of analysts. While CMS has developed and begun using IDR, the repository does not include all the planned data, such as Medicaid and shared systems data. Program officials attribute this lack of data to insufficient planning, which did not consider unexpected obstacles or allow time for contingencies. In addition, the agency has developed and deployed One PI, but the system is being used by less than 7 percent of the intended user community and does not yet provide as many tools as planned. According to agency officials, plans to train and deploy the system to a broad community of users were disrupted when resources dedicated to these activities were redirected to address a need to improve the user training program. Further, plans and schedules for completing the remaining work have not been finalized, and CMS has not identified risks and obstacles to project schedules that may affect its ability to ensure broad use and full implementation of the systems. Until program officials finalize plans and develop reliable schedules for providing all planned data and capabilities and ensuring that One PI gains broader use throughout the program integrity community, CMS will remain at risk of experiencing additional delays in reaching widespread use and full implementation of the systems. Consequently, the agency may miss an opportunity to effectively use these IT solutions to enhance its ability to detect fraud, waste, and abuse in the Medicare and Medicaid programs.

IDR Has Been Developed and Is in Use, but Does Not Yet Include All Data Needed to Enhance Program Integrity Efforts

IDR has been in use by CMS and contractor program integrity analysts since September 2006 and currently incorporates data related to claims for reimbursement of services under Medicare Parts A, B, and D. Specifically, CMS incorporated Part D data into IDR in September 2006, as planned, and incorporated Parts A and B data by the end of fiscal year 2008. The primary source of these data is CMS’s National Claims History database, from which data are extracted on a weekly basis. Other supplemental data were incorporated into IDR that are used to conduct program integrity analyses, including drug code data that are obtained from daily and weekly updates of data from CMS’s Drug Data Processing System, and claims- related data about physicians that are retrieved from National Provider Index databases on a daily basis. Additionally, IDR contains data about beneficiaries that are extracted daily from the Medicare Beneficiary Database and health plan contract and benefit data that are obtained on a weekly basis from CMS’s Health Plan Management Systems. According to IDR program officials with the Office of Information Services, the integration of these data into IDR established a centralized source of data previously accessed from multiple disparate system files.

CMS reported to OMB in 2010 that the agency had spent almost $48 million to establish IDR and incorporate the existing data since the program was initiated. Table 2 provides the actual costs of developing and implementing IDR for each year since fiscal year 2006, as reported to us by CMS officials.

Although the agency has been incorporating data from various data sources since 2006, IDR does not yet include all the data that were planned to be incorporated by the end of 2010 and that are needed to support enhanced program integrity initiatives. Specifically, the shared systems data that are needed to allow predictive analyses of claims are not incorporated. Without this capability, program integrity analysts are not able to access data from IDR that would help them identify and prevent payment of fraudulent claims. Additionally, IDR does not yet include the Medicaid data that are critical to analysts’ ability to detect fraud, waste, and abuse in the Medicaid program.

According to IDR program officials, the shared systems data were not incorporated into IDR because, although initial program integrity requirements included the incorporation of these data by July 2008, funding for the development of the software and acquisition of the hardware needed to meet this requirement was not approved until the summer of 2010. Since then, IDR program officials have developed project plans and identified users’ requirements, and plan to incorporate shared systems data by November 2011.

With respect to Medicaid data, program officials stated that the agency has not incorporated these data into IDR because the original plans and schedules for obtaining Medicaid data did not account for the lack of a mandate or funding for states to provide Medicaid data to CMS, or the variations in the types and formats of data stored in disparate state Medicaid systems. In this regard, program officials did not consider risks to the program’s ability to collect the data and did not include additional time to allow for contingencies. Consequently, the IDR program officials were not able to collect the data from the states as easily as they expected and, therefore, did not complete this activity as originally planned.

In addition to the IDR program, in December 2009, CMS initiated another agencywide program intended to, among other things, identify ways to collect Medicaid data from the many disparate state systems and incorporate the data into a single data store. As envisioned by CMS, this program, the Medicaid and Children’s Health Insurance Program Business Information and Solutions program, or MACBIS, is to include activities in addition to providing expedited access to current data from state Medicaid programs. For example, the MACBIS initiative is also intended to result in the development of a national system to address the needs of federal and state Medicaid partners, along with technical assistance and training for states on the use of the system. Once established, the MACBIS system data would then be incorporated into IDR and made accessible to program integrity analysts. According to program planning documentation, this enterprisewide initiative is expected to cost about $400 million through fiscal year 2016. However, plans for this program are not final, and funds for integrating Medicaid data into IDR have not yet been requested.

According to agency planning documentation, as a result of efforts to be initiated under the MACBIS program, CMS intends to incorporate Medicaid data for all 50 states into IDR by the end of fiscal year 2014. Program integrity officials stated that they plan to work with three states during 2011 to test the transfer and use of Medicaid data to help CMS determine the data that are available in those states’ systems. The Center for Program Integrity is also working with Medicaid officials to establish a test environment to begin integrating state Medicaid data into IDR. Despite establishing these high-level milestones, the agency has not finalized detailed plans for incorporating the Medicaid data that include reliable schedules that identify all the necessary activities and resources for completing these efforts or the risks associated with efforts to collect and standardize data from 50 independent systems that differ in design, technology, and other characteristics dictated by state policies.

Table 3 shows the original planned dates for incorporating the various types of data and the data that were incorporated into IDR as of the end of fiscal year 2010.

While CMS has identified target dates for incorporating the remaining data, best practices, such as those described in our cost estimation guide, emphasize the importance of establishing reliable program schedules that include all activities to be performed, assign resources (labor, materials, etc.) to those activities, and identify risks and their probability and build appropriate reserve time into the schedule. However, the IDR schedule we reviewed did not identify all activities and necessary resources or include a schedule risk analysis. Such an analysis could have helped CMS identify and prepare for obstacles, such as those previously encountered in trying to incorporate Medicaid data into IDR and expected to be encountered as CMS initiates efforts to collect and standardize data from 50 state systems. Without establishing a reliable schedule for future efforts to incorporate new data sources, the agency will be at greater risk of schedule slippages, which could result in additional delays in CMS’s efforts to incorporate all the data sources into IDR that are needed to support enhanced program integrity efforts.

One PI Has Been Developed but Deployed to Few Users and With Less Functionality Than Planned

According to program officials, user acceptance testing of the One PI system was completed in February 2009, and the system was deployed in September 2009 as originally planned. This initial deployment of One PI consisted of a portal that provided Web-based access to analytical tools used by program integrity analysts to retrieve and analyze data stored in IDR.

CMS reported to OMB that the agency had spent almost $114 million to develop the existing features and functionality of the One PI system by the end of fiscal year 2010. Table 4 provides information on the actual costs of developing One PI since fiscal year 2006, as reported to us by CMS officials.

As currently implemented, the system provides access to two analytical tools—Advantage Suite and Business Objects. Documented specifications of the One PI system described Advantage Suite as a commercial, off-the- shelf decision support tool that is used to perform data analysis to, for example, detect patterns of activities that may identify or confirm suspected cases of fraud, waste, or abuse. According to program officials and the One PI users to whom we spoke, program integrity analysts use Advantage Suite to analyze claims data retrieved from IDR and create standard and custom reports that combine data about costs and quality of services, providers, and beneficiaries. The results of this level of analysis may be used to generate leads for further analysis with Business Objects, which provides users extended capabilities to perform more complex analyses of data by allowing customized queries of claims data across the three Medicare plan types. It also allows the user to create ad hoc queries and reports for nonroutine analysis.

For example, an analyst could use Advantage Suite to identify potentially fraudulent trends in ambulance services. He or she could use the tool to gather data about claims for ambulance services and medical treatments, and then use Business Objects to conduct further analysis to determine associations between the two types of services. If the analyst found claims for ambulance travel costs but no corresponding claims for medical treatment, the analyst may conclude that the billings for those services were possibly fraudulent.

Figure 3 provides a simplified view of the IDR and One PI environment as currently implemented.

While program officials deployed the One PI portal and two analytical tools to CMS and contractor program integrity analysts, the system was not being used as widely as planned. Program planning documentation from August 2009 indicated that One PI program officials planned for 639 program integrity staff and analysts to be trained and using the system by the end of fiscal year 2010; however, CMS confirmed that by the end of October 2010 only 42 of those intended users were trained to use One PI, and 41 were actively using the portal and tools. These users represent less than 7 percent of the original intended users. Of these, 31 were contractors and 10 were CMS staff who performed analyses of claims to detect potential cases of fraud, waste, and abuse. Table 5 describes the analysts planned to be and actually using One PI at the end of fiscal year 2010.

According to One PI program officials, the system was not being used by the intended number of program integrity analysts because the office had not trained a sufficient number of analysts to use the system. Similarly, although CMS contractually requires Medicare program integrity contractors to use the system, officials stated that they could not enforce this requirement because they also had not trained enough of their program integrity contractors.

Although One PI program plans emphasized the importance of effective training and communications, program officials responsible for implementing the system acknowledged that their initial training plans and efforts were insufficient. According to the officials, they initially provided training for the all the components of the system—the portal, tools, and use of IDR data—in a 3-and-a-half-day course. However, they realized that the trainees did not effectively use One PI after completing the training. Consequently, program officials initiated activities and redirected resources to redesign the One PI training plan in April 2010, and began to implement the new training program in July of that year. The redesigned program includes courses on each of the system components and allows trainees to use the components to reinforce learning before taking additional courses. For example, the redesigned plan includes a One PI portal overview and data training webinars that users must complete before attending instructor-led training on Advantage Suite and Business Objects. The new plans also incorporate the use of “data coaches” who provide hands-on help to analysts, such as assistance with designing queries. Additionally, the plans require users to complete surveys to evaluate the quality of the training and their ability to use the tools after they complete each course.

As program officials took the initiative and time to redesign the training program, this effort caused delays in CMS’s plans to train the intended number of users. Since the new training program was implemented, the number of users has not yet significantly increased, but the number of contractor analysts requesting training has increased. Specifically, One PI officials told us that 62 individuals had signed up to be trained in 2011, and that the number of training classes for One PI was increased from two to four per month. The officials also stated that they planned to reach out to and train more contractors and staff from the HHS OIG and the Department of Justice to promote One PI. They anticipated that 12 inspectors general and 12 law enforcement officials would be trained and using One PI by the end of May 2011.

Nonetheless, while these activities indicate some progress toward increasing the number of One PI users, the number of users expected to be trained and to begin using the system represents a small fraction of the population of 639 intended users. Additionally, One PI program officials had not yet made detailed plans and developed schedules for completing training of all the intended users. Further, although program officials had scheduled more training classes, they have not established deadlines for contractor analysts to attend training so that they are able to fulfill the contractual requirement to use One PI. Unless the agency takes more aggressive steps to ensure that its program integrity community is trained, it will not be able to require the use of the system by its contractors, and the use of One PI may remain limited to a much smaller group of users than the agency intended. As a result, CMS will continue to face obstacles in its efforts to deploy One PI to the intended number of program integrity users as the agency continues to develop and implement additional features and functionalities in the system.

Additionally, although efforts to develop and implement One PI were initiated in 2006 and the Advantage Suite and Business Objects tools are fully developed, implemented, and in use, the One PI system does not yet include additional analytical functionality that CMS initially planned to implement by the end of 2010. Program documentation for the system includes plans for future phases of One PI development to incrementally add new analytical tools, additional sources of data, and expanded portal functionality, such as enhanced communications support, and specifically included the integration of a third tool by the end of fiscal year 2010. However, program officials have not yet identified users’ needs for functionality that could be provided by another tool, such as the capability to access and analyze more data from IDR than the current implementation of the system provides. According to program officials, they intend to determine users’ needs for additional functionality when the system becomes more widely used by agency and contractor analysts who are able to identify deficiencies and define additional features and functionality needed to improve its effectiveness.

Additionally, as with IDR, in developing the One PI schedule estimate that was provided to OMB in 2010, program officials did not complete a risk assessment for the schedule that identified potential obstacles to the program. As a result, they lacked information needed to plan for additional time to address contingencies when obstacles arose. As the program office makes plans for deploying the system to the wide community of program integrity analysts and implementing additional tools, it is crucial that officials identify potential obstacles to the schedules and the risks they may introduce to the completion of related activities. For example, an analysis that identified the risk that resources would need to be redirected to other elevated priorities, such as user training, could have informed managers of the need to include additional time and resources in the schedule to help keep the development and deployment of One PI on track. Unless program officials complete a risk assessment of schedules for ongoing and future activities, CMS faces risks of perpetuating delays in establishing widespread use of One PI and achieving full implementation of the system for increased rates of fraud, waste, and abuse detection.

CMS Is Not Yet Positioned to Fully Meet Goals and Objectives for Detecting Fraud, Waste, and Abuse through the Use of IDR and One PI

Our prior work emphasized agencies’ need to ensure that IT investments actually produce improvements in mission performance. As we have reported, agencies should forecast expected benefits and then measure actual financial benefits accrued through the implementation of IT programs. Further, OMB requires agencies to report progress against performance measures and targets for meeting them that reflect the goals and objectives of the programs. To do this, performance measures should be outcome-based, developed with stakeholder input, and monitored and compared to planned results. Additionally, industry experts describe the need for performance measures to be developed with stakeholders’ input early in a project’s planning process to provide a central management and planning tool and to monitor the performance of the project against plans and stakeholders’ needs.

CMS Has Made Limited Progress toward Meeting Program Integrity Goals and Objectives through the Use of IDR

As stated in program planning documentation, IDR’s overall goal is to integrate Medicare and Medicaid data so that CMS and its partners may access the data from a single source. Specifically, the implementation of IDR was expected to result in financial benefits associated with the program’s goal to transition from a data environment of stove-piped, disparate databases and systems to an integrated data environment.

Officials with the Office of Information Services stated that they developed estimates of financial benefits expected to be realized through the use of IDR. In 2006, program officials projected financial benefits from IDR of $152 million at an estimated cost of $82 million, or a net benefit of about $70 million. In 2007 these officials revised their projection of total financial benefits to $187 million based on their estimates of the amount of improper payments they expected to be recovered as a result of analyzing data provided by IDR. The resulting net benefit expected from implementing IDR was estimated to be $97 million in 2010 due to changes in program cost estimates.

Table 6 includes CMS’s estimated financial benefits, costs, and net benefits reported to OMB for the lifecycle of the program from fiscal year 2006 to 2010.

However, as of March 2011, program officials had not identified actual financial benefits of implementing IDR based on the recovery of improper payments. In our discussions with the Office of Information Services, program officials stated they determined that deploying IDR led to the avoidance of IT costs as a result of the retirement of several legacy systems attributable to the implementation of IDR. However, they had not quantified these or any other financial benefits. Until officials measure and track financial benefits related to program goals, CMS officials cannot be assured that the use of the system is helping the agency prevent or recover funds lost as a result of improper payments of Medicare and Medicaid claims.

Additionally, while program officials defined and reported to OMB performance targets for IDR related to some of the program’s goals, they do not reflect its goal to provide a single source of Medicare and Medicaid data for program integrity efforts. Although progress made to date in implementing IDR supports the program’s goals to transition CMS to an integrated data environment, program officials have not defined and reported to OMB performance measures to gauge the extent to which the program is meeting this goal. Specifically, IDR officials defined performance measures for technical indicators, such as incorporating Medicare data into the repository, making the data available for analysis, and reducing the number of databases CMS must support, but they have not defined measures and targets that reflect the extent to which all the data needed to support program integrity initiatives are incorporated into a single source, including the Medicaid and shared systems data which have not yet been incorporated into IDR. Further, the IDR performance measures do not reflect indicators that may lead to the program’s ability to achieve the financial benefits defined by the agency’s program integrity initiatives.

In discussing this matter, IDR officials stated that the performance measures for the program are only intended to track progress toward implementing technical capabilities of the system, such as the amount of data from specific sources incorporated into the repository and made available through software tools to analysts. They do not define performance indicators, measures, and targets for incorporating data from future sources of data until plans are made and funds are provided by the agency’s business offices to begin activities to implement new functionalities into IDR. IDR program officials also stated that they do not define or track business-related performance indicators for achieving specific program integrity goals; rather, they depend upon business owners to measure and track these indicators based upon the use of IDR data to achieve business goals. However, without performance measures that reflect business owners’ and other stakeholders’ needs for the program to deliver a single source of all Medicare and Medicaid data needed to conduct analyses, and lacking measures that reflect the success of the program toward achieving financial benefits projected for program integrity initiatives, program officials lack key management information needed to ensure that the data and infrastructure components provided by IDR enhance CMS’s ability to meet its program integrity goals and objectives. Without this assurance, the effectiveness of the system’s capability to increase rates of fraud, waste, and abuse detection and, consequently, decrease the amount of money lost to improper payments of claims will remain unknown.

CMS Is Not Yet Positioned to Demonstrate Improvements in Its Ability to Meet Goals and Objectives for Detecting Fraud, Waste, and Abuse through the Use of One PI

The Center for Program Integrity’s overall goal for One PI was to provide robust tools for accessing a single source of information to enable consistent, reliable, and timely analyses to improve the agency’s ability to detect fraud, waste, and abuse. Achieving this goal was intended to result in the recovery of significant funds lost each year from improper payments of Medicare and Medicaid claims. In September 2007, program officials projected financial benefits from implementing One PI—nearly $13 billion over the 10-year lifecycle of the project. According to program officials, these benefits were expected to accrue from the recovery of improper payments of Medicare and Medicaid claims and reduced program integrity contractor expenditures for supporting IT required to maintain separate databases.

In September 2007, One PI officials projected and reported to OMB benefits of nearly $13 billion. They subsequently revised this estimate to approximately $21 billion. Program officials told us that increases in the projected financial benefits were made based on assumptions that accelerated plans to integrate Medicare and Medicaid data into a central data repository would enable One PI users to identify increasing numbers of improper payments sooner than previously estimated, thus allowing the agency to recover more funds lost due to payment errors. Table 7 provides data CMS reported to OMB on estimated benefits and costs, actual costs as of the end of fiscal year 2010, and net benefits projected to be realized as a result of implementing One PI from fiscal year 2007 through 2010.

However, the current implementation of One PI has not yet produced outcomes that position the agency to identify or measure financial benefits. Therefore, the net financial benefit of developing and implementing One PI remains unknown. Center for Program Integrity officials stated that at the end of fiscal year 2010—over a year after deploying One PI—it was too early to determine whether the program has provided any financial benefits because, since the program had not met its goal for widespread use of One PI, there were not enough data available to quantify financial benefits attributable to the use of the system. These officials anticipated that as the user community is expanded, they will be able to begin to identify and measure financial and other benefits of using the system. However, the officials also indicated that they had not yet defined mechanisms for determining the amount of money recovered as a result of detecting improper payments through the use of One PI. As with IDR, until the agency quantifies and tracks the progress it is making in delivering benefits intended to be realized through widespread use of One PI, CMS officials cannot be assured of the cost-effectiveness of implementing One PI to help the agency meet its goal to enable consistent, reliable, and timely analyses of data to improve the agency’s ability to detect fraud, waste, and abuse.

Additionally, in discussion groups held with active One PI users, program integrity analysts identified several issues that confirmed the agency’s limited progress toward meeting the goals of the program. For example, while several users told us that the One PI system can support their work, they recognized limited progress toward the establishment of a single source of information and analysis tools for all fraud, waste, and abuse activities. Further, One PI users stated that the system enabled analysts to access national data not otherwise accessible to them and supported analysis across different Medicare programs. They also noted that the tools offered by One PI provided more functionality than other tools they use. However, of the analysts in the discussion groups, most did not use One PI as their only source of information and analysis for detecting improper payments. Rather, to help conduct their work, they relied on other analysis tools provided by CMS or their companies, along with data from CMS claims processing contractors or from private databases created by other contractors.

One PI users in the discussion groups also told us that they use other tools because they are more familiar with those tools. Additionally, they stated that other databases sometimes provide data that are not currently accessible through One PI and IDR, such as demographic data about providers. Program integrity analysts further stated that they only use One PI as a cross-check of data and analysis from their own systems because they are not yet convinced that One PI can be used as a replacement for or adjunct to those data sources and tools.

Further, CMS officials have not developed quantifiable measures for meeting the program’s goals. CMS officials defined and reported to OMB performance measures and targets toward meeting the program’s goals for enabling timely analyses of data to detect cases of fraud, waste, and abuse, but have not yet been able to quantify measures for these indicators. For example, performance measures and targets for One PI include increases in the detection of improper payments for Medicare Parts A and B claims. However, according to program integrity officials, measures had not yet been quantified because they had not yet identified ways to determine the extent to which increases in the detection of errors could be attributed to the use of One PI. Additionally, the limited use of the system has not generated enough data to quantify the amount of funds recovered from improper payments. Moreover, measures of One PI’s program performance do not accurately reflect the current state of the program. Specifically, indicators to be measured for the program include the number of states using One PI (for Medicaid integrity purposes) and decreases in the Medicaid payment error rate, but One PI does not have access to those data because they are not yet incorporated into IDR. Therefore, these performance indicators are not relevant to the current implementation of the system.

Finally, CMS officials did not consult external system users (e.g., program integrity contractors) in developing measures of One PI’s effectiveness. According to industry experts, developing performance measures with stakeholder input early in the planning process can provide a mechanism for gauging the effectiveness of outcomes toward meeting business needs and achieving program goals as a program progresses. However, CMS officials did not consult external users of the system about how they would measure its effectiveness. According to program officials, program integrity stakeholders within CMS were involved in the development of the performance measures; however, external users of the system were not asked to provide input when it may have been used to establish an effective performance tracking tool, such as when defining ways to determine whether One PI meets stakeholders’ needs. For example, program officials told us that they intend to determine user satisfaction, a performance measure reported to OMB, by conducting surveys at the end of training sessions. However, these surveys were conducted before the analysts actually used the system in their work and were focused on satisfaction with the training itself. In this case, involvement of external stakeholders when defining the measure could have led to more effective ways to determine user satisfaction, such as surveying analysts based on their experiences resulting from the use of One PI after a certain period of time defined by stakeholders.

Until they define measurable performance indicators and targets that reflect the goals and objectives of CMS’s program integrity initiatives, agency officials will continue to lack the information needed to ensure that the implementation of One PI helps improve the agency’s ability to identify improper payments and to detect cases of fraud, waste, and abuse. Additionally, when lacking stakeholders’ input into the process for determining measures of successful performance, One PI program officials may miss an opportunity to obtain information needed to define meaningful measures that reflect the success of the program toward meeting users’ and the agency’s needs. Because it lacks meaningful outcome-based performance measures and effective methods for tracking progress toward meeting performance targets, CMS does not have the information needed to ensure that the system is useful to the extent that benefits realized from the implementation of One PI help the agency meet program integrity goals.

Conclusions

IDR and One PI program officials have made progress in developing and implementing IDR and One PI to support CMS’s program integrity initiatives, but the systems do not yet provide all the data and functionality initially planned. Additionally, CMS program integrity officials have not yet taken appropriate actions to ensure the use of IDR and One PI on a widespread basis for program integrity purposes. Further, program officials have not defined plans and reliable schedules for incorporating the additional data into IDR that are needed to support its program integrity goals. Until the agency takes these steps, it cannot ensure that ongoing development, implementation, and deployment efforts will provide the data and technical capabilities needed to improve program integrity analysts’ capabilities for detecting potential cases of fraud, waste, and abuse.

Furthermore, because the systems are not being used as planned, CMS program integrity officials are not yet in a position to determine the extent to which the systems are providing financial benefits or supporting the agency’s initiatives to meet its program integrity goals and objectives. Until it does so, CMS officials will lack the means to determine whether the use of the systems contributes to the agency’s goal of reducing the number and amounts of improper payments made as a result of fraudulent, wasteful, or abusive claims for Medicare and Medicaid services. Furthermore, the contribution of IDR and One PI to the agency’s efforts to save billions of dollars each year attributable to improper payments made due to fraud, waste, and abuse in the Medicare and Medicaid programs will remain unknown.

Recommendations for Executive Action

To help ensure that the development and implementation of IDR and One PI are successful in helping the agency meet the goals and objectives of its program integrity initiatives, we are recommending that the Administrator of CMS take the following seven actions: finalize plans and develop schedules for incorporating additional data into IDR that identify all resources and activities needed to complete tasks and that consider risks and obstacles to the IDR program; implement and manage plans for incorporating data in IDR to meet schedule milestones; establish plans and reliable schedules for training all program integrity analysts intended to use One PI; establish and communicate deadlines for program integrity contractors to complete training and use One PI in their work; conduct training in accordance with plans and established deadlines to ensure schedules are met and program integrity contractors are trained and able to meet requirements for using One PI; define any measurable financial benefits expected from the implementation of IDR and One PI; and with stakeholder input, establish measurable, outcome-based performance measures for IDR and One PI that gauge progress toward meeting program goals.

Agency Comments and Our Evaluation

In written comments on a draft of this report, signed by HHS’s Assistant Secretary for Legislation and reprinted in appendix II, CMS stated that it concurred with all of our recommendations and identified steps agency officials were taking to implement them. Among these were actions to further refine training plans to better ensure that program integrity contractors are trained and able to meet requirements to use One PI, along with efforts to define measurable financial benefits expected from augmenting the data in IDR. If these and other identified actions are implemented in accordance with our recommendations, CMS will be better positioned to meet the goals and objectives of its program integrity initiatives. The agency also provided technical comments, which were incorporated as appropriate.

As we agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution of it until 30 days from the date of this letter. At that time, we will send copies of this report to appropriate congressional committees, the Administrator of CMS, and other interested parties. The report will also be available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staffs have questions about this report, please contact me at (202) 512-6304 or melvinv@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

The objectives of our review were to (1) assess the extent to which the Centers for Medicare and Medicaid Services (CMS) has developed and implemented the Integrated Data Repository (IDR) and One Program Integrity (One PI) systems and (2) determine the agency’s progress toward achieving defined goals and objectives for using the systems to help detect fraud, waste, and abuse in the Medicare and Medicaid programs.

To assess the extent to which IDR and One PI have been developed and implemented, we collected and analyzed agency documentation that described planning and management activities. Specifically, we assessed project management plans and artifacts that described the status of the systems, such as program management review briefings to technical review boards, and memoranda approving continued development and implementation of the systems at key decision points in the systems’ lifecycles. We observed the operation of CMS’s data center where IDR is installed and viewed a demonstration of the One PI portal and analytical tools. We also discussed with officials from CMS’s Office of Information Services and Center for Program Integrity plans for and progress made toward developing and implementing the systems. We focused our analysis on the extent to which the development and implementation of IDR and One PI met system and business requirements and plans for deploying the systems to CMS’s program integrity analysts.

To assess the agency’s processes for defining system requirements, we reviewed IDR and One PI requirements management plans, system requirements, and documentation that traces requirements to functionality provided by the systems at different stages of implementation. Program documents we reviewed include the 2007 IDR Medicare Program Integrity Requirements, the 2006 One PI Startup Findings Draft, the 2010 One PI Requirements Management Plan, and detailed software requirements specifications for One PI. In addition, we discussed with IDR and One PI program officials their requirements development and management processes and procedures. We then assessed the department’s current approach to requirements development and management against best practices identified in the Software Engineering Institute’s Capability Maturity Model Integration.

To assess schedule estimates of the IDR and One PI programs, we used criteria defined in GAO’s cost estimating and assessment guide to determine the extent to which relevant schedules were prepared in accordance with best practices that are fundamental to estimating reliable schedules. We identified information reported to the Office of Management and Budget (OMB) by CMS in fiscal year 2010 that defined program schedule estimates for the remaining lifecycles of the programs through 2016. We collected and analyzed program documentation that supported these estimates, such as work breakdown structures and staffing estimates.

To assess each program’s schedule estimates, we rated the IDR and One PI program management offices’ implementation of nine scheduling best practices defined in our guidance. Based on these criteria, we analyzed the One PI integrated master schedule and the IDR validation, along with supporting documentation, and used commercially available software tools to assess the schedules. Specifically, we determined whether each schedule was developed by identifying and including critical elements of reliable scheduling best practices, such as identifying all resources needed to conduct activities, and whether risk assessment and contingency plans had been conducted for the schedules.

We shared our guidance, the criteria against which we evaluated the program’s schedule estimates, as well as our preliminary findings with program officials. We then discussed our preliminary assessment results with the program management officials. When warranted, we updated our analyses based on the agency response and additional documentation provided to us. We also analyzed changes to the program schedules over time.

To determine the reliability of the data used to assess schedule estimates, we used a scheduling analysis software tool that identified missing logic and constraints, and checked for specific problems that could hinder the schedule’s ability to dynamically respond to changes. We examined the schedule data to identify any open-ended activities (i.e., activities with no predecessor or successors), and searched for activities with poor logic, such as activities with constraints that keep the schedule rigid (e.g., start no earlier than, finish no later than, etc.). We found the data sufficiently reliable for the purposes of this review.

To determine the number of system end users for One PI, we identified the universe of analysts trained to use One PI by examining documentation provided by CMS. Specifically, we obtained a list of trained users from the Center for Program Integrity. From that list, we selected program integrity analysts whom CMS identified as using the system to conduct analyses of IDR data to identify potential cases of fraud, waste, and abuse. We then compared this selection of analysts to data generated by the One PI system that recorded user login data from January 3, 2010, through October 16, 2010, to identify the current population of One PI users. Through this analysis, we identified 41 trained program integrity analysts who had used the system during the designated time period, including 8 Medicare drug integrity contractors, 23 zone program integrity and program safeguard contractors, and 10 CMS program integrity analysts.

To ensure that the data that we used to identify One PI users were reliable, we held discussions with CMS officials who were knowledgeable of the user community and mechanisms for accessing the system. We discussed with them the list of trained end users and the computer-generated login information provided by the system. We also discussed the reliability of the computer-generated system login information. Specifically, agency officials confirmed that the data reported by the system were complete and accurate and that the method we used to identify active users—an analysis of system login data—was valid.

To determine the extent to which the IDR and One PI programs have achieved defined goals and objectives for using the systems to help detect fraud, waste, and abuse, we collected CMS’s analyses of projected costs and benefits for IDR and One PI. We also collected and assessed data reported on the costs and benefits realized through the current implementation of the systems. To do so, we compared (1) actual costs and benefits attributed to each system through fiscal year 2010 and (2) current estimated total lifecycle costs and benefits for each system. We calculated the expected net benefit by subtracting estimated and actual system costs from estimated and actual system benefits for each system. To understand how costs and benefits for each system were derived, we met with officials from the Office of Information Services and from the Center for Program Integrity and discussed CMS’s processes for estimating and tracking costs and benefits of both IDR and One PI. We also obtained from agency officials documentation about and descriptions of qualitative benefits provided by both systems. Additionally, we reviewed planning documents that described the goals and objectives of both programs, along with other documentation that described actions taken to address program goals and objectives. We reviewed and assessed supporting documentation for the measures, which the agency reported to OMB as having been met.

To determine if CMS’s approach to developing performance measures for IDR and One PI was consistent with federal guidance, we examined documents describing CMS’s approach and held discussions with program officials about practices they followed when defining performance measures and targets. We compared program officials’ practices to guidance defined by OMB. We also compared the performance measures defined for the two programs to CMS’s goals and objectives for program integrity initiatives to determine if the IDR and One PI measures supported intended outcomes of agencywide efforts to better detect fraud, waste, and abuse. We supplemented our documentation review with interviews of officials from the Center for Program Integrity and the Office of Information Services to obtain additional information about the development of current and future performance measures for IDR and One PI. During our interviews, we discussed performance measures and strategic goals and initiatives for One PI and IDR, and the extent to which the agency involved internal and external stakeholders in the development of performance measures.

To obtain information about the extent to which One PI has been deployed and is being used by a broad community of program integrity analysts to meet CMS’s goals and objectives, we invited the 41 users we identified in addressing the first objective of this engagement to participate in facilitated discussions about the data and tools needed to support fraud, waste, and abuse detection. Thirty-two of those 41 users attended the discussion group meetings. During those meetings, we discussed the following topics: usage of One PI tools and data from IDR, comparison and contrasting of One PI and IDR with other tools and data sets, and benefits and challenges of using One PI and IDR for detecting fraud, waste, and abuse. We also discussed users’ needs for analytical tools and data and for systems training. After those discussions, we sent written questions to all 32 discussion group participants to obtain more detailed information about their use of analytical tools and data sources. Thirty-one participants responded and provided additional supplementary information about their use of One PI and IDR.

For each of the objectives, we assessed the reliability of the data we analyzed through interviews with agency officials knowledgeable of the user community and training program, mechanisms for accessing the systems, and the methods for tracking and reporting costs and schedules of the IDR and One PI programs. We found the data sufficiently reliable for the purposes of this review.

We conducted this performance audit from June 2010 through June 2011 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Health and Human Services

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Teresa F. Tucker (Assistant Director), Sheila K. Avruch (Assistant Director), April W. Brantley, Clayton Brisson, Neil J. Doherty, Amanda C. Gill, Kendrick M. Johnson, Lee A. McCracken, Terry L. Richardson, Karen A. Richey, and Stacey L. Steele made key contributions to this report.