Medicare Transaction System: Strengthened Management and Sound Development Approach Critical to Success
Released: 1995-11-16T00:00:00

Letter

Mts: an Important Vision

HCFA’s vision, which we support, is for a single, unified system to replace the nine current systems now used by Medicare, the nation’s largest health insurer, serving about 37 million Americans. The goals of MTS are to better protect program funds from waste, fraud, and abuse; allow better oversight of Medicare contractors’ operations; improve service to beneficiaries and providers; and reduce administrative expenses. At present, HCFA expects MTS to be fully operational in September 1999, and to process over 1 billion claims and pay $288 billion in benefits per year by 2000. These are ambitious goals, and we realize that developing such a system is complex and challenging.

Currently, when legislative or administrative initiatives result in revised payment or coverage policies, each of the nine automated systems maintained by Medicare contractors to process claims must be modified. An integrated system would eliminate the need for such cumbersome and costly multiple processes. In January 1994, HCFA awarded a contract to GTE Government Systems Corporation to design, develop, and implement the new automated system for processing claims. Two related contracts were awarded: to Intermetrics, Inc., in April 1994 for what is known as independent verification and validation, or IV&V—a separate technical check on GTE’s work; and to SETA Corporation in September 1995 for systems testing.

Best Practices: Disciplined Management Process Essential for Success

Over the last 12 years, the federal government has spent more than $200 billion on information technology, and we have evaluated hundreds of these projects. On the basis of this work, we have determined that two basic, recurring problems constrain the ability of organizations to successfully develop large systems: (1) failure to adequately select, plan, prioritize, and control information system projects; and (2) failure to take advantage of business process improvements that can significantly reduce costs, improve productivity, and provide better services to customers.

These problems have often led to meager results in federal agency efforts to design, develop, and acquire complex information systems. For example, after investing over 12 years of effort, the Federal Aviation Administration (FAA) chose to cut its losses in its problem-plagued Advanced Automation System by cancelling or extensively restructuring elements of this modernization of the nation’s air traffic control system. The reasons for FAA’s problems included the failure to (1) accurately estimate the project’s technical complexity and resource requirements, (2) finalize system requirements, and (3) adequately oversee contractor activities.

Similarly, our work on IRS’ Tax Systems Modernization, designed to automate selected tax-processing functions, identified several weaknesses. For example, IRS lacked (1) a disciplined process for managing definition of requirements, and (2) a management process for controlling software development. These problems caused significant rework and delays.

Last year, to help federal agencies improve their chances of success, we completed a study of how successful private and public organizations reached their goals of acquiring information systems that significantly improved their ability to carry out their missions. Our report describes an integrated set of fundamental management practices that were instrumental in producing success. The active involvement of senior managers, focusing on minimizing project risks and maximizing return on investment, was essential. To accomplish these objectives, senior managers in successful organizations consistently followed these practices—which have become known as best practices—to ensure that they received information needed to make timely and appropriate decisions.

Among others, one key practice is for executives to manage information systems as investments rather than expenses. This requires using disciplined investment control processes that provide quantitative and qualitative information that senior managers can use to continuously monitor costs, benefits, schedules, and risks; and to ensure that structured systems-development methodologies are used throughout the system’s life cycle.

A consensus has emerged within the administration and the Congress that better investment decisions on information technology projects are needed to help the government improve service. Important changes recently made to several laws and executive policy guidance are instituting best-practice approaches of leading organizations into the federal government. This month, the Office of Management and Budget will issue guidance that describes an analytical framework for making information technology investment decisions. Developed in cooperation with GAO, this guidance calls for agencies to implement management practices to select, control, and evaluate information technology investments throughout their life cycles.

Mts Displays Early Symptoms of Unnecessary Risk

HCFA has not yet instituted a set of well-defined investment control processes to measure the quality of development efforts and monitor progress and problems. This situation has contributed to a series of problems related to requirements-definition, schedule, and costs; these problems raise concerns that MTS may suffer the same fate as many other complex systems—extensive delays, large cost increases, and the inability to achieve potential benefits.

First, HCFA has not sufficiently followed sound practices in defining MTS project requirements. As a result, HCFA has twice redirected the approach and, 2 years into the contract, requirements definition at the appropriate level of specificity has not been completed. Requirements, which are defined during the analysis phase of a project, document the detailed functions and processes the system is expected to perform and the performance level to be achieved. They are intended to correct deficiencies in the current system and take advantage of opportunities to improve program economy, efficiency, and service. Because requirements provide the foundation for designing, developing, testing, and implementing the system, it is critical that they be precisely defined to avoid ambiguity and overlap, and that they completely and logically describe all features of the planned system. Using an appropriate methodology to define requirements significantly reduces risk that requirements defects will cause technical problems.

Originally, HCFA’s plans called for GTE to document the current systems’ requirements, while HCFA staff defined new or future requirements for MTS. However, in September 1994, HCFA concluded that GTE’s analysis of the current systems did not contain enough detail to fully describe the current systems’ requirements. HCFA then directed GTE to provide additional detail. In September 1995, HCFA concluded that the products GTE was developing were too detailed, and again directed GTE to refocus its efforts—this time, however, on assisting HCFA staff in defining future MTS requirements.

On the basis of our experience in evaluating other systems, such multiple redirections in the analysis phase of a major project indicate that HCFA’s process to control requirements lacks discipline. HCFA currently lacks an effective process for managing requirements, and has not provided adequate guidance to staff responsible for defining requirements. These deficiencies have also been cited by the IV&V contractor as an area of significant risk.

Because of problems in completing the definition of requirements, and HCFA’s plans to implement a fully functional MTS in September 1999, HCFA is proceeding into the next phase of system development, the design phase, before requirements have been completed. HCFA plans to select an MTS design alternative by the end of this calendar year, but requirements are not scheduled to be completed until September 1996. Because design alternatives are used to determine how the system will be structured, if the alternatives do not reflect key requirements, the system’s future capabilities may be seriously constrained. The IV&V contractor pointed out that HCFA’s plan to select the system design in parallel with defining system requirements also increases risks that the system will not meet important goals.

HCFA officials told us they believe that MTS requirements are sufficiently defined to prepare high-level system-design alternatives, but the IV&V contractor disagrees. To support critical design decisions, requirements need to be sufficiently detailed to include such functions and processes as performance levels and response times. When we reviewed HCFA’s preliminary set of requirements, we found that many of them did not contain enough detail.

Second, HCFA’s development schedule for MTS contains significant overlap—or concurrency—among the various system-development phases: analysis, design, programming, testing, validation, and implementation. As shown in figure 1, the April 1994 MTS schedule—an early estimate by HCFA—is used only to illustrate the sequential nature of these phases. The November 1995 schedule shows extensive concurrency; for example, the analysis and design phases are occurring simultaneously during the period from July 1994 to September 1996.

In our January 1994 report on MTS, we stated that if a contractor advances too far into a succeeding system-development phase before sufficient progress has been made in the previous phase, the risk that technical problems will occur is significantly increased. Senior HCFA officials recently told us that the MTS schedule contains concurrency because it is important to deploy the system before the end of the century; otherwise, significant costs would be incurred to modify existing systems. What is needed is quantifiable information on this cost, compared with an assessment of the risks of concurrency. HCFA has not, however, implemented a formal process to assess and manage system-development risks. The IV&V contractor has also cited this lack of a formal risk-assessment process as a problem.

In addition, while HCFA’s MTS schedule has been revised several times because of the redirection of requirements definition in the analysis phase, the initial and final system-implementation dates have remained largely unchanged. As a result, the time scheduled to complete the rest of the system-development phases to meet those dates is now significantly compressed. For example, because HCFA did not adjust the initial operating capability date, it is now scheduled, at one point in a 1-year period, to work concurrently on the remaining development phases—design, programming, testing, and validation. On the basis of our previous work on large systems-development efforts, we believe that failure to allow for sufficient time to complete system-development phases increases risk and will likely result in reduced systems capability.

Moreover, HCFA has not developed an integrated schedule that reflects both HCFA and contractor activities, work products, and time frames needed to perform these activities. Such a schedule provides an important tool for closely monitoring progress and problems in completing various activities. Without detailed insight about the actual status of all development activities, management will not have the information it needs to make timely decisions. HCFA’s IV&V contractor also cited concerns about the lack of an integrated schedule baseline for MTS. HCFA officials agreed that such a schedule is important.

Finally, HCFA has not sufficiently developed disciplined processes to adequately monitor progress in achieving cost and benefit objectives, which are important to managing projects as investments. The estimated MTS project costs, pegged by HCFA at $151 million in 1992, have not been updated since then, and HCFA is not tracking internal costs associated with the project, such as personnel, training, and travel. According to HCFA officials, they plan to update their cost estimate next year, to reflect their current understanding of MTS’ capabilities. Similarly, except for estimated administrative savings of $200 million a year during the first 6 years of operation (1997-2002), HCFA has not yet quantified other important expected benefits of MTS, such as targets for reducing fraud, waste, and abuse, and improving services to beneficiaries and providers. Without current information on costs and potential benefits, HCFA executives will not be in the best position to realistically monitor performance or identify and maximize the system’s true return on investment.

Conclusions

We have seen an inescapable pattern in agencies’ development of information systems: even on a small scale, those that are not developed according to sound practices encounter major, expensive problems later on. The larger the project, the bigger the risk. It takes serious, sustained effort and disciplined management processes to effectively manage system development. Effective oversight greatly reduces exposure to risk; without it, risk is dramatically and needlessly increased. The risks we see in the development of MTS can be substantially reduced if HCFA management implements some of the best practices that have been proven effective in other organizations: managing systems as investments, changing information management practices, creating line manager ownership, better managing resources, and measuring performance. HCFA still has time to correct these deficiencies. We are encouraged by HCFA’s expression of interest in learning about how to implement the best practices in systems development used by successful organizations, and look forward to working with them.

This concludes our statement, Mr. Chairmen. We will be happy to respond to any questions you or other members of the subcommittees may have at this time.

The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.