Financial Management Systems: HHS Faces Many Challenges in Implementing Its Unified Financial Management System
Published: 2004-09-30T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: GAO has previously reported on systemic problems the federal government faces in achieving the goals of financial management reform and the importance of using disciplined processes for implementing financial management systems. As a result, the Subcommittee on Government Efficiency and Financial Management, House Committee on Government Reform, asked GAO to review and evaluate the agencies' plans and ongoing efforts for implementing financial management systems. The results of GAO's review of the Department of Health and Human Services' (HHS) ongoing effort to develop and implement the Unified Financial Management System (UFMS) are discussed in detail in the report Financial Management Systems: Lack of Disciplined Processes Puts Implementation of HHS' Financial System at Risk (GAO-04-1008). In this report, GAO makes 34 recommendations focused on mitigating risks associated with the project. In light of this report, the Subcommittee asked GAO to testify on the challenges HHS faces in implementing UFMS.
section_title: What GAO Found
paragraphs: HHS had not effectively implemented several disciplined processes, which are accepted best practices in systems development and implementation, and had adopted other practices, that put the project at unnecessary risk. Although the implementation of any major system is not a risk-free proposition, organizations that follow and effectively implement disciplined processes can reduce these risks to acceptable levels. While GAO recognized that HHS had adopted some best practices related to senior level support, oversight, and phased implementation, GAO noted that HHS had focused on meeting its schedule to the detriment of disciplined processes. GAO found that HHS had not effectively implemented several disciplined processes to reduce risks to acceptable levels, including requirements management, testing, project management and oversight using quantitative measures, and risk management. Compounding these problems are departmentwide weaknesses in information technology management processes needed to provide UFMS with a solid foundation for development and operation, including investment management, enterprise architecture, and information security. GAO also identified human capital issues that significantly increase the risk that UFMS will not fully meet one or more of its cost, schedule, and performance objectives, including staffing and strategic workforce planning. HHS stated that it had an aggressive implementation schedule, but disagreed that a lack of disciplined processes is placing the UFMS program at risk. GAO firmly believes if HHS continues to follow an approach that is schedule-driven and shortcuts key disciplined processes, it is unnecessarily increasing its risk. GAO stands by its position that adherence to disciplined processes is crucial, particularly with a project of this magnitude and importance. HHS indicated that it plans to delay deployment of significant functionality associated with its UFMS project for at least 6 months. This decision gives HHS a good opportunity to effectively implement disciplined processes to enhance the project's opportunity for success.

section_title: Letter
paragraphs: We are pleased to be here today to discuss the efforts by the Department of  Health and Human Services (HHS) to develop and implement its Unified  Financial Management System (UFMS). We would like to thank the  Subcommittee for having this hearing.  Hearings such as this one today  foster meaningful financial management reform.  Our work focused on  whether the UFMS project was being managed in a way that best ensures  long-term success of this important project.  At the time of our review, the  complete implementation of UFMS was targeted for 2007 and the estimated  total project cost was over $700 million.  Not only must the system  ultimately replace 5 accounting systems, but it must also interface with  about 110 other systems.  By any measure, this is a major undertaking that  brings with it a degree of risk.  Risk, though, can be managed and reduced  to acceptable levels through the use of disciplined processes, which in  short, represent best practices in system development and implementation  that have proven their value in the past.
Our report, which was prepared at the request of the Subcommittee and is  being released at this hearing, discusses in detail the issues we identified  with the UFMS project and includes 34 recommendations that focus on  mitigating project risk.  Our testimony today will (1) highlight the  importance of adhering to disciplined processes for a system development  and implementation effort such as UFMS, (2) summarize our findings on  HHS’ management of the UFMS project, and (3) provide our perspective on  actions needed for HHS to mitigate the risk to this project and move  forward.
subsections: section_title: Disciplined Processes Are Key to Successful System Implementation
paragraphs: The ability to produce the information needed to efficiently and effectively  manage the day-to-day operations of the federal government and provide  accountability to taxpayers and the Congress has been a long-standing  challenge for federal agencies.  To help address this challenge, many  agencies are in the process of replacing their core financial systems as part  of their financial management system improvement efforts.  Although the  implementation of any major system is not a risk-free proposition,  organizations that follow and effectively implement disciplined processes  can reduce these risks to acceptable levels.  The use of the term acceptable  levels acknowledges the fact that any systems acquisition has risks and will  suffer the adverse consequences associated with defects.  However,  effective implementation of the disciplined processes reduces the potential  for risks to occur and helps prevent those that do occur from having any  significant adverse impact on the cost, timeliness, and performance of the  project. A disciplined software development and acquisition process can  maximize the likelihood of achieving the intended results (performance)  within established resources (costs) on schedule.
Although there is no standard set of practices that will ever guarantee  success, several organizations, such as the Software Engineering Institute  (SEI) and the Institute of Electrical and Electronic Engineers (IEEE), as  well as individual experts have identified and developed the types of  policies, procedures, and practices that have been demonstrated to reduce  development time and enhance effectiveness.  The key to having a  disciplined system development effort is to have disciplined processes in  multiple areas, including project planning and management, requirements  management, configuration management, risk management, quality  assurance, and testing.  Effective processes should be implemented in each  of these areas throughout the project life cycle because change is constant.   Effectively implementing the disciplined processes necessary to reduce  project risks to acceptable levels is hard to achieve because a project must  effectively implement several best practices, and inadequate  implementation of any one practice may significantly reduce or even  eliminate the positive benefits of the others.
Successfully acquiring and implementing a new financial management  system requires a process that starts with a clear definition of the  organization’s mission and strategic objectives and ends with a system that  meets specific information needs.  We have seen many system efforts fail  because agencies started with a general need, such as improving financial  management, but did not define in precise terms (1) the specific problems  they were trying to solve, (2) what their operational needs were, and  (3) what specific information requirements flowed from these operational  needs.  Instead, they plunged into the acquisition and implementation  process in the belief that these specifics would somehow be defined along  the way.  The typical result was that systems were delivered well past  anticipated milestones; failed to perform as expected; and, accordingly,  were overbudget because of required costly modifications.
Undisciplined projects typically show a great deal of productive work at  the beginning of the project, but the rework associated with defects begins  to consume more and more resources.  In response, processes are adopted  in the hopes of managing what later turns out to have been unproductive  work.  Generally, these processes are “too little, too late” because sufficient  foundations for building the systems were not established or not  established adequately. Experience has shown that projects for which  disciplined processes are not implemented at the beginning are forced to  implement them later when it takes more time and the processes are less  effective.
A major consumer of project resources in undisciplined efforts is rework  (also known as thrashing).  Rework occurs when the original work has  defects or is no longer needed because of changes in project direction.   Disciplined organizations focus their efforts on reducing the amount of  rework because it is expensive.  Experts have reported that fixing a defect  during the testing phase costs anywhere from 10 to 100 times the cost of  fixing it during the design or requirements phase.  Projects that are unable  to successfully address their rework will eventually only be spending their  time on rework and the associated processes rather than on productive  work.  In other words, the project will continually find itself reworking  items.
section_title: HHS Had Not Effectively Implemented Disciplined Processes, Information Technology Management Practices, and Human Capital Planning
paragraphs: We found that HHS had adopted some best practices in its development of  UFMS.  The project had support from senior officials and oversight by  independent experts, commonly called independent verification and  validation (IV&V) contractors.  We also view HHS’ decision to follow a  phased implementation to be a sound approach.
However, at the time of our review, HHS had not effectively implemented  several disciplined processes essential to reducing risks to acceptable  levels and therefore key to a project’s success, and had adopted other  practices that put the project at unnecessary risk.  HHS officials told us that  they had carefully considered the risks associated with implementing  UFMS and that they had put in place strategies to manage these risks and to  allow the project to meet its schedule within budget.  However, we found  that HHS had focused on meeting its schedule to implement the first phase  of the new system at the Centers for Disease Control and Prevention (CDC)  in October 2004, to the detriment of disciplined processes and thus had  introduced unnecessary risks that may compromise the system’s cost,  schedule, and performance.  We would now like to briefly highlight a few of  the key disciplined processes that HHS had not fully implemented at the  time of our review.  These matters are discussed in detail in our report.
Requirements management. Requirements are the specifications that  system developers and program managers use to design, develop, and  acquire a system. Requirements management deficiencies have  historically been a root cause of systems that do not meet their cost,  schedule, and performance objectives. Effective requirements  management practices are essential for ensuring the intended  functionality will be included in the system and are the foundation for  testing.  We found significant problems in HHS’ requirements  management process and that HHS had not developed requirements that  were clear and unambiguous.
Testing. Testing is the process of executing a program with the intent of  finding errors. Without adequate testing, an organization (1) is taking a  significant risk that substantial defects will not be detected until after  the system is implemented and (2) does not have reasonable assurance  that new or modified systems will function as planned.  We found that  HHS faced challenges in implementing a disciplined testing program,  because, first of all, it did not have an effective requirements  management system that produced clear, unambiguous requirements  upon which to build its testing efforts.  In addition, HHS had scheduled  its testing activities, including those for converting data from existing  systems to UFMS, late in the implementation cycle leaving little time to  correct defects identified before the scheduled deployment in October  2004.
Project management and oversight using quantitative measures. We  found that HHS did not have quantitative metrics that allowed it to fully  understand (1) its capability to manage the entire UFMS effort; (2) how  problems in its management processes would affect the UFMS cost,  schedule, and performance objectives; and (3) the corrective actions  needed to reduce the risks associated with the problems identified with  its processes.  Such quantitative measures are essential for adequate  project management oversight.  Without such information, HHS  management can only focus on the project schedule and whether  activities have occurred as planned, not on whether the substance of the  activities achieved their system development objectives.  As we note in  our report, this is not an effective practice.
Risk management. We noted that HHS routinely closed its identified  risks on the premise that they were being addressed. Risk management  is a continuous process to identify, monitor, and mitigate risks to ensure  that the risks are being properly controlled and that new risks are  identified and resolved as early as possible.  An effective risk  management process is designed to mitigate the effects of undesirable  events at the earliest possible stage to avoid costly consequences.
In addition, HHS’ effectiveness in managing the processes associated with  its data conversion and UFMS interfaces will be critical to the success of  this project.  For example, CDC’s ability to convert data from its existing  systems to the new system will be crucial to helping ensure that UFMS will  provide the kind of data needed to manage CDC’s programs and operations.   The adage “garbage in garbage out” best describes the adverse impact.   Furthermore, HHS expects that once UFMS is fully deployed, the system  will need to interface with about 110 other systems, of which about 30  system interfaces are needed for the CDC deployment.  Proper  implementation of the interfaces between UFMS and the other systems it  receives data from and sends data to is essential for providing data integrity  and ensuring that UFMS will operate as it should and provide the  information needed to help manage its programs and operations.
Compounding these UFMS-specific problems are departmentwide  weaknesses we have previously reported in information technology (IT)  investment management, enterprise architecture, and information  security.  Specifically, HHS had not established the IT management  processes needed to provide UFMS with a solid foundation for  development and operation.  Such IT weaknesses increase the risk that  UFMS will not achieve planned results within the estimated budget and  schedule.  We will now highlight the IT management weaknesses that HHS  must overcome:  Investment management. HHS had weaknesses in the processes it uses  to select and control its IT investments. Among the weaknesses we  previously identified, HHS had not (1) established procedures for the  development, documentation, and review of IT investments by its  review boards or (2) documented policies and procedures for aligning  and coordinating investment decision making among its investment  management boards. Until HHS addresses weaknesses in its selection or  control processes, IT projects like UFMS will face an increased  likelihood that the projects will not be completed on schedule and  within estimated costs.
Enterprise architecture. While HHS is making progress in developing an  enterprise architecture that incorporates UFMS as a central component,  most of the planning and development of the UFMS IT investment had  occurred without the guidance of an established enterprise architecture.   An enterprise architecture is an organizational blueprint that defines  how an entity operates today and how it intends to operate in the future  and invest in technology to transition to this future state.  Our  experience with other federal agencies has shown that projects  developed without the constraints of an established enterprise  architecture are at risk of being duplicative, not well integrated,  unnecessarily costly to maintain and interface, and ineffective in  supporting missions.
Information security. HHS had not yet fully implemented the key  elements of a comprehensive security management program and had  significant and pervasive weaknesses in its information security  controls.  The primary objectives of information security controls are to  safeguard data, protect computer application programs, prevent  unauthorized access to system software, and ensure continued  operations.  Without adequate security controls, UFMS cannot provide  reasonable assurance that the system is protected from loss due to  errors, fraud and other illegal acts, disasters, and incidents that cause  systems to be unavailable.
Finally, we believe it is essential that an agency take the necessary steps to  ensure that it has the human capital capacity to design, implement, and  operate a financial management system.  We found that staff shortages and  limited strategic workforce planning have resulted in the project not having  the resources needed to effectively design, implement, and operate UFMS.   We identified the following weaknesses:  Staffing. HHS had not filled positions in the UFMS Program  Management Office that were identified as needed.  Proper human  capital planning includes identifying the workforce size, skills mix, and  deployment needed for mission accomplishment and to create  strategies to fill the gaps.  Scarce resources could significantly  jeopardize the project’s success and have led to several key UFMS  deliverables being significantly behind schedule.
Strategic workforce planning. HHS had not yet fully developed key  workforce planning tools, such as the CDC skills gap analysis, to help  transform its workforce so that it can effectively use UFMS.  Strategic  workforce planning focuses on developing long-term strategies for  acquiring, developing, and retaining an organization’s total workforce  (including full- and part-time federal staff and contractors) to meet the  needs of the future.  Strategic workforce planning is essential for  achieving the mission and goals of the UFMS project.  By not identifying  staff with the requisite skills to operate such a system and by not  identifying gaps in needed skills and filling them, HHS has not optimized  its chances for the successful implementation and operation of UFMS.
section_title: Action Is Needed to Mitigate Risk
paragraphs: To address the range of problems we have just highlighted, our report  includes 34 recommendations that focus on mitigating the risks associated  with this project. We made 8 recommendations related to the initial  deployment of UFMS at CDC that are specifically tied to implementing  critical disciplined processes.  In addition, we recommended that until  these 8 recommendations are substantially addressed, HHS delay the initial  deployment. The remaining 25 recommendations were centered on  developing an appropriate foundation for moving forward and focused on  (1) disciplined processes, (2) IT security controls, and (3) human capital  issues.
In its September 7, 2004, response to a draft of our report, HHS disagreed  regarding management of the project and whether disciplined processes  were being followed. In its comments, HHS characterized the risk in its  approach as the result, not of a lack of disciplined processes, but of an  aggressive project schedule. From our perspective, this project  demonstrated the classic symptoms of a schedule-driven effort for which  key processes had been omitted or shortcutted, thereby unnecessarily  increasing risk. As we mentioned at the outset of our testimony, this is a  multiyear project with an estimated completion date in fiscal year 2007 and  a total estimated cost of over $700 million.  With a project of this  magnitude and importance, we stand by our position that it is crucial for  the project to adhere to disciplined processes that represent best practices.   Therefore, in order to mitigate its risk to an acceptable level, we continue  to believe it is essential for HHS to adopt and effectively implement our 34  recommendations.
In commenting on our draft report, HHS also indicated that actions had  either been taken, were under way, or were planned that address a number  of our recommendations.  In addition, HHS subsequently contacted us on  September 23, 2004, to let us know that it had decided to delay the  implementation of a significant amount of functionality associated with the  CDC deployment from October 2004 until April 2005 in order to address the  issues that had been identified with the project.  HHS also provided us with  copies of IV&V reports and other documentation that had been developed  since our review.  Delaying implementation of significant functionality at  CDC is a positive step forward given the risks associated with the project.   This delay, by itself, will not reduce the risk to an acceptable level, but will  give HHS a chance to implement the disciplined processes needed to do so.
HHS will face a number of challenges in the upcoming 6 months to address  the weaknesses in its management of the project that were discussed in our  report.  At a high level, the key challenge will be to implement an event  driven project based on effectively implemented disciplined processes,  rather than a schedule-driven project. It will be critical as well to address  the problems noted in the IV&V reports that were issued during and  subsequent to our review. If the past is prologue, taking the time to adhere  to disciplined processes will pay dividends in the long term.
Mr. Chairman, this concludes our statement. We would be pleased to  answer any questions you or other members of the Subcommittee may  have at this time.
section_title: Contacts and Acknowledgments
paragraphs: For further information about this statement, please contact Jeffrey C.  Steinhoff, Managing Director, Financial Management and Assurance, who  may be reached at (202) 512-2600 or by e-mail at steinhoffj@gao.gov, or  Keith A. Rhodes, Chief Technologist, Applied Research and Methodology  Center for Engineering and Technology, who may be reached at (202) 512- 6412 or by e-mail at rhodesk@gao.gov. Other key contributors to this  testimony include Kay Daly, Michael LaForge, Chris Martin, and Mel  Mench.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. It may be reproduced and distributed in its entirety without further  permission from GAO. However, because this work may contain copyrighted images or  other material, permission from the copyright holder may be necessary if you wish to  reproduce this material separately.