Financial Management Systems: DHS Has an Opportunity to Incorporate Best Practices in Modernization Efforts
Released: 2006-03-29T00:00:00

Letter

Lessons Learned in Recurring Failures of Federal Agency Financial Management System Implementations

In our recent report, we summarize many of the agencies’ financial management system implementation failures that have been previously reported by us and inspectors general (IG). Our work and that of the IGs over the years has shown that agencies have failed to employ accepted best practices in systems development and implementation (commonly referred to as disciplined processes) that can collectively reduce the risk associated with implementing financial management systems. In our report, we identified key causes of failures within several recurring themes, including disciplined processes and human capital management. DHS would be wise to study the lessons learned through other agencies’ costly failures and consider building a strong foundation for successful financial management system implementation, as we will discuss later in our testimony.

Disciplined Processes Have Not Been Fully Employed

From our review of over 40 prior reports, we identified weaknesses in the following areas of disciplined processes.

Requirements management. Ill-defined or incomplete requirements have been identified by many system developers and program managers as a root cause of system failure. It is critical that requirements—functions the system must be able to perform—be carefully defined and flow from the concept of operations (how the organization’s day-to-day operations are or will be carried out to meet mission needs). In our previous work, we have found agencies with a lack of a concept of operations, vague and ambiguous requirements, and requirements that are not traceable or linked to business processes.

Testing. Complete and thorough testing is essential to provide reasonable assurance that new or modified systems will provide the capabilities in the requirements. Testing is the process of executing a program with the intent of finding errors. Because requirements provide the foundation for system testing, they must be complete, clear, and well documented to design and implement an effective testing program. Absent this, an organization is taking a significant risk that substantial defects will not be detected until after the system is implemented. Industry best practices indicate that the sooner a defect is recognized and corrected, the cheaper it is to fix. In our work, we have found flawed test plans, inadequate timing of testing, and ineffective systems testing.

Data conversion. In its white paper on financial system data conversion, the Joint Financial Management Improvement Program (JFMIP) identified data conversion as one of the critical tasks necessary to successfully implement a new financial system. JFMIP also noted that if data conversion is done right, the new system has a much greater opportunity for success. On the other hand, converting data incorrectly or entering unreliable data from a legacy system has lengthy and long-term repercussions. The adage “garbage in, garbage out” best describes the adverse impact. Examples of problems we have reported on include agencies that have not properly developed and implemented good data conversion plans, have planned the data conversion too late in the project, and have not reconciled account balances.

Risk management. According to leading systems acquisition organizations, risk management is a process for identifying potential problems before they occur and adjusting the acquisition to decrease the chance of their occurrence. Risks should be identified as early as possible and a risk management process should be developed and put in place. Risks should be identified, analyzed, mitigated, and tracked to closure. Effectively managing risks is one way to minimize the chances of project cost, schedule, and performance problems from occurring. We have reported that agencies have not fully implemented effective risk management practices, including shortcomings in identifying and tracking risks.

Project management. Effective project management is the process for planning and managing all project-related activities, such as defining how components are interrelated, defining tasks, estimating and obtaining resources, and scheduling activities. Project management allows the performance, cost, and schedule of the overall program to be continually measured, compared with planned objectives, and controlled. We have reported on a number of project management problems including inadequate project management structure, schedule-driven projects, and lack of performance metrics and oversight.

Quality assurance. Quality assurance provides independent assessments of whether management process requirements are being followed and whether product standards and requirements are being satisfied. This process includes, among other things, the use of independent verification and validation (IV&V). We and others have reported on problems related to agencies’ use of IV&V including specific functions not being performed by the IV&V, the IV&V contractor not being independent, and IV&V recommendations not being implemented.

Inadequate implementation of disciplined processes can manifest itself in many ways when implementing a financial management system. While full deployment has been delayed at some agencies, specific functionality has been delayed or flawed at other agencies. The following examples illustrate some of the recurring problems related to the lack of disciplined processes in implementing financial management systems.

In May 2004, we reported significant flaws in requirements management and testing that adversely affected the initial development and implementation of the Army’s Logistics Modernization Program (LMP), in which the Army estimated that it would invest about $1 billion. These flaws also hampered efforts to correct the operational difficulties experienced at the Tobyhanna Army Depot. In June 2005, we reported that the Army had not effectively addressed its requirements management and testing problems, and data conversion weaknesses had hampered the Army’s ability to address the problems that needed to be corrected before the system could be fielded to other locations. The Army lacked reasonable assurance that (1) system problems experienced during the initial deployment and causing the delay of future deployments had been corrected and (2) LMP was capable of providing the promised system functionality. Subsequent deployments of the system have been delayed.

We reported in February 2005 that our experience with major systems acquisitions, such as the Office of Personnel Management’s (OPM) Retirement Systems Modernization (RSM) program, has shown that having sound disciplined processes in place increases the likelihood of the acquisitions meeting cost and schedule estimates as well as performance requirements. However, we found that many of the processes in these areas for RSM were not sufficiently developed, were still under development, or were planned for future development. For example, OPM lacked needed processes for developing and managing requirements, planning and managing project activities, managing risks, and providing sound information to investment decision makers. Without these processes in place, RSM was at increased risk of not being developed and delivered on time and within budget and falling short of promised capabilities.

In August 2004, the Department of Veterans Affairs (VA) IG reported that the effect of transferring inaccurate data to VA’s new core financial system at a pilot location interrupted patient care and medical center operations. This raised concerns that similar conversion problems would occur at other VA facilities if the conditions identified were not addressed and resolved nationwide prior to roll out. Some of the specific conditions the IG noted were that contracting and monitoring of the project were not adequate, and the deployment of the new system encountered multiple problems, including those related to software testing, data conversion and system interfaces, and project management. As a result of these problems, patient care was interrupted by supply outages and other problems. The inability to provide sterile equipment and needed supplies to the operating room resulted in the cancelation of 81 elective surgeries for a week in both November 2003 and February 2004. In addition, the operating room was forced to operate at two-thirds of its prior capacity. Because of the serious nature of the problems raised with the new system, VA management decided to focus on transitioning back to the previous financial management software at the pilot location and assembled a senior leadership team to examine the results of the pilot and make recommendations to the VA Secretary regarding the future of the system.

Human Capital Management Problems Impede Financial Systems Development and Deployment

We are concerned that federal agencies’ human capital problems are eroding the ability of many agencies—and threatening the ability of others—to perform their IT missions economically, efficiently, and effectively. For example, we found that in the 1990s, the initial rounds of downsizing were set in motion without considering the longer-term effects on agencies’ IT performance capacity. Additionally, a number of individual agencies drastically reduced or froze their hiring efforts for extended periods. Consequently, following a decade of downsizing and curtailed investments in human capital, federal agencies currently face skills, knowledge, and experience imbalances, especially in their IT workforces. Without corrective action, these imbalances will worsen, especially in light of the numbers of federal civilian workers becoming eligible to retire in the coming years. In this regard, we are emphasizing the need for additional focus on the following three key elements of human capital management.

Strategic workforce planning. Having staff with the appropriate skills is key to achieving financial management improvements, and managing an organization’s employees is essential to achieving results. It is important that agencies incorporate strategic workforce planning by (1) aligning an organization’s human capital program with its current and emerging mission and programmatic goals and (2) developing long-term strategies for acquiring, developing, and retaining an organization’s total workforce to meet the needs of the future. This incorporates a range of activities from identifying and defining roles and responsibilities, to identifying team members, to developing individual competencies that enhance performance. We have reported on agencies without a sufficient human capital strategy or plan, skills gap analysis, or training plans.

Human resources. Having sufficient numbers of people on board with the right mix of knowledge and skills can make the difference between success and failure. This is especially true in the IT area, where widespread shortfalls in human capital have contributed to demonstrable shortfalls in agency and program performance. We have found agency projects with significant human resource challenges, including addressing personnel shortages, filling key positions, and developing and retaining staff with the required competencies.

Change management. According to leading IT organizations, organizational change management is the process of preparing users for the business process changes that will accompany implementation of a new system. An effective organizational change management process includes project plans and training that prepare users for impacts the new system might have on their roles and responsibilities and a process to manage those changes. We have reported on various problems with agencies’ change management, including transition plans not being developed, business processes not being reengineered, and customization not being limited.

The following examples illustrate some of the recurring problems related to human capital management in implementing financial management systems.

We first reported in February 2002 that the Internal Revenue Service (IRS) had not defined or implemented an IT human capital strategy for its Business Systems Modernization (BSM) program and recommended that IRS address this weakness. In June 2003, we reported that IRS had made important progress in addressing our recommendation, but had yet to develop a comprehensive multiyear workforce plan. IRS also had not hired, developed, or retained sufficient human capital resources with the required competencies, including technical skills, in specific mission areas. In September 2003, the Treasury Inspector General for Tax Administration reported that IRS’s Modernization and IT Services organization had made significant progress in developing its human capital strategy but had not yet (1) identified and incorporated human capital asset demands for the modernized organization, (2) developed detailed hiring and retention plans, or (3) established a process for reviewing the human capital strategy development and monitoring its implementation. We most recently reported in July 2005 that IRS had taken some steps in the right direction. However, until IRS fully implements its strategy, it will not have all of the necessary IT knowledge and skills to effectively manage the BSM program or to operate modernized systems. Consequently, the risk of BSM program and project cost increases, schedule slippages, and performance problems is increased.

We reported, in September 2004, that staff shortages and limited strategic workforce planning resulted in the Department of Health and Human Services (HHS) not having the resources needed to effectively design and operate its new financial management system. HHS had taken the first steps in strategic workforce planning. For example, the Centers for Disease Control and Prevention (CDC), where the first deployment was scheduled, was the only operating division that had prepared a competency report, but a skills gap analysis and training plan for CDC had not been completed. In addition, many government and contractor positions on the implementation project were not filled as planned. While HHS and the systems integrator had taken measures to acquire additional human resources for the implementation of the new financial management system, we concluded that scarce resources could significantly jeopardize the project’s success and lead to several key deliverables being significantly behind schedule. In September 2004, HHS decided to delay its first scheduled deployment at CDC by 6 months in order to address these and other issues.

DHS Faces Serious Financial Management Challenges

DHS faces unique challenges in attempting to develop integrated financial management systems across the breadth of such a large and diverse department. DHS was established by the Homeland Security Act of 2002, as the 15th Cabinet Executive Branch Department of the United States government. DHS inherited a myriad of redundant financial management systems from 22 diverse agencies along with 180,000 employees, about 100 resource management systems, and 30 reportable conditions identified in prior component financial audits. Of the 30 reportable conditions, 18 were so severe they were considered material weaknesses. Among these weaknesses were insufficient internal controls or processes to reliably report financial information such as revenue, accounts receivable, and accounts payable; significant system security deficiencies; financial systems that required extensive manual processes to prepare financial statements; and incomplete policies and procedures necessary to complete basic financial management activities.

DHS received a disclaimer of opinion on its financial statements for fiscal year 2005, and the independent auditors also reported that DHS’s financial management systems did not substantially comply with the requirements of FFMIA. The disclaimer was primarily due to financial reporting problems at five components. The five components include Immigration and Customs Enforcement (ICE), the United States Coast Guard (Coast Guard), State and Local Government Coordination and Preparedness (SLGCP), the Transportation Security Administration (TSA), and Emergency Preparedness and Response (EPR). Further, ICE is an accounting service provider for other DHS components, and it failed to adequately maintain both its own accounting records and those of other DHS components during fiscal year 2005.

The auditors’ fiscal year 2005 report discusses 10 material weaknesses, two other reportable conditions in internal control, and instances of noncompliance with seven laws and regulations. Among the 10 material weaknesses were inadequate financial management and oversight at DHS components, primarily ICE and Coast Guard; decentralized financial reporting at the component level; significant general IT and application control weaknesses over critical financial and operational data; and the lack of accurate and timely reconciliation of fund balance with treasury accounts. The results of the auditors’ tests of fiscal year 2005 compliance with certain provisions of laws, regulations, contracts, and grant agreements disclosed instances of noncompliance. The DHS auditors reported instances of noncompliance with 31 U.S.C. § 3512(c),(d), commonly known as the Federal Managers’ Financial Integrity Act of 1982 (FMFIA); the Federal Financial Management Improvement Act of 1996 (FFMIA), Pub. L. No. 104-208, div. A, § 101(f), title VIII, 110 Stat. 3009, 3009-389 (Sept. 30, 1996); the Federal Information Security Management Act of 2002 (FISMA), Pub. L. No. 107-347, title III, 116 Stat. 2899, 2946 (Dec. 17, 2002); the Single Audit Act, as amended (codified at 31 U.S.C. §§ 7501-7507), and other laws and regulations related to OMB Circular No. A-50, Audit Follow-up, as revised (Sept. 29, 1982); the Improper Payments Information Act of 2002, Pub. L. No. 107-300, 116 Stat. 2350 (Nov. 26, 2002); the Department of Homeland Security Financial Accountability Act of 2004, Pub. L. No. 108-330, 118 Stat. 1275 (Oct. 16, 2004); and the Government Performance and Results Act of 1993 (GPRA), Pub. L. No. 103-62, 107 Stat. 285 (Aug. 3, 1993).

Although DHS inherited many of the reportable conditions and noncompliance issues discussed above, the department’s top management, including the CFO, is ultimately responsible for ensuring that progress is made in the area of financial management.

In August 2003, DHS began the “electronically Managing enterprise resources for government effectiveness and efficiency” (eMerge) program at an estimated cost of $229 million. The eMerge that the acquisition of eMerge was in the early stages and continued focus and follow through, among other things, would be necessary for it to be successful.

According to DHS officials, because the project was not meeting its performance goals and timeline, DHS officials began considering whether to continue the project and in Spring 2005 started looking at another strategy. DHS officials told us they decided to change the strategy for its eMergeacquisition and development activities on eMerge had stopped and the blanket purchase agreement with the systems integrator expired. DHS officials added that the eMerge using a shared services approach, which allows its components to choose among three DHS providers of financial management services and the Department of the Treasury’s Bureau of the Public Debt, which was identified by OMB as a governmentwide financial management center of excellence. DHS officials told us that although a departmentwide concept of operations and migration plan were still under development, they expected progress to be made in the next 5 years. As we will discuss later, a departmentwide concept of operations document would help DHS and others understand such items as how DHS will migrate the various entities to these shared service providers and how it will obtain the departmental information necessary to manage the agency from these disparate operations. DHS officials acknowledged that they needed to first address the material weaknesses at the proposed shared service providers before component agencies migrate to them.

The Building Blocks of Successful Financial Management System Implementations

The key for federal agencies, including DHS, to avoid the long-standing problems that have plagued financial management system improvement efforts is to address the foremost causes of those problems and adopt solutions that reduce the risks associated with these efforts to acceptable levels. Although it appears that DHS will adopt a shared services approach to meet its needs for integrated financial management systems, implementing this approach will be complex and challenging, making the adoption of best practices even more important for this undertaking. Based on industry best practices, we identified four key concepts that will be critical to DHS’s ability to successfully complete its planned migration to shared service providers. Careful consideration of these four concepts, each one building upon the next, will be integral to the success of DHS’s strategy. The four concepts are (1) developing a concept of operations, (2) defining standard business processes, (3) developing a migration strategy for DHS components, and (4) defining and effectively implementing disciplined processes necessary to properly manage the specific projects. We will now highlight the key issues to be considered for each of the four areas.

Concept of Operations Provides Foundation

As we discussed previously, a concept of operations defines how an organization’s day-to-day operations are (or will be) carried out to meet mission needs. The concept of operations includes high-level descriptions of information systems, their interrelationships, and information flows. It also describes the operations that must be performed, who must perform them, and where and how the operations will be carried out. Further, it provides the foundation on which requirements definitions and the rest of the systems planning process are built. Normally, a concept of operations document is one of the first documents to be produced during a disciplined development effort and flows from both the vision statement and the enterprise architecture. According to the Institute of Electrical and Electronic Engineers (IEEE) standards, a concept of operations is a user- oriented document that describes the characteristics of a proposed system from the users’ viewpoint. The key elements that should be included in a concept of operations are major system components, interfaces to external systems, and performance characteristics such as speed and volume.

Another key element of a concept of operations is a transition strategy that is useful for developing an understanding of how and when changes will occur. Not only is this needed from an investment management point of view, it is a key element in the human capital problems discussed previously that revolved around change management strategies. Describing how to implement DHS’s approach for using shared service providers for its financial management systems, as well as the process that will be used to deactivate legacy systems that will be replaced or interfaced with a new financial management system, are key aspects that need to be addressed in a transition strategy.

Key Issues for DHS to Consider What is considered a financial management system? Are all the components using a standard definition?

Who will be responsible for developing a DHS-wide concept of operations, and what process will be used to ensure that the resulting document reflects the departmentwide solution rather than individual component agency stove-piped efforts?

How will DHS’s concept of operations be linked to its enterprise architecture?

How can DHS obtain reliable information on the costs of its financial management systems investments?

Standard Business Processes Promote Consistency

Business process models provide a way of expressing the procedures, activities, and behaviors needed to accomplish an organization’s mission and are helpful tools to document and understand complex systems. Business processes are the various steps that must be followed to perform a certain activity. For example, the procurement process would start when the agency defines its needs, and issues a solicitation for goods or services, and would continue through contract award, receipt of goods and services, and would end when the vendor properly receives payment. The identification of preferred business processes would be critical for standardization of applications and training and portability of staff.

To maximize the success of a new system acquisition, organizations need to consider the redesign of current business processes. As we noted in our Executive Guide: Creating Value Through World-class Financial Management, leading finance organizations have found that productivity gains typically result from more efficient processes, not from simply automating old processes. Moreover, the Clinger-Cohen Act of 1996 requires agencies to analyze the missions of the agency and, based on the analysis, revise mission-related and administrative processes, as appropriate, before making significant investments in IT used to support those missions. Another benefit of what is often called business process modeling is that it generates better system requirements, since the business process models drive the creation of information systems that fit in the organization and will be used by end users. Other benefits include providing a foundation for agency efforts to describe the business processes needed for unique missions, or developing subprocesses to support those at the departmentwide level.

Key Issues for DHS to Consider Who will be responsible for developing DHS-wide standard business processes that meet the needs of its component agencies?

How will the component agencies be encouraged to adopt new processes, rather than selecting other methods that result in simply automating old ways of doing business?

How will the standard business processes be implemented by the shared service providers to provide consistency across DHS?

What process will be used to determine and validate the processes needed for DHS agencies that have unique needs?

Strategy for Implementing the Financial Management Shared Services Approach Will Be Key

Although DHS has a goal of migrating agencies to a limited number of shared service providers, it has not yet articulated a clear and measurable strategy for achieving this goal. In the context of migrating to shared service providers, critical activities include (1) developing specific criteria for requiring component agencies to migrate to one of the providers rather than attempting to develop and implement their own stove-piped business systems; (2) providing the necessary information for a component agency to make a selection of a shared service provider for financial management; (3) defining and instilling new values, norms, and behaviors within component agencies that support new ways of doing work and overcoming resistance to change; (4) building consensus among customers and stakeholders on specific changes designed to better meet their needs; and (5) planning, testing, and implementing all aspects of the transition from one organizational structure and business process to another.

Finally, sustained leadership will be key to a successful strategy for moving DHS components towards consolidated financial management systems. In our Executive Guide: Creating Value Through World-class Financial Management, we found that leading organizations made financial management improvement an entitywide priority by, among other things, providing clear, strong executive leadership. We also reported that making financial management a priority throughout the federal government involves changing the organizational culture of federal agencies. Although the views about how an organization can change its culture can vary considerably, leadership (executive support) is often viewed as the most important factor in successfully making cultural changes. Top management must be totally committed in both words and actions to changing the culture, and this commitment must be sustained and demonstrated to staff. As pressure mounts to do more with less, to increase accountability, and to reduce fraud, waste, abuse, and mismanagement, and efforts to reduce federal spending intensify, sustained and committed leadership will be a key factor in the successful implementation of DHS’s financial management systems.

Key Issues for DHS to Consider What guidance will be provided to assist DHS component agencies in adopting a change management strategy that reduces the risks of moving to a shared service provider?

What processes will be put in place to ensure that individual component agency financial management system investment decisions focus on the benefits of standard processes and shared service providers?

What process will be used to facilitate the decision-making process used by component agencies to select a provider?

How will component agencies incorporate strategic workforce planning in the implementation of the shared service provider approach?

Disciplined Processes Will Help Ensure Successful Implementation

Once the concept of operations and standard business processes have been defined and a migration strategy is in place, the use of disciplined processes, as discussed previously, will be a critical factor in helping to ensure that the implementation is successful. The key to avoiding long- standing implementation problems is to provide specific guidance to component agencies for financial management system implementations, incorporating the best practices identified by the Software Engineering Institute, the IEEE, the Project Management Institute, and other experts that have been proven to reduce risk in implementing systems. Such guidance should include the various disciplined processes such as requirements management, testing, data conversion and system interfaces, risk and project management, and related activities, which have been problematic in the financial systems implementation projects we and others have reviewed.

Disciplined processes have been shown to reduce the risks associated with software development and acquisition efforts to acceptable levels and are fundamental to successful system implementations. The principles of disciplined IT systems development and acquisition apply to shared services implementation, such as that contemplated by DHS. A disciplined software implementation process can maximize the likelihood of achieving the intended results (performance) within established resources (costs) on schedule. For example, disciplined processes should be in place to address the areas of data conversion and interfaces, two of the many critical elements necessary to successfully implement a new system—the lack of which have contributed to the failure of previous agency efforts. Further details on disciplined processes can be found in appendix III of our recently issued report.

Key Issues for DHS to Consider How can existing industry standards and best practices be incorporated into DHS- wide guidance related to financial management system implementation efforts, including migrating to shared service providers?

What actions will be taken to reduce the risks and costs associated with data conversion and interface efforts?

What oversight process will be used to ensure that modernization efforts effectively implement the prescribed policies and procedures?

Concluding Observations

In closing, the best practices we identified are interrelated and interdependent, collectively providing an agency with a better outcome for its system deployment—including cost savings, improved service and product quality, and ultimately, a better return on investment. The predictable result of DHS and other agencies not effectively addressing these best practices is projects that do not meet cost, schedule, and performance objectives. There will never be a 100 percent guarantee that a new system will be fully successful from the outset. However, risk can be managed and reduced to acceptable levels through the use of disciplined processes, which in short represent best practices that have proven their value in the past. We view the application of disciplined processes to be essential for DHS’s systems modernization efforts. Based on industry best practices, the following four concepts would help ensure a sound foundation for developing and implementing a DHS-wide solution for the complex financial management problems it currently faces: (1) developing a concept of operations that expresses DHS’s view of financial management and how that vision will be realized, (2) defining standard business processes, (3) developing an implementation strategy, and (4) defining and effectively implementing applicable disciplined processes. If properly implemented, the best practices discussed here today and in our recently issued report will help reduce the risk associated with a project of this magnitude and importance to an acceptable level. With DHS at an important crossroads in the implementation of the eMergefoundation on which to base its efforts and avoid the problems that have plagued so many other federal agencies faced with the same challenge.

Mr. Chairmen, this concludes our prepared statement. We would be happy to respond to any questions you or other Members of the Subcommittees may have at this time.

Contacts and Acknowledgments

For information about this testimony, please contact McCoy Williams, Director, Financial Management and Assurance, at (202) 512-9095 or at williamsm1@gao.gov, or Keith A. Rhodes, Chief Technologist, Applied Research and Methods, who may be reached at (202) 512-6412 or at rhodesk@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this testimony. Individuals who made key contributions to this testimony include Kay Daly, Assistant Director; Chris Martin, Senior-Level Technologist; Francine DelVecchio; Mike LaForge; and Chanetta Reed. Numerous other individuals made contributions to the GAO reports cited in this testimony.

This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.