Financial Management Systems: Additional Efforts Needed to Address Key Causes of Modernization Failures
Released: 2006-03-15T00:00:00

Letter

Background

OMB plays a central role in setting federal financial management policy and guidance. The CFO Act of 1990 established OMB’s Office of Federal Financial Management (OFFM), which has responsibility to provide overall direction and leadership to the executive branch on financial management matters by establishing financial management policies and requirements, and by monitoring the establishment and operation of federal government financial management systems. Among the key issues OFFM addresses in addition to financial management systems, are agency and governmentwide financial reporting, asset management, grants management, improper payments, performance measurement, single audits, and travel and purchase cards. Within OFFM, the Federal Financial Systems Branch is responsible for orchestrating all of the elements of the financial systems governmentwide into a coherent, coordinated architecture. These elements include agency financial management systems and JFMIP standards; interfaces between agency financial systems and other systems that support business processes (e.g., human resources systems, procurement systems, databases supporting performance management); common financial management services, including e-Travel, e-Learning, Contractor Central Registry, Intragovernmental Payment and Collection System, and Electronic Certification System; and governmentwide accounting and other data consolidation systems.

Another office in OMB, the Office of Electronic Government and Information Technology, has responsibility for providing overall leadership and direction to the executive branch on electronic government. In particular, this OMB office oversees implementation of IT throughout the federal government, including monitoring and consulting on agency technology efforts; advising the OMB Director on the performance of IT investments, as well as identifying opportunities for joint agency and governmentwide IT projects; and overseeing the development of enterprise architectures within and across agencies, which is being fulfilled through the Federal Enterprise Architecture. This office also shares statutory IT management responsibilities with the Office of Information and Regulatory Affairs, which OMB was required to establish under the Paperwork Reduction Act of 1995.

Finally, OMB is the preparer of the President’s budget and provides instructions to executive branch agencies to submit budget-related information in accordance with the requirements of OMB Circular No. A-11, Preparation, Submission and Execution of the Budget. OMB is responsible for reviewing and evaluating IT spending across the federal government and uses the IT spending information submitted by the agencies during the budget formulation process to review requests for agency financial management systems and other IT spending. Major agency IT investments are reported to OMB individually. OMB Circular No. A-11 defines a major IT investment as a system or project that requires special management attention because of its importance to an agency’s mission, or has significant program or policy implications, among other criteria. Financial management systems costing more than $500,000 annually are considered major IT investments. OMB Circular No. A-11 also requires agencies to use Exhibit 300, Capital Asset Plan and Business Case, to describe the business case for the investment, which serves as the primary means of justifying IT investment proposals as well as managing IT investments once they are funded.

Elements of IT Management

Best practices are tried and proven methods, processes, techniques, and activities that organizations define and use to minimize risks and maximize chances for success. As we have previously reported, using best practices related to IT acquisitions can result in better outcomes—including cost savings, improved service and product quality, and ultimately, a better return on investment. We and others, such as the Software Engineering Institute (SEI), have identified and promoted the use of a number of best practices associated with acquiring IT systems. For the purposes of this report, we have identified various elements of IT management and categorized them as disciplined processes, human capital and other IT management practices that are critical elements for minimizing the risks related to financial management system implementations. These areas are interrelated and interdependent, collectively providing an agency with a comprehensive understanding both of current business approaches and of efforts (under way or planned) to change these approaches and a means to implement those changes. Understanding the relationships among these areas can help an agency determine how it is applying its resources, analyze how to redirect these resources in the face of change, implement such redirections, and measure success. With this decision-making capability, the agency is better positioned to deploy financial management systems and direct appropriate responses to unexpected changes in its environment. The following sections provide additional background information on the key elements of IT management discussed in this report, including disciplined processes, human capital and other IT management practices.

Disciplined Processes

Disciplined processes are fundamental to successful systems implementation efforts and have been shown to reduce the risks associated with software development and acquisition to acceptable levels. A disciplined software development and acquisition process can maximize the likelihood of achieving the intended results (performance) within established resources (costs) on schedule. Although there is no standard set of practices that will ever guarantee success, several organizations, such as the SEI and the Institute of Electrical and Electronic Engineers (IEEE), as well as individual experts, have identified and developed the types of policies, procedures, and practices that have been demonstrated to reduce development time and enhance effectiveness. The key to having a disciplined system development effort is to have disciplined processes in multiple areas, including requirements management, testing, data conversion and system interfaces, configuration, risk and project management, and quality assurance. Effective processes should be implemented in each of these areas throughout the project life cycle because change is constant. Effectively implementing the disciplined processes necessary to reduce project risks to acceptable levels is difficult to achieve because a project must effectively implement several best practices, and inadequate implementation of any one may significantly reduce or even negate the positive benefits of the others.

Figure 1 shows how organizations that do not effectively implement the disciplined processes lose the productive benefits of their efforts as a project continues through its development and implementation cycle. Although undisciplined projects show a great deal of what appears to be productive work at the beginning of the project, the rework associated with defects begins to consume more and more resources. In response, processes are adopted in the hopes of managing what later turns out, in reality, to have been unproductive work. Generally, these processes are “too little, too late” because sufficient foundations for building the systems were not done or not done adequately. Experience in both the private sector and the government has shown that projects for which disciplined processes are not implemented at the beginning then must be implemented later, when it takes more time and they are less effective.

As shown in figure 1, a major consumer of project resources in undisciplined efforts is rework (also known as thrashing). Rework occurs when the original work has defects or is no longer needed because of changes in project direction. Disciplined organizations focus their efforts on reducing the amount of rework because it is expensive. Fixing a requirements defect after the system is released costs anywhere from 10 to 100 times the cost of fixing it when the requirements are defined. Projects that do not successfully address rework will eventually spend even more effort on rework and the associated processes rather than on productive work. In other words, the project will continually require reworking items.

Human Capital Management

People—human capital—are a critical element to transforming organizations to meet the challenges of the 21st century. Recognizing this, we first added strategic human capital management as a governmentwide high-risk issue in January 2001, and although progress has been made, continued to include it on the latest high-risk list issued in January 2005. Strategic human capital management for financial management projects includes organizational planning, staff acquisition, and team development. Human capital planning is necessary for all stages of the system implementation. It is important that agencies incorporate strategic workforce planning by (1) aligning an organization’s human capital program with its current and emerging mission and programmatic goals and (2) developing long-term strategies for acquiring, developing, and retaining an organization’s total workforce to meet the needs of the future. This incorporates a range of activities from identifying and defining roles and responsibilities, to identifying team members, to developing individual competencies that enhance performance. It is essential that an agency take the necessary steps to ensure that it has the human resources to design, implement, and operate a financial management system. In addition, organizational change management, which is the process of preparing users for the business process changes that usually accompany implementation of a new system, is another important human capital element.

Strategic workforce planning is essential for achieving the mission and goals of financial management system projects. As we have reported, there are five key principles that strategic workforce planning should address: Involve top management, employees, and other stakeholders in developing, communicating, and implementing the strategic workforce plan.

Determine the critical skills and competencies that will be needed to achieve current and future programmatic results.

Develop strategies that are tailored to address gaps in the number, deployment, and alignment of human capital approaches for enabling and sustaining the contributions of all critical skills and competencies. Build the capability needed to address administrative, educational, and other requirements important to support workforce planning strategies.

Monitor and evaluate the agency’s progress toward its human capital goals and the contribution that human capital results have made toward achieving programmatic results.

Having adequate and sufficient human resources with the requisite training and experience to successfully implement a financial management system is another critical success factor. According to OMB, qualified federal IT project managers are our first line of defense against the cost overruns, schedule slippage, and poor performance that threaten agencies’ ability to deliver efficient and effective services to citizens. In July 2004, OMB issued a memorandum to help agencies comply with fiscal year 2005 budget guidance that instructed agencies to ensure “by September 30, 2004, all major projects are managed by project managers qualified in accordance with CIO Council guidance.” The CIO Council’s Federal IT Project Manager Guidance Matrix and Federal IT Project Management Validation define levels of complexity for IT projects/systems, identify appropriate competencies and experience, suggest education and training sources, and serve as a tool for validating IT project manager credentials. IT project managers are expected to achieve and demonstrate baseline skills in applicable competency areas listed in the Office of Personnel Management (OPM) Interpretive Guidance for Project Manager Positions. The OMB memorandum also required agencies to submit a plan to meet the guidance on project manager qualifications and document the approach, milestones, and schedule. The plans should also follow OPM’s Workforce Planning Model and Human Capital Assessment and Accountability Framework.

Changing an organization’s business processes is not an easy task. Managing culture and process change in large, diverse, organizationally and geographically decentralized agencies is a much greater challenge. Frequently, the greatest difficulties lie not in managing the technical or operational aspects of change, but in managing the human dimensions of change. Some experts caution that unless planning and accountability for change management are given a separate focus, the efforts will not be managed well. Management roles in implementing a new system include establishing business goals, realistic expectations, accountability, and leading cultural change necessary to accept the capabilities of a new system. During the implementation phase especially, agency executives must be in the forefront in dealing with the social, psychological, and political resistance to changing the way work is done. Executives must also recognize that their own roles and responsibilities may need to undergo change as well.

Other IT Management Practices

Weaknesses in other IT management processes also increase the risks associated with financial management system implementation efforts. Developing an enterprise architecture, establishing IT investment management policies, and addressing information security weaknesses are critical to ensuring successful system implementation.

OMB Circular No. A-130, which establishes executive branch policies pursuant to the Paperwork Reduction Act of 1995 and the Clinger-Cohen Act of 1996 among other laws, requires agencies to use architectures. A well-defined enterprise architecture provides a clear and comprehensive picture of the structure of any enterprise by providing models that describe in business and technology terms how the entity operates today and predicts how it will operate in the future. It also includes a plan for transitioning to this future state. Enterprise architectures are integral to managing large-scale programs. Managed properly, an enterprise architecture can clarify and help optimize the interdependencies and relationships among an organization’s business operations and the underlying IT infrastructure and applications that support these operations. Employed in concert with other important management controls, architectures can greatly increase the chances that organizations’ operational and IT environments will be configured to optimize mission performance. To aid agencies in assessing and improving enterprise architecture management, we issued guidance establishing an enterprise architecture management framework. The underpinning of this framework is a five-stage maturity model outlining steps toward achieving a stable and mature process for managing the development, maintenance, and implementation of an enterprise architecture.

IT investment management provides for the continuous identification, selection, control, life-cycle management, and evaluation of IT investments. The Clinger-Cohen Act lays out specific aspects of the process that agency heads are to implement to maximize the value of the agency’s IT investments. In addition, OMB and GAO have issued guidance for agencies to use in implementing the Clinger-Cohen Act requirements for IT investment management. For example, we issued guidance establishing an IT investment management framework. This framework is also a maturity model composed of five progressive stages of maturity that an agency can achieve in its IT investment management capabilities. These stages range from creating investment awareness to developing a complete investment portfolio to leveraging IT for strategic outcomes. The framework can be used both to assess the maturity of an agency’s investment management processes and as a tool for organizational improvement.

The Federal Information Security Management Act of 2002 provides the overall framework for ensuring the effectiveness of information security controls that support federal operations and assets and requires agencies and OMB to report annually to the Congress on their information security programs. OMB Circular No. A-130 also requires agencies to protect information commensurate with the risk and magnitude of the harm that would result from the loss, misuse, or unauthorized access to or modification of such information. The reliability of operating environments, computerized data, and the systems that process, maintain, and report these data is a major concern to federal entities that have distributed networks that enable multiple computer processing units to communicate with each other. Such distributed networks increase the risk of unauthorized access to computer resources and possible data alteration. Effective departmentwide information security controls will help reduce the risk of loss due to errors, fraud, and other illegal acts, disasters, or incidents that cause systems to be unavailable. Inadequate security and controls can adversely affect the reliability of the operating environments in which financial management systems and their applications operate.

Agencies’ Failure to Follow Best Practices in Three Key Areas Has Hampered Successful Implementation of Financial Management Systems

We reviewed numerous prior GAO and IG reports and identified several problems related to agencies’ implementation of financial management systems in three recurring and overarching themes: disciplined processes, human capital and other IT management practices. Simply put, the agencies were not following best practices in these three critical areas. The predictable result of not effectively addressing these three areas has been numerous agency systems throughout the federal government that did not meet their cost, schedule, and performance objectives. We have issued governmentwide reports on other IT management practices including agencies’ enterprise architecture, IT investment management, and information security and therefore will not be addressing those issues further in this report. However, broad-based actions are needed to address the problems repeatedly experienced at the agencies as they continue to struggle to implement new financial management systems. Many of the systems we reviewed had at least one problem in each of the three critical areas. While there was some overlap in these three areas, we selected examples that best illustrate the specific problems in each area.

Disciplined Processes Have Not Been Fully Used

From our review of over 40 prior reports, we identified a number of key problem areas in disciplined processes related to requirements management, testing, data conversion and system interfaces, risk management, and project management activities. Inadequate implementation of disciplined processes can manifest itself in many ways when implementing a financial management system and the failure to properly implement disciplined processes in one area can undermine the work in all the other areas and cause significant problems. Table 2 summarizes and provides examples for some of the problems we identified from prior reports that can be expected when agencies do not effectively implement the disciplined processes necessary to manage their financial management system implementation projects.

The following provides more specific details on three of the examples of financial management system implementation problems related to the lack of disciplined processes.

In May 2004, we first reported our concerns with the requirements management and testing processes used by the Army in the implementation of the Logistics Modernization Program and the problems being encountered after it became operational in July 2003.

At the time of our initial report, the Army decided that future deployments would not go forward until they had reasonable assurance that the deployed system would operate as expected for a given deployment. However, as we reported in June 2005, the Army had not effectively addressed its requirements management and testing problems and data conversion weaknesses had hampered the Army’s ability to address the problems that need to be corrected before the system can be fielded to other locations. For example, the system cannot properly recognize revenue nor bill customers. Data conversion problems resulted in general ledger account balances that were not properly converted to the new system in July 2003, and these differences remained unresolved almost 18 months later. These weaknesses adversely affected the Army’s ability to set the prices for the work performed at the Tobyhanna Army Depot. In addition, data conversion problems resulted in excess items being ordered and shipped to Tobyhanna. As noted in our June 2005 report, three truckloads of locking washers (for bolts) were mistakenly ordered and received, and subsequently returned, because of data conversion problems. As a result of the problems, the Army has implemented error-prone, time-consuming manual workarounds as a means to minimize disruption to critical operations; however, the depot’s financial management operations continue to be adversely affected by systems problems.

NASA has struggled to implement a modern integrated financial management system. After two failed efforts over 12 years and about $180 million, NASA embarked on a third effort that is expected to cost about $983 million. We have previously identified problems and made recommendations to NASA related to requirements, testing, and project management as well as problems with human capital and other IT management issues related to this effort. For example, NASA had not implemented quantitative metrics to help gauge the effectiveness of its requirements management process. Such metrics would be particularly important for NASA to address the root causes of system defects and be reasonably assured that its processes would result in a system that meets its business needs. However, in our September 2005 report, we found that overall progress implementing our recommendations had been slow. From our perspective, of the 45 recommendations we made in prior reports, NASA had taken sufficient action to close 3 recommendations and had partially implemented 13, but 29 recommendations remained open. Furthermore, in November 2004, NASA’s independent auditor reported that NASA’s new financial system, which was implemented in June 2003, could not produce auditable financial statements for fiscal year 2004 and did not comply with the requirements of FFMIA. Key areas of concern included the core financial module’s inability to (1) produce transaction-level detail in support of financial statement account balances, (2) identify adjustments or correcting entries, and (3) correctly and consistently post transactions to the right accounts.

In August 2004, the VA IG reported that the effect of transferring inaccurate data to its new core financial system at a pilot location interrupted patient care and medical center operations. This raised concerns that similar conversion problems would occur at other VA facilities if the conditions identified were not addressed and resolved nationwide prior to roll out. Some of the specific conditions the IG noted were that contracting and monitoring of the project were not adequate, and the deployment of the new system encountered multiple problems including those related to software testing, data conversion and system interfaces, and project management. When the new financial system was deployed at the pilot location in October 2003, it did not function as project managers had expected because of inaccurate or incomplete vendor and inventory system data. As a result of these problems, patient care was interrupted by supply outages and other problems. The inability to provide sterile equipment and needed supplies to the operating room resulted in the cancelation of 81 elective surgeries for a week in both November 2003 and February 2004. In addition, the operating room was forced to operate at two-thirds of its prior capacity. Because of the serious nature of the problems raised with the new system, VA management decided to focus on transitioning back to the previous financial management software at the pilot location and assemble a senior leadership team to examine the results of the pilot and make recommendations to the VA Secretary regarding the future of the system.

Human Capital Management Problems Impede Financial Systems Development and Deployment

Effective human capital management is critical to the success of systems implementations. As we previously reported in our Executive Guide: Creating Value Through World-class Financial Management, having staff with the appropriate skills is key to achieving financial management improvements, and managing an organization’s employees is essential to achieving results. By not identifying staff with the requisite skills to implement such systems and by not identifying gaps in needed skills and filling them, agencies reduce their chances of successfully implementing and operating new financial management systems. For example, in our prior report on building the IT workforce, we found that in the 1990s the initial rounds of downsizing were set in motion without considering the longer-term effects on agencies’ IT performance capacity. Additionally, a number of individual agencies drastically reduced or froze their hiring efforts for extended periods. Consequently, following a decade of downsizing and curtailed investments in human capital, federal agencies face skills, knowledge, and experience imbalances, especially in their IT workforces. Without corrective action, this situation will worsen, especially in light of the numbers of federal civilian workers becoming eligible to retire in the coming years. In this regard, we are emphasizing the need for additional focus on key problem areas we identified from prior reports including strategic workforce planning, human resources, and change management. Examples for some of the human capital management problems we identified in prior reports that hamper the implementation of new financial management systems are summarized in table 3.

The following provides more specific details on two of the examples of the types of human capital management problems we found.

In May 2002, we first reported that the Customs Modernization Office did not have the people in place to perform critical system acquisition functions and did not have an effective strategy for meeting its human capital needs. Customs had decided to compress its time frame for delivering its new system from 5 to 4 years and was taking a schedule- driven approach to acquiring the system because of the system’s national importance. This exacerbated the level of project risk by introducing more overlap among incremental system releases and stretching critical resources. In our most recent report issued in March 2005, we found that although Customs had developed a staffing plan, it had not been approved and was already out of date because the modernization office subsequently implemented a reorganization that transferred government and contractor personnel to the modernization office. We also observed that changes in roles and responsibilities had the modernization office and the contractor sharing development duties of the new system. Finally, Customs developed a revised organizational change approach with new change management activities, but key actions associated with the revised approach were not planned for implementation because the funding request for fiscal year 2005 did not fully reflect the revised approach. In July 2004, Customs extended delivery of the last release from fiscal year 2007 to fiscal year 2010, adding a new release for screening and targeting, and increasing the life-cycle cost estimate by about $1 billion to $3.1 billion. The new schedule reflected less overlap between future releases. While Customs, which is now under the Department of Homeland Security, has taken important actions to help address release-by-release cost and schedule overruns that we previously identified, we concluded that it was unlikely that these actions would prevent the past pattern of overruns from recurring because the Department of Homeland Security had relaxed system quality standards, so that milestones were being passed despite material system defects, and because correcting these defects will ultimately require the program to expend resources, such as people and test environments, at the expense of later system releases (some of which are now under way).

We reported, in September 2004, that staff shortages and limited strategic workforce planning resulted in HHS not having the resources needed to effectively design and operate its new financial management system. HHS had taken the first steps in strategic workforce planning. For example, the Centers for Disease Control and Prevention (CDC), where the first deployment was scheduled, was the only operating division that had prepared a competency report, but a skills gap analysis and training plan for CDC had not been completed. In addition, many government and contractor positions on the implementation project were not filled as planned. For example, an independent verification and validation contractor reported that some key personnel filled multiple positions and their actual available time was inadequate to perform the allocated tasks. As a result, some personnel were overworked, which, according to the independent verification and validation contractor could lead to poor morale. The organization chart for the project showed that the project team was understaffed and that several integral positions were vacant or filled with part-time detailees. While HHS and the systems integrator had taken measures to acquire additional human resources for the implementation of the new financial management system, we concluded that scarce resources could significantly jeopardize the project’s success and lead to several key deliverables being significantly behind schedule. In September 2004, HHS decided to delay its first scheduled deployment at CDC by 6 months in order to address these and other issues identified with the project.

Other IT Management Practices Were Not Fully Implemented

We identified a number of key problems related to other IT management practices. Specifically, we found that in planning and developing new financial management systems, agencies had not adequately considered their existing IT management processes and framework. Through our research into IT management best practices and our evaluation of agency IT management performance, we have identified a set of essential and complementary management disciplines. These include key areas where we found problems such as enterprise architecture, investment management, and information security, among others. Using the results of this research and evaluation, we have developed various management frameworks and guides and reported on numerous IT management weaknesses at individual agencies. Table 4 summarizes and provides examples for some of the key problems we found described in prior reports on financial management system implementations related to other IT management areas not previously discussed.

The following provides more specific details on two of the examples of other problems related to IT management that have had an impact on financial management system implementation projects.

For several years, we have reported that deficiencies in DOD’s enterprise architecture and IT investment management policies are contributing factors to DOD’s stovepiped, duplicative, and nonintegrated systems environment. In May 2004, we reported that we had not seen any significant change in the content of DOD’s architecture or in DOD’s approach to investing billions of dollars annually in existing and new systems. Few actions had been taken to address prior recommendations, which were aimed at improving DOD’s plans for developing the next version of the architecture and implementing the institutional means for selecting and controlling both planned and ongoing business systems investments. In April 2005, we reported that DOD still did not have an effective departmentwide management structure for controlling business investments despite DOD requesting over $13 billion in fiscal year 2005 to operate, maintain, and modernize its existing duplicative business systems. In addition, because DOD lacked a well-defined business enterprise architecture and transition plan, billions of dollars continued to be at risk of being spent on systems that would be duplicative, not interoperable, cost more to maintain than necessary, and would not optimize mission performance and accountability. In July 2005, we reported that despite spending almost 4 years and about $318 million, DOD still did not have an effective architecture program, and as a result its modernization program remained a high risk.

We reported, in February 2005, that OPM had implemented selected processes in the areas of systems acquisition, investment management, and information security; however, many processes were not sufficiently developed, were still under development, or were planned for future development. Although OPM had an executive steering committee chaired by the deputy associate director of the Center for Retirement and Insurance Services that acted as an IT investment management board for the new retirement system, program officials were not aware of formal policies or procedures guiding the board’s oversight responsibilities or activities. Agency officials stated that they would define such a governance structure for the retirement system project during the contract award process. In addition, the agency had not yet developed security plans for the licensed technology and data conversion portions of the new system. Agency officials said they did not have detailed security requirements for the licensed technology portion of the new system, although the request for proposals identified the need for high-level security requirements. They planned to develop detailed security requirements after awarding the licensed technology contract to a vendor. Without fully developed security plans and security requirements for the licensed technology and data conversion portions of the new system, OPM increased the risk that both it and its vendors would not meet information security needs for these portions of the program expected to be implemented in fiscal year 2008.

Federal Initiatives Under Way to Improve System Implementations

As the federal organization with key responsibility for federal financial management systems, OMB has undertaken a number of initiatives related to acquiring and implementing financial management system capabilities. Some of these initiatives are in collaboration with the CIO and CFO Councils and are broad-based attempts to reform financial management operations across the federal government. While reforming federal financial management is an undertaking of tremendous complexity, it presents great opportunities for improvements in financial management system implementations and related business operations.

Notably, OMB has developed and continues to evolve governmentwide Federal Enterprise Architecture products and has required a mapping of agency architectures to this federal architecture as part of the budget review process. Another key OMB initiative is referred to as the lines of business and promotes streamlining common systems to enhance the government’s performance and services, such as establishing centers of excellence to consolidate financial management activities for major agencies through cross-servicing arrangements. The advantages of this approach are many, including the implementation of standard business processes and focusing system acquisition, development, and maintenance activities at select agencies or entities with experience that have the necessary resources to reduce the risks associated with such efforts. Furthermore, certain activities and responsibilities performed by JFMIP prior to its termination have been reassigned to OMB’s OFFM, the Financial Systems Integration Office, and a CFO Council Committee providing guidance and oversight. However, as discussed in the next section, we identified four key concepts that are not yet fully developed and integrated in OMB’s initiatives and related processes. Table 5 highlights some of the foremost initiatives under way at OMB and their potential strengths.

Federal Enterprise Architecture

In 2002, OMB established the Federal Enterprise Architecture Program Management Office to develop a Federal Enterprise Architecture according to a collection of five reference models. These models are intended to facilitate governmentwide improvement through cross-agency analysis and the identification of duplicative investments, gaps, and opportunities for collaboration, interoperability, and integration within and across government agencies. According to OMB, the result will be a more citizen-centered, customer-focused government that maximizes technology investments to better achieve mission outcomes. The Federal Enterprise Architecture reference models are summarized in table 6.

In May 2005, the five reference models were combined into the Consolidated Reference Model document to compose a framework for describing important elements of the Federal Enterprise Architecture in a common and consistent way. OMB views the Federal Enterprise Architecture not as a static model, but as a program, built into the annual budget process to repeatedly and consistently improve all aspects of government service delivery. OMB officials acknowledged that they are still mapping out the Federal Enterprise Architecture and making it more robust and recognized that some lines of business have fleshed out their areas in more detail than others. In prior testimony on the Federal Enterprise Architecture, we recognized that OMB and the CIO Council have made important progress, but that hard work lies ahead to ensure that the Federal Enterprise Architecture is appropriately described, matured, and used. The development of the Federal Enterprise Architecture has continued to evolve and OMB has been promoting the adoption of the Federal Enterprise Architecture. For example, for the fiscal year 2007 budget submission, agencies will be required to use predetermined codes to link their major IT investments on Exhibit 53 to the Federal Enterprise Architecture. For fiscal year 2005, agencies were required to use the Federal Enterprise Architecture Performance Reference Model to identify performance measurements for each new major IT investment. As we have previously testified, questions remain regarding the nature of the Federal Enterprise Architecture, the relationship of agency enterprise architectures to the Federal Enterprise Architecture, and the security aspects of the Federal Enterprise Architecture. Therefore, we will not be addressing these issues further from a governmentwide perspective in this report.

Lines of Business

Building upon the efforts of the Federal Enterprise Architecture program, OMB and designated agency task forces have launched the lines of business initiative. This initiative seeks to develop business-driven common solutions for six lines of business that span across the federal government. OMB and the lines of business task forces plan to use enterprise architecture-based principles and best practices to identify common solutions for business processes or technology-based shared services to be made available to government agencies. Driven from a business perspective rather than a technology focus, the solutions are expected to address distinct business improvements to enhance the government’s performance and services for citizens. The end results of the lines of business efforts are expected to save taxpayer dollars, reduce administrative burden, and significantly improve service delivery.

We have long supported and called for such initiatives to standardize and streamline common systems, which can reduce costs and, if done correctly, can also improve accountability. OMB officials from both OFFM and the Electronic Government office told us that they worked collaboratively to develop the financial management line of business along with an interagency task force. The interagency task force recommended the establishment of governmentwide service providers in the areas of financial management and human resources management. The financial management line of business raises a number of issues that have far- reaching implications for the government and private sector application service providers. This concept has commonly been used in the private sector where application service providers provide services such as payroll, sales force automation, and human resource applications to many corporate clients. The interagency task force analysis estimated that savings of more than $5 billion can be expected over a 10-year time frame through consolidation of financial management and human resources systems and the standardization and optimization of associated business processes and functions. To help realize these benefits, OMB evaluated agencies’ business cases submitted as part of the fiscal year 2006 budget process. On the basis of the review, the following four agencies were designated as governmentwide financial management application service providers, which OMB refers to as centers of excellence.

Department of the Interior (National Business Center) General Services Administration Department of the Treasury (Bureau of the Public Debt’s Administrative Resource Center)

Department of Transportation (Enterprise Services Center)

The National Business Center, the General Services Administration, and the Bureau of the Public Debt have significant experience providing financial management services to other federal entities. For a number of years, these entities have provided financial management services— primarily to smaller federal agencies such as the Nuclear Regulatory Commission, the Office of Government Ethics, and the Panama Canal Commission. The Department of Transportation plans to utilize its newly implemented financial management system to provide services to other agencies. OMB officials told us that, at a minimum, centers of excellence must be able to support, or must use, core financial system software that has passed the most recent qualification test of the Financial Systems Integration Office, which is the current entity that performs many of the roles and responsibilities of the former JFMIP Program Management Office as we discuss below. Centers of excellence may provide related maintenance, interfaces with feeder systems, and transaction processing.

Other services may also be offered, including hosting and other financial applications such as payroll and travel. OMB also indicated that it plans to explore using private sector application service providers to serve as centers of excellence.

OMB expects to manage the migrations of agencies to centers of excellence using the agencies’ business cases submitted as part of the annual budget process. According to OMB, agencies that submit business cases with proposals to develop new financial systems or significantly update or enhance current financial systems are prime candidates for moving to a financial management center of excellence. The general principle OMB plans to follow is that agencies should migrate to a financial management center of excellence when it is cost effective to do so and they have maximized the return on investment in the current system, which averages about 5 to 7 years. OMB officials told us that several major executive branch agencies are considering moving to a financial management center of excellence.

In August 2005, OPM was the first large agency to announce its plans to move to a designated center of excellence. At the time of our review, OPM was still in the planning phase; although it had selected the Bureau of the Public Debt as the provider, it did not yet have a project plan. OPM officials recognized that moving to a center of excellence at the beginning of a fiscal year and not converting mid-year was a best practice they planned to follow. In addition, at the time of our review, the Environmental Protection Agency was in the planning and acquisition phase of its Financial System Modernization Project. As part of its best- value determination, the Environmental Protection Agency was considering the designated centers of excellence as well as private sector providers for software, integration, and hosting and had issued a draft request for quotations. Also, OMB officials stated that they helped the National Gallery of Art in preparing its solicitation for a new system, and the agency recently selected a private sector firm as its application service provider. OMB expects that most agencies will move to a center of excellence or private sector firm within the next 7 to 8 years. In OMB Circular No. A-11, for fiscal year 2007 OMB has asked agencies to provide an overview of their current and future financial management systems framework, including migration strategies for moving to a financial management center of excellence.

JFMIP Realignment

In an effort to eliminate duplicative roles and streamline financial management improvement efforts, the four principals of JFMIP agreed to realign JFMIP’s responsibilities for financial management policy and oversight as described in a December 2004 OMB memorandum. Some of the former responsibilities of JFMIP, such as issuing systems requirements, were to be placed under the authority of OFFM and a renamed CFO Council committee—the Financial Systems Integration Committee. As a result of the realignment, JFMIP ceased to exist as a separate organization, although the principals will continue to meet at their discretion consistent with the Budget and Accounting Procedures Act of 1950 (codified, in part, at 31 U.S.C. §3511(d)).

Under the realignment announced in December 2004, the JFMIP Program Management Office was to report to the chair of the CFO Council’s Financial Systems Integration Committee. This reporting relationship subsequently changed. At the request of the OMB Controller, the CFO at the Department of Labor now chairs the Financial Systems Integration Committee and is the leading agency sponsor of the financial management line of business. Two subcommittees were also established under the announced realignment: Configuration Control Subcommittee—to focus on interface Transaction Processing Standardization Subcommittee—to support interagency development of functional requirements for the software certification process.

OMB officials indicated that the roles and responsibilities of the two subcommittees under the Financial Systems Integration Committee will likely continue to evolve. However, the full committee will periodically evaluate the subcommittees and whether they are well aligned and still needed or if additional subcommittees are needed.

Other significant responsibilities of the former JFMIP Program Management Office, which was previously managed by the JFMIP executive director using funds provided by the CFO Council, were shifted to the Financial Systems Integration Office (FSIO), which was established with staff from the original JFMIP Program Management Office. The FSIO will now report to the FSIO executive director, who will report to the OMB Controller. Before the realignment, the JFMIP Program Management Office was responsible for the testing and certification of commercial off- the-shelf (COTS) core financial systems for use by federal agencies and coordinating the development and publication of functional requirements for financial management systems, among other things. OMB officials expect that the FSIO will continue to focus on core financial systems and still be responsible for certification and testing of core systems, but they plan to evaluate the effectiveness of the certification and testing function. In addition, OMB has recognized the need for standardization and the inclusion of key stakeholders in developing systems requirements and processes, but considers it a long-term goal. The FSIO will develop systems requirements and the Financial Systems Integration Committee will be responsible for advising OFFM on the systems requirements. OFFM will now be responsible for issuing new systems requirements.

According to OMB officials, the FSIO is reassessing the realignment plan described in the December 2004 OMB memorandum and recently developed foundational materials including the mission statement, goals, objectives, performance indicators, scope of activities, prioritization of work, budget, organizational chart, and communication plan. According to OMB officials, resources at FSIO will be aligned under the priorities identified and the office will be structured according to the new priorities. The FSIO will identify its needs for additional staff and determine how many are needed and what skill sets are appropriate. The FSIO will continue defining its priorities and evaluating the effectiveness of processes and its plans will continue to evolve. While OMB has taken steps to accomplish the Federal Enterprise Architecture, lines of business, and JFMIP realignment initiatives, as discussed in the next section, it is generally at the early stages of implementation and a firm foundation has not yet been established to address the long-standing problems that have impeded success.

Broad-Based Actions Needed to Implement Financial Management Systems Governmentwide

The key for federal agencies to avoid the long-standing problems that have plagued financial management system improvement efforts is to address the foremost causes of those problems and adopt solutions that reduce the risks associated with these efforts to acceptable levels. Although OMB has articulated an approach for reforming financial management systems governmentwide under its financial management line of business and JFMIP realignment initiatives, implementing these initiatives will be complex and challenging. OMB has correctly recognized that enhancing the government’s ability to implement financial management systems that are capable of providing accurate, reliable, and timely information on the results of operations needs to be addressed as a governmentwide solution, rather than as individual agency stove-piped efforts designed to meet a given entity’s needs. However, OMB has not yet fully defined and implemented the processes needed to successfully complete these initiatives. Specifically, based on industry best practices, we identified four key concepts that are not yet fully developed and integrated in OMB’s initiatives and related processes. While OMB has addressed certain elements of these best practices in its initiatives, many specific steps are not yet completed. Careful consideration of these four concepts, each one building upon the next, will be integral to the success of OMB’s initiatives and will help break the cycle of failure in implementing financial management systems. The four concepts are (1) developing a concept of operations, (2) defining standard business processes, (3) developing a strategy for ensuring that agencies are migrated to a limited number of application service providers in accordance with OMB’s stated approach, and (4) defining and effectively implementing disciplined processes necessary to properly manage the specific projects. The following sections highlight the key issues to be considered for each of the four areas.

Concept of Operations Provides Foundation

What is considered a financial management system?

Who will be responsible for developing a governmentwide concept of operations and what process will be used to ensure that the resulting document reflects the governmentwide solution rather than individual agency stove-piped efforts?

How will the concept of operations be linked to the Federal Enterprise Architecture?

How can the federal government obtain reliable information on the costs of its financial management systems investments?

A concept of operations defines how an organization’s day-to-day operations are (or will be) carried out to meet mission needs. The concept of operations includes high-level descriptions of information systems, their interrelationships, and information flows. It also describes the operations that must be performed, who must perform them, and where and how the operations will be carried out. Further, it provides the foundation on which requirements definitions and the rest of the systems planning process are built. Normally, a concept of operations document is one of the first documents to be produced during a disciplined development effort and flows from both the vision statement and the enterprise architecture. According to the IEEE standards, a concept of operations is a user-oriented document that describes the characteristics of a proposed system from the users’ viewpoint. The key elements that should be included in a concept of operations are major system components, interfaces to external systems, and performance characteristics such as speed and volume.

In the case of federal financial management systems, another key element for the concept of operations would be a clear definition and scope of the financial management activities to be included. One problem with the current OMB approach for reporting is that systems that have historically been considered part of financial management, such as payroll and inventory management, are not captured under the financial management line of business when a particular agency reports IT investments to OMB as part of the annual budget submission for inclusion in the Budget of the United States Government. This is because the Federal Enterprise Architecture coding structure for agencies to use when transmitting IT investment information to OMB calls for only IT investments that support certain financial system functions to be identified as a financial management system. An effective concept of operations would help identify these omissions.

Financial management systems are defined by OMB in Circulars No. A-11 and A-127 in similar terms to that found in statutes such as FFMIA. This definition is also similar to that used by DOD to define a defense business system as provided by the fiscal year 2005 Defense Authorization Act. These various sources generally consider financial management systems to be financial systems and the financial portion of mixed systems that support the interrelationships and interdependencies between budget, cost, and management functions, and the information associated with business activities. A mixed system is an information system that supports both financial and nonfinancial functions of the federal government. At DOD, for example, an estimated 80 percent of the information needed to prepare annual financial statements comes from mixed systems such as logistics, personnel, and procurement systems that are outside of the responsibility of the DOD CFO. In contrast, the Federal Enterprise Architecture’s Business Reference Model defines a financial management system as one that uses financial information to measure, operate, and predict the effectiveness and efficiency of an entity’s activities in relation to its objectives. These differences illustrate that a consistent definition of financial management systems is not being used across the federal government.

One of the key challenges faced by OMB when evaluating financial management system implementation efforts is capturing all financial management system investments and their related costs. The fiscal year 2006 budget requests for IT spending totaled about $65.2 billion. Our analysis showed that, of this amount, only $3.9 billion, less than 6 percent, is reflected under the financial management mission as defined by OMB using the definition of a financial management system in its Federal Enterprise Architecture. A more comprehensive analysis of financial management system investments using the definition in OMB Circular No. A-127 that includes mixed systems such as payroll and inventory and including those considered by DOD as business systems brings the total to about $20 billion. Payroll and inventory management systems clearly support financial management activities, but these systems are not included in the financial management line of business within the Federal Enterprise Architecture framework. The payroll and inventory systems are reflected under the human resource management and supply chain management lines of business, respectively.

Because of these differing definitions, the total number of systems and the respective costs associated with financial management system implementation efforts are difficult to capture. OMB officials stated that they are currently revising OMB Circular No. A-127 and will consider clarifying the definition to ensure that it is consistent with FFMIA. In addition, an effective concept of operations would help bridge this gap and facilitate the monitoring of the activity related to financial management systems. Addressing this issue would be a key factor in developing a foundation for the lines of business initiative to consolidate federal financial management systems under a limited number of application service providers.

An effective concept of operations would describe, at a high level (1) how all of the various elements of federal financial systems and mixed systems relate to each other, and (2) how information flows from and through these systems. Further, a concept of operations would provide a useful tool to explain how financial management systems at the agency and governmentwide levels can operate cohesively. It would be geared to a governmentwide solution rather than individual agency stove-piped efforts. Further, it would provide a road map that can be used to (1) measure progress and (2) focus future efforts. OMB officials told us that they had developed a concept of operations, but did not know when it would be released or if it meets the criteria in the IEEE standards. Because the federal government has lacked such a document, a clear understanding of the interrelationships among federal financial systems and how the application service provider concept fits into this framework has not yet been achieved.

While the Federal Enterprise Architecture, when fully populated, could provide some of this perspective, a concept of operations document presents these items from a user’s viewpoint in nontechnical terms. Such a document would be invaluable in getting various stakeholders, including those at the agency and governmentwide levels, the software vendors, and the three branches of the federal government, to understand how the financial systems are expected to operate cohesively and how they fit into “the big picture.” A concept of operations from this perspective would clarify which financial management systems should be operated at an agency level and which ones would be handled at a governmentwide level and how those two would integrate. In addition, it could identify the nature and extent of skills needed to effectively operate these systems. This would play a part in resolving some of the human capital management problems discussed previously.

Another key element of a concept of operations is a transition strategy that is useful for developing an understanding of how and when changes will occur. Not only is this needed from an investment management point of view, it is a key element in the human capital problems discussed previously that revolved around change management strategies. Describing how to implement OMB’s approach for outsourcing financial management systems and the process that will be used to deactivate legacy systems that will be replaced or interfaced with a new financial management system are key aspects that need to be addressed in a transition strategy. This, in turn, allows the agencies to begin taking the necessary actions to integrate this approach into their investment management and change management processes.

Standard Business Processes Promote Consistency

How can governmentwide standard business processes be developed to meet the needs of federal agencies?

How can agencies be encouraged to adopt new processes, rather than selecting other methods that result in simply automating old ways of doing business?

How will the standard business processes be implemented by the application service providers to provide consistency across government agencies and among the application service providers?

What process will be used to determine and validate the processes needed for agencies that have unique needs?

Business process models provide a way of expressing the procedures, activities, and behaviors needed to accomplish an organization’s mission and are helpful tools to document and understand complex systems. Business processes are the various steps that must be followed to perform a certain activity. For example, the procurement process would start when the agency defines its needs, issues a solicitation for goods or services and would continue through contract award, receipt of goods and services, and would end when the vendor properly receives payment. The identification of preferred business processes would be critical for standardization of applications and training and portability of staff, as well as for the software vendor community to use for software design and implementation purposes. Without standard processes, the federal government will continue to spend funds to develop individual agency stove-piped efforts that may or may not meet a given entity’s needs.

To maximize the success of a new system acquisition, organizations need to consider the redesign of current business processes. As we noted in our Executive Guide: Creating Value Through World-class Financial Management, leading finance organizations have found that productivity gains typically result from more efficient processes, not from simply automating old processes. Moreover, the Clinger-Cohen Act of 1996 requires agencies to analyze the missions of the agency and, based on the analysis, revise mission-related and administrative processes, as appropriate, before making significant investments in information technology used to support those missions. Another benefit of what is often called business process modeling is that it generates better system requirements, since the business process models drive the creation of information systems that fit in the organization and will be used by end users. Other benefits include (1) providing a foundation for agency efforts to describe the business processes needed for unique missions, or to develop subprocesses to support those at the governmentwide level and (2) describing the business processes of the federal government to the vendor community for standardization. While in many cases, government business processes will be identical or very similar to processes used by the private sector, these standards should also describe processes unique to federal accounting.

However, according to OMB officials, the lines of business initiative is moving forward even though this important key issue has not yet been addressed. OMB officials believed that for standardized processes, it is important to get buy-in as the processes are developed, and not force the process from the top. OMB officials we talked with recognized that standardization of business processes is important, but they did not want to wait to deploy the financial management line of business initiative until standard business processes had been developed. OMB planned to task the newly created CFO Council Transaction Processing Standardization Subcommittee with the responsibility for developing standard federal business processes. Because this key issue has not been addressed, and the other key issues flow from it, little has been done to address those important considerations. From our perspective, adopting standardized processes is a fundamental step needed for all financial system implementations, but especially for making the financial management line of business initiative successful. Otherwise, we believe that there is a much greater risk of the continued proliferation of nonstandard business processes that would not result in a marked improvement from the current environment.

Strategy for Implementing the Financial Management Line of Business Initiative Will Be Key

What guidance will be provided to assist agencies in adopting a change management strategy that reduces the risks of moving to the application service provider approach?

What processes will be put in place to ensure that agency financial management system investment decisions focus on the benefits of standard processes and application service providers?

What process will be used to facilitate the decision-making process used by agencies to select a given provider?

How will agencies incorporate strategic workforce planning in the implementation of the application service provider approach?

Although OMB has a goal of migrating agencies to a limited number of application service providers within the next 7 to 8 years to deliver the standard business processes, rather than funding individual agency efforts, it has not yet articulated a clear and measurable strategy for achieving this goal. This is important because there has been a historical tendency for agencies and units within agencies to view their needs as urgent and resist standardization. Decisive action will be needed to ensure that agencies adopt the application service provider concept and that agencies do not continue to attempt to develop and implement their own financial management systems. OMB has been proactive since the beginning of the financial management line of business initiative in describing the goals of the initiative by making speeches, discussing the initiative with the media, including it in the President’s budget request, and highlighting it on its Web site. However, there are limited tools and guidance available and OMB has not provided centers of excellence with standard document templates needed to minimize risk, provide assurance, and develop understandings with customers on topics such as service level agreements and concept of operations. A service level agreement is critical for both the application service providers and the agencies to be held accountable for their respective parts of the agreement. Much work remains to develop a change management strategy that addresses key activities needed to minimize the risk associated with the implementation of the financial management line of business initiative.

Change management in the context of migrating federal agencies to an application service provider will need to include activities such as (1) developing specific criteria for requiring agencies to migrate to an application service provider rather than attempting to develop and implement their own stove-piped business systems; (2) providing the necessary information for an agency to make a selection of an application service provider; (3) defining and instilling new values, norms, and behaviors within agencies that support new ways of doing work and overcoming resistance to change; (4) building consensus among customers and stakeholders on specific changes designed to better meet their needs; and (5) planning, testing, and implementing all aspects of the transition from one organizational structure and business process to another.

According to leading IT organizations, organizational change management is the process of preparing users for the business process changes that will accompany implementation of a new system. An effective organizational change management process includes project plans and training that prepare users for impacts the new system might have on their roles and responsibilities and a process to manage those changes. We have reported on various problems with agencies’ change management including the failure to develop transition plans, reengineer business processes, and limit customization. In addition, one CFO Council member told us that from his perspective systems do not fail, but there is an implementation failure because of (1) ineffective coordination and communication between the CFO and CIO offices, (2) excessive modification of COTS systems, (3) business processes not being reengineered correctly, completely, or timely, and (4) a lack of authority and leadership for the CFO and project management offices to make the implementation work.

With regard to establishing criteria for transitioning agencies to an application service provider, we note that providing governmentwide financial services is not a new concept to the federal government. One of the 24 Presidential Electronic Government initiatives is e-payroll, which was intended to consolidate 22 federal payroll systems into 4 federal payroll providers to simplify and standardize federal human resources/payroll policies and procedures to better integrate payroll, human resources, and finance functions. Numerous agencies had targeted their payroll operations for costly modernizations, and according to OMB, by consolidating duplicative payroll modernization efforts, an estimated $1.1 billion can be saved over the next decade in future IT investments given the economies of scale and cost avoidance. Federal agencies already have or will be migrating to one of the four selected payroll providers to process payroll and pay employees.

OMB officials told us they learned from the e-payroll initiative that directing and forcing change as they had done with the e-payroll effort was not palatable to federal agencies. The agencies preferred having choices on timing the move and on having options for various providers. As a result, for the financial management line of business initiative, they do not plan to establish a migration path or time table. Further, processes have not been put in place to facilitate agency decisions on selecting a provider or focusing investment decisions on the benefits of standard processes and application service providers. It is not clear how this will impact the adoption of this initiative. Given the pressures to reduce budgets, discipline with respect to following a clear migration path will be essential. Without such a migration path, while some agencies may readily migrate to a center of excellence or application service provider to minimize the tremendous undertaking of implementing or significantly upgrading a financial system, other agencies will likely perpetuate the waste of taxpayer dollars previously described related to failed system implementation efforts.

The need for clear criteria on migrating agencies to the financial management line of business initiative is highlighted by the following example.

In fiscal year 2004, the Department of Justice embarked on implementing a new core financial system and is not planning to move to a center of excellence. OMB officials stated that they were not requiring Justice to move to a center of excellence because it had unique needs and was already far enough along in its attempt to modernize and consolidate the financial systems used throughout the agency. OMB officials also speculated that Justice might eventually become a center of excellence that focuses on law enforcement agencies and addresses the law enforcement community’s unique needs. According to a supporting document of the Analytical Perspectives, Budget of the United States Government, Fiscal Year 2006, Justice spent about $6.9 million on modernizing its core financial system in fiscal year 2004. Further, Justice planned to spend $23.1 million for modernization during fiscal year 2005, and expects fiscal year 2006 modernization costs to more than triple to $72.5 million. In October 2004, the IG reported that little progress had been made in implementing the new system and continued to report financial management and systems as a top management challenge. Thus, it is not clear why Justice should continue with its financial systems development project when the cost is expected to significantly escalate and significant challenges remain.

Further, the application service provider concept will still require that agencies address long-standing human capital problems by incorporating elements of strategic workforce planning such as (1) aligning an organization’s human capital program with its current and emerging mission and programmatic goals and (2) developing long-term strategies for acquiring, developing, and retaining an organization’s total workforce to meet the needs of the future. This includes a range of activities from identifying and defining roles and responsibilities, to identifying team members, to developing individual competencies that enhance performance. To maintain and enhance the capabilities of IT staff, organizations should develop and implement a human capital strategy that, among other things, includes assessing competencies and skills needed to effectively perform IT operations to support agency mission and goals, inventorying the competencies and skills of current IT staff to identify gaps in needed capabilities, and developing and implementing plans to fill the gap between requirements and current staffing.

As we have testified, having sufficient numbers of people on board with the right mix of knowledge and skills can make the difference between success and failure. This is especially true in the IT area, where widespread shortfalls in human capital have contributed to demonstrable shortfalls in agency and program performance. According to Building the Work Force Capacity to Successfully Implement Financial Systems, the roles needed on an implementation team are consistent across financial system implementation projects and include a project manager, systems integrator, functional experts, information technology manager, and IT analysts. Many of these roles require the dedication of full-time staff for one or more of the project’s phases.

Finally, sustained leadership will be key to a successful strategy for moving federal agencies towards consolidated financial management systems. In our Executive Guide: Creating Value Through World-class Financial Management, we found that leading organizations made financial management improvement an entitywide priority by, among other things, providing clear, strong executive leadership. We also reported that making financial management a priority throughout the federal government involves changing the organizational culture of federal agencies. Although the views about how an organization can change its culture can vary considerably, leadership (executive support) is often viewed as the most important factor in successfully making cultural changes. Top management must be totally committed in both words and actions to changing the culture, and this commitment must be sustained and demonstrated to staff. In addition, a recent best practice guide on shared services stated that it is not enough for management to merely support the financial operations’ shared service implementation—top management must provide the leadership structure to ensure that the transition is successful. Because the tenure of political appointees is relatively short, the current and future administrations must continue a strong emphasis on top-notch financial management.

Disciplined Processes Will Help Ensure Successful Implementations

How can existing industry standards and best practices be incorporated into governmentwide guidance related to financial management system implementation efforts, including migrating to an application service provider?

What actions will be taken to reduce the risks and costs associated with data conversion and interface efforts?

What oversight process will be used to ensure that modernization efforts effectively implement the prescribed policies and procedures?

Once the concept of operations and standard business processes have been defined and a migration strategy is in place, individual agencies will have to work closely with the selected application service provider or systems integrator to help ensure that the implementation is successful. Although application service providers may provide a COTS solution, effective implementation and testing processes are still required to ensure that the system delivers the desired functionality on time and within budget. As previously discussed, a partnership between the CIO and CFO offices, as well as with those program management offices responsible for financial or mixed systems such as payroll and inventory, is critical for success. Agencies have frequently struggled to implement key best practices when implementing COTS financial management systems. The key to avoiding these long-standing implementation problems is to provide specific guidance to agencies for financial management system implementations, incorporating the best practices identified by the SEI, the IEEE, the Project Management Institute, and other experts that have been proven to reduce risk in implementing systems. Such guidance should include the various disciplined processes such as requirements management, testing, data conversion and system interfaces, risk and project management, and related activities, which have been problematic in the financial systems implementation projects we reviewed.

Disciplined processes have been shown to reduce the risks associated with software development and acquisition efforts to acceptable levels and are fundamental to successful system implementations. The principles of disciplined IT systems development and acquisition of services apply to shared services implementation. A disciplined software implementation process can maximize the likelihood of achieving the intended results (performance) within established resources (costs) on schedule. For example, disciplined processes should be in place to address the areas of data conversion and interfaces, two of the many critical elements necessary to successfully implement a new system that have contributed to the failure of previous agency efforts. The former JFMIP provided guidance on data conversion, and the Configuration Control Subcommittee under the CFO Council’s Financial Systems Integration Committee was tasked with focusing on interface requirements. However, a standard set of practices will be needed to guide the migration from legacy systems to new systems and application service providers. Further details on disciplined processes needed can be found in appendix III.

In addition, oversight to help ensure that the disciplined processes are in place and operating as intended will be a critical factor in the success of the implementation of new and consolidated financial management systems. Currently, OMB guidance requires agencies to have qualified project managers and to use earned value management tools for major IT investments. However, OMB only performs limited reviews of agencies’ financial management systems implementations. OFFM officials told us that these reviews vary considerably in scope and that one of their goals is to provide more structure to the reviews. OMB’s review depends on the agency and the phase of the project, and generally does not focus on implementation of the disciplined processes used. Industry experts agree that the best indicator of whether risks have been reduced to an acceptable level is an assessment of the disciplined processes in place. For example, in the area of requirements management, disciplined processes would help ensure (1) the requirements document contains all the requirements identified by the customer, as well as those needed for the definition of the system, (2) the requirements fully describe the software functionality to be delivered, (3) the requirements are stated in clear terms that allow for quantitative evaluation, and (4) traceability among various documents is maintained. Proper oversight would entail verification of these requirements-related disciplined processes.

In addition to problems with the structure and scope of OMB’s current system reviews, we noted that OFFM has a staff of only four employees dedicated to reviewing federal executive branch agency projects to implement financial management systems. These four staff also have other time-consuming duties such as developing a coherent, coordinated architecture and issuing federal financial system requirements. As a result, the current level of detail in the existing system reviews is necessarily limited. Moreover, there is limited follow-up by OMB on suggested improvements they have made to agency officials, and there is not any impetus for agencies to implement suggested improvements. For example, OFFM officials told us that they advised an agency that there were numerous disadvantages to deploying a new financial management system mid-year. Nonetheless, the agency deployed the system at mid-year and has faced problems by doing so. The FSIO also has a limited number of staff to perform its numerous financial management policy and oversight activities and is currently reassessing its priorities and available resources. Given the range of OMB’s leadership roles and its relatively small size as part of the Executive Office of the President, it is not realistic to expect OMB to be able to carry out a comprehensive review function. Instead, agencies could be required to have their financial management system projects undergo independent verification and validation reviews to ensure that the projects adequately implemented the disciplined processes needed to manage the risks to acceptable levels. OMB could then review reports produced as a result of the independent verification and validation process to leverage its oversight efforts. Accordingly, OMB could then focus its oversight efforts on the projects with the greatest risks.

Conclusions

Because the federal government is one of the largest and most complex organizations in the world, operating, maintaining, and modernizing its financial management systems represent a monumental challenge— technically and cost-wise. The past paradigm must be changed from one in which each federal agency attempts to implement systems that, in many cases, are to perform redundant functions and have all too often resulted in failure, have been delayed, and cost too much. Thus, a more holistic governmentwide approach as OMB has been advocating is necessary to address the key causes of failure. OMB has recognized the seriousness of the problems. Its primary initiative related to the use of a limited number of application service providers is a step in the right direction. This initiative is in the early stage and does not yet include basic elements that are integral to its success. Based on industry best practices, the following four concepts would help ensure a sound foundation for developing and implementing a governmentwide solution for long-standing financial management system implementation failures: (1) developing a concept of operations that ties in other systems, (2) defining standard business processes, (3) developing a strategy for ensuring that agencies are migrated to a limited number of application service providers, and (4) defining and effectively implementing applicable disciplined processes. As pressure mounts to do more with less, to increase accountability, and to reduce fraud, waste, abuse, and mismanagement, and efforts to reduce federal spending intensify, sustained and committed leadership will be a key factor in the successful implementation of these governmentwide initiatives. However, regardless of the approach taken, the adherence to disciplined processes in systems development and acquisition will be at the core of successfully addressing the key causes of financial management system implementation failures.

Recommendations for Executive Action

To help reduce the risks associated with financial management system implementation efforts and facilitate the implementation of the financial management line of business and JFMIP realignment initiatives across the government, we recommend that the Director of OMB take the following 18 actions. This would entail placing a high priority on fully integrating into its approach the following concepts and underlying key issues, all of which are related to the fundamental disciplines in systems implementation: Developing a concept of operations. This would include identifying the interrelationships among federal financial systems and how the application service provider concept fits into this framework, prescribing which financial management systems should be operated at an agency level and which should be operated at a governmentwide level and how those would integrate, and defining financial management systems in the Federal Enterprise Architecture to be more consistent with the similar definitions used in FFMIA and OMB Circulars No. A-11 and No. A-127.

Defining standard business processes. This would include describing the standard business processes that are needed to meet federal agencies’ needs, developing a process to identify those that are needed to meet unique agency needs, requiring application service providers to adopt standard business processes to provide consistency, and encouraging agencies to embrace new processes.

Developing a strategy for ensuring that agencies are migrated to a limited number of application service providers in accordance with OMB’s stated approach. This would include articulating a clear goal and criteria for ensuring agencies are subject to the application service provider concept and cannot continue developing and implementing their own stove-piped systems, establishing a migration path or time table for when agencies should migrate to an application service provider, providing the necessary information for an agency to select an application service provider, and developing guidance to assist agencies in adopting a change management strategy for moving to application service providers.

Defining and effectively implementing disciplined processes necessary to properly manage the specific projects. This would include providing specific guidance to agencies on disciplined processes for providing a standard set of practices to guide the migrations from legacy systems to new systems and application service providers, and developing processes to facilitate oversight and review that allow for a more structured review and follow-up of agencies’ financial system implementation projects.

Agency Comments and Our Evaluation

We received written comments on a draft of this report from the Controller of OMB, which are reprinted in appendix IV. The Controller agreed with our recommendations and described the approach and steps that OMB is taking to improve financial management system modernization efforts. As OMB moves forward to address the recommendations in our report, it is important that it prioritize its efforts and focus on the concepts and underlying key issues we discussed, such as adequately defining and implementing disciplined processes. We are encouraged that OMB plans to issue additional guidance outlining the fundamental risk-reduction approaches that agencies can implement when acquiring and implementing financial systems. It will be critical that the guidance stresses the importance of this standard set of practices. We continue to believe that careful consideration of all the building blocks and key issues we identified will be integral to the success of OMB’s initiatives. OMB also provided additional oral comments which we incorporated as appropriate.

We are sending copies of this report to the Chairman and Ranking Minority Member, Senate Committee on Homeland Security and Governmental Affairs, and other interested congressional committees. We are also sending a copy to the Director of OMB. Copies will also be made available to others upon request. The report will also be available at no charge on GAO’s Web site at http://www.gao.gov.

If you or your staff have any questions about this report, please contact McCoy Williams, Director, Financial Management and Assurance, who may be reached at (202) 512-9095 or by e-mail at williamsm1@gao.gov, or Keith A. Rhodes, Chief Technologist, Applied Research and Methods, who may be reached at (202) 512-6412 or by e-mail at rhodesk@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix V.

Appendix I: Scope and Methodology

To determine the key causes for financial management system implementation failures, we conducted database searches of GAO and inspector general (IG) Web sites to identify reports issued by any GAO teams or IGs that could be relevant. We summarized and analyzed prior GAO reports on commercial off-the-shelf financial management system implementations within the last 5 years. We performed a content analysis of the GAO and IG reports to determine if causes for the financial management system implementation problems were included. We discussed the relevant GAO report findings and current status with the key staff that worked on the reports. In addition, we identified other potential data sources, such as key industry groups and well-known national experts for information they had on this topic. We also interviewed key Office of Management and Budget (OMB) officials and had discussions with other interested parties such as Chief Financial Officers (CFO) Council representatives.

To identify the significant governmentwide initiatives that are currently under way that impact financial management systems implementation failures, we interviewed key OMB officials and reviewed relevant OMB policies, guidance, and memorandums related to the initiatives. We also interviewed CFO Council representatives to discuss the initiatives to reform federal financial management systems. In addition, we interviewed Office of Personnel Management officials to discuss their plans to migrate to a financial management center of excellence. We also reviewed reports from various authors and governmentwide forums where participants provided their perspectives on governmentwide initiatives.

To provide our views on actions that can be taken to help improve the management and control of agency financial management system modernization efforts, we analyzed the GAO and IG reports we had identified as relevant to the topic to highlight the actions called for in those reports. Further, we reviewed material from key industry groups and national experts to identify any potential solutions posed by those groups, lessons learned, and relevant best practices. We took into consideration those governmentwide initiatives that were currently under way and the perspectives provided by authors and participants in governmentwide forums. In addition, during our consultations with various GAO stakeholders, and external groups such as OMB and the CFO Council, we obtained their perspectives on the actions needed to address the problems.

We conducted our work in Washington, D.C., from January 2005 through October 2005, in accordance with U.S. generally accepted government auditing standards. We did not evaluate the federal government’s overall IT strategy or whether a particular agency selected the most appropriate financial management system. Because we have previously provided agencies with specific recommendations in individual reports, we are not making additional recommendations to them in this report. We requested comments on a draft of this report from the Director of OMB or his designee. Written comments from OMB are reprinted in appendix IV and evaluated in the Agency Comments and Our Evaluation section.

Appendix II: IG Reports Reviewed

Department of the Treasury Office of Inspector General. The Modernization Program Is Establishing a Requirements Management Office to Address Development and Management Problems. Reference No. 2005-20-023. Washington, D.C.: January 19, 2005.

Department of Transportation Office of Inspector General. Consolidated Financial Statements for Fiscal Years 2004 and 2003. Report FI-2005- 009. Washington, D.C.: November 15, 2004.

Department of Housing and Urban Development Office of Inspector General. Fiscal Year 2004 Review of Information Systems Controls in Support of the Financial Statements Audit. Report 2005-DP-0001. Washington, D.C.: October 19, 2004.

Department of Justice Office of Inspector General. The Drug Enforcement Administration’s Management of Enterprise Architecture and Information Technology Investments. Report 04-36. Washington, D.C.: September 2004.

Department of Veterans Affairs Office of Inspector General. Issues at VA Medical Center Bay Pines, Florida and Procurement and Deployment of the Core Financial and Logistics System. Report 04-01371-177. Washington, D.C.: August 11, 2004.

Department of Energy Office of Inspector General. Management of the Federal Energy Regulatory Commission’s Information Technology Program. Report DOE/IG-0652. Washington, D.C.: June 2004.

Department of Justice Office of Inspector General. The Federal Bureau of Investigation’s Implementation of Information Technology Recommendations. Report 03-36. Washington, D.C.: September 2003.

Small Business Administration Office of Inspector General. Audit of SBA’s Acquisition, Development and Implementation of the Joint Accounting and Administrative Management System. Report 3-32. Washington, D.C.: June 30, 2003.

Department of Energy Office of Inspector General. Audit Report on Business Management Information System. Report DOE/IG-0572. Washington, D.C.: November 2002.

Department of the Interior Office of Inspector General. Developing the Department of the Interior’s Information Technology Capital Investment Process: A Framework for Action. Report 2002-I-0038. Washington, D.C.: August 2002.

Department of Defense Office of Inspector General. Development of the Defense Finance and Accounting Service Corporate Database and other Financial Management Systems. Report D-2002-014. Washington, D.C.: November 7, 2001.

Department of Transportation Office of Inspector General. Implementing a New Financial Management System. Report FI-2001-074. Washington, D.C.: August 7, 2001.

Appendix III: Disciplined Processes

Disciplined Processes Are Key to Successful Financial Management System Implementation Efforts

Disciplined processes have been shown to reduce the risks associated with software development and acquisition efforts to acceptable levels and are fundamental to successful system implementations. A disciplined software implementation process can maximize the likelihood of achieving the intended results (performance) within established resources (costs) on schedule. Although a standard set of practices that will guarantee success does not exist, several organizations, such as the Software Engineering Institute (SEI) and the Institute of Electrical and Electronic Engineers (IEEE), and individual experts, have identified and developed the types of policies, procedures, and practices that have been demonstrated to reduce development time and enhance effectiveness. The key to having a disciplined system development effort is to have disciplined processes in multiple areas, including requirements management, testing, data conversion and system interfaces, configuration management, risk management, project management, and quality assurance.

Requirements Management

Requirements are the specifications that system developers and program managers use to design, develop, and acquire a system. They need to be carefully defined, consistent with one another, verifiable, and directly traceable to higher-level business or functional requirements. It is critical that they flow directly from the organization’s concept of operations (how the organization’s day-to-day operations are or will be carried out to meet mission needs).

According to the IEEE, a leader in defining the best practices for such efforts, good requirements have several characteristics, including the following: The requirements fully describe the software functionality to be delivered. Functionality is a defined objective or characteristic action of a system or component. For example, for grants management, a key functionality includes knowing (1) the funds obligated to a grantee for a specific purpose, (2) the cost incurred by the grantee, and (3) the funds provided in accordance with federal accounting standards.

The requirements are stated in clear terms that allow for quantitative evaluation. Specifically, all readers of a requirement should arrive at a single, consistent interpretation of it.

Traceability among various requirement documents is maintained.

Requirements for projects can be expressed at various levels depending on user needs. They range from agencywide business requirements to increasingly detailed functional requirements that eventually permit the software project managers and other technicians to design and build the required functionality in the new system. Adequate traceability ensures that a requirement in one document is consistent with and linked to applicable requirements in another document.

The requirements document contains all of the requirements identified by the customer, as well as those needed for the definition of the system.

Studies have shown that problems associated with requirements definition are key factors in software projects that do not meet their cost, schedule, and performance goals. Examples include the following: A 1988 study found that getting a requirement right in the first place costs 50 to 200 times less than waiting until after the system is implemented to get it right.

A 1994 survey of more than 8,000 software projects found that the top three reasons that projects were delivered late, over budget, and with less functionality than desired all had to do with requirements management.

A 1994 study found that, on average, there is about a 25-percent increase in requirements over a project’s lifetime, which translates into at least a 25-percent increase in the schedule.

A 1997 study noted that between 40 and 60 percent of all defects found in a software project could be traced back to errors made during the requirements development stage.

Testing

Testing is the process of executing a program with the intent of finding errors. Because requirements provide the foundation for system testing, they must be complete, clear, and well documented to design and implement an effective testing program. Absent this, an organization is taking a significant risk that substantial defects will not be detected until after the system is implemented. As shown in figure 2, there is a direct relationship between requirements and testing.

Although the actual testing occurs late in the development cycle, test planning can help disciplined activities reduce requirements-related defects. For example, developing conceptual test cases based on the requirements derived from the concept of operations and functional requirements stages can identify errors, omissions, and ambiguities long before any code is written or a system is configured. Disciplined organizations also recognize that planning the testing activities in coordination with the requirements development process has major benefits.

Although well-defined requirements are critical for implementing a successful testing program, disciplined testing efforts for projects have several characteristics, which include the following: Testers who assume that the program has errors are likely to find a greater percentage of the defects present in the system. This is commonly called the testing mindset.

Test plans and scripts that clearly define what the expected results should be when the test case is properly executed and the program does not have a defect that would be detected by the test case. This helps to ensure that defects are not mistakenly accepted.

Processes that ensure test results are thoroughly inspected.

Test cases that include exposing the system to invalid and unexpected conditions as well as the valid and expected conditions. This is commonly referred to as boundary condition testing.

Testing processes that determine if a program has unwanted side effects. For example, a process should update the proper records correctly but should not delete other records.

Systematic gathering, tracking, and analyzing statistics on the defects identified during testing.

Although these processes may appear obvious, they are often overlooked in testing activities.

Data Conversion and System Interfaces

Data conversion is defined as the modification of existing data to enable them to operate with similar functional capability in a different environment. It is one of the many critical elements necessary to successfully implement a new system. Because of the difficulty and complexity associated with financial systems data conversion, highly skilled staff are needed. There are three primary phases in a data conversion: 1. Pre-conversion activities prior to and leading up to the conversion, such as determining the scope and approach or method, developing the conversion plan, performing data cleanup and validation, ensuring data integrity, and conducting necessary analysis and testing. 2. Cutover activities to convert the legacy data to the new system, such as testing system process and data edits, testing system interfaces (both incoming and outgoing), managing the critical path, supervising workload completion, and reconciliation. 3. Post-installation activities such as verifying data integrity, conducting final disposition of the legacy system data, and monitoring the first reporting cycle.

There are also specific issues that apply uniquely to converting data as part of the replacement of a financial system, including identifying specific open transactions and balances to be established, analyzing and reconciling transactions for validation purposes, and establishing transactions and balances in the new system through an automated or manual process.

Further, consideration of various data conversion approaches and implications are important. Some considerations to be taken into account for the system conversion are the timing of the conversion (beginning-of- the-year, mid-year, or incremental) and other options such as direct or flash conversions, parallel operations, and pilot conversions. In addition, agencies should consider different data conversion options for different categories of data when determining the scope and time lines such as opting not to conduct a data conversion, processing new transactions and activity only, establishing transaction balances in the new system for reporting converting open transactions from the legacy system, and recording new activity on closed prior year transactions.

Validation and adjustment of open transactions and data in the legacy system are essential prerequisites to the conversion process and have often been problematic. When data conversion is done right, the new system can flourish. However, converting data incorrectly has lengthy and long-term repercussions.

System interfaces operate on an ongoing basis linking various systems and provide data that are critical to day-to-day operations, such as obligations, disbursements, purchase orders, requisitions, and other procurement activities. Testing the system interfaces in an end-to-end manner is necessary so agencies can have reasonable assurance that the system will be capable of providing the intended functionality. Systems that lack appropriate system interfaces often rely on manual reentry of data into multiple systems, convoluted systems, or both. According to the SEI, a widely recognized model for evaluating the interoperability of systems is the Levels of Information System Interoperability. This model focuses on the increasing levels of sophistication of system interoperability. Efforts at the highest level of this model—enterprise-based interoperability—are systems that can provide multiple users access to complex data simultaneously, data and applications are fully shared and distributed, and data have a common interpretation regardless of format. This is in contrast to the traditional interface strategies that are more aligned with the lowest level of the SEI model. Data exchanged at this level rely on electronic links that result in a simple electronic exchange of data.

Configuration Management According to the SEI, configuration management is defined as a discipline applying technical and administrative direction and surveillance to (1) identify and document the functional and physical characteristics of a configuration item, (2) control changes to those characteristics, (3) record and report change processing and implementation status, and (4) verify compliance with specified requirements. The purpose of configuration management is to establish and maintain the integrity of work products. Configuration management involves the processes of identifying the configuration of selected work products that compose the baselines at given points in time, controlling changes to configuration items, building or providing specifications to build work products from the maintaining the integrity of baselines, and providing accurate status and current configuration data to developers, integrators, and end users.

The work products placed under configuration management include the products that are delivered to the customer, designated internal work products, acquired products, tools, and other items that are used in creating and describing these work products.

For COTS systems, configuration management focuses on ensuring that changes to the requirements or components of a system are strictly controlled to ensure the integrity and consistency of system requirements or components. Two of the key activities for configuration management include ensuring that (1) project plans explicitly provide for evaluation, acquisition, and implementation of new, often frequent, product releases and (2) modification or upgrades to deployed versions of system components are centrally controlled, and unilateral user release changes are precluded. Configuration management recognizes that when using COTS products, it is the vendor, not the acquisition or implementing organization, that controls the release of new versions and that new versions are frequently released.

Risk Management

Risk and opportunity are inextricably related. Although developing software is a risky endeavor, risk management processes should be used to manage the project’s risks to acceptable levels by taking the actions necessary to mitigate the adverse effects of significant risks before they threaten the project’s success. If a project does not effectively manage its risks, then the risks will manage the project.

Risk management is a set of activities for identifying, analyzing, planning, tracking, and controlling risks. Risk management starts with identifying the risks before they can become problems. If this step is not performed well, then the entire risk management process may become a useless exercise since one cannot manage something that one does not know anything about. As with the other disciplined processes, risk management is designed to eliminate the effects of undesirable events at the earliest possible stage to avoid the costly consequences of rework.

After the risks are identified, they need to be analyzed so that they can be better understood and decisions can be made about what actions, if any, will be taken to address them. Basically, this step includes activities such as evaluating the impact on the project if the risk does occur, determining the probability of the event occurring, and prioritizing the risk against the other risks. Once the risks are analyzed, a risk management plan is developed that outlines the information known about the risks and the actions, if any, which will be taken to mitigate those risks. Risk monitoring is a continuous process because both the risks and actions planned to address identified risks need to be monitored to ensure that the risks are being properly controlled and that new risks are identified as early as possible. If the actions envisioned in the plan are not adequate, then additional controls are needed to correct the deficiencies identified.

Project Management

Effective project management is the process for planning and managing all project-related activities, such as defining how components are interrelated, defining tasks, estimating and obtaining resources, and scheduling activities. Project management allows the performance, cost, and schedule of the overall program to be continually measured, compared with planned objectives, and controlled. Project management activities include planning, monitoring, and controlling the project.

Project planning is the process used to establish reasonable plans for carrying out and managing the software project. This includes (1) developing estimates of the resources needed for the work to be performed, (2) establishing the necessary commitments, and (3) defining the plan necessary to perform the work. Effective planning is needed to identify and resolve problems as soon as possible, when it is the cheapest to fix them. According to one author, the average project expends about 80 percent of the time on unplanned rework—fixing mistakes that were made earlier in the project. Recognizing that mistakes will be made in a project is an important part of planning. According to this author, successful system development activities are designed so that the project team makes a carefully planned series of small mistakes to avoid making large, unplanned mistakes. For example, spending the time to adequately analyze three design alternatives before selecting one results in time spent analyzing two alternatives that were not selected. However, discovering that a design is inadequate after development can result in code that must be rewritten, at a cost greater than analyzing the three alternatives in the first place. This same author notes that a good rule of thumb is that each hour a developer spends reviewing project requirements and architecture saves 3 to 10 hours later in the project.

Project monitoring and control help to understand the progress of the project and determine when corrective actions are needed based on the project’s performance. Best business practices indicate that a key facet of project management and oversight is the ability to effectively monitor and evaluate a project’s actual performance, cost, and schedule against what was planned. In order to perform this critical task, the accumulation of quantitative data or metrics is required and can be used to evaluate a project’s performance. An effective project management and oversight process uses quantitative data or metrics to understand matters such as (1) whether the project plan needs to be adjusted and (2) oversight actions that may be needed to ensure that the project meets its stated goals and complies with agency guidance. For example, an earned value management system is one metric that can be employed to better manage and oversee a system project. An earned value management system attempts to compare the value of work accomplished during a given period with the work scheduled for that period. With ineffective project oversight, management can only respond to problems as they arise.

Agency management can also perform oversight functions, such as project reviews and participation in key meetings, to help ensure that the project will meet the agency needs. Management can use independent verification and validation reviews to provide it with assessments of the project’s software deliverables and processes. Although independent of the developer, verification and validation is an integral part of the overall development program and helps management mitigate risks. This core element involves having an independent third party—such as an internal audit function or a contractor that is not involved with any of the system implementation efforts—verify and validate that the systems were implemented in accordance with the established business processes and standards. Doing so provides agencies with needed assurance about the quality of the system, which is discussed in more detail in the following section.

Quality Assurance

Quality assurance is defined as a set of procedures designed to ensure that quality standards and processes are adhered to and that the final product meets or exceeds the required technical and performance requirements. Quality assurance is a widely used approach in the software industry to improve upon product delivery and the meeting of customer requirements and expectations. The SEI indicates that quality assurance should begin in the early phases of a project to establish plans, processes, standards, and procedures that will add value to the project and satisfy the requirements of the project and the organizational policies. Quality assurance provides independent assessments, typically performed by an independent verification and validation or internal audit team, of whether management process requirements are being followed and whether product standards and requirements are being satisfied. Some of the widely used quality assurance activities include defect tracking, technical reviews, and system testing.

Defect tracking–keeping a record of each defect found, its source, when it was detected, when it was resolved, how it was resolved (fixed or not), and so on.

Technical reviews–reviewing user interface prototypes, requirements specifications, architecture, designs, and all other technical work products.

System testing–executing software for the purpose of finding defects, typically performed by an independent test organization or quality assurance group.

According to one author, quality assurance activities might seem to result in a lot of overhead, but in actuality, exactly the opposite is true. If defects can be prevented or removed early, a significant schedule benefit can be realized. For example, studies have shown that reworking defective requirements, design, and code typically consumes 40 to 50 percent of the total costs of software development projects. An effective quality assurance approach is to detect as many defects as possible as early as possible to keep the costs of corrections down. However, enormous amounts of time can be saved by detecting defects earlier than during system testing.

Appendix IV: Comments from the Office of Management and Budget

Appendix IV: Comments from the Office of Management and Budget o Facilitate stronger internal controls that ensure integrity in accounting and other o Reduce costs by providing a competitive alternative for agencies to acquire, develop, implement, and operate financial management systems through shared service solutions; o Standardize systems, business processes and data elements; and o Provide for seamless data exchange between and among Federal agencies by implementing a common language and structure for financial information and system interfaces. 3. What are the critical milestones that must be accomplished in order to achieve the vision and goals of the FMLOB?

Federal agencies have begun implementing the FMLOB initiative by actively migrating to shared service providers and initiating solutions to integrate financial data among and between agency business systems. Nothing in this memorandum changes the expectation that agencies will continue to take all the necessary steps (in the earliest possible timeframes) to meet FMLOB objectives. The milestones described below, therefore, are intended to facilitate, not delay, agency efforts.

As depicted in Attachment 1, the critical milestones of the FMLOB can be broken down into three stages – (i) transparency and standardization; (ii) competitive environment and seamless data integration; and (iii) results.

Stage 1: Transparency and Standardization. In order to enable a competitive environment where agencies have more options and leverage in choosing a financial system, and in order to facilitate seamless integration of financial data among agency business systems, additional transparency and standardization is required.

Transparency: In determining the best options available when modernizing financial systems, the Federal financial community must have clarity on how to evaluate the performance and cost of shared service alternatives (i.e., Centers of Excellence (COE)) as well as clarity on what steps Federal agencies are expected to undertake in order to migrate to a COE. As described in more detail below, a COE is a shared service solution where a single entity provides financial management services for multiple organizations. In order to achieve additional transparency, two specific projects (with associated milestones) will be undertaken: Establishment of Common Performance Measures – This project will result in standard quality and cost measures for agencies to benchmark and compare the performance of financial system alternatives.

Development of Migration Planning Guidance – This project will result in comprehensive guidance that helps Federal agencies describe, prepare for, and manage an agency’s migration to a COE. This guidance will also include a definition of the full range of services to be provided by all COEs and a description of the “rules of 2 engagement,” including templates for service level agreements outlining provider and client responsibilities.

Standardization: In order to mitigate the cost and risk of migrations to a COE and to ensure that financial data can be shared across agency business systems, the Federal government must ensure greater standardization of business processes, interfaces, and data. To this end, two specific projects (with associated milestones) will be undertaken: Development of Standard Business Processes – This project will result in government- wide common business rules, data components, and policies for funds control, accounts payable, accounts receivable, and fixed assets.

Creation of a Common Government-wide Accounting Code – This project will result in a uniform accounting code structure, layout, and definitions.

Once established, all agencies will be expected to adopt these common processes on a schedule agreed upon between the agency and OMB. See Attachment 2 for additional details on the priority projects related to the transparency and standardization initiatives described above.

Stage 2: Competitive Environment and Seamless Data Integration. In order to enable improved performance of financial systems, the FMLOB envisions more competitive alternatives for financial systems and an environment where financial data can be more easily compared and aggregated across agencies.

Competitive Environment: To enable improved cost, quality, and performance of financial systems, Federal agencies must have competitive options available for financial systems. The COE framework is intended to help achieve these results. A COE is a shared service solution where a single entity provides financial management services for multiple organizations. When the FMLOB is successful, there will be a limited number of stable and high performing COEs that provide competitive alternatives for agencies investing in financial system modernizations. The economies of scale and skill of a COE will allow it to provide Federal agencies with a lower risk, lower cost, and increased service quality alternative for financial system modernization efforts.

Notably, a competitive environment is sustainable if Federal agencies have the ability to migrate from one solution to a more competitive or better performing alternative that is offered. The transparency and standardization efforts described above will lay the foundation for facilitating better portability of agency systems from one solution to another.

Seamless Data Integration: The standardization efforts, associated with Stage 1 of the FMLOB initiative, will enable financial data to be easily compared and aggregated across agencies. For 3 example, the development of a common government-wide accounting code will assist in the intra-governmental reconciliation process by requiring that all common types of financial data be accounted for in a similar format. A common structure will also enable easier transmission of financial reports to OMB and Treasury and assist these central agencies with aggregating similar-type data on a government-wide basis. Seamless and standardized data exchange will enable the government to streamline operations through more efficient information management and increased data accuracy.

In addition, seamless data integration will reduce the costs and risks of establishing interfaces between agency business systems. By requiring standard core business processes, rules, data definitions, and a common government-wide accounting code, interfacing systems, such as travel, will not have to be specifically designed for each agency. This will save agencies money and enable them to more easily migrate between different system solutions.

Stage 3: Results. When the FMLOB is fully realized, agencies’ data will be more timely and accurate for decision-making and there will be improved government-wide stewardship and accounting. More timely and accurate data will result from the standardization and seamless data integration efforts, including the implementation of centralized interfaces between core financial systems and other systems. These efforts will focus on promoting strong internal controls and ensuring the integrity of accounting data. The easy exchange of data between federal agencies will increase federal managers’ stewardship abilities.

There will also be a reduction of government-wide information technology costs and risks. These benefits will be the result of shared-service solutions, also assisted by the standardization and seamless data integration efforts. Shared-service solutions will enable economies of scale by centrally locating, or consolidating, solution assets and reusing Federal and commercial subject matter expertise through common acquisitions, interface development, and application management. The reduction in the number of agencies implementing their own systems will reduce the risks, and associated costs, of systems implementations. 4. What governance structure will be in place to ensure accountability for successful completion of priority FMLOB initiatives?

As depicted in Attachment 3, FSIO will have direct responsibility for completing priority projects under the FMLOB. OMB, in consultation with the Financial Systems Integration Committee (FSIC) of the CFO Council, will provide oversight and guidance to FSIO on priorities and expected performance in meeting these priorities.

OMB will continue its role as Executive Sponsors of the FMLOB. The FSIC chair will be the lead agency sponsor for the FMLOB. A liaison from the CIO community and the Executive Director of FSIO will serve on the FSIC and support the FSIC chair in his/her responsibilities as they relate to the FMLOB. Going forward, FSIO will coordinate the collection and expenditure of FMLOB funds. 4 The FSIC will assist OMB in evaluating and monitoring FSIO’s progress in completing FMLOB projects and provide feedback to OMB and FSIO. As appropriate, members of the FSIC will participate in working groups to assist FSIO with completing deliverables. The FSIC will evaluate its current subcommittee structure to assess whether changes are needed to best meet these objectives.

The updated governance structure ensures that the FSIO, FMLOB, and the FSIC do not operate in separate stovepipes. In addition, responsibility for work products will now rest with FSIO, where full time dedicated staff will be held accountable for achieving FMLOB milestones. 5. What is the status of the realignment of JFMIP to FSIO?

In December of 2004, the JFMIP Principals voted to modify the roles and responsibilities of the JFMIP Program Office, now FSIO. As a result, OMB and the FSIC were given an increased management and oversight role in the activities of FSIO. OMB and the FSIC have worked closely with FSIO staff to update FSIO’s mission statement and define FSIO’s scope of activities and priorities for FY 2006.

In terms of mission and scope, FSIO has three major areas of responsibilities: (a) continuing its primary role of core financial system requirements development, testing, and certification; (b) providing support to the Federal financial community by taking on special priority projects as determined by the OMB Controller, CFO Council, and the FSIO Executive Director, and (c) conducting outreach through the annual financial management conference and other related activities.

Most importantly, the projects that FSIO undertakes will directly reflect the priorities of the CFO Community and OMB. As noted above, the priority projects to be undertaken in the near term will relate to the transparency and standardization initiatives of the FMLOB. Other projects that were previously under FSIO’s purview – acquisition, budget formulation, and property system requirements – have been transitioned to the Chief Acquisition Council, the Budget Officers Advisory Council, and the Federal Real Property Council, respectively, for their consideration and completion.

Also, effective January 2006, the FSIO office will be transferred from the General Service Administration’s (GSA) Office of the Chief Financial Officer to the Office of Government-wide Policy, Office of Technology Strategy (OTS). There are several significant benefits of this move: lower administrative cost through shared resources (rent, supplies, equipment, etc.) permanent SES in place to provide leadership to FSIO staff access to immediate resources and expertise on IT, administrative management, contract management, testing, etc. fits well with current mission and stakeholder focused model of OTS 5 6. What specific actions are expected of Federal agencies?

As described above, a central goal of the FMLOB is that financial system investments will be at lower risk and lower cost as agencies leverage the economies offered by shared service solutions (i.e., COEs). To this end, OMB has instituted a policy that agencies seeking to modernize their financial system must either be designated a public COE or must migrate to a COE (public, private, or a combination of both). Although exceptions to this policy will be made in limited situations when an agency demonstrates compelling evidence of a best value and lower risk alternative, it is OMB’s intent to avoid investments in “in-house” solutions wherever possible so that the shared service framework can fully achieve potential and anticipated returns.

To the extent we require any specific action on your part to carry out the priority initiatives and milestones outlined above, we will communicate such requests through subsequent memos from OMB or the FSIC.

Performance Measures Performance Measures Performance Measures Performance Measures Performance Measures Performance Measures Project Description: Develop a Project Description: Develop a Project Description: Develop a Project Description: Develop a Project Description: Develop a Project Description: Develop a series of measures (i.e., cost, series of measures (i.e., cost, series of measures (i.e., cost, series of measures (i.e., cost, series of measures (i.e., cost, series of measures (i.e., cost, quality, timeliness) to assess the quality, timeliness) to assess the quality, timeliness) to assess the quality, timeliness) to assess the quality, timeliness) to assess the quality, timeliness) to assess the performance of those services. performance of those services. performance of those services. performance of those services. performance of those services. performance of those services. Measures will address the Measures will address the Measures will address the Measures will address the Measures will address the Measures will address the major service areas offered by a major service areas offered by a major service areas offered by a major service areas offered by a major service areas offered by a major service areas offered by a financial service organization financial service organization financial service organization financial service organization financial service organization financial service organization (e.g., Center of Excellence). (e.g., Center of Excellence). (e.g., Center of Excellence). (e.g., Center of Excellence). (e.g., Center of Excellence). (e.g., Center of Excellence). For each measure, identify For each measure, identify For each measure, identify For each measure, identify For each measure, identify For each measure, identify corresponding benchmarks for corresponding benchmarks for corresponding benchmarks for corresponding benchmarks for corresponding benchmarks for corresponding benchmarks for equivalent services (either equivalent services (either equivalent services (either equivalent services (either equivalent services (either equivalent services (either internal or outsourced) offered by internal or outsourced) offered by internal or outsourced) offered by internal or outsourced) offered by internal or outsourced) offered by internal or outsourced) offered by public and private organizations. public and private organizations. public and private organizations. public and private organizations. public and private organizations. public and private organizations. In order to compare In order to compare In order to compare In order to compare In order to compare In order to compare performance across the Federal performance across the Federal performance across the Federal performance across the Federal performance across the Federal performance across the Federal government, it must be possible government, it must be possible government, it must be possible government, it must be possible government, it must be possible government, it must be possible to normalize the measures (e.g., to normalize the measures (e.g., to normalize the measures (e.g., to normalize the measures (e.g., to normalize the measures (e.g., to normalize the measures (e.g., cost per transaction, cycle-time). cost per transaction, cycle-time). cost per transaction, cycle-time). cost per transaction, cycle-time). cost per transaction, cycle-time). cost per transaction, cycle-time).

Common Accounting Code Common Accounting Code Common Accounting Code Common Accounting Code Common Accounting Code Common Accounting Code Project Description: Develop a Project Description: Develop a Project Description: Develop a Project Description: Develop a Project Description: Develop a Project Description: Develop a common accounting code common accounting code common accounting code common accounting code common accounting code common accounting code structure, including an applicable structure, including an applicable structure, including an applicable structure, including an applicable structure, including an applicable structure, including an applicable set of definitions, which all federal set of definitions, which all federal set of definitions, which all federal set of definitions, which all federal set of definitions, which all federal set of definitions, which all federal agencies’ new financial agencies’ new financial agencies’ new financial agencies’ new financial agencies’ new financial agencies’ new financial management systems must management systems must management systems must management systems must management systems must management systems must adhere. The common accounting adhere. The common accounting adhere. The common accounting adhere. The common accounting adhere. The common accounting adhere. The common accounting code structure will: code structure will: code structure will: code structure will: code structure will: code structure will: menu of services offered by menu of services offered by menu of services offered by menu of services offered by menu of services offered by menu of services offered by COEs; COEs; COEs; COEs; COEs; COEs; project plan templates; project plan templates; project plan templates; project plan templates; project plan templates; project plan templates; service level agreement service level agreement service level agreement service level agreement service level agreement service level agreement templates; templates; templates; templates; templates; templates; rules of engagement (i.e., relevant rules of engagement (i.e., relevant rules of engagement (i.e., relevant rules of engagement (i.e., relevant rules of engagement (i.e., relevant rules of engagement (i.e., relevant contractual and competitive criteria contractual and competitive criteria contractual and competitive criteria contractual and competitive criteria contractual and competitive criteria contractual and competitive criteria involved in migration) involved in migration) involved in migration) involved in migration) involved in migration) involved in migration) due diligence checklist (clarified due diligence checklist (clarified due diligence checklist (clarified due diligence checklist (clarified due diligence checklist (clarified due diligence checklist (clarified and updated); and updated); and updated); and updated); and updated); and updated); comparison of approaches on comparison of approaches on comparison of approaches on comparison of approaches on comparison of approaches on comparison of approaches on public vs. private solutions; public vs. private solutions; public vs. private solutions; public vs. private solutions; public vs. private solutions; public vs. private solutions; migration risk areas (interfaces, migration risk areas (interfaces, migration risk areas (interfaces, migration risk areas (interfaces, migration risk areas (interfaces, migration risk areas (interfaces, data conversion and clean up, data conversion and clean up, data conversion and clean up, data conversion and clean up, data conversion and clean up, data conversion and clean up, testing, financial reporting, change testing, financial reporting, change testing, financial reporting, change testing, financial reporting, change testing, financial reporting, change testing, financial reporting, change to business processes); and to business processes); and to business processes); and to business processes); and to business processes); and to business processes); and human capital planning. human capital planning. human capital planning. human capital planning. human capital planning. human capital planning.

Project Description: Develop Project Description: Develop Project Description: Develop Project Description: Develop Project Description: Develop Project Description: Develop a standard set of business a standard set of business a standard set of business a standard set of business a standard set of business a standard set of business practices for core financial practices for core financial practices for core financial practices for core financial practices for core financial practices for core financial management functions (funds, management functions (funds, management functions (funds, management functions (funds, management functions (funds, management functions (funds, payments, receipts, fixed payments, receipts, fixed payments, receipts, fixed payments, receipts, fixed payments, receipts, fixed payments, receipts, fixed assets) to be adopted by all assets) to be adopted by all assets) to be adopted by all assets) to be adopted by all assets) to be adopted by all assets) to be adopted by all federal agencies. The federal agencies. The federal agencies. The federal agencies. The federal agencies. The federal agencies. The document/model will include: document/model will include: document/model will include: document/model will include: document/model will include: document/model will include: sequenced activities for core sequenced activities for core sequenced activities for core sequenced activities for core sequenced activities for core sequenced activities for core business processes; business processes; business processes; business processes; business processes; business processes; data objects participating in a data objects participating in a data objects participating in a data objects participating in a data objects participating in a data objects participating in a business activity; business activity; business activity; business activity; business activity; business activity; relationships among the relationships among the relationships among the relationships among the relationships among the relationships among the objects as they exist in the objects as they exist in the objects as they exist in the objects as they exist in the objects as they exist in the objects as they exist in the actual business activities; actual business activities; actual business activities; actual business activities; actual business activities; actual business activities; data elements and definitions data elements and definitions data elements and definitions data elements and definitions data elements and definitions data elements and definitions stored about these objects; stored about these objects; stored about these objects; stored about these objects; stored about these objects; stored about these objects; and and and and and and business rules governing business rules governing business rules governing business rules governing business rules governing business rules governing these objects. these objects. these objects. these objects. these objects. these objects.

Appendix V: GAO Contacts and Staff Acknowledgments

GAO Contacts

Staff Acknowledgments

In addition to the contacts named above, Kay Daly, Assistant Director; Chris Martin, Senior-Level Technologist; Francine DelVecchio; Mike LaForge; and Chanetta Reed made key contributions to this report.

Related GAO Products

DOD Business Systems Modernization: Navy ERP Adherence to Best Business Practices Critical to Avoid Past Failures. GAO-05-858. Washington, D.C.: September 29, 2005.

Financial Management: Achieving FFMIA Compliance Continues to Challenge Agencies. GAO-05-881. Washington, D.C.: September 20, 2005.

Business Modernization: Some Progress Made toward Implementing GAO Recommendations Related to NASA’s Integrated Financial Management Program. GAO-05-799R. Washington, D.C.: September 9, 2005.

Business Systems Modernization: Internal Revenue Service’s Fiscal Year 2005 Expenditure Plan. GAO-05-774. Washington, D.C.: July 22, 2005.

DOD Business Systems Modernization: Long-standing Weaknesses in Enterprise Architecture Development Need to Be Addressed. GAO-05-702. Washington, D.C.: July 22, 2005.

Army Depot Maintenance: Ineffective Oversight of Depot Maintenance Operations and System Implementation Efforts. GAO-05-441. Washington, D.C.: June 30, 2005.

DOD Business Systems Modernization: Billions Being Invested without Adequate Oversight. GAO-05-381. Washington, D.C.: April 29, 2005.

Internal Revenue Service: Assessment of the Fiscal Year 2006 Budget Request. GAO-05-566. Washington, D.C.: April 27, 2005.

Information Technology: OMB Can Make More Effective Use of Its Investment Reviews. GAO-05-276. Washington, D.C.: April 15, 2005.

Information Technology: Customs Automated Commercial Environment Program Progressing, but Need for Management Improvements Continues. GAO-05-267. Washington, D.C.: March 14, 2005.

Office of Personnel Management: Retirement Systems Modernization Program Faces Numerous Challenges. GAO-05-237. Washington, D.C.: February 28, 2005.

DOD Systems Modernization: Management of Integrated Military Human Capital Program Needs Additional Improvements. GAO-05-189. Washington, D.C.: February 11, 2005.

Department of Defense: Further Actions Are Needed to Effectively Address Business Management Problems and Overcome Key Business Transformation Challenges. GAO-05-140T. Washington, D.C.: November 18, 2004.

Business Systems Modernization: IRS’s Fiscal Year 2004 Expenditure Plan. GAO-05-46. Washington, D.C.: November 17, 2004.

Financial Management: Improved Financial Systems Are Key to FFMIA Compliance. GAO-05-20. Washington, D.C.: October 1, 2004.

Financial Management Systems: HHS Faces Many Challenges in Implementing Its Unified Financial Management System. GAO-04- 1089T. Washington, D.C.: September 30, 2004.

Financial Management Systems: Lack of Disciplined Processes Puts Implementation of HHS’ Financial System at Risk. GAO-04-1008. Washington, D.C.: September 23, 2004.

Information Technology: DOD’s Acquisition Policies and Guidance Need to Incorporate Additional Best Practices and Controls. GAO-04-722. Washington, D.C.: July 30, 2004.

Department of Defense: Financial and Business Management Transformation Hindered by Long-standing Problems. GAO-04-941T. Washington, D.C.: July 8, 2004.

Information Security: Agencies Need to Implement Consistent Processes in Authorizing Systems for Operation. GAO-04-376. Washington, D.C.: June 28, 2004.

DOD Business Systems Modernization: Billions Continue to Be Invested with Inadequate Management Oversight and Accountability. GAO-04- 615. Washington, D.C.: May 27, 2004.

Information Technology: The Federal Enterprise Architecture and Agencies’ Enterprise Architectures Are Still Maturing. GAO-04-798T. Washington, D.C.: May 19, 2004.

National Aeronautics and Space Administration: Significant Actions Needed to Address Long-standing Financial Management Problems. GAO-04-754T. Washington, D.C.: May 19, 2004.

DOD Business Systems Modernization: Limited Progress in Development of Business Enterprise Architecture and Oversight of Information Technology Investments. GAO-04-731R. Washington, D.C.: May 17, 2004.

Information Technology: Early Releases of Customs Trade System Operating, but Pattern of Cost and Schedule Problems Needs to Be Addressed. GAO-04-719. Washington, D.C.: May 14, 2004.

Information Technology Investment Management: A Framework for Assessing and Improving Process Maturity (Version 1.1). GAO-04-394G. Washington, D.C.: March 2004.

Information Technology Management: Governmentwide Strategic Planning, Performance Measurement, and Investment Management Can Be Further Improved. GAO-04-49. Washington, D.C.: January 12, 2004.

Human Capital: Key Principles for Effective Strategic Workforce Planning. GAO-04-39. Washington, D.C.: December 11, 2003.

Business Modernization: NASA’s Challenges in Managing Its Integrated Financial Management Program. GAO-04-255. Washington, D.C.: November 21, 2003.

Business Modernization: NASA’s Integrated Financial Management Program Does Not Fully Address Agency’s External Reporting Issues. GAO-04-151. Washington, D.C.: November 21, 2003.

Business Modernization: Disciplined Processes Needed to Better Manage NASA’s Integrated Financial Management Program. GAO-04-118. Washington, D.C.: November 21, 2003.

Information Technology: Architecture Needed to Guide NASA’s Financial Management Modernization. GAO-04-43. Washington, D.C.: November 21, 2003.

Information Technology: Leadership Remains Key to Agencies Making Progress on Enterprise Architecture Efforts. GAO-04-40. Washington, D.C.: November 17, 2003.

DOD Business Systems Modernization: Important Progress Made to Develop Business Enterprise Architecture, but Much Work Remains. GAO-03-1018. Washington, D.C.: September 19, 2003.

Business Systems Modernization: IRS Has Made Significant Progress in Improving Its Management Controls, but Risks Remain. GAO-03-768. Washington, D.C.: June 27, 2003.

Business Modernization: Improvements Needed in Management of NASA’s Integrated Financial Management Program. GAO-03-507. Washington, D.C.: April 30, 2003.

Department of Housing and Urban Development: Status of Efforts to Implement an Integrated Financial Management System. GAO-03-447R. Washington, D.C.: April 9, 2003.

Information Technology: A Framework for Assessing and Improving Enterprise Architecture Management (Version 1.1). GAO-03-584G. Washington, D.C.: April 2003.

DOD Business Systems Modernization: Continued Investment in Key Accounting Systems Needs to be Justified. GAO-03-465. Washington, D.C.: March 28, 2003.

DOD Business Systems Modernization: Improvements to Enterprise Architecture Development and Implementation Efforts Needed. GAO-03- 458. Washington, D.C.: February 28, 2003.

Customs Service Modernization: Automated Commercial Environment Progressing, but Further Acquisition Management Improvements Needed. GAO-03-406. Washington, D.C.: February 28, 2003.

Customs Service Modernization: Third Expenditure Plan Meets Legislative Conditions, but Cost Estimating Improvements Needed. GAO-02-908. Washington, D.C.: August 9, 2002.

DOD Financial Management: Important Steps Underway but Reform Will Require a Long-term Commitment. GAO-02-784T. Washington, D.C.: June 4, 2002.

Customs Service Modernization: Management Improvements Needed on High-Risk Automated Commercial Environment Project. GAO-02-545. Washington, D.C.: May 13, 2002.

Tax Administration: IRS Continues to Face Management Challenges in its Business Practices and Modernization Efforts. GAO-02-619T. Washington, D.C.: April 15, 2002.

Business Systems Modernization: IRS Needs to Better Balance Management Capacity with Systems Acquisition Workload. GAO-02-356. Washington, D.C.: February 28, 2002.

Human Capital: Building the Information Technology Workforce to Achieve Results. GAO-01-1007T. Washington, D.C.: July 31, 2001.

Business Systems Modernization: Results of Review of IRS’ March 2001 Expenditure Plan. GAO-01-716. Washington, D.C.: June 29, 2001.

Information Technology: DLA Should Strengthen Business Systems Modernization Architecture and Investment Activities. GAO-01-631. Washington, D.C.: June 29, 2001.

Customs Service Modernization: Results of Review of First Automated Commercial Environment Expenditure Plan. GAO-01-696. Washington, D.C.: June 5, 2001.

Information Technology: Architecture Needed to Guide Modernization of DOD’s Financial Operations. GAO-01-525. Washington, D.C.: May 17, 2001.

District of Columbia: Weaknesses in Financial Management System Implementation. GAO-01-489. Washington, D.C.: April 30, 2001.

Tax Systems Modernization: Results of Review of IRS’ Third Expenditure Plan. GAO-01-227. Washington, D.C.: January 22, 2001.

Tax Systems Modernization: Results of Review of IRS’ August 2000 Interim Spending Plan. GAO-01-91. Washington, D.C.: November 8, 2000.

Indian Trust Funds: Improvements Made in Acquisition of New Asset and Accounting System But Significant Risks Remain. GAO/AIMD-00- 259. Washington, D.C.: September 15, 2000.

Tax Systems Modernization: Results of Review of IRS’ March 7, 2000, Expenditure Plan. GAO/AIMD-00-175. Washington, D.C.: May 24, 2000.

Executive Guide: Creating Value Through World-class Financial Management. GAO/AIMD-00-134. Washington, D.C.: April 2000.

Indian Trust Funds: Interior Lacks Assurance That Trust Improvement Plan Will Be Effective. GAO/AIMD-99-53. Washington, D.C.: April 28, 1999.

Federal Information System Controls Audit Manual, Volume I: Financial Statement Audits. GAO/AIMD-12.19.6. Washington, D.C.: January 1999.

District of Columbia: Status of Efforts to Develop a New Financial Management System. GAO/AIMD-97-101R. Washington, D.C.: July 9, 1997.

Information Security: Opportunities for Improved OMB Oversight of Agency Practices. GAO/AIMD-96-110. Washington, D.C.: September 24, 1996.