District of Columbia: Weaknesses in Financial Management System Implementation
Released: 2001-04-30T00:00:00

Letter

Background

On June 21, 1995, we testified that the District’s financial records were inadequate and that the City did not have the most basic financial data, including the status of its expenditures against budgeted amounts, the amount of bills owed, or the balance of cash available. As a result, District managers did not have fundamental financial information necessary to help control spending and costs and to estimate budget and cash needs. Given the long-standing problems with the District’s financial management, we recommended that the Authority study the accounting and financial management information needs of the District.

Subsequently, the Authority and the District requested and the Congress approved funds to assess the need for implementing a new financial management system. The 1996 District of Columbia Appropriations Act (Public Law 104-134) authorized the District of Columbia government to spend $28 million of its revenues to implement a replacement for its existing financial management system. Of the $28 million, $2 million was provided for a needs analysis and assessment of the existing financial management environment. Public Law104-134 made the remaining $26 million available to procure the necessary hardware; install new software; and perform system conversion, testing, and training after the needs analysis and the assessment were received by the Congress.

The District is now in its fourth year of implementing its new financial management system. On September 4, 1997, the Authority awarded a 1- year contract with four option years potentially totaling $21 million to design and install a state-of-the-art financial management system for the District. The District began working with the contractor in September 1997 and piloted the new system at five agencies beginning February 1, 1998, with the goal of District-wide implementation of SOAR on October 1, 1998.

SOAR consists of commercial, off-the-shelf applications used by state and local governments. SOAR was expected to strengthen control over appropriations and spending plans; enhance tracking of grants and projects; automate and streamline the financial management process; record obligations as incurred; make and track payments and disbursements; monitor performance measures by program and organization; prepare timely, accurate, and reliable financial reports; expedite the month-end closing process; and provide the ability to input and control data on-line.

SOAR is the District’s central general ledger system and includes the five following components: The Relational Standard Accounting and Reporting System (R*STARS), the core accounting module, provides general ledger capabilities as well as budgetary control, cash management, expenditures/payables, revenue/receivables, and budget execution functions.

The Advanced Purchasing and Inventory Control System (ADPICS) is integrated with R*STARS and provides a comprehensive system of materials management encompassing requisition/purchase transactions, accounts payable, and inventory processing.

The Performance Budgeting module supports the development of operating and capital budgets and provides information on program costs and performance measures.

The Fixed Assets System supports accounting, management, and control over capital and controllable assets.

The Executive Information System (EIS) is a high-level analysis tool for program and financial management that enables data modeling, creates analyses for “what if” scenarios, and offers the flexibility to generate ad hoc reports.

In addition to SOAR, there are other critical feeder systems that make up the District’s financial management system. Figure 1 shows the various SOAR components within the District’s overall financial management system.

Scope and Methodology

To fulfill our objectives, we analyzed the District’s financial management systems project and program management plans, work breakdown implementation schedules, project cost tracking documents, contract records, meeting minutes, and briefing reports; reviewed the District’s budget formulation process, budget process manual, and the Government Finance Officers Association’s report on the District’s budget office organizational structure; reviewed reports issued by the Inspector General of the District; and reviewed audit reports and related documents describing financial management system implementation activities and weaknesses identified during financial statement audits of the District covering fiscal years 1995 through 2000.

We met with the following District personnel: Chief Financial Officer; Deputy CFO for the Office of Budget and Planning; Deputy CFO for the Office of Financial Operations and Systems; SOAR Program Management Office officials; Director, Mission Support, OCFO; Director, Information Systems Administration, Office of Tax and Revenue; Office of the Inspector General officials; Representatives from the Authority; agency CFOs and financial staff; and Office of Procurement staff.

We also met with contractors responsible for implementing the new system.

Our work was performed from September 2000 through February 2001 in accordance with generally accepted government auditing standards. We requested and obtained comments on a draft of this report from the Mayor of the District of Columbia and the District’s Chief Financial Officer. These comments are discussed in a later section of this report and reprinted in appendix II.

Major Components of the District’s System of Accounting and Reporting Are Not Operational

Financial management requires that financial and program managers be accountable for financial results of actions taken, control over the government’s financial resources, and protection of assets. The District’s Office of the Chief Financial Officer (OCFO) has responsibility for effective financial management in the District. In order to meet the above requirements, financial management systems must be in place to process and record financial events effectively and efficiently and to provide complete, timely, reliable, and consistent information for decisionmakers and the public.

Over the past several years, the District has undertaken a number of initiatives designed to improve its financial management environment. However, a number of key initiatives have still not been completed or have been placed on hold, and some are still being revised. For example, the District has yet to complete implementation of SOAR, the personnel and payroll system, the procurement system, and the tax system. The District is also in the process of reengineering its budget process before deciding whether to implement a new budget process and fully integrate the budget data within SOAR. In addition, the District has not yet implemented the fixed asset module of SOAR.

The original implementation schedule indicated that SOAR would be fully implemented by September 30, 1998, and the external feeder system and the related interfaces were scheduled for completion by April 1999. The SOAR implementation, however, has been marked by delays. As we noted in our April 1998 report, we found severe weaknesses in critical implementation processes, including lack of requirements development and project management. For example, the District did not have an organizational policy for establishing and managing software-related requirements and no clear assignment of responsibility for requirements development and management. As a result, the District did not have assurance that the new financial management system or any other software acquisition project undertaken would be conducted in a disciplined manner. Further, studies have shown that problems associated with requirements management are key factors in software projects that do not meet their cost, schedule, and performance goals and that these systems can cost many times more than expected if done improperly.

Performance Budgeting Module Put on Hold

The Performance Budgeting module included in SOAR was expected to facilitate the management of the entire budget formulation process, from budget submission to final review and approval. One of the primary objectives of the Performance Budgeting module is the automation of the annual budget development process to replace the present cumbersome, manual process. The module was also expected to provide financial reporting at various program levels as needed and contains a performance measures feature to capture information for performance measures. District officials anticipated that the performance measures feature in the module would be able to provide comparative and cost information for all levels of programs throughout the District. Comparative and accurate cost information would enable stakeholders to make more informed decisions, eventually providing better service to the citizens of the District. District officials anticipated that performance measures, once they were developed for programs in the District, could be maintained and tracked easily and accurately in the module.

At the time of our review, the District had decided to suspend the implementation of the Performance Budgeting module. District officials told us that in addition to the need for reengineering the budget process, the current design of the Performance Budgeting system does not fully support the District’s information needs. Specifically, the Performance Budgeting module is not a fully integrated product of the core accounting system and will require multiple and frequent updates to the two systems in order to maintain a fully integrated system as updates or upgrades occur to one or both systems; the current, nonintegrated design of the Performance Budgeting module limits both agency budget staff and the Office of Budget and Planning (OBP) from fully using information from the core accounting system; and inadequate reporting tools within the Performance Budgeting module prevent OBP from generating essential budget reports on a timely basis to the stakeholders.

In addition to the above financial system issues, District officials stated that use of the Performance Budgeting module at this time presents a number of other technical support system challenges and the current condition of the District’s financial support systems needs to be stabilized before implementation. For example, The lack of a stable unified Wide Area Network (WAN) precludes District agencies from accessing the system and developing their fiscal year 2002 budgets. However, according to the SOAR Project Plan, the necessary infrastructure analysis was completed on January 6, 1998.

Current client server resources prevent agency budget staff from concurrently accessing the Performance Budgeting module during peak budget season, which can result in users being locked out or knocked off the system.

In addition to the above factors, further review by OBP has resulted in a decision by the District to reengineer the budget formulation process and develop a requirements definition for the new process and then select a new software solution to deliver the results.

Fixed Assets Module Has Not Been Implemented

The Fixed Assets module was intended to track the acquisition, transfer, disposition, and maintenance of the District’s capitalized assets (such as personal property, equipment, and buildings) and support accounting, management, and control over these assets, which totaled a reported $3.1 billion in fiscal year 2000. The District’s implementation schedule for this module has continually been revised. For example, originally scheduled to be implemented in February 1999, the Fixed Assets module is now planned to be implemented by the end of fiscal year 2001—over 2 years after the initially planned implementation. According to the SOAR Program Management Office (PMO), the delay was the result of multiple competing priorities such as the Year 2000 conversion and the production of the Comprehensive Annual Financial Report (CAFR) combined with the problem of limited personnel resources. Currently, fixed assets are recorded by each agency using a variety of methods, including manually updated ledger books and off-line automated tools. Such methods, however, are error-prone and could lead to incorrect recording and reporting of assets. Once the Fixed Assets module is implemented, according to District officials, all District agencies will use this same tool to account for assets. However, the SOAR PMO does not have a documented comprehensive plan for managing the implementation of the Fixed Assets module.

Significant Related Financial Systems Are Not Fully Operational

Although the financial management system was scheduled to become fully integrated in April 1999, significant external feeder systems, including personnel and payroll functions, the procurement system, and the tax system, are not fully integrated with SOAR as originally planned because the feeder systems have not been completed.

In addition to the feeder systems not being completed, current monitoring of interface development is not documented. For example, we asked the SOAR PMO for an update on the status of the interface development and we were given the most recent interface status report, which was dated September 14, 1998. The SOAR PMO was unable to provide an explanation as to why the status report was not being updated more frequently even though work was ongoing. In our July 1997 report, we raised concerns about the District’s failure to focus more broadly on its financial requirements, such as those stemming from the need to integrate SOAR with feeder systems. We noted that the District had not defined how the interfaces would work or what data would be provided for each feeder system. In addition, as we reported in October 1997, the District’s time frames for implementing its systems seemed to be ambitious in light of the complex nature of the District’s financial management structure and the lack of identified and confirmed requirements for several key systems, such as the feeder systems.

Weaknesses and Uncertainties Surround Personnel/Payroll System

The District acquired and developed a new personnel and payroll system in order to improve the quality of its business processes and to replace an aging legacy system. As we noted in a 1995 testimony, personnel information on the District’s 40,000 employees has long been error-prone and inconsistent. Beginning in 1991, the District created an action plan to acquire an automated human resources management information system, called the Comprehensive Automated Personnel and Payroll System (CAPPS). CAPPS was estimated to cost about $13 million to develop and was expected to be deployed by December 1999. The District had anticipated that CAPPS would provide more robust human resources capability than the prior legacy system, such as providing on-line funding data at the agency level, budgetary and spending controls at the position level, and accurate accounting of expenses, such as overtime.

However, as we reported in December 1999, the District did not effectively plan for CAPPS. We noted that the District did not develop a project management plan and a risk management plan; obtain agreement from the acquisition team, system users, and the contractor on detailed requirements for CAPPS; or establish a configuration control process to control the changes that were made to data tables connected to the software package that the District acquired for CAPPS. By not implementing these critical management processes, the District lacked the means to establish realistic time frames for CAPPS, track development along those time frames, and ensure that changes being made to CAPPS were consistent and in line with business requirements. In fact, since beginning the CAPPS initiative in 1991, the District has had to continually revise its CAPPS implementation deadline.

As a result of these delays, some District agencies are using the CAPPS system, while others continue to use the old personnel and payroll system, the Unified Personnel and Payroll System (UPPS), until a plan for a new payroll processing system can be developed. DC Public Schools, the Fire Department, and a few smaller agencies are using CAPPS to process payroll, while the remaining District agencies process their payroll through UPPS. Relying on dual systems in this manner leads to lack of standardization and creates unnecessary effort and inefficiencies. In addition, CAPPS is not electronically integrated with SOAR, and both CAPPS and UPPS data must undergo a conversion process before interfacing with SOAR, creating further inefficiencies in processing and reporting payroll-related costs.

Furthermore, the District is in the process of determining whether it will continue with CAPPS or whether an entirely new system is needed. In an August 11, 2000, letter to the Mayor, the District CFO stated that “the new CAPPS system, for a number of reasons, is compromised beyond repair. While it continues to pay people accurately, it has been customized to the point that its basic architecture has been destroyed. Underlying calculations necessary to make retirement computations and W-2s are likely fatally compromised.” This combination of weaknesses and uncertainties surrounding CAPPS further call into question the District’s ability to resolve implementation issues and its ability to pay its employees accurately and on time and account for their retirement and benefits.

Integrated Tax System Is Not Fully Implemented

According to a District official, the Integrated Tax System (ITS) is intended to be a complete reengineering of the Office of Tax and Revenue’s (OTR) business process, at an estimated cost of about $63 million. It came about because of serious concerns related to business processes, collection of delinquent accounts, tax compliance/discovery, data purification, work flow management, and the integration of revenue management with other key governmental functions (for example, unemployment compensation, business registration and regulation, and child support enforcement).

The old system consisted of a stand-alone, nonintegrated system for each major tax category—business, real property, and individual. For example, if a taxpayer was due a refund from an individual tax return but owed property taxes, there was no linkage under the old structure that would allow these two tax systems to interact to offset one another. District officials told us that the Integrated Tax System will allow the District to integrate all tax types under one system. On November 13, 2000, the District’s Business Tax module became operational and interfaced with SOAR. The real property and individual tax modules are expected to be completed and integrated with SOAR in January 2002. Until the real property and individual tax modules are implemented, the District will continue the cumbersome practice of manually entering data for property and individual taxes into SOAR.

The District Relies on Two Procurement Systems

In July 2000, the District’s Inspector General reported that the District was using two distinctly different procurement systems. The Office of Contracts and Procurement (OCP) purchased PRISM/OCP Express as its procurement package, while the CFO uses ADPICS for procurement transactions. As a result, various offices rely on different systems to process procurements. OCP has spent at least $14 million since it began the process of implementing a new procurement system 13 years ago. A July 2000 DC Office of Inspector General report stated that PRISM/OCP Express and SOAR had interface problems and that procurement data were being maintained in both systems. Furthermore, reports generated with procurement data must be developed with the coordination of the responsible agencies and after reconciliation of data from both systems. According to a District official, this situation makes it difficult for them to track procurements and payments, and the use of both systems has produced inefficiencies, duplication, and waste within the District. An OCP official told us that they recently hired a contractor to review OCP Express capabilities and determine what modifications, if any, could be made to enhance its functionality. According to District officials, until the District completes its assessment of the procurement system, it plans to continue using ADPICS to process procurement transactions through SOAR as ADPICS is the only available mechanism for entering procurement transactions into SOAR.

Computer Security Weaknesses Place Information at Risk

We also reported in our January 31, 2001, report that serious and pervasive computer security weaknesses place the DC Highway Trust Fund and other District financial, payroll, personnel, and tax information at risk of inadvertent or deliberate misuse, fraudulent use, and unauthorized alteration or destruction without detection. A primary reason for the District’s information system control problems was that it did not have a comprehensive computer security planning and management program. An effective program includes guidance and procedures for assessing risks, establishing appropriate policies and related controls, raising awareness of prevailing risks and mitigating controls, and evaluating the effectiveness of established controls. Such a program, if implemented effectively, would provide the District with a solid foundation for resolving existing computer security problems and managing its information security risks. District management stated that it has recognized the seriousness of the weaknesses we identified and expressed its commitment to improving information system controls.

Status of District Efforts to Improve Its Budget Process

As discussed earlier, the District has placed the implementation of the Performance Budgeting system module on hold. A District official told us that it needs to conduct business process reengineering of the budget process before making a decision on the appropriate solution for its budget and related reporting needs. In the meantime, the District continues to rely on a manual, cumbersome process each year to develop the budget.

District budget officials have told us that a number of improvements have been made to the budget process for fiscal year 2002 that they believe will address some of the problems they faced in developing the fiscal year 2001 budget. Table 1 compares the fiscal year 2001 budget process and results to the planned fiscal year 2002 budget process, modifications, and expected results, as provided by the District CFO. However, the District will enter its budget formulation process for fiscal year 2002 without an implemented financial system for gathering and formulating its budget data. Furthermore, the District will not have adequate program-level cost and budget results data for fiscal year 2001 for use in formulating its fiscal year 2002 budget.

Because the District was in the early stages of budget formulation, we were not able to assess whether these improvements will achieve their intended results. Further, the District was unable to provide us with any documentation to support that it had undertaken a structured and disciplined approach to implementing these actions.

In order to address its budget processes and systems, the District contracted with the Government Finance Officers Association (GFOA), a leader in state and local budgeting and finance, to help evaluate how well OBP was structured to carry out its budget and financial functions. The report, which GFOA issued on November 2, 2000, focused on an organizational review of the budget office internally and as it related to other District agencies.

GFOA found that OBP faced organizational and personnel management issues. For example, the study cited the following issues: organizationally, there is little communication or coordination between OBP divisions during major budget cycle periods; OBP lacks clear organizational and management policies regarding budget development and execution; staff need training on the current financial management system and the acquisition of analytical tools to perform financial analyses; and OBP staff have not had fiscal analyses sufficiently incorporated into their typical duties, such as expenditure and revenue analyses, cost-benefit analyses, and program outcome analyses. OBP officials stated that, as a result of the GFOA recommendations, they have made a number of staffing and organizational changes, which were consistent with many of the initiatives for change they had already started. OBP officials stated that they have taken the following actions: created the position of Chief Operating Officer to increase overall program efficiency and accountability in three programs: data management, organization management, and communications; realigned the Associate Deputy CFO (ADCFO) position to sharpen OBP’s quality of forecasting and long-term fiscal planning ability with ADCFO assuming responsibility for the functions of economic analysis, budget execution and reporting, and legislative affairs, which are three new branches; assigned the Economic Analysis Branch the lead on improving performance-based budgeting for the District; designated the Legislative Affairs Branch as the leading provider of legislative support to OCFO and other stakeholders, providing legislative, legal, and policy analysis to ensure that the goals of District stakeholders are achieved; gave the Organizational Management Branch responsibility for supervising and coordinating general office operations and building morale and reducing attrition through reform, orientation, recruitment, outreach efforts, training, and career development; and reorganized the Operating and Capital Budget divisions, which under the old structure functioned independently of each other with little or no collaboration or interaction, into a two-pronged division that strategically links operating and capital budget operations.

According to District OBP officials, the above changes have improved budget forecasting activities as well as provided greater control over budget execution and fiscal oversight.

We agree that OBP needed to address personnel and related organizational issues in order to better align its operations and facilitate and enhance its ability to carry out its budgetary and financial responsibilities. However, because these efforts were only in the early stages of implementation, we could not assess their impact on OBP’s operations or on its ability to successfully implement a budgeting system in the future.

Until the District develops a disciplined and structured approach to its business process reengineering efforts for its budget process, it will continue to develop its budget using a process that is cumbersome and inefficient. Until a budget formulation and execution system is implemented and fully integrated with its financial systems, the District’s budget is not likely to reflect the cost of services at the program level because the District currently does not have a way of measuring those costs.

SOAR User Expectations Have Not Been Fully Met

As originally envisioned, SOAR was expected to provide general ledger, grants management, fixed assets management, budget execution, cash management, and budget formulation functionality. SOAR was expected to strengthen control over appropriations and spending, provide enhanced tracking of grants and projects, automate and streamline the financial management process, record obligations as incurred, track payments and disbursements, monitor performance measures by program and organization, prepare timely, accurate, and reliable financial reports, expedite the month-end closing process, and provide the ability to input and control data on-line. Overall, the District and its residents were expected to benefit from improved financial management and reporting of public services and resources. Our discussions with SOAR pilot agencies indicate that these expectations have not been realized.

SOAR Currently Does Not Meet Financial Information Needs

Five District agencies were used as pilot agencies during the systems implementation— the District of Columbia Public Schools (DCPS), Department of Public Works (DPW), Department of Human Services (DHS), Metropolitan Police Department (MPD), and Financial and Technical Services. The pilot program provided an opportunity for agencies with unique requirements to customize the implementation of SOAR in their agencies. The pilot program also provided a test and “exercise” of the implementation process as well as a test of the function of the new system. The pilot was designed to identify problems and develop solutions prior to full implementation.

We contacted each pilot agency to obtain officials’ views on SOAR’s current operational status. In conjunction with our discussion with the pilot agencies, we reviewed the expectations communicated in a December 1997 presentation in which the OCFO provided a detailed list of the former financial management system’s deficiencies along with the anticipated capabilities of the new financial management system that were expected to remedy these deficiencies. The following are examples of current operational issues identified by officials at the pilot agencies where anticipated resolutions did not materialize.

All five pilot agencies reported that they need more recent data to more fully use the SOAR EIS, a high-level management tool capable of generating ad hoc reports. Currently, data in EIS is updated weekly, even though the expectation for the new system was that users would have “real time, on-line information.”

Four of the five pilot agencies indicated that the lack of full integration between the core accounting system and feeder systems is a problem. For example, two agencies said that they have to spend extra time reconciling payroll transactions between CAPPS and SOAR in order to ensure that the payroll data in both systems is accurate and complete.

Four of the five pilot agencies indicated that the help desk facility needs improvement because it does not provide adequate assistance.

Four of the five pilot agencies indicated that the SOAR training was not tailored to their specific needs.

Three out of the five pilot agencies indicated that they need enhanced project costing capabilities. According to one agency official, SOAR does not provide cost information on specific programs or activities. This results in agencies having to maintain information outside the system in an attempt to track program costs.

Our work as well as other studies have shown that problems associated with requirements management are key factors in software projects that do not meet their cost, schedule, and performance goals. By not clearly identifying and defining user requirements up front, the financial management system is currently not able to fulfill the financial management and reporting needs of its users.

SOAR Training Was Not Comprehensive

Training is a critical component of successful implementation of a new financial management system and can be accomplished through a formal program, on-the-job training, and the use of experts assigned to each agency. We previously reported that from January 1998 through April 1999, 42 percent of SOAR users did not attend scheduled training. In December 2000, the SOAR PMO identified a core training curriculum consisting of nine courses. According to the SOAR PMO, less than 50 percent of the SOAR user community had completed the new core training curriculum.

The SOAR Program Director told us that conflicting priorities contributed to the low attendance rate, including the large amount of effort needed to complete the District’s annual financial statements and prepare for its annual audit. According to the same official, another factor was the lack of complete buy-in by the users of the new system. In addition, a District official stated that many reported transaction errors resulted from a lack of understanding of the impact of transactions and their effect on general ledger accounts, coupled with the learning curve experienced with the implementation of the new system. Furthermore, several pilot agency officials stated that the overall training was generic and not specific to the District’s needs.

In our June 1999 report, we stated that the District planned to pilot a job certification program for employees in financial positions. Under this program, employees would be certified for financial positions based on training and testing. In November 2000, the SOAR PMO told us that they are still reviewing the details of implementing a certification program. The SOAR PMO also said that a comprehensive training plan for financial management personnel for fiscal year 2001 does not exist. The SOAR Steering Committee had determined that there was a need for more District user access to individuals with enhanced SOAR expertise.

At the same time, the District is in the process of implementing a new program called the “Super Users” program. The goal of the program is to develop a team of “super users,” individuals with advanced SOAR skills, to serve as mentors and providers of on-the-job training to users. District officials said they had recently selected eight individuals for this program, and a recruitment effort is underway to identify eight more individuals for the program.

SOAR Does Not Capture Cost of Specific Programs

The current system does not have the capability to capture the costs of specific District programs or activities. Project cost accounting is important in determining whether specific programs or activities are achieving their goals within budget. To compensate for the lack of a project cost accounting capability, agencies are capturing and maintaining information outside SOAR manually or by using other software applications. According to District officials, there are no plans to implement a separate module, such as activity-based costing. Comparative and accurate cost information would enable stakeholders to make more informed decisions, which would, in turn, provide better service to the citizens of the District through improved service delivery. Currently, various District activities, such as the cost of trash collections, motor vehicle inspection, clinical care, and street repairs, must be calculated manually, which is inefficient, time-consuming, and prone to error. District officials anticipated that the performance measures featured in the performance budget module would be able to provide comparative and cost information for all levels of programs throughout the District. However, it is unlikely that the District will have a solution for its program cost needs until after a budgeting system is implemented and integrated with SOAR.

SOAR Project Costs Have Significantly Exceeded Original Estimates

In September 1997, the District planned to spend $26 million (plus related costs for personnel and space) to implement SOAR. According to the Deputy CFO, as of March 8, 2001—almost 3-1/2 years later—the actual cost had climbed to about $41 million, an increase of about $15 million.

According to the PMO, one of the reasons for the increased costs was the need to provide knowledge transfer (this refers to agency-specific implementation assistance, enhanced training, and help desk operations) at the major agencies and transition assistance, which were not originally included in the contract. According to another District official, the increase in implementation costs also resulted from the District initially not completely understanding user requirements. Consequently, the District found it necessary to implement additional requirements after the fact. For example, after the SOAR contract was awarded, the University of the District of Columbia (UDC) determined that SOAR could satisfy its requirements for higher education accounting and opted to install SOAR, rather than a separate new system, at a cost of $1 million. Also, according to a District official, the District underestimated the level of support and time required to implement SOAR. Accordingly, the District contracted for additional implementation support. Further, additional District users required access to EIS and thus required the purchase of additional EIS licenses, licenses which cost the District approximately $900,000.

District Did Not Follow a Structured Approach

Acquiring an effective information system must begin with a clear definition of the organization’s mission and strategic goals. To assure a solid foundation for the District’s system, we offered a structured approach in our July 1997 report based on three building blocks especially important in the early stages of a project: concept of operations, requirements definition, and alternatives analysis. A concept of operations— the panorama of a system’s purpose and the environment in which it will function—is the basis on which specific requirements— functions the system must be able to perform—are developed. With a complete set of well-defined requirements based on a clear concept of operations, District leaders could make an accurate analysis of how well available alternatives would meet the needs of the government and its citizens.

In April 1998, after the District had contracted for and begun development of its new system, a more detailed study led us again to the conclusion that the District’s process, while strong in some areas, was undisciplined and immature. Our conclusion was based on a well- recognized model of software process assessment, the Software Engineering Institute’s Software Acquisition Capability Maturity Model (SA-CMM). In our recommendations, we stressed the need for written policy and documentation, the development of a management plan, the assignment of responsibility for areas of planning and development, and requirements development in the key process areas of software acquisition planning, requirements development and management, project management, contract tracking and oversight, evaluation, and acquisition risk management.

Although the processes spelled out in the SA-CMM model and our recommendations are detailed, rigorous, and time-consuming, they are, in the long run, cost-effective and vital elements of success. Unfortunately, in its efforts to meet an overly ambitious time schedule, the District has spent considerably more money than planned to acquire a system that—6 years after we began our review and started making recommendations— now serves as yet another cautionary example of the risks entities run when they choose to short-cut a disciplined approach to the planning, acquisition, and management of a financial management system.

Another key component of successful financial management for the District is conducting a comprehensive assessment of its human capital needs for its financial management functions. In our Executive Guide: Creating Value Through World-class Financial Management, we outlined three critical elements for developing first-rate financial management staff teams. These elements were: (1) determining required skills and competencies, (2) measuring the gap between what the organization needs and what it has, and (3) developing strategies and detailed plans to address current or expected future deficiencies. We reported that having staff with appropriate skills is key to achieving financial management improvements, and managing an organization’s employees is essential to achieving results. Entities that focus on valuing employees and aligning their policies to support organizational performance goals start with a human capital assessment. The results of such an assessment could help to determine the resources needed to successfully implement financial management improvements. And, as we reported in September 2000,performing a self-assessment of human capital needs helps organization leaders understand the strengths and limitations of their human capital information systems. These data can help an organization develop a profile of its human capital. Further, because human capital information can spotlight areas of concern before they develop into crises, gathering these data is an indispensable part of effective risk management. Without a formal assessment of its requirements and needs, and a strategy for addressing them, the District’s efforts can become piecemeal, incomplete, and ineffective. The challenges the District already faces in implementing its financial management system could be exacerbated by its lack of a human capital assessment for its financial management functions. By not identifying staff with the requisite skills to implement such a system and by not identifying gaps in needed skills and filling them, the District has reduced its chances for successful implementation.

Most of Our Prior Recommendations Related to the Financial Management System Have Not Been Implemented

In reports dating back to 1995, we have reported on the District’s financial situation and efforts to improve its financial management, including the implementation of SOAR. For example, in 1995, we recommended that the Mayor clean up existing data in the financial systems and place special emphasis on ensuring that basic accounting policies and procedures are followed and that the Authority study the accounting and financial management information needs of the District. In a July 9, 1997, status report, we noted that the District had not completed three key elements of its system acquisition: concept of operations, requirements definition, and alternatives analysis. We also noted that the time frames for completing several of these important tasks were unknown and that the District had adopted an overly ambitious schedule.

On April 30, 1998, we reported that the District’s software acquisition processes, while having some strengths, were not mature when compared to standards established by the Software Engineering Institute. Weaknesses noted in the report included: (1) requirements definition problems; (2) project management and oversight weaknesses; (3) lack of an effective evaluation process; (4) lack of a formal risk management process; and (5) failure to meet some milestones. In that report, we made six recommendations for strengthening the processes that relate to the SOAR project and improving future software acquisition efforts. As discussed earlier, in our December 1999 report on the CAPPS system, we noted that by not implementing critical management processes, the District lacked the means to establish realistic time frames for CAPPS, track development along those time frames, and ensure that changes being made to CAPPS were consistent and in line with business requirements. We made six recommendations to the District that focused on the need to implement effective management controls and processes for maintaining, operating, and protecting CAPPS.

Together, those reports established a framework for actions the District needed to take in order to (1) improve its financial management and (2) avoid costly failure when acquiring a new financial management system that would successfully meets its needs. In our reports, we indicated that major improvements in the District’s financial management and other management information can only be realized if they are part of an overall assessment of processes, people, and equipment.

As of April 3, 2001, the District had implemented 3 of the 16 recommendations we made since 1995, and critical recommendations have yet to be fully addressed. The District had action in process on another 13 recommendations. Table 2 in appendix I provides the implementation status of recommendations made in our reports and testimony on SOAR implementation covering fiscal years 1996 through 2000.

Conclusions

The District continues to face significant challenges in its efforts to put in place a financial management framework that ensures timely and reliable financial data on the cost of the District’s operations. In its efforts to meet an overly ambitious schedule, the District has spent considerably more money than planned to acquire a system that—6 years after we began our review and started making recommendations —now serves as yet another cautionary example of the risks involved in not following a disciplined approach to the planning, acquisition, and implementation of a financial management system.

Almost 4 years after the District’s acquisition of its core financial management system, SOAR and related systems are in various stages of implementation and some elements have been put on hold. The current mix of components involves duplication of effort and requires cumbersome manual processing instead of automated interfaces. Staff members who use the system are inadequately trained. In its current state, the system is unable to produce relevant, useful, timely, and reliable information adequate to the needs of government officials for assessing the costs of programs, measuring program performance, and making well- informed decisions in forming the city’s budget and in providing city services.

With a system that is still incomplete more than 2 years after the planned date for citywide implementation, the District has already spent over 50 percent more than originally planned for SOAR implementation. The project continues to experience implementation delays, and the final cost of the complete financial management system cannot yet be determined.

Disciplined acquisition and implementation processes are designed to prevent the types of problems experienced by the District in its financial management systems implementation. The key to a disciplined system development effort is the use of disciplined processes in multiple areas, including requirements management, project planning, project tracking and oversight, quality assurance, configuration management, and risk management. These key areas have been the focus of our recommendations in reports dating back to 1995. The District has not yet completed action on most of these recommendations and has failed to institute the disciplined approach needed to ensure the successful implementation and management of a financial management system.

The District’s difficulties reflect the experience of other entities that have attempted to build a financial management system without first laying a solid foundation. Essential to that foundation is the definition of requirements. A system cannot be counted on to fill needs that have not been clearly defined. When those needs are later identified, retrofitting software can cost significantly more than the same work done during original development. The District continues to develop its system without clearly defined user requirements.

Although the District recently received its fourth consecutive unqualified or “clean” audit opinion on its financial statements, the financial information needed by decisionmakers to measure and manage performance requires greater precision and more timely access than that needed to satisfy a financial audit. Furthermore, to continue achieving clean opinions without the support of an efficient financial management system, officials and staff will be forced each year to devote extraordinary effort at the expense of other city government operations. As the city moves toward greater financial independence, the weakness of its financial management system may become increasingly difficult to overcome.

As we recently reported in our Executive Guide, to provide meaningful information to decisionmakers, entities must develop systems that support the partnership between finance and operations. Entities must ensure that the systems accurately measure the program costs and that they provide decisionmakers and line managers with timely, accurate financial information on the quality and efficiency of business processes and performance. Entities must also identify their human capital needs by conducting a human capital assessment in order to develop human capital strategies to address current and future risks faced by the entity. Such an assessment is critical to helping entities establish the systems and processes needed to successfully improve financial management and accountability.

District officials need to take time now to assess the current status of the city’s financial management system, to identify problems, and to establish a disciplined process to address these problems through the completion of its financial systems implementation. As we discussed in our Executive Guide, financial management improvement needs to be an entitywide priority—in this case, the District—overseen by leadership that is in control of the process and accountable for its success. Financial and program managers need to be able to rely on the system for adequate, timely cost and performance information needed to manage costs, measure performance, make program funding decisions, and analyze outsourcing or privatization options. With such information District decisionmakers will have the tools they need to meet the demands of managing the city’s finances efficiently and serving its citizens effectively.

Recommendations

Before moving forward on the implementation of the District’s financial management system, we recommend that the Mayor, in concert with the Chief Financial Officer, take the following actions: Assess the status of current financial management operations, including financial management policies and procedures and systems acquisition and development policy and procedures, and determine whether the current systems have the capability of meeting the District’s financial management needs.

Develop an overall concept of operations which clearly articulates overall quantitative and qualitative system characteristics to the user, developer, and other organizational elements and which facilitates understanding of the user organizations, missions, and organizational objectives from an integrated systems point of view.

Develop an action plan based on that assessment and the overall concept of operations that addresses any identified weaknesses, including the necessary systems and procedural changes, and that specifies a disciplined process with milestones and clear accountability. Incorporate our prior, open recommendations in the action plan to address the key areas of requirements development and management, project planning, project tracking and oversight, quality assurance, and training as they apply to components of the system that are not yet fully implemented, including the fixed asset module, performance budgeting, personnel and payroll, procurement, integrated tax, and all interfaces.

Determine the competencies required at leadership, management, and functional levels for financial and nonfinancial managers and develop appropriate training. Strictly enforce the implementation of the training curriculum and mandate attendance at user training sessions.

Conduct an assessment of the District’s human capital needs for financial management in order to strategically develop its financial management team to successfully address the current weaknesses in financial management systems, as well as to support the District’s overall mission, goals, and objectives.

Complete the reengineering of the budget process in conjunction with the implementation of a budget and project costing system.

Agency Comments and Our Evaluation

In commenting on a draft of this report, the Chief Financial Officer of the District agreed with our recommendations and provided additional details on four areas: (1) our prior recommendations, (2) our recommendation about assessing human capital needs, (3) our recommendation regarding the budget process, and (4) implementation of SOAR. With respect to the CFO’s comments on SOAR, the results of our work showed that the CFO’s characterization of the progress made to date was overly optimistic. The CFO’s comments are reprinted in appendix II. A representative from the Mayor’s Office also reviewed this draft, along with the CFO’s comments, and stated that the Mayor’s Office had no further comments.

In regard to implementing our prior recommendations, the CFO stated that the District is taking action as described in a March 12, 2001, letter to us. This letter was in response to our recent request that the District provide us an update on the actions it had taken to address recommendations from our December 1999 report. As part of our ongoing work in the District, we will be evaluating these actions to determine whether they satisfactorily address our prior recommendations.

Concerning our recommendation that the District assess its human capital needs, the CFO noted that the District had taken the initial step in conducting a human capital assessment by engaging a professional services firm to help review the District’s organizational structure and identify performance measures and best practices. However, as the CFO noted, the professional services firm review provides the groundwork for the first phase of the CFO’s financial management performance measures program. A complete human capital assessment will be an essential part of the CFO’s improved financial management leadership and support.

Although the CFO agreed with our recommendation that the District complete the reengineering of the budget process in conjunction with the implementation of a budget and project costing system, the CFO took exception to certain statements pertaining to our finding. Specifically, the CFO disagreed with our statement that the fiscal year 2002 budget formulation process did not have the benefit of an “implemented financial system gathering and formulating its budget data” and that it would not have “adequate program-level cost and budget results data for fiscal year 2001.” The CFO stated that the fiscal year 2002 budget process did in fact integrate data from several financial systems: CAPPS, UPPS, and SOAR. However, this reliance on compiling data generated from multiple, nonintegrated systems contributed to our finding that the District relies on a cumbersome process to generate financial information. Instead of relying on one unified system to reliably and routinely provide information as needed, the District must compile information from various different systems, which creates inefficiencies and rework.

The CFO also stated that the District has an updated timetable and comprehensive plan for fully implementing the SOAR system. However, at the time we had finalized our report, the District had not provided us with a plan. In addition, as discussed in our report, the District’s performance budget module has been put on hold and the fixed asset module is incomplete. Both of these modules are key components of the SOAR system. Further, the implementation of systems that feed into SOAR— personnel and payroll, procurement, and tax —is incomplete and the systems lack electronic interfaces with SOAR. Also, it is uncertain whether the currently envisioned successor to the personnel and payroll system— CAPPS—will be retained or whether an entirely new system is needed, and we were unable to obtain updated timetables and comprehensive plans for the implementation of these key feeder systems. As we recommended, the District needs to formulate a comprehensive plan that includes details of estimated dates, actions needed, and assignment of responsibilities for the completion of these modules and related systems.

The CFO also stated that the District’s annual financial statements are an output of the SOAR system and thus reliable, auditable financial data is available from SOAR and the Executive Information System. However, as we noted in our report, the financial information needed by decisionmakers to measure and manage performance requires greater precision and more timely access than that needed to satisfy a financial audit. Financial and program managers need to be able to rely on the system for adequate, timely cost and performance information needed to manage costs, measure performance, make program funding decisions, and analyze outsourcing or privatization options.

The CFO also stated that the core SOAR implementation was delivered on schedule and performs as intended. He further noted that the increased costs of the SOAR implementation were directly associated with changes in scope not cost overruns. As we noted in our report, however, the District’s performance budget module has been put on hold and the fixed asset module is incomplete. Both of these modules are key components of the SOAR system. Further, as discussed above, the implementation of systems that feed into SOAR—personnel and payroll, procurement, and tax—is incomplete and the systems lack electronic interfaces with SOAR. As we also discussed, many of the cost increases were also the result of additional requirements and the District not completely identifying user requirements up-front.

Finally, in regard to the CFO’s comment that not all SOAR users are required to complete all core modules, according to the SOAR PMO, less than 50 percent of the SOAR user community had completed the new core training curriculum.

We are sending copies of this report to Senator Mike DeWine, Senator George Voinovich, Senator Mary Landrieu, Senator Richard J. Durbin, Representative Chaka Fattah, Representative Constance A. Morella, and Representative Eleanor Holmes Norton in their capacities as Chairmen or Ranking Minority Members of Senate and House Subcommittees. We are also sending copies of this report to Anthony A. Williams, Mayor of the District of Columbia; Natwar Gandhi, Chief Financial Officer of the District of Columbia; Charles Maddox, Inspector General of the District of Columbia; Deborah K. Nichols, District of Columbia Auditor; Suzanne Peck, Chief Technology Officer; and Alice Rivilin, Chairman of the District of Columbia Financial Responsibility and Management Assistance Authority. Copies will be made available to others upon request.

Please contact me at (202) 512-2600 or Jeanette Franzel at (202) 512-9406 or by e-mail at franzelj@gao.gov if you have any questions about this report. Other major contributors to this report were Richard Cambosos, Linda Elmore, Maxine Hattery, Jeffrey Isaacs, John C. Martin, Meg Mills, and Norma Samuel.

Appendix I: Status of Prior Years’ Recommendations Pertaining to SOAR Implementation

Date and report number To the District of Columbia Financial Responsibility and Management Assistance Authority (Authority) June 21, 1995 GAO/T-AIMD-95-176

Status as reported by District officials 1. Study the accounting and financial management information needs of the District of Columbia government.

Completed. The Authority has (1) performed site visits and benchmarking analysis of accounting and financial management information systems similar to that used in the District; (2) hired a consultant with extensive business process reengineering and systems implementation experience to analyze the District’s financial management information systems implementation effort; and (3) created a System of Accounting and Reporting (SOAR) Steering Committee, headed by the Chair of the Authority, which includes the Mayor, CFO, Chief Technology Officer, Inspector General, and a DC Council member. Action in process. According to the Authority, it is (1) assessing the functionality of the SOAR Performance Budgeting module; and (2) including performance budgeting as an agenda item for the SOAR Steering Committee. See items 3 through 8 below.

Recommendation addresses life-cycle support of the software; and (7) develop a written policy for software acquisition planning. 4. Requirements Development and Management: (1) develop an organizational policy for establishing and managing software- related requirements; (2) clearly assign responsibility for requirements development and management; (3) document other resource requirements or resources expended for requirements development activities; (4) develop the capability to trace between contractual requirements and the contractor’s work products; and (5) develop measurements to determine the status of the requirements development and management activities.

Status as reported by District officials alternatives and cost-benefit analysis of out-sourcing the data center and upgrading versus replacing the current system. Action in process. The Authority is: (1) emphasizing the importance of implementing and enforcing clear policies and lines of accountability through the SOAR Steering Committee; (2) requiring that the District provide documentation and justification for resources requested or expended; and (3) emphasizing the importance of explicitly linking contractor payments to specific deliverables through the use of work breakdown structures. Action in process. The Authority has emphasized developing quantitative measures of project performance. Examples include emphasis on the internal transactions files and timeliness and accuracy of payroll data. Action in process. The Authority has emphasized to the SOAR PMO the importance of developing clear project deliverables and matching these to costs through the development of work breakdown structures.

5. Project Management: (1) develop a written policy for the execution of the software project; (2) authorize the project manager to independently alter either the performance, cost, or schedule; and (3) require that measurements be taken to determine the status of project management activities. 6. Contract Tracking and Oversight: (1) develop written policy for contract tracking and oversight activities for the financial management system project; (2) support the project team with contracting specialists; (3) require that the project team review the contractor’s planning documents (for example, the project management plan, software risk management plan, software engineering plan, configuration management plan); (4) assign someone responsibility for maintaining the integrity of the contract; and (5) take measurements to determine the status of contract tracking and oversight activities. 7. Evaluation: (1) develop written policy for managing the evaluation of acquired software products and services; (2) develop a documented evaluation plan; (3) develop evaluation requirements in conjunction with system requirements; (4) assess the contractor’s performance for compliance with evaluation requirements; (5) develop measurements to determine the status of evaluation activities; and (6) ensure that the Authority and project manager review the status of evaluation activities.

Action in process. Utilizing benchmarking performance management and best practices, the District is establishing a program to ensure adherence to best technology practices for all future critical systems software acquisitions. Policies and plans are being developed within the framework of a long-term technology blueprint.

Recommendation 8. Acquisition Risk Management: (1) develop written policy for software acquisition risk management; (2) designate a group to be responsible for coordinating software acquisition risk management activities; (3) define resource requirements for acquisition risk management; (4) ensure that individuals designated to perform software acquisition risk management have adequate experience and training; (5) integrate software acquisition risk management activities; (6) develop a software acquisition risk management plan in accordance with a defined software acquisition process; (7) develop a documented acquisition risk management plan and conduct risk management as an integral part of the solicitation, project performance management, and contract performance management processes; (8) track and control software acquisition risk handling actions until the risks are mitigated; and (9) ensure that risk management activities are reviewed by the Authority and the project manager.

Status as reported by District officials Action in process. The Authority has: (1) created the SOAR Steering Committee, responsible for coordinating a variety of activities including software acquisition risk management; (2) emphasized the importance of ensuring that information technology employees within the District are properly screened, certified, and qualified; and (3) hired a consultant to review and assess overall SOAR acquisition and implementation performance, including risk management.

9. Clean up existing data in the financial systems and place special emphasis on ensuring that basic accounting principles and procedures are followed. 10. Establish a process of accountability for implementation of management initiatives.

Completed. The District has cleaned up its financial data and has continued to place an emphasis on accounting principles and policies. The OCFO obtained contractual assistance to work with the District agencies, identify required adjustments to the SOAR system balances, and ensure that these adjustments were properly recorded and reflected in SOAR. In addition, the District reestablished the Committee for Financial Excellence charged to build an infrastructure that supports a strong financial base. Action completed. According to the then- Interim Chief Financial Officer, all management reform and the reporting of initiatives is done by the Chief Management Officer. This monthly reporting process captures information by agency, including funding; expense; cost saving; and project activity, including phase, duration, Status as reported by District officials start date, and completion date.

See items 11 through 15 below.

11. Develop and maintain a risk management plan. 12. Develop a requirements baseline and obtain agreement between the program office and the system users.

According to the Director, Enterprise Office, action is in process to address this recommendation. According to the Director, Enterprise Office, action is in process to address this recommendation. According to the Director, Enterprise Office, action is in process to address this recommendation.

13. Implement a configuration control process to control and document further modifications being made to CAPPS. The process should (1) clearly define and assess the effects of modifications on future product upgrades before the modification is approved, (2) clearly document the software products that are placed under configuration management, and (3) maintain the integrity and traceability of the configuration throughout the system life cycle. 14. Develop and implement a life-cycle support plan, assign responsibility for life- cycle maintenance, and develop an estimate of maintenance and operation costs for CAPPS. 15. Develop and implement a security plan based on a realistic risk assessment of CAPPS security vulnerabilities.

According to the Director, Enterprise Office, action is in process to address this recommendation.

16. Develop a centralized file for contract task orders and other contract documentation related to CAPPS.

According to the Director, Enterprise Office, action is in process to address this recommendation. According to the Director, Enterprise Office, action is in process to address this recommendation.

As discussed in the report body, CAPPS, while not a component of SOAR, is a critical interface to the system.

Appendix II: Comments From the District of Columbia

The following are GAO’s comments on the District of Columbia’s April 3, 2001 letter.

GAO Comments

1. See the “Agency Comments and Our Evaluation” section of this report. 2. The report has been changed to show 9 instead of 10 training modules.

As stated in our report, according to the SOAR PMO, less than 50 percent of the SOAR user community had completed the new core training curriculum.

Ordering Information

The first copy of each GAO report is free. Additional copies of reports are $2 each. A check or money order should be made out to the Superintendent of Documents. VISA and MasterCard credit cards are also accepted.

Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

Orders by mail: U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Orders by visiting: Room 1100 700 4th St., NW (corner of 4th and G Sts. NW) Washington, DC 20013 Orders by phone: (202) 512-6000 fax: (202) 512-6061 TDD (202) 512-2537 Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.

To Report Fraud, Waste, and Abuse in Federal Programs

Web site: http://www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov 1-800-424-5454 (automated answering system)